Summary

Add flags to further control AOT operations, as initially defined in JEP 483. The new flags will allow a single command (invocation of the JVM) to perform a combined "train and assemble" function.

Problem

In JEP 483 an AOT cache must be requested by issuing a pair of commands (invocations of the JVM), which communicate by a temporary file which the user must name and manage (ie., delete after completion).

We define a new JVM invocation mode, a combined training and assembly operation.

The additional flags of this CSR are necessary to trigger the new combined operation, and to control its details.

See the JEP https://bugs.openjdk.org/browse/JDK-8350022 for more details on the new operation, including its motivation and envisioned usage.

See the CSR https://bugs.openjdk.org/browse/JDK-8338243 for details on the pre-existing flags of JEP 483.

Solution

Extend the existing AOTCache command option with one directional inflection, AOTCacheOutput.

Then, adjust default behaviors (under AOTMode=auto and AOTMode=record) to invoke an extra sub-command of the JVM (with AOTMode=create), when the user runs a Java application with AOTCacheOutput. This will assist the user by orchestrating the steps necessary to creating an AOT cache in one command invocation.

Also, add a new environment variable JDK_AOT_VM_OPTIONS, closely akin to JAVA_TOOL_OPTIONS, to control fine details of the second sub-command (AOT cache assembly). All initial VM options are passed to both sub-commands, and JDK_AOT_VM_OPTIONS is picked up only by the second sub-command.

Specification

New VM flag:

--- a/src/hotspot/share/cds/cds_globals.hpp
+++ b/src/hotspot/share/cds/cds_globals.hpp
@@ -119,6 +119,9 @@
           "Cache for improving start up and warm up")                       \
           constraint(AOTCacheConstraintFunc, AtParse)                       \
                                                                             \
+  product(ccstr, AOTCacheOutput, nullptr,                                   \
+          "Specifies the file name for writing the AOT cache")              \
+                                                                            \
   product(bool, AOTInvokeDynamicLinking, false, DIAGNOSTIC,                 \
           "AOT-link JVM_CONSTANT_InvokeDynamic entries in cached "          \
           "ConstantPools")                                                  \

Java man page update:

 --- a/src/java.base/share/man/java.md
 +++ b/src/java.base/share/man/java.md
 @@ -3995,8 +3995,8 @@
    can be used, but the "archived module graph" feature will be disabled. This can lead to increased
    start-up time.

 -To diagnose problems with the above options, you can add `-Xlog:cds` to the application's VM
 -arguments. For example, if `--add-modules jdk.jconcole` was specified during archive creation
 +To diagnose problems with the AOT options, you can add `-Xlog:aot` to the application's VM
 +arguments. For example, if `--add-modules jdk.jconsole` was specified during archive creation
  and `--add-modules jdk.incubator.vector` is specified during runtime, the following messages will
  be logged:

 @@ -4045,31 +4045,66 @@

  The AOT cache can be used with the following command-line options:

 -`-XX:AOTCache:=`*cachefile*
 +`-XX:AOTCache=`*cachefile*
  :   Specifies the location of the AOT cache. The standard extension for *cachefile* is `.aot`.
 -    If `-XX:AOTCache` is specified but `-XX:AOTMode` is not specified,
 -    then `AOTMode` will be given the value of `auto`.
 +    This option cannot be used together with `-XX:AOTCacheOutput`.

 -`-XX:AOTConfiguration:=`*configfile*
 +    This option is compatible with `AOTMode` settings of `on`, `create`, or `auto` (the default).
 +    The *cachefile* is read in AOT modes `on` and `auto`, and is ignored by mode `off`.
 +    The *cachefile* is written by AOT mode `create`.  In that case, this option is
 +    equivalent to `-XX:AOTCacheOutput=`*cachefile*.
 +
 +`-XX:AOTCacheOutput=`*cachefile*
 +:   Specifies the location of the AOT cache to write. The standard extension for *cachefile* is `.aot`.
 +    This option cannot be used together with `-XX:AOTCache`.
 +
 +    This option is compatible with `AOTMode` settings of `record`, `create`, or `auto` (the default).
 +
 +`-XX:AOTConfiguration=`*configfile*
  :   Specifies the AOT Configuration file for the JVM to write to or read from.
 -    This option can be used only with `-XX:AOTMode=record` and `-XX:AOTMode=create`.
      The standard extension for *configfile* is `.aotconfig`.

 -`-XX:+AOTMode:=`*mode*
 -:   *mode* must be one of the following: `off`, `record`, `create`, `auto`, or `on`.
 +    This option is compatible with `AOTMode` settings of `record`, `create`, or `auto` (the default).
 +    The *configfile* is read by AOT mode `create`, and written by the other applicable modes.
 +    If the AOT mode is `auto`, then `AOTCacheOutput` must also be present.

 --   `off`: no AOT cache is used.
 +`-XX:AOTMode=`*mode*
 +:   Specifies the AOT Mode for this run.
 +    *mode* must be one of the following: `auto`, `off`, `record`, `create`, or `on`.

 --   `record`: Execute the application in the Training phase.
 -    `-XX:AOTConfiguration=`*configfile* must be specified. The JVM gathers
 -     statistical data and stores them into *configfile*.
 +-   `auto`: This AOT mode is the default, and takes effect if no `-XX:AOTMode` option
 +    is present.  It automatically sets the AOT mode to `record`, `on`, or `off`, as follows:
 +     - If `-XX:AOTCacheOutput=`*cachefile* is specified, the AOT mode is changed to `record`
 +       (a training run, with a subsequent `create` operation).
 +     - Otherwise, if an AOT cache can be loaded, the AOT mode is changed to `on` (a production run).
 +     - Otherwise, the AOT mode is changed to `off` (a production run with no AOT cache).

 --   `create`: Perform the Assembly phase. `-XX:AOTConfiguration=`*configfile*
 -     and `-XX:AOTCache=`*cachefile*  must be specified. The JVM reads the statistical
 -     data from *configfile* and writes the optimization artifacts into *cachefile*.
 +-   `off`: No AOT cache is used.
 +    Other AOT command line options are ignored.
 +
 +-   `record`: Execute the application in the training phase.
 +     At least one of `-XX:AOTConfiguration=`*configfile* and/or
 +     `-XX:AOTCacheOutput=`*cachefile* must be specified.
 +     If `-XX:AOTConfiguration=`*configfile* is specified, the JVM gathers
 +     statistical data and stores them into *configfile*.
 +     If `-XX:AOTConfiguration=`*configfile* is not specified, the JVM uses
 +     a temporary file name, which may be the string `AOTCacheOutput+".config"`,
 +     or else a fresh implementation-dependent temporary file name.
 +     If `-XX:AOTCacheOutput=`*cachefile* is specified, a second JVM process is launched
 +     to perform the Assembly phase to write the optimization artifacts into *cachefile*.
 +
 +     Extra JVM options can be passed to the second JVM process using the environment
 +     variable `JDK_AOT_VM_OPTIONS`, with the same format as the environment variable
 +     `JAVA_TOOL_OPTIONS`, which is
 +     [defined by JVMTI](https://docs.oracle.com/en/java/javase/24/docs/specs/jvmti.html#tooloptions).
 +
 +-   `create`: Perform the Assembly phase. `-XX:AOTConfiguration=`*configfile* must be
 +     specified.
 +     The JVM reads history and statistics
 +     from *configfile* and writes the optimization artifacts into *cachefile*.
       Note that the application itself is not executed in this phase.

 --   `auto` or `on`: These modes should be used in the Production phase.
 +-   `on`: Execute the application in the Production phase.
       If `-XX:AOTCache=`*cachefile* is specified, the JVM tries to
       load *cachefile* as the AOT cache. Otherwise, the JVM tries to load
       a *default CDS archive* from the JDK installation directory as the AOT cache.
 @@ -4086,7 +4121,7 @@
         Since the AOT cache is an optimization feature, there's no guarantee that it will be
         compatible with all possible JVM options. See [JEP 483](https://openjdk.org/jeps/483),
         section **Consistency of training and subsequent runs** for a representative
 -       list of scenarios that may be incompatible with the AOT cache for JDK 24.
 +       list of scenarios that may be incompatible with the AOT cache.

         These scenarios usually involve arbitrary modification of classes for diagnostic
         purposes and are typically not relevant for production environments.
 @@ -4093,7 +4128,7 @@

    When the AOT cache fails to load:

 -.    - If `AOTMode` is `auto`, the JVM will continue execution without using the
 +.    - If `AOTMode` was originally `auto`, the JVM will continue execution without using the
         AOT cache. This is the recommended mode for production environments, especially
         when you may not have complete control of the command-line (e.g., your
         application's launch script may allow users to inject options to the command-line).
 @@ -4103,7 +4138,7 @@
       - If `AOTMode` is `on`, the JVM will print an error message and exit immediately. This
         mode should be used only as a "fail-fast" debugging aid to check if your command-line
         options are compatible with the AOT cache. An alternative is to run your application with
 -.      `-XX:AOTMode=auto -Xlog:cds` to see if the AOT cache can be used or not.
 +.      `-XX:AOTMode=auto -Xlog:cds,aot` to see if the AOT cache can be used or not.

. `-XX:+AOTClassLinking`
. :   If this option is enabled, the JVM will perform more advanced optimizations (such
 @@ -4120,6 +4155,16 @@
     When `-XX:AOTMode` *is not used* in the command-line,  `AOTClassLinking` is disabled by
     default to provide full compatibility with traditional CDS options such as `-Xshare:dump.

 +The first occurrence of the special sequence `%p` in `*configfile* and `*cachefile* is replaced
 +with the process ID of the JVM process launched in the command-line, and likewise the
 +first occurrence of `%t` is replace by the JVM's startup timestamp.
 +(After replacement there must be no further occurrences of `%p` or `%t`, to prevent
 +problems with sub-processes.)  For example:
 +
 +>   `java -XX:AOTConfiguration=foo%p.aotconfig -XX:AOTCacheOutput=foo%p.aot -cp foo.jar Foo`
 +
 +will create two files: `foopid123.aotconfig` and `foopid123.aot`, where `123` is the
 +process ID of the JVM that has executed the application `Foo`.

[1] The new ccstr option AOTCacheOutput supplies the same data as AOTCache, but is always interpreted in the output direction. As such, it is illegal for operation modes which read an AOT cache file, notably AOTMode=on (a production run), and it is legal for AOTMode=create (as an alternative to the non-directional AOTCache).

It is illegal to specify both AOTCache and AOTCacheOutput on the same command line.

[2] AOTCacheOutput, when no AOTMode option is present (or either AOTMode=auto or AOTMode=record is explicitly present), requests the new behavior, of combined training and assembly. In this case, the presence of AOTCacheOutput switches on AOTMode=record, which makes a training run as before. After that, a second sub-command operates with AOTMode=create, assembling the AOT cache named by the AOTCacheOutput option.

During combined training and assembly, the VM invocation is automatically split (transparently to the user) into two VM invocations. One sub-command operates with AOTMode=record, and the second sub-commmand operates with AOTMode=create. They share all other options passed into the original command.

Rationale: It is much easier to explain one command splitting into two sub-commands running under two pre-existing modes, than to explain the interactions of a new combined mode with all other VM features. That is why we choose not to invent a new mode such as record+create.

[3] During combined training and assembly, any AOTConfiguration option present (from JEP 483) works as previously documented. That is, the named file is written, then read, and not deleted. But as a new behavior, if AOTConfiguration is not specified on the command line, the VM chooses a temporary file name and uses a file of temporary name to transmit AOT configuration from the first sub-command to the second. This file is deleted after use. (The temporary name may either be the fixed name AOTCacheOutput+".config", or else an implementation-dependent temporary file name, not interfering with any existing file.) Indeed, the VM implementation is free to use any appropriate means, instead of a temporary file, for transmitting this data between sub-commands.

[4] During combined training and assembly, the training sub-command (AOTMode=record) does not act on the AOTCacheOutput option; it applies only to the cache assembly sub-command (AOTMode=create).

[5] The pre-existing environment variable JAVA_TOOL_OPTIONS takes effect as usual when the VM starts up. Sub-commands may be thought of as receiving edited versions of the original VM options, as if via JAVA_TOOL_OPTIONS, and debugging mechanisms (based on -Xlog) will provide a view of the flags being passed to the sub-commands. Note that only the training sub-command receives the Java application main class and arguments.

[6] If the value of AOTMode is determined to be create after normal parsing of the JVM options, the JVM will also read the value of JDK_AOT_VM_OPTIONS and parse it in the same way as JAVA_TOOL_OPTIONS. Thus, JDK_AOT_VM_OPTIONS can modify settings parsed from the earlier options, during any assembly phase. If JDK_AOT_VM_OPTIONS itself contains an AOTMode option, it must select the AOT mode create.

Rationale: Maintainers of the jruby launcher have requested that the JVM handle the details of AOT configuration management and sub-command launching, but launchers sometimes need fine control over sub-commands that they invoke directly or indirectly. This environment variable gives a necessary hook for such fine control. The same argument applies more generally to script-based use of the JVM, where both $JAVA_TOOL_OPTIONS and $JDK_AOT_VM_OPTIONS environment variables are likely to be useful for injecting common options used by the whole script. The specific use of $JDK_AOT_VM_OPTIONS is also likely to be useful for testing, diagnosis, and research related to AOT technologies.

[7] In many file options of the HotSpot JVM, the first occurrence of the special string %p, and/or %t, is replaced by the JVM's PID, and/or startup timestamp. This replacement occurs on the AOTCache, AOTCacheOutput, and AOTConfiguration options as well. It is always the user’s job to find such unpredictably-named files after they are produced. (After replacement there must be no further occurrences of %p or %t, to prevent problems with sub-processes.) The replacement is done in the parent sub-command, not the child sub-command, so that the interpolated numeral always corresponds to the JVM process that the user launched. This expansion rule applies to all AOT modes, not just the new behaviors documented here.

Note: This is a slightly incompatible change from JDK 24, in that some file names containing the substring %p are not currently expanded. This substring %p, standard to HotSpot, is chosen so as to be unlikely to occur unintentionally.

Notes on the Resulting User Model

With these changes in place, the combined AOT story runs as follows.

If the AOTMode option is not mentioned, it starts with a default setting (of auto, as documented in JEP 483), and is then ergonomically reset to be consistent with the other AOT options. If AOTCacheOutput is present, the mode is ergonomically set to record and then create, implementing combined training and assembly. Also, in this case, if no AOTConfiguration name is given, the configuration output and input is managed as a temporary file.

Note that while AOTCache denotes either input or output (depending on AOTMode), the new options AOTCacheOutput fully disambiguates data flow direction into or out of AOT cache files, regardless of mode. Thus, AOTCacheOutput will always create the named AOT cache; it will never load it.

Note also that in AOTMode=create only, the -XX:AOTCache=newcache.aot option specifies an output AOT cache file, not an input. In that mode, the output direction may be specified explicitly as well, using the new option -XX:AOTCacheOutput=newcache.aot.

If the -XX:AOTConfiguration option is present on the command line, that file name will be used, and the file will be retained after the Java launcher exits. Otherwise, it will be deleted (if it ever existed) when the cache assembly sub-command is finished.

Thus, the AOTMode and AOTConfiguration options are not needed for simple AOT cache creation. At a minimum, with the given set of default behaviors, an AOTCacheOutput option suffices to trigger creation of an AOT cache from a training run.

Notes on the Environment Variable

In some advanced use cases, the AOT cache assembly process may need special assembly-specific command line options that differ from the training run. This is handled directly in the old two-command workflow, by simply handing the assembly-specific command line options to the second explicit command (assembly) but not the first (training). With the one-command workflow, assembly-specific options can be passed via the JDK_AOT_VM_OPTIONS environment variable.

There is currently no corresponding proposal for passing options only to the first sub-command (training run). That is because most options passed to the training run can be modified by subsequent settings (in JDK_AOT_VM_OPTIONS) of the same option.

A combined training and assembly command applies all command line options first to the training run. Then, the options are also passed to the AOT assembly phase, as if by JAVA_TOOL_OPTIONS. After these options are processed, any options in the environment string JDK_AOT_VM_OPTIONS are also applied to the AOT assembly phase.

A script or launcher may launch many JVM instances. If it is desired to create AOT caches for all of them, the option -XX:AOTCacheOutput=cache%p.aot can be added into the JAVA_TOOL_OPTIONS environment string. For example:

JAVA_TOOL_OPTIONS='-Xlog:arguments=args%p.txt -XX:AOTCacheOutput=cache%p.aot' bash launch_many_java_programs.sh

The syntax of the JAVA_TOOL_OPTIONS (and thus JDK_AOT_VM_OPTIONS) environment variable is documented here:

https://docs.oracle.com/en/java/javase/24/docs/specs/jvmti.html#tooloptions