Uploaded image for project: 'JDK'
  1. JDK
  2. JDK-8292892

Javadoc index descriptions are not deterministic



    • Bug
    • Resolution: Fixed
    • P4
    • 20
    • 15, 16, 17, 18, 19, 20
    • tools
    • 15
    • b16


      The descriptions of index entries in the JDK API Specification are not deterministic. Their differences reveal an underlying bug unrelated to the goal of reproducible builds.

      There are 55,782 index entries in the JDK 20 API Specification, but 56 of them get different descriptions depending on whether I run the build on my local workstation, on a remote Launchpad build machine, or in a QEMU/KVM virtual machine. Of those 56 index entries, 40 are static variables of classes in the package 'java.util.jar'.

      For example, each build defines a single entry for LOCCRC in the L-Index file and in the member search index, but the descriptions for the entry differ as shown below.

      Local Workstation

        LOCCRC - Static variable in class java.util.jar.JarEntry
        Search: java.util.zip.JarEntry.LOCCRC -> File not found

      Remote Launchpad

        LOCCRC - Static variable in class java.util.jar.JarOutputStream
        Search: java.util.zip.JarOutputStream.LOCCRC -> File not found

      Virtual Machine

        LOCCRC - Static variable in class java.util.jar.JarInputStream
        Search: java.util.zip.JarInputStream.LOCCRC -> File not found

      When I list the source files of the package in directory order (unsorted) on my local workstation, JarEntry is the first class found that inherits the LOCCRC variable:

        $ ls -1U ~/opt/jdk-20/src/java.base/java/util/jar/

      On the virtual machine, JarInputStream is the first such class found:

        $ ls -1U ~/opt/jdk-20/src/java.base/java/util/jar/

      At first, this issue appeared to be the usual file-ordering problem of reproducible builds. Yet the LOCCRC variable is inherited by four of the classes in the 'java.util.jar' package: JarEntry, JarFile, JarInputStream, and JarOutputStream. The variable is also inherited by four classes in the 'java.util.zip' package: ZipEntry, ZipFile, ZipInputStream, ZipOutputStream.

      The underlying issue is that the LOCCRC index entry, and others like it, should be listed once for each inheriting class. For LOCCRC, that would require eight entries. The problem seems to occur when documenting members inherited from classes with package access. Such members are to be documented as though they were declared in the inheriting class. See JDK-4780441 for details.


      System information for my local workstation running Ubuntu 20.04.4 LTS is listed below:

        $ uname -a
        Linux tower 5.15.0-46-generic #49~20.04.1-Ubuntu SMP
          Thu Aug 4 19:15:44 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux

        $ ldd --version
        ldd (Ubuntu GLIBC 2.31-0ubuntu9.9) 2.31

        $ getconf GNU_LIBPTHREAD_VERSION
        NPTL 2.31

        $ $HOME/opt/jdk-20/bin/java --version
        openjdk 20-ea 2023-03-21
        OpenJDK Runtime Environment (build 20-ea+11-661)
        OpenJDK 64-Bit Server VM (build 20-ea+11-661, mixed mode, sharing)


      I was able to reproduce the problem by building on three different systems with the following processors:

        * Local Workstation: 4-core Intel Xeon CPU E3-1225 v5
        * Remote Launchpad: 4-core AMD EPYC-Rome Processor
        * Virtual Machine: Single-core Intel Core Processor (Skylake, IBRS)

      I can't be certain that the processor played a role in the ordering of files in their directories, but it may have affected the timing of the process that created them.


      The builds of the JDK are identical.


      The builds are different, but they differ only in their Javadoc API index files and the corresponding 'member-search-index.js' file. There are 56 entries in the index that differ:

        $ git diff --numstat --shortstat local remote
        11 11 {local => remote}/index-12.txt
        1 1 {local => remote}/index-13.txt
        5 5 {local => remote}/index-18.txt
        1 1 {local => remote}/index-19.txt
        2 2 {local => remote}/index-20.txt
        1 1 {local => remote}/index-21.txt
        22 22 {local => remote}/index-3.txt
        12 12 {local => remote}/index-5.txt
        1 1 {local => remote}/index-8.txt
         9 files changed, 56 insertions(+), 56 deletions(-)

      The differences occur for index entries that are identified on my local workstation as:

        * Methods in class java.awt.BufferCapabilities.FlipContents
        * Methods in class java.time.chrono.HijrahDate
        * Methods in class jdk.incubator.vector.ByteVector
        * Static variables in class java.util.jar.JarEntry

      They are:

        java.time.chrono.HijrahDate.until(Temporal, TemporalUnit)
        jdk.incubator.vector.ByteVector.castShape(VectorSpecies<F>, int)
            VectorOperators.Conversion<Byte, F>, VectorSpecies<F>, int)
            VectorOperators.Conversion<Byte, F>, int)

      I attached the following two files that show all of the differences:

        * index-local-vs-remote.diff - Compares 'api/index-files/*.html'
        * search-local-vs-remote.diff - Compares 'api/member-search-index.js'

      I made the comparisons easier by converting the HTML files to plain text with 'w3m' and by expanding the JavaScript file using the 'js-beautify' tool, as in the examples below:

        $ w3m -dump -cols 10000 -T text/html index-12.html > index-12.txt
        $ js-beautify --end-with-newline -o search-local.js \


      I used the following shell script to narrow the scope of packages while testing:

        # Runs Javadoc for testing

        # The JDK home directory and its extracted source files

        "$jdk_dir/bin/java" --patch-module jdk.javadoc=target/classes \
            jdk.javadoc.internal.tool.Main \
            --source-path "$jdk_src/java.base" \
            -d tmp/doc -notimestamp -Xdoclint:none \
            java.util.jar "$@"


      I don't have a workaround, but I do have a fix. My pull request will follow shortly.


        Issue Links



              jgneff John Neffenger
              jgneff John Neffenger
              0 Vote for this issue
              3 Start watching this issue