InvalidJarIndexException due to bug in sun.misc.JarIndex.merge()

FULL PRODUCT VERSION :
java version "1.6.0_16"
Java(TM) SE Runtime Environment (build 1.6.0_16-b01)
Java HotSpot(TM) Server VM (build 14.2-b01, mixed mode)

(but it is probably present also in other versions)

A DESCRIPTION OF THE PROBLEM :
Summary: sun.misc.JarIndex.merge() calls toIndex.add() with package/directory name as first parameter but sun.misc.JarIndex.add(String fileName, String jarName) expects a filename; this can lead to possibly invalid package names being added.

(Note: I got the sourcecode for the sun.misc classes from openjdk 6 b17)

Longer Description:

Class sun.misc.JarIndex has a method
    public void merge(JarIndex toIndex, String path)

At the end of the loop in this method
    toIndex.add(packageName, jarName);
is being called.

As the variable name suggests and my own debugging output (resulting from using a custom version of this class with some logging added and a custom bootclasspath for the JVM) supports "packageName" is the name of a package/directory normally.

But sun.misc.JarIndex.add(String fileName, String jarName) expects (as the parameter name suggests and the code of the add() method shows) a resource/file name, from which it then tries to create a package/directory name by cutting off the part after the last '/'.

Thus, if (as happened in our project) for example
    toIndex( "dir1/dir2/packagedir", "somejar.jar" )
is called, the add() method will wrongly interpret "dir1/dir2/packagedir" as a resource/file name, turn it into "dir1/dir2" and then memorizes that this possibly actually invalid/non-existent package (as there may not be any files/resources directly in "dir1/dir2" - as was the case with us) can be found in "somejar.jar" though that actually isn't the case.

Thus if later sun.misc.URLClassPath$JarLoader, method
    getResource(final String name, boolean check, Set<String> visited)
is called with "name" something like "dir1/dir2/packagedir$classname" (which can happen e.g. with the Groovy parser/compiler which seems to try to find various variants of classes/classnames) gets "somejar.jar" as one of the entries in "jarFilesList" as one of the JARs that are supposed to contain that class.

The method validIndex() which is called in the loop there and which actually directly loops over all entries in the JARs then is not able to find any entry for "dir1/dir2" in "somejar.jar" (if that package does not contain any files/resources, as mentioned above) and throws an "InvalidJarIndexException".

The bug occurs only in specific situations (merge() must actually be called, package structure must be "right", etc.) but is then of course reproducible every time.

As a sidenote, the message of the InvalidJarIndexException should really provide some info (which is easily available at that point) about what resource getResource() was trying to find and in which JAR, instead of simply (re-)stating "invalid index". Had we had this info available right from the start it would have saved us already a whole day of analysis.

On another sidenote, it's great that the sourcecode of the JVM is freely available, otherwise we probably still wouldn't have made *any* progress with this problem :-)


REPRODUCIBILITY :
This bug can be reproduced occasionally.

CUSTOMER SUBMITTED WORKAROUND :
Without patch to sun.misc.JarIndex: Do not use any JAR indices/INDEX.LIST .

  Suggested patch/fix for JarIndex:

Split add() into add() and addPackage():

public void add(String fileName, String jarName) {
    String packageName;
    int pos;
    if((pos = fileName.lastIndexOf("/")) != -1) {
        packageName = fileName.substring(0, pos);
    } else {
        packageName = fileName;
    }
    addPackage(packageName, jarname);
}

public void addPackage(String packageName, String jarName) {
    // add the mapping to indexMap
    addToList(packageName, jarName, indexMap);

    // add the mapping to jarMap
    addToList(jarName, packageName, jarMap);
}

And change the call in merge() to

public void merge(JarIndex toIndex, String path) {
// [...lines omitted...]
            toIndex.addPackage(packageName, jarName);
        }
    }
}