Possibly a race condition bug in 1.1.5 in Class.getResourceAsStream() when operating on JAR files. We have been able to reproduce the bug in a small test program on 1.1.5 for WIN32, but have also seen it happen in our product when running on Solaris 1.1.5, although this program will only fail for us on WIN32.

SYMPTOMS

The bug manifests itself in a number of different ways, but has only ever been seen when more than one thread is trying to load data (and, optionally, classes) from a JAR. The actual errors encountered may be any one of the following:

    a. getResourceAsStream() may return null, even though the
       resource exists.

    b. The returned stream may be corrupt, having been loaded
       from the wrong place in the JAR.

    c. A thread attempting to load a class from the JAR may
       fail (after 'a' or 'b' above), throwing a NoClassDefFoundError.

The most likely explanations (guessingly wildly, now) are:

    1. For some reason, the JVM may not be closing the JAR file in
       each thread in a timely manner.

    2. A race condition in the JVM may be allowing the seek pointer
       within the JAR to be moved unexpectedly.

LOOK FOR THE BUG ON *YOUR* SYSTEM!

I include two simple test programs to demonstrate the bug. The first,
BugGen.java, generates the JAR of data files needed for the test.
The second, BugTest.java, is used to actually demonstrate the bug.

Both programs can be compiled and the JAR file can be created using
the following script:

javac -d . BugGen.java
java BugGen 200
javac -d . BugTest.java
jar c0vf bugtest.jar *.class bugtest

After running the above, the bug can be demonstrated using the
BugTest program with the following script:

# This will fail on WIN32 within a few minutes.
jre -cp bugtest.jar bugtest.classes.BugTest 200

The damning nail-in-the-coffin can be seen by running the same
program, but now reading from the loose files instead of the JAR:

# This will succeed
jre -cp . bugtest.classes.BugTest 200

Here are the two test programs needed for the above scripts:

//
//

import java.io.BufferedWriter;
import java.io.File;
import java.io.FileWriter;
import java.io.PrintWriter;
import java.util.Random;

/**
 * Create the data files used by the actual bug tester.
 */
public class BugGen
{
    static final int MAX_DATAFILE_BYTES = 25 * 1024;

    /**
     * Instantiate the application object and run the program.
     */
    public static void main (String [] args)
    {
        (new BugGen()).run (args);
    }

    /**
     * Create the data files needed by the bug tester.
     */
    public void run (String [] args)
    {
        try
        {
            // Gross check for number of arguments
            if (args.length != 1)
                throw new Exception (
                    "Usage: java GenBugTest nDataFiles");

            // Parse arguments
            int nDataFiles = Integer.parseInt (args[0]);

            // Generate the data files needed to run the test.
            genDataFiles (nDataFiles);
        }
        catch (Exception e)
        {
            e.printStackTrace ();
        }
    }

    /**
     * Create the specified number of data files for test harness.
     */
    public void genDataFiles (int nDataFiles) throws Exception
    {
        File dataDir = new File (new File ("bugtest"), "data");
        dataDir.mkdirs ();

        for (int id = 1; id <= nDataFiles; id++)
            genDataFile (dataDir, id);
    }

    /**
     * Create a data file for the bug test harness.
     */
    public void genDataFile (File dir, int id) throws Exception
    {
        String fileName = "data" + id;
        System.out.println ("Generating '" + fileName + "'");

        File dataFile = new File (dir, fileName);
        dataFile.delete ();

        BufferedWriter fw = null;

        try
        {
            fw = new BufferedWriter (new FileWriter (dataFile));

            Random rand = new Random ();

            int wantBytes =
                (int) (rand.nextDouble() * MAX_DATAFILE_BYTES);

            while (wantBytes > 0)
            {
                String strData = "abcdefghijklmnopqrstuvwxyz";
                fw.write (strData, 0, strData.length());
                fw.newLine ();

                wantBytes -= strData.length() + 1;
            }
        }
        finally
        {
            if (fw != null)
                fw.close();
        }
    }
}

// ----------------------------------------------------------------

//
//

package bugtest.classes;

import java.io.BufferedReader;
import java.io.File;
import java.io.InputStream;
import java.io.InputStreamReader;
import java.net.URL;
import java.util.Random;

/**
 * This class runs the actual test that demonstrates the bug.
 */
public class BugTest extends Thread
{
    static Object lock = new Object();
    static Random _rand = new Random ();
    int _threadId = 0;
    int _nDataFiles = 0;
    int _numReps = 0;
    int _dataFileId = 0;

    /**
     * The test program starts a certain number of threads, each of
     * which attempts to process a certain number of data files.
     */
    public static void main (String [] args)
    {
        try
        {
            // Gross check for number of arguments
            if (args.length < 1 || args.length > 3)
                throw new Exception (
                    "Usage: java BugTest nDataFiles nReps nThreads");

            // Parse arguments
            int nDataFiles = Integer.parseInt (args[0]);

            // How many data files should each thread process?
            int numReps = 1000;

            if (args.length >= 2)
                numReps = Integer.parseInt (args[1]);

            // How many simultaneous threads will there be?
            int numThreads = 9;

            if (args.length >= 3)
                numThreads = Integer.parseInt (args[2]);

            // Create each thread and set it on its merry way.
            for (int threadId = 1; threadId <= numThreads; threadId++)
                (new BugTest(threadId, nDataFiles, numReps)).start();
        }

        catch (Exception e)
        {
            outError (0, e);
        }
    }

    /**
     * Construct a new thread with the parameters needed for the
     * run() method.
     */
    BugTest (int threadId, int nDataFiles, int numReps)
    {
        _threadId = threadId;
        _nDataFiles = nDataFiles;
        _numReps = numReps;
        _dataFileId = 1 + (int) (_rand.nextDouble()*_nDataFiles);
    }

    /**
     * Run a test run, processing a number of data files.
     */
    public void run ()
    {
        try
        {
            for (int count = 0; count < _numReps; count ++)
                loadNextFile ();
        }
        catch (Throwable e)
        {
            outError (_threadId, e);
        }
    }

    /**
     * Load and process another data file.
     */
    public void loadNextFile () throws Exception
    {
        // Advance to the next data file, looping around at end.
        if (++_dataFileId > _nDataFiles)
            _dataFileId = 1;

        // Use '/' rather than File.separator for getResource paths.
        String resSep = "/";

        String strFile =
            resSep + "bugtest" +
            resSep + "data" +
            resSep + "data" + _dataFileId;

        // Load the data file with the specified name.
        loadDataFile (strFile);
    }

    /**
     * Use getResourceAsStream to load the specified data file.
     */
    public void loadDataFile (String fileName) throws Exception
    {
        InputStream inStream = null;
        BufferedReader inReader = null;

        try
        {
            // Dancing bear.
            printLine (fileName);

            // Open the file as a stream.
            inStream = getClass().getResourceAsStream (fileName);

            if (inStream == null)
                throw new Exception (
                    "Failed to open data file '" + fileName +
                    "'. (inStream == null)"
                );

            // Get ready to read lines of text.
            inReader =
                new BufferedReader (
                    new InputStreamReader (
                        inStream
                    )
                );

            // Read the file, looking for expected data, until
            // you hit the end of the stream.

            String line = null;

            while ((line = inReader.readLine ()) != null)
            {
                if (! line.equals ("abcdefghijklmnopqrstuvwxyz"))
                    throw new Exception (
                        "Did not read expected data from data file." +
                        " (line == '" + line + "')"
                    );
            }
        }
        finally
        {
            // Be sure to close the stream that we opened.
            if (inReader != null)
                inReader.close ();
            else if (inStream != null)
                inStream.close();
        }
    }

    /**
     * Complain about any error that we encountered.
     */
    public static void outError (
        int threadId,
        Throwable thrown)
    {
        synchronized (lock)
        {
            System.out.print (threadId + ": ");
            thrown.printStackTrace ();
            System.exit (-1);
        }
    }

    /**
     * Print a message, prefaced by the thread number.
     */
    public void printLine (String messsage)
    {
        synchronized (lock)
        {
            System.out.println (_threadId + ": " + messsage);
        }
    }
}


Here's a workaround: place your calls to load resources from the JAR file in synchronized blocks. If all of the calls are coming from instances of the same class, then add this variable to the class:

static final Object mutex = new Object();

then you can place the code that reads from the JAR in blocks like this:

code...
synchronized (mutex) {
jar reading code...
}

Otherwise you can simply synchronize the methods that read from the JAR, create a single method
to do your reading and synchronize it, or, well, you get the idea.

Just to be clear, I agree that the appropriate methods in the jar classes should be synchronized, but
I don't think this is a bug of the epic proportions you're making it out to be.


When we first came upon this bug, it was often manifesting itself as classes failing to load properly
after the open jar file had gotton confused. In my mind, I had put this bug down as getResourceAsStream
in one thread interfering with a class load in another thread at least some of the time, as opposed to
a getResourceAsStream interfering with another getResourceAsStream. However, my test program
certainly doesn't demonstrate that, so I'm going to have to go back and prove it one way or the other
with a revised test.

If, as you suggest, it's simply getResourceAsStream itself that isn't thread-safe, then your workaround
will probably be Totally Adequate (high praise indeed), assuming that the cl_ss l__ders don't call
getResourceAsStream somewhere in their bowels, and also assuming that none of the third party
classes that we use (which we don't have the sources for) don't create their own threads and then
call getResourceAsStream themselves. I'll look into this Monday 03/31/98. Thanks.


(Having performed the tests...)
Sadly, the solution proposed above did not eliminate the problem, so the question is still open for others to solve.


jdcinclude
nancy.schorr@eng 1998-05-11