Name: jl125535 Date: 11/12/2003


FULL PRODUCT VERSION :
$ java -version
java version "1.4.2"
Java(TM) 2 Runtime Environment, Standard Edition (build 1.4.2-b28)
Java HotSpot(TM) 64-Bit Server VM (build 1.4.2-b28, mixed mode)


FULL OS VERSION :
$ uname -a
Linux bat03glnxi64 2.4.18-e.31smp #1 SMP Wed May 21 17:47:59 EDT 2003 ia64 unknown

$ cat /etc/redhat-release
Red Hat Linux Advanced Workstation release 2.1AW (Derry)


A DESCRIPTION OF THE PROBLEM :
The library libzip.so shipped with this JDK re-exports a number of symbols from
zlib version 1.1.3. I believe there are 43 symbols in all, the first of which
is _tr_align, the last being zlibVersion. The list can be obtained via:

$ nm -D libzip.so | grep ' T ' | tail -43
0000000000026b70 T _tr_align
0000000000027bb0 T _tr_flush_block
000000000001d060 T _tr_init
0000000000026620 T _tr_stored_block
0000000000028b30 T _tr_tally
000000000000e160 T adler32
0000000000045ee0 T allocZip
000000000000f250 T crc32
0000000000012c90 T deflate
0000000000014770 T deflateCopy
0000000000014120 T deflateEnd
000000000000fd30 T deflateInit2_
000000000000fc20 T deflateInit_
0000000000011f50 T deflateParams
0000000000011ab0 T deflateReset
0000000000010ff0 T deflateSetDictionary
000000000000f230 T get_crc_table
000000000002d8e0 T inflate
000000000002cf00 T inflateEnd
000000000002d120 T inflateInit2_
000000000002d800 T inflateInit_
000000000002ccc0 T inflateReset
000000000002fbf0 T inflateSetDictionary
0000000000030030 T inflateSync
00000000000307b0 T inflateSyncPoint
00000000000313c0 T inflate_blocks
0000000000038b80 T inflate_blocks_free
0000000000030d30 T inflate_blocks_new
00000000000308d0 T inflate_blocks_reset
0000000000038f70 T inflate_blocks_sync_point
000000000003c7a0 T inflate_codes
0000000000041fa0 T inflate_codes_free
000000000003c4c0 T inflate_codes_new
0000000000042db0 T inflate_fast
0000000000042080 T inflate_flush
0000000000038e00 T inflate_set_dictionary
000000000003b530 T inflate_trees_bits
000000000003b9e0 T inflate_trees_dynamic
000000000003c380 T inflate_trees_fixed
000000000002caa0 T zError
000000000002cb30 T zcalloc
000000000002cc30 T zcfree
000000000002ca80 T zlibVersion

As near as I can tell, no other libraries in the JDK actually require these
symbols, and therefore they could be internal to libzip instead of being
exported.

The problem comes when I try to use the invocation API from a C/C++ program
which also uses zlib. The latest version of zlib is 1.1.4, and the two
versions cannot be mixed. Doing so results in a the VM being unable to find
classes due to zlib confusion like this:

java.lang.InternalError
        at java.util.zip.Inflater.init(Native Method)
        at java.util.zip.Inflater.<init>(Inflater.java:79)
        at java.util.zip.ZipFile.getInflater(ZipFile.java:264)
        at java.util.zip.ZipFile.getInputStream(ZipFile.java:209)
        at java.util.zip.ZipFile.getInputStream(ZipFile.java:184)
        at java.util.jar.JarFile.getInputStream(JarFile.java:359)
        at sun.misc.URLClassPath$5.getInputStream(URLClassPath.java:616)
        at sun.misc.Resource.getBytes(Resource.java:57)
        at java.net.URLClassLoader.defineClass(URLClassLoader.java:248)
        at java.net.URLClassLoader.access$100(URLClassLoader.java:55)
        at java.net.URLClassLoader$1.run(URLClassLoader.java:194)
        at java.security.AccessController.doPrivileged(Native Method)
        at java.net.URLClassLoader.findClass(URLClassLoader.java:187)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:289)
        at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:274)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:235)
        at java.lang.ClassLoader.loadClassInternal(ClassLoader.java:302)

I'm hoping that you can internalize the zlib symbols in the next release of
the JDK, and/or move to the latest version of zlib which is 1.1.4. In theory
either would solve my problem. Internalizing the symbols is probably the
best long-term solution. That would be consistent with what is done in the
Solaris JDK. The Linux linker uses the same model for exported symbols that
Solaris does.

<Subsequent Communication from Customer>
I did some more investigation over the past month or so,
and it's not a version difference (1.1.3 vs. 1.1.4) as I initially
suspected. Rather it's an _LP64 patch that I believe Sun has
applied to zlib which makes the zlib type "uLong" to be 32 bits.
That patch (coupled with an export of the symbols) is hazardous to
any application which links against an unpatched zlib and also
loads the VM.

Do you have plans to get your changes into zlib's distribution?

Removing the export of these symbols will help a great deal either
way.
</Subsequent>

STEPS TO FOLLOW TO REPRODUCE THE PROBLEM :
Using the Linux ia64 SDK:

Create a simple invocation API example, one which starts the VM, calls FindClass and invokes its static main. Set up the class path to look inside of a jar file, in order to exercise the zip library. Confirm that this works.

Next link the executable against zlib 1.1.4 using -lz. On a Redhat 2.1 box, this library exists in /usr/lib already so merely adding -lz on the link line should be enough.

Run the program and watch it fail to find the class.

A complete example (Makefile + source code) is included below.


EXPECTED VERSUS ACTUAL BEHAVIOR :
EXPECTED -
When it works, the example prints "Hello World".

When it fails (linked against libz) it reports "Cannot find Hello class.."
ACTUAL -
greg@bat03glnxi64 787$ gmake
==================================================
Without -lz, the example works fine:
LD_LIBRARY_PATH=/hub/ia64/apps/java/jdk1.4.2/b28/jre/lib/ia64/server:/hub/ia64/apps/java/jdk1.4.2/b28/jre/lib/ia64 ./works
Hello World
 
==================================================
With -lz, we cannot find classes inside jar files:
LD_LIBRARY_PATH=/hub/ia64/apps/java/jdk1.4.2/b28/jre/lib/ia64/server:/hub/ia64/apps/java/jdk1.4.2/b28/jre/lib/ia64 ./busted
Cannot find Hello class..


REPRODUCIBILITY :
This bug can be reproduced always.

---------- BEGIN SOURCE ----------
# Hopefully your web form respects tabs. Otherwise you might have to
# edit the pasted makefile to change some spaces back to tabs. Once
# you have the Makefile, simply type make to create the java and C code
# necessary to reproduce the problem.

# Modify JDK to point to yours
JDK := /hub/ia64/apps/java/jdk1.4.2/b28
JRE := $(JDK)/jre
LD_LIB_PATH := $(JRE)/lib/ia64/server:$(JRE)/lib/ia64

# Either one works
# CC := ecc
CC := gcc
CFLAGS := -g -I$(JDK)/include -I$(JDK)/include/linux

.PHONY : test
test : Hello.jar works busted
        @echo '=================================================='
        @echo 'Without -lz, the example works fine: '
        LD_LIBRARY_PATH=$(LD_LIB_PATH) ./works
        @echo ' '
        @echo '=================================================='
        @echo 'With -lz, we cannot find classes inside jar files:'
        LD_LIBRARY_PATH=$(LD_LIB_PATH) ./busted

works : invoke.c Makefile
        $(CC) $(CFLAGS) -o $@ $< $(LIBDEPS) -ldl

busted : invoke.c Makefile
        $(CC) $(CFLAGS) -o $@ $< $(LIBDEPS) -ldl -lz

invoke.c : Makefile
        grep '^# C ' Makefile | sed "s/^# C //" > $@

Hello.java : Makefile
        grep '^# JAVA ' Makefile | sed "s/^# JAVA //" > $@

Hello.class : Hello.java
        $(JDK)/bin/javac $<

Hello.jar : Hello.class
        $(JDK)/bin/jar cf $@ $<
        $(JDK)/bin/jar tf $@

.PHONY : clean
clean :
        rm -f works busted Hello.* invoke.c

# JAVA public class Hello {
# JAVA public static void main(String[] args) {
# JAVA System.out.println("Hello World");
# JAVA }
# JAVA }

# C #include <jni.h>
# C #include <dlfcn.h>
# C #include <stdlib.h>
# C
# C void die(const char *msg) { puts(msg); exit(1); }
# C
# C int main(void)
# C {
# C JavaVMInitArgs vm_args;
# C JNIEnv *env = NULL;
# C void *lib = NULL;
# C JavaVM *jvm;
# C int (*InitProc)(JavaVM**,JNIEnv**,JavaVMInitArgs*) = NULL;
# C jmethodID mid;
# C jclass cls;
# C
# C JavaVMOption options[1];
# C options[0].optionString = "-Djava.class.path=Hello.jar";
# C vm_args.version = JNI_VERSION_1_2;
# C vm_args.options = options;
# C vm_args.nOptions = 1;
# C
# C lib = dlopen("libjava.so", RTLD_NOW | RTLD_GLOBAL);
# C if (lib == NULL) die("Cannot open libjava.so..\n");
# C
# C InitProc = (int(*)(JavaVM**,JNIEnv**,JavaVMInitArgs*))
# C dlsym(lib,"JNI_CreateJavaVM");
# C
# C if (InitProc == NULL) die("Cannot find symbols..\n");
# C if (InitProc(&jvm, &env, &vm_args)) die("Cannot start the VM..\n");
# C
# C cls = (*env)->FindClass(env, "Hello");
# C if (cls == 0) die("Cannot find Hello class..\n");
# C
# C mid = (*env)->GetStaticMethodID(env,cls,"main","([Ljava/lang/String;)V");
# C if (mid == 0) die("Can't find Hello.main..\n");
# C
# C (*env)->CallStaticVoidMethod(env, cls, mid, NULL);
# C (*jvm)->DestroyJavaVM(jvm);
# C exit(0);
# C }

---------- END SOURCE ----------

CUSTOMER SUBMITTED WORKAROUND :
A real hack is to replace zlib 1.1.4 (libz.so.1) with a symlink to the
copy of libzip.so that you ship with the JDK. This only works if the
JVM is also in-process, however, so implementing this work-around for
our customers would be a real mess.

I'm successfully using the 1.4.2 JVM internally using this workaround,
but we cannot make any progress with a real product until this issue is resolved. (We also need a redistributable JRE, which isn't yet available as of the time of this writing..)
(Incident Review ID: 202008)
======================================================================
###@###.### 2003-11-13