Summary:

1. On unix, System.loadLibrary and dlopen look in the same places.

    On win32 they do not. The differences are that
    System.loadLibrary looks in
        ..../jre/bin always
        ..../bin if ..../bin/java is used to run java

    while LoadLibrary will look in
        .../bin if ..../bin/java is used to run java
    or .../jre/bin if ..../jre/bin/java is used to run java

    ie, if ../jdkxxx/bin/java.exe is used to run java, the .dlls in
    the jdkxxx/jre/bin dir won't be found.

    I don't think the java spec says that System.loadLibrary should look in
    the same place for libs that the native shared library function does,
    but it would seem reasonable that it does. And, some attempt has been
    made to make this be true. (eg, the setup of java.library.path attempts
    to mimic the dlopen and LoadLibrary search places.)

2. On unix, dlopen looks in the same place whether .../jdkxxx/bin/java
    or .../jdkxxx/jre/bin/java is run.

    On win32, LoadLibrary doesn't look in the same place.
    If .../jdkxxx/bin/java.exe is run, LoadLibrary looks in .../jdkxxx/bin.
    If .../jdkxxx/jre/bin/java.exe is run, LoadLibrary looks in .../jdkxxx/jre/bin.
============================

Here is some more info on the subject.



To: ###@###.###
Cc: ###@###.###,
    ###@###.###,
    scott.seligman
Subject: Re: Code review please
References: <###@###.###>
<###@###.###>
X-Mailer: VM 6.72 under 21.1 (patch 3) "Acadia" XEmacs Lucid
FCC: INBOX.sent
--text follows this line--

Neal, thanks for the quick review. To make a long story short, I've decided
to apply a more localized fix.

Scott, this msg contains lots of info about how shared libs are
currently handled in the VM and points out some problems in this area.
Is there some place to file it away as interesting information and
something we should cleanup some day?



Neal M Gafter writes:
 > Jim Holmlund wrote:
 > > Neal and Tim, could you code review this please?
 > >
 > > file:/net/mmm/export/mmm/ws/m/b2/service_sdk_baseline/webrev.sharedLibs/index.html
  >
 > I'm uncomfortable changing the user's PATH.

I am too, a little.

 > This will be visible to Runtime.exec,
 > and therefore this is a user-visible change requiring CCC approval
 > at least.

I don't see why approval would be needed:

- We wouldn't be doing anything to win32 PATH that we aren't already doing
  to unix LD_LIBRARY_PATH. I can't see that the spec says anything
  about either env. var.

- It is true that changing PATH could change what cmd Runtime.exec
  executes because we would be prepending ..../jdkxx/jre/bin onto the
  path. So, for example, if a windows user had a program named
  "tnameserv" on his/her PATH and did
       Runtime.exec( "tnameserv");
  after this change, we would run .../jdkxxx/jre/bin/tnameserv.exe
  instead of the user's tnameserv. This change would be true only for the .exe
  files in ..../jdkxxx/jre/bin. While this would be a change from
  current, I don't see that the result is any different than the solaris
  case where we change LD_LIBRARY_PATH which could cause user code
  executed by Runtime.exec to get one of our libs instead of theirs.

 > Why
 > isn't this a problem for all the other DLLs in jre/bin? I suspect the problem is
 > more likely in the way we attempt to load the DLL.
 >

No other VM code (that I have found) uses dlopen/LoadLibrary without abs paths to load native libs.
They all find the library pathname themselves and then call dlopen/LoadLibrary
with the absolute path. The problem with this is that there are different ways
of finding the abs path of a library. Here is one way, from hpi_solaris.cpp:

    :
#if 1 //HotSpot change
    char *thread_type = strdup("native_threads");

    os::jvm_path(buf, sizeof buf);
#else //HotSpot change
    char *thread_type = getenv("_JVM_THREADS_TYPE");

    if (thread_type) {
/* Set by .java_wrapper. Remove for the sake of exec'd VMs. */
thread_type = strdup(thread_type);
        putenv("_JVM_THREADS_TYPE=");
    } else {
/* Default thread type for the invocation API. */
        thread_type = strdup("native_threads");
    }

    dladdr((void *)InitializeHPI, &dlinfo);
    strcpy(buf, (char *)dlinfo.dli_fname);
#endif //HotSpot change

#ifdef PRODUCT
    const char * hpi_lib = "/libhpi.so";
#else
    char * ptr = strrchr(buf, '/');
    assert(strstr(ptr, "/libjvm") == ptr, "invalid library name");
    const char * hpi_lib = strstr(ptr, "_g") ? "/libhpi_g.so" : "/libhpi.so";
#endif

    *(strrchr(buf, '/')) = '\0'; /* get rid of /libjvm.so */
    *(strrchr(buf, '/') + 1) = '\0'; /* get rid of hotspot */
    strcat(buf, thread_type);
    strcat(buf, hpi_lib);
    /* we use RTLD_NOW because of bug 4032715 */
    if (TraceHPI) tty->print("Loading HPI %s ", buf);
    hpi_handle = dlopen(buf, RTLD_NOW);

This is obviously dependent on lots of things that change (PRODUCT, dir structure, ...).
I've seen other code that does things that are similar to this only slightly
different.


Below, after the ---- line, is what I was able to find out about how dlls are loaded.
The first part of this tells how the various vars and properties are set. The second
part tells how these settings are used to load shared libs. The third points out
inconsistencies in
- the solaris implmentation vs the win implementation
- the win implementation when ..../jdkxxx/bin/java.exe is used to run java vs.
  when ..../jre/bin/java.exe is used.
- the JPDA frontend and the JPDA backend

My fix will fix these inconsistencies and make our handling of shared libs more
uniform. I think it would be a first step in cleaning up how shared libs are handled.

Are you convinced? :-) Well, I no longer am. I am going to abandon this
fix because:

- It has occured to me that maybe a more correct fix would be to fix java.c
  to NOT set LD_LIBRARY_PATH on solaris, and add some internal API to
  load shared libs, and change all the code to use that API instead of doing
  different things as is done now.

- Tim pointed out that this issue is complicated by the 64 bit version which
  has shared libs in yet another directory. I don't have time to delve into
  issues that that might raise.

- I can easily fix our JPDA bug by just adding a band aid like the one above
  to our code that loads our transport libs. Such a fix is a lot more localized
  (ie. safer) than changing the launcher. I don't like doing this because
  after awhile the product becomes a rats-nest of band-aids. But, I guess
  there is a time and a place for everything.


I also hope that we are adequately testing the 64 bit release to verify that
all code that loads shared libs works when the shared libs are in a different
directory.


--------------------------------------

LD_LIBRARY_PATH is set in sdk .../src/share/bin/java.c to:
        $JVMPATH (directory portion only)
      $JRE/lib/$ARCH
      $JRE/../lib/$ARCH
        followed by the user's previous $LD_LIBRARY_PATH, if any

   JVMPATH is the path name of the libjvm.so file
   ** NOTE that I suspect that $JRE/../lib/$ARCH is put onto LD_LIBRARY_PATH
      because the JPDA shared libs used to be in that dir. They aren't any longer
      so this could probably be removed.

   JRE is set in sdk .../src/solaris/bin/java_md.c. It is the
   pathname of the .../jdkxxx/jre dir if there is one, else just the path
   of the dir that contains bin/java.

   NOTE that java.c does NOT modify PATH on windows.


Sys props are set here:
   hs .../src/os/solaris/vm/os_solaris.cpp:os::get_system_properties (sets them first time)
   hs .../src/share/vm/runtime/jvm.cpp: JVM_InitProperties

 - dll_dir is the dir that contains the shared libs, eg, ...\jre\bin on windows
   or .../jre/lib/sparc on solaris.
   - On solaris, this is set by using dladdr to find the pathname of libjvm.so and then doing
     ../.., ie, .../jre/lib/sparc/client/libvmg.so becomes .../jre/lib/sparc

  - On windows, this is java.home\bin

 - java.home is .../jdkxxx/jre (if there is one), no matter whether .../jre/bin/java or .../bin/java is run.
   - On unix, this is set by doing 'dll_dir'/../.. . So, ..../jre/lib/sparc becomes
     ..../jre
   - On windows, this is set by calling GetModuleFileName which I think returns the pathname
     of jvm.dll and doing ../../.., eg, ....\jre\bin\client/jvm.dll becomes ....\jre.

 - java.library.path is:
   - solaris: LD_LIBRARY_PATH:/usr/lib
   - win:
     * 1. The directory from which application is loaded.
          -ie, the dir containing 'java'.
          - This is obtained by calling GetModuleFileName(NULL...) which returns the pathname
            of java.exe that was used to create the process.
            ********************
            So, running ..../bin/java.exe and ..../jre/bin/java.exe will cause this
            to have different values.

     * 2. The current directory
     * 3. System directory (GetSystemDirectory)
     * 4. Windows directory (GetWindowsDirectory)
     * 5. The PATH environment variable
     This is the same as where LoadLibrary looks.

 - sun.boot.library.path is dll_dir

Exampe:
- solaris: /jdk1.4/bin/java is run.
  Then:
       LD_LIBRARY_PATH = /jdk1.4/bin/jre/lib/sparc/client:/jdk1.4/jre/lib/sparc:/jdk1.4/jre/../lib/sparc
       dll_dir = /jdk1.4/jre/lib/sparc
       java.home = /jdk1.4/jre
       java.library.path = LD_LIBRARY_PATH:/usr/lib
       sun.boot.library.path = dll_dir

- win32: d:/jdk1.4/bin/java is run:
       dll_dir = d:/jdk1.4/jre/bin
       java.home = d:/jdk1.4/jre
       java.library.path = d:/jdk1.4/bin;.;<system dir>;<windows dir>;$PATH
       sun.boot.library.path = dll_dir

- win32: d:/jdk1.4/jre/bin/java is run:
       dll_dir = d:/jdk1.4/jre/bin
       java.home = d:/jdk1.4/jre
       java.library.path = d:/jdk1.4/jre/bin;.;<system dir>;<windows dir>;$PATH
       sun.boot.library.path = dll_dir

------------------

With all that in mind, shared libs are located in three different ways:
1. System.loadLibrary. This looks in
        java.libary.path
        sun.boot.library.path
    This is used to load most of the shared libs.
    NOTE that this allows all the shared libs in the shared lib dir to be
    found (they are in the sun.boot.library.path).
    NOTE: The JPDA FE uses this to load the shmem transport if there is one.

2. -Xrunxxxx looks in dll_dir. (see: hs../runtime/thread.cpp)

3. dlopen (solaris) searches LD_LIBRARY_PATH.

    LoadLibrary (win32) searches: ( See hs .../src/os/win32/vm/os_win32.cpp)
    /* Win32 library search order (See the documentation for LoadLibrary):
     *
     * 1. The directory from which application is loaded.
          -ie, the dir containing 'java'.
     * 2. The current directory
     * 3. System directory (GetSystemDirectory)
     * 4. Windows directory (GetWindowsDirectory)
     * 5. The PATH environment variable
     */

     dlopen and LoadLibrary are used in the JPDA BE to load the transport shared
     libs. They can also be used by native code. They aren't used by other hotspot
     code at this time.

------------------------------------

This all means the following:

1. On unix, System.loadLibrary and dlopen look in the same places.

    On win32 they do not. The differences are that
    System.loadLibrary looks in
        ..../jre/bin always
        ..../bin if ..../bin/java is used to run java

    while LoadLibrary will look in
        .../bin if ..../bin/java is used to run java
    or .../jre/bin if ..../jre/bin/java is used to run java

    ie, if ../jdkxxx/bin/java.exe is used to run java, the .dlls in
    the jdkxxx/jre/bin dir won't be found.

    I don't think the java spec says that System.loadLibrary should look in
    the same place for libs that the native shared library function does,
    but it would seem reasonable that it does. And, some attempt has been
    made to make this be true. (eg, the setup of java.library.path attempts
    to mimic the dlopen and LoadLibrary search places.)

2. On unix, dlopen looks in the same place whether .../jdkxxx/bin/java
    or .../jdkxxx/jre/bin/java is run.

    On win32, LoadLibrary doesn't look in the same place.
    If .../jdkxxx/bin/java.exe is run, LoadLibrary looks in .../jdkxxx/bin.
    If .../jdkxxx/jre/bin/java.exe is run, LoadLibrary looks in .../jdkxxx/jre/bin.

3. On windows, The JPDA frontend doesn't look for transport libs in the same
    place that the JPDA backend does. The frontend (ie, code that runs in a
    debugger) uses System.loadLibrary while the backend uses LoadLibrary.

--------------------------------
From tbell:


When we have a chance to revisit the layout of libraries
and executables, could we make the JRE a proper subset
of JDK? Currently everything in jre/bin is duplicated
in bin:


% find $JAVA_HOME -name java -print
/export/home2s6/opt/j2sdk-1_4_0-beta_refresh-b71/jre/bin/java
/export/home2s6/opt/j2sdk-1_4_0-beta_refresh-b71/jre/bin/sparcv9/java
/export/home2s6/opt/j2sdk-1_4_0-beta_refresh-b71/bin/java
/export/home2s6/opt/j2sdk-1_4_0-beta_refresh-b71/bin/sparcv9/java

(Same for keytool, orbd, policytool, etc...)

This can be the source of surprises such as Jim found - where
one executable works in a certain environment and the other
fails.


  Tim Bell x53241