Uploaded image for project: 'JDK'
  1. JDK
  2. JDK-7066948

non-deterministic canonicalization cache behavior under ${java.home}



    • generic
    • solaris_nevada


      [dep, 13Jul2011]

        We recently had a problem where the Solaris-packaged java help
        (delivered as an extension) didn't work when used with a strict
        security policy... but only on some machines. On machines that
        worked, java would load the java help jar from


        On machines that did not, it would load the jar from


        The former is a symlink to the latter. For correct application of
        the security policy, it is necessary for the extension to load from
        the extension directory, not from /usr/jdk/packages. A similar
        problem is discussed in (4141872).

        The problem here is slightly different. On these two machines, we
        are getting different results when Java canonicalizes the symlink.
        On one machine, we get the symlink, on the other, we get the target.
        The problem boils down to the implementation of the canonicalization
        cache, as illustrated by a test case that does nothing but call
        getCanonicalFile() on the above symlink:

          # On the functioning machine:

          $ java -Dsun.io.useCanonPrefixCache=true Test
          $ java -Dsun.io.useCanonPrefixCache=false Test

          # On the malfunctioning machine:

          $ java -Dsun.io.useCanonPrefixCache=true Test
          $ java -Dsun.io.useCanonPrefixCache=false Test

        It turns out that the two machines -- despite having identical java
        installations -- had one subtle difference in the installed bits: the
        raw contents of the /usr/jdk/instances/jdk1.6.0/jre/lib/ext directory
        are ordered differently (observable using "ls -1U"). This is
        important because when constructing the URLClassLoader for
        java.ext.dirs, sun.misc.Launcher just calls list() on each directory
        to enumerate the jars within.

        The canonicalization cache, meanwhile, will cache path prefixes found
        under ${java.home}, passing through all subsequent files found under
        a previously canonicalized directory. This means that the results
        from the canonicalization algorithm differ depending on the order
        lookups are performed. If the first file canonicalized in a
        directory is a real file, subsequently canonicalized symlinks will be
        returned unmodified -- i.e. as if they are real files. On the other
        hand, if the first file canonicalized in a directory is a symlink (or
        more precisely, all symlinks in a directory canonicalized before a
        regular file is canonicalized), it will be resolved to its target
        prior to canonicalization.

        To put all the pieces together, we were seeing failures on those
        machines where chance resulted in the jhall.jar symlink being the
        first .jar file in the extension directory. This resulted in an
        attempt to load the extension from outside the extension directory,
        which in turn was considered a violation of the security policy
        that was in place.

        It is unfortunate that we must rely on the canonicalization cache in
        the first place -- as discussed in 4141872, manually resolving
        symlinks before determining if something falls in your sandbox
        defeats one of the primary uses of symlinks in Unix. That said, if
        we must rely on the canonicalization cache, it needs to at least
        behave in a consistent fashion.


        Issue Links



              bchristi Brent Christian
              duke J. Duke
              0 Vote for this issue
              6 Start watching this issue