There is an internal document with samples to modify config files and command line instructions to disable the following solaris native pkcs11 libraries :

   /usr/lib/security/pkcs11_kernel.so.1
   /usr/lib/security/pkcs11_kms.so.1
   /usr/lib/security/pkcs11_softtoken.so.1
   /usr/lib/security/pkcs11_softtoken_extra.so.1
   /usr/lib/security/sparcv9/pkcs11_kernel.so.1
   /usr/lib/security/sparcv9/pkcs11_kms.so.1
   /usr/lib/security/sparcv9/pkcs11_softtoken.so.1
   /usr/lib/security/sparcv9/pkcs11_softtoken_extra.so.1

In java for solaris the wrapper library libj2pkcs11.so is loaded by sunpkcs11.jar using sunpkcs11-solaris.cfg config file

SUNWj5rt:
    /usr/jdk/instances/jdk1.5.0/jre/lib/ext/sunpkcs11.jar
    /usr/jdk/instances/jdk1.5.0/jre/lib/security/sunpkcs11-solaris.cfg
    /usr/jdk/instances/jdk1.5.0/jre/lib/sparc/libj2pkcs11.so
SUNWj5rtx:
    /usr/jdk/instances/jdk1.5.0/jre/lib/sparcv9/libj2pkcs11.so

SUNWj6rt:
    /usr/jdk/instances/jdk1.6.0/jre/lib/ext/sunpkcs11.jar
    /usr/jdk/instances/jdk1.6.0/jre/lib/security/sunpkcs11-solaris.cfg
    /usr/jdk/instances/jdk1.6.0/jre/lib/sparc/libj2pkcs11.so
SUNWj6rtx:
    /usr/jdk/instances/jdk1.6.0/jre/lib/sparcv9/libj2pkcs11.so

SUNWj7rt:
    /usr/jdk/instances/jdk1.7.0/jre/lib/ext/sunpkcs11.jar
    /usr/jdk/instances/jdk1.7.0/jre/lib/security/sunpkcs11-solaris.cfg
    /usr/jdk/instances/jdk1.7.0/jre/lib/sparc/libj2pkcs11.so
SUNWj7rtx:
    /usr/jdk/instances/jdk1.7.0/jre/lib/sparcv9/libj2pkcs11.so

SUNWj8rt:
    /usr/jdk/instances/jdk1.8.0/jre/lib/ext/sunpkcs11.jar
    /usr/jdk/instances/jdk1.8.0/jre/lib/security/sunpkcs11-solaris.cfg
    /usr/jdk/instances/jdk1.8.0/jre/lib/sparcv9/libj2pkcs11.so

Detailed info can be added or corrected in Java PKCS#11 Reference Guide
http://docs.oracle.com/javase/7/docs/technotes/guides/security/p11guide.html

Please add documentation about debug option -Djava.security.debug=sunpkcs11

Doc ID 1510935.1
------------------------
This document describes how to disable PKCS11 so that the Sun crypto provider is used in its stead.

You can disable PKCS11 in two ways:

A) Disable PKCS11 for one Java process: Start or restart the Java process with the Java command line flag: -Dsun.security.pkcs11.enable-solaris=false

or

B) Disable PKCS11 for all Java processes run with a particular Java installation by editing the $JAVA_HOME/jre/lib/security/java.security file as follows:

                 1) Comment out the first security provider, PKCS11

                      # List of providers and their preference orders (see above):
                           #security.provider.1=sun.security.pkcs11.SunPKCS11 ${java.home}/lib/security/sunpkcs11-solaris.cfg
                             security.provider.2=sun.security.provider.Sun
                             security.provider.3=sun.security.rsa.SunRsaSign
                             security.provider.4=com.sun.net.ssl.internal.ssl.Provider
                             security.provider.5=com.sun.crypto.provider.SunJCE
                             security.provider.6=sun.security.jgss.SunProvider
                             security.provider.7=com.sun.security.sasl.Provider
                             security.provider.8=org.jcp.xml.dsig.internal.dom.XMLDSigRI
                             security.provider.9=sun.security.smartcardio.SunPCSC

                 2) Renumber the rest of the listed security providers, such that number 2 becomes number 1, number 3 becomes number 2, and so on, as follows:

                      # List of providers and their preference orders (see above):
                           #security.provider.1=sun.security.pkcs11.SunPKCS11 ${java.home}/lib/security/sunpkcs11-solaris.cfg
                             security.provider.1=sun.security.provider.Sun
                             security.provider.2=sun.security.rsa.SunRsaSign
                             security.provider.3=com.sun.net.ssl.internal.ssl.Provider
                             security.provider.4=com.sun.crypto.provider.SunJCE
                             security.provider.5=sun.security.jgss.SunProvider
                             security.provider.6=com.sun.security.sasl.Provider
                             security.provider.7=org.jcp.xml.dsig.internal.dom.XMLDSigRI
                             security.provider.8=sun.security.smartcardio.SunPCSC

                 3) Start or restart your Java processes which run on this installation of Java.

 
Note: You must renumber the provider list, as the JVM will stop searching the list of providers if security.provider.1 is not present.

Many problems with PKCS11 occur in the SecureRandom mechanism and can be resolved by disabling this mechanism only, rather than the entire PKCS11 provider. To disable the SecureRandom mechanism only, you can add SecureRandom to the list of disabled mechanisms in the $JAVA_HOME/jre/lib/security/sunpkcs11-solaris.cfg file:

$ more sunpkcs11-solaris.cfg
...
disabledMechanisms = {

  SecureRandom

  CKM_MD2
  CKM_MD5
  CKM_SHA_1
...
}
 
Please note that the portion of the sunpkcs11-solaris.cfg shown above is just an example meant to show you where to place the mechanism to be disabled.