ADDITIONAL SYSTEM INFORMATION :
OS: Aix

bash-5.2# uname -a
AIX aix-1core-4g-100g 2 7 00CC88274C00

Happening with all java versions (I tried with Jdk8, Jdk11 and Jdk17 atleast)

java -version
openjdk version "11.0.19" 2023-04-18
OpenJDK Runtime Environment Temurin-11.0.19+7 (build 11.0.19+7)
OpenJDK 64-Bit Server VM Temurin-11.0.19+7 (build 11.0.19+7, mixed mode)

A DESCRIPTION OF THE PROBLEM :
when using keytool command with PKCS#11 to list keypairs, getting below exception-

Using with SoftHSM2 module for testing here (built locally on the machine itself).


bash-5.2# /home/anshuman/softwares/java/jdk-11.0.19+7/bin/keytool -J-Djava.security.debug=sunpkcs11 -keystore NONE -storetype PKCS11 -storepass anshuman -providerClass sun.security.pkcs11.SunPKCS11 -providerArg /home/anshuman/clang-pkcs11.cfg -list
SunPKCS11 loading /home/anshuman/clang-pkcs11.cfg
sunpkcs11: Initializing PKCS#11 library /home/anshuman/workspaces/github.com/opendnssec/SoftHSMv2/build-clang/src/lib/libsofthsm2.so
Information for provider SunPKCS11-STM
Library info:
  cryptokiVersion: 2.40
  manufacturerID: SoftHSM
  flags: 0
  libraryDescription: Implementation of PKCS11
  libraryVersion: 2.06
All slots: 1483903152, 1
Slots with tokens: 1483903152, 1
Slot info for slot 1483903152:
  slotDescription: SoftHSM slot ID 0x587290b0
  manufacturerID: SoftHSM project
  flags: CKF_TOKEN_PRESENT
  hardwareVersion: 2.06
  firmwareVersion: 2.06
Token info for token in slot 1483903152:
  label: Token0
  manufacturerID: SoftHSM project
  model: SoftHSM v2
  serialNumber: d19bc157d87290b0
  flags: CKF_RNG | CKF_LOGIN_REQUIRED | CKF_USER_PIN_INITIALIZED | CKF_RESTORE_KEY_NOT_NEEDED | CKF_TOKEN_INITIALIZED
  ulMaxSessionCount: CK_EFFECTIVELY_INFINITE
  ulSessionCount: CK_UNAVAILABLE_INFORMATION
  ulMaxRwSessionCount: CK_EFFECTIVELY_INFINITE
  ulRwSessionCount: CK_UNAVAILABLE_INFORMATION
  ulMaxPinLen: 255
  ulMinPinLen: 4
  ulTotalPublicMemory: CK_UNAVAILABLE_INFORMATION
  ulFreePublicMemory: CK_UNAVAILABLE_INFORMATION
  ulTotalPrivateMemory: CK_UNAVAILABLE_INFORMATION
  ulFreePrivateMemory: CK_UNAVAILABLE_INFORMATION
  hardwareVersion: 2.06
  firmwareVersion: 2.06
  utcTime: 2024021915354900
Mechanism CKM_AES_CBC:
  ulMinKeySize: 16
  ulMaxKeySize: 32
  flags: 131840 = CKF_ENCRYPT | CKF_DECRYPT | CKF_WRAP
Mechanism CKM_AES_CBC_ENCRYPT_DATA:
  ulMinKeySize: 0
  ulMaxKeySize: 0
  flags: 524288 = CKF_DERIVE
Mechanism CKM_AES_CBC_PAD:
  ulMinKeySize: 16
  ulMaxKeySize: 32
  flags: 393984 = CKF_ENCRYPT | CKF_DECRYPT | CKF_WRAP | CKF_UNWRAP
Mechanism CKM_AES_CMAC:
  ulMinKeySize: 16
  ulMaxKeySize: 32
  flags: 10240 = CKF_SIGN | CKF_VERIFY
Mechanism CKM_AES_CTR:
  ulMinKeySize: 16
  ulMaxKeySize: 32
  flags: 768 = CKF_ENCRYPT | CKF_DECRYPT
Mechanism CKM_AES_ECB:
  ulMinKeySize: 16
  ulMaxKeySize: 32
  flags: 768 = CKF_ENCRYPT | CKF_DECRYPT
Mechanism CKM_AES_ECB_ENCRYPT_DATA:
  ulMinKeySize: 0
  ulMaxKeySize: 0
  flags: 524288 = CKF_DERIVE
Mechanism CKM_AES_GCM:
  ulMinKeySize: 16
  ulMaxKeySize: 32
  flags: 768 = CKF_ENCRYPT | CKF_DECRYPT
Mechanism CKM_AES_KEY_GEN:
  ulMinKeySize: 16
  ulMaxKeySize: 32
  flags: 32768 = CKF_GENERATE
Mechanism CKM_AES_KEY_WRAP:
  ulMinKeySize: 16
  ulMaxKeySize: 2147483648
  flags: 393216 = CKF_WRAP | CKF_UNWRAP
Mechanism CKM_AES_KEY_WRAP_PAD:
  ulMinKeySize: 1
  ulMaxKeySize: 2147483648
  flags: 393216 = CKF_WRAP | CKF_UNWRAP
Mechanism CKM_CONCATENATE_BASE_AND_DATA:
  ulMinKeySize: 1
  ulMaxKeySize: 512
  flags: 524288 = CKF_DERIVE
Mechanism CKM_CONCATENATE_BASE_AND_KEY:
  ulMinKeySize: 1
  ulMaxKeySize: 512
  flags: 524288 = CKF_DERIVE
Mechanism CKM_CONCATENATE_DATA_AND_BASE:
  ulMinKeySize: 1
  ulMaxKeySize: 512
  flags: 524288 = CKF_DERIVE
Mechanism CKM_DES2_KEY_GEN:
  ulMinKeySize: 0
  ulMaxKeySize: 0
  flags: 32768 = CKF_GENERATE
Mechanism CKM_DES3_CBC:
  ulMinKeySize: 0
  ulMaxKeySize: 0
  flags: 131840 = CKF_ENCRYPT | CKF_DECRYPT | CKF_WRAP
Mechanism CKM_DES3_CBC_ENCRYPT_DATA:
  ulMinKeySize: 0
  ulMaxKeySize: 0
  flags: 524288 = CKF_DERIVE
Mechanism CKM_DES3_CBC_PAD:
  ulMinKeySize: 0
  ulMaxKeySize: 0
  flags: 393984 = CKF_ENCRYPT | CKF_DECRYPT | CKF_WRAP | CKF_UNWRAP
Mechanism CKM_DES3_CMAC:
  ulMinKeySize: 0
  ulMaxKeySize: 0
  flags: 10240 = CKF_SIGN | CKF_VERIFY
Mechanism CKM_DES3_ECB:
  ulMinKeySize: 0
  ulMaxKeySize: 0
  flags: 768 = CKF_ENCRYPT | CKF_DECRYPT
Mechanism CKM_DES3_ECB_ENCRYPT_DATA:
  ulMinKeySize: 0
  ulMaxKeySize: 0
  flags: 524288 = CKF_DERIVE
Mechanism CKM_DES3_KEY_GEN:
  ulMinKeySize: 0
  ulMaxKeySize: 0
  flags: 32768 = CKF_GENERATE
Mechanism CKM_DES_CBC:
  ulMinKeySize: 0
  ulMaxKeySize: 0
  flags: 131840 = CKF_ENCRYPT | CKF_DECRYPT | CKF_WRAP
Mechanism CKM_DES_CBC_ENCRYPT_DATA:
  ulMinKeySize: 0
  ulMaxKeySize: 0
  flags: 524288 = CKF_DERIVE
Mechanism CKM_DES_CBC_PAD:
  ulMinKeySize: 0
  ulMaxKeySize: 0
  flags: 393984 = CKF_ENCRYPT | CKF_DECRYPT | CKF_WRAP | CKF_UNWRAP
Mechanism CKM_DES_ECB:
  ulMinKeySize: 0
  ulMaxKeySize: 0
  flags: 131840 = CKF_ENCRYPT | CKF_DECRYPT | CKF_WRAP
Mechanism CKM_DES_ECB_ENCRYPT_DATA:
  ulMinKeySize: 0
  ulMaxKeySize: 0
  flags: 524288 = CKF_DERIVE
Mechanism CKM_DES_KEY_GEN:
  ulMinKeySize: 0
  ulMaxKeySize: 0
  flags: 32768 = CKF_GENERATE
Mechanism CKM_DH_PKCS_DERIVE:
  ulMinKeySize: 512
  ulMaxKeySize: 10000
  flags: 524288 = CKF_DERIVE
Mechanism CKM_DH_PKCS_KEY_PAIR_GEN:
  ulMinKeySize: 512
  ulMaxKeySize: 10000
  flags: 65536 = CKF_GENERATE_KEY_PAIR
Mechanism CKM_DH_PKCS_PARAMETER_GEN:
  ulMinKeySize: 512
  ulMaxKeySize: 10000
  flags: 32768 = CKF_GENERATE
Mechanism CKM_DSA:
  ulMinKeySize: 512
  ulMaxKeySize: 1024
  flags: 10240 = CKF_SIGN | CKF_VERIFY
Mechanism CKM_DSA_KEY_PAIR_GEN:
  ulMinKeySize: 512
  ulMaxKeySize: 1024
  flags: 65536 = CKF_GENERATE_KEY_PAIR
Mechanism CKM_DSA_PARAMETER_GEN:
  ulMinKeySize: 512
  ulMaxKeySize: 1024
  flags: 32768 = CKF_GENERATE
Mechanism CKM_DSA_SHA1:
  ulMinKeySize: 512
  ulMaxKeySize: 1024
  flags: 10240 = CKF_SIGN | CKF_VERIFY
Mechanism CKM_DSA_SHA224:
  ulMinKeySize: 512
  ulMaxKeySize: 1024
  flags: 10240 = CKF_SIGN | CKF_VERIFY
Mechanism CKM_DSA_SHA256:
  ulMinKeySize: 512
  ulMaxKeySize: 1024
  flags: 10240 = CKF_SIGN | CKF_VERIFY
Mechanism CKM_DSA_SHA384:
  ulMinKeySize: 512
  ulMaxKeySize: 1024
  flags: 10240 = CKF_SIGN | CKF_VERIFY
Mechanism CKM_DSA_SHA512:
  ulMinKeySize: 512
  ulMaxKeySize: 1024
  flags: 10240 = CKF_SIGN | CKF_VERIFY
Mechanism CKM_ECDH1_DERIVE:
  ulMinKeySize: 112
  ulMaxKeySize: 521
  flags: 524288 = CKF_DERIVE
Mechanism CKM_ECDSA:
  ulMinKeySize: 112
  ulMaxKeySize: 521
  flags: 26224640 = CKF_SIGN | CKF_VERIFY | CKF_EC_F_P | CKF_EC_UNCOMPRESS
Mechanism CKM_EC_EDWARDS_KEY_PAIR_GEN:
  ulMinKeySize: 255
  ulMaxKeySize: 448
  flags: 65536 = CKF_GENERATE_KEY_PAIR
Mechanism CKM_EC_KEY_PAIR_GEN:
  ulMinKeySize: 112
  ulMaxKeySize: 521
  flags: 26279936 = CKF_GENERATE_KEY_PAIR | CKF_EC_F_P | CKF_EC_UNCOMPRESS
Mechanism CKM_EDDSA:
  ulMinKeySize: 255
  ulMaxKeySize: 448
  flags: 10240 = CKF_SIGN | CKF_VERIFY
Mechanism CKM_GENERIC_SECRET_KEY_GEN:
  ulMinKeySize: 1
  ulMaxKeySize: 2147483648
  flags: 32768 = CKF_GENERATE
Mechanism CKM_MD5:
  ulMinKeySize: 0
  ulMaxKeySize: 0
  flags: 1024 = CKF_DIGEST
Mechanism CKM_MD5_HMAC:
  ulMinKeySize: 16
  ulMaxKeySize: 512
  flags: 10240 = CKF_SIGN | CKF_VERIFY
Mechanism CKM_MD5_RSA_PKCS:
  ulMinKeySize: 512
  ulMaxKeySize: 16384
  flags: 10240 = CKF_SIGN | CKF_VERIFY
Mechanism CKM_RSA_PKCS:
  ulMinKeySize: 512
  ulMaxKeySize: 16384
  flags: 404224 = CKF_ENCRYPT | CKF_DECRYPT | CKF_SIGN | CKF_VERIFY | CKF_WRAP | CKF_UNWRAP
Mechanism CKM_RSA_PKCS_KEY_PAIR_GEN:
  ulMinKeySize: 512
  ulMaxKeySize: 16384
  flags: 65536 = CKF_GENERATE_KEY_PAIR
Mechanism CKM_RSA_PKCS_OAEP:
  ulMinKeySize: 512
  ulMaxKeySize: 16384
  flags: 393984 = CKF_ENCRYPT | CKF_DECRYPT | CKF_WRAP | CKF_UNWRAP
Mechanism CKM_RSA_PKCS_PSS:
  ulMinKeySize: 512
  ulMaxKeySize: 16384
  flags: 10240 = CKF_SIGN | CKF_VERIFY
Mechanism CKM_RSA_X_509:
  ulMinKeySize: 512
  ulMaxKeySize: 16384
  flags: 11008 = CKF_ENCRYPT | CKF_DECRYPT | CKF_SIGN | CKF_VERIFY
Mechanism CKM_SHA1_RSA_PKCS:
  ulMinKeySize: 512
  ulMaxKeySize: 16384
  flags: 10240 = CKF_SIGN | CKF_VERIFY
Mechanism CKM_SHA1_RSA_PKCS_PSS:
  ulMinKeySize: 512
  ulMaxKeySize: 16384
  flags: 10240 = CKF_SIGN | CKF_VERIFY
Mechanism CKM_SHA224:
  ulMinKeySize: 0
  ulMaxKeySize: 0
  flags: 1024 = CKF_DIGEST
Mechanism CKM_SHA224_HMAC:
  ulMinKeySize: 28
  ulMaxKeySize: 512
  flags: 10240 = CKF_SIGN | CKF_VERIFY
Mechanism CKM_SHA224_RSA_PKCS:
  ulMinKeySize: 512
  ulMaxKeySize: 16384
  flags: 10240 = CKF_SIGN | CKF_VERIFY
Mechanism CKM_SHA224_RSA_PKCS_PSS:
  ulMinKeySize: 512
  ulMaxKeySize: 16384
  flags: 10240 = CKF_SIGN | CKF_VERIFY
Mechanism CKM_SHA256:
  ulMinKeySize: 0
  ulMaxKeySize: 0
  flags: 1024 = CKF_DIGEST
Mechanism CKM_SHA256_HMAC:
  ulMinKeySize: 32
  ulMaxKeySize: 512
  flags: 10240 = CKF_SIGN | CKF_VERIFY
Mechanism CKM_SHA256_RSA_PKCS:
  ulMinKeySize: 512
  ulMaxKeySize: 16384
  flags: 10240 = CKF_SIGN | CKF_VERIFY
Mechanism CKM_SHA256_RSA_PKCS_PSS:
  ulMinKeySize: 512
  ulMaxKeySize: 16384
  flags: 10240 = CKF_SIGN | CKF_VERIFY
Mechanism CKM_SHA384:
  ulMinKeySize: 0
  ulMaxKeySize: 0
  flags: 1024 = CKF_DIGEST
Mechanism CKM_SHA384_HMAC:
  ulMinKeySize: 48
  ulMaxKeySize: 512
  flags: 10240 = CKF_SIGN | CKF_VERIFY
Mechanism CKM_SHA384_RSA_PKCS:
  ulMinKeySize: 512
  ulMaxKeySize: 16384
  flags: 10240 = CKF_SIGN | CKF_VERIFY
Mechanism CKM_SHA384_RSA_PKCS_PSS:
  ulMinKeySize: 512
  ulMaxKeySize: 16384
  flags: 10240 = CKF_SIGN | CKF_VERIFY
Mechanism CKM_SHA512:
  ulMinKeySize: 0
  ulMaxKeySize: 0
  flags: 1024 = CKF_DIGEST
Mechanism CKM_SHA512_HMAC:
  ulMinKeySize: 64
  ulMaxKeySize: 512
  flags: 10240 = CKF_SIGN | CKF_VERIFY
Mechanism CKM_SHA512_RSA_PKCS:
  ulMinKeySize: 512
  ulMaxKeySize: 16384
  flags: 10240 = CKF_SIGN | CKF_VERIFY
Mechanism CKM_SHA512_RSA_PKCS_PSS:
  ulMinKeySize: 512
  ulMaxKeySize: 16384
  flags: 10240 = CKF_SIGN | CKF_VERIFY
Mechanism CKM_SHA_1:
  ulMinKeySize: 0
  ulMaxKeySize: 0
  flags: 1024 = CKF_DIGEST
Mechanism CKM_SHA_1_HMAC:
  ulMinKeySize: 20
  ulMaxKeySize: 512
  flags: 10240 = CKF_SIGN | CKF_VERIFY
sunpkcs11: login succeeded
Exception in thread "main" java.lang.OutOfMemoryError
        at jdk.crypto.cryptoki/sun.security.pkcs11.wrapper.PKCS11.C_FindObjects(Native Method)
        at jdk.crypto.cryptoki/sun.security.pkcs11.P11KeyStore.findObjects(P11KeyStore.java:2680)
        at jdk.crypto.cryptoki/sun.security.pkcs11.P11KeyStore.mapLabels(P11KeyStore.java:2293)
        at jdk.crypto.cryptoki/sun.security.pkcs11.P11KeyStore.engineLoad(P11KeyStore.java:769)
        at java.base/java.security.KeyStore.load(KeyStore.java:1479)
        at java.base/sun.security.tools.keytool.Main.doCommands(Main.java:1010)
        at java.base/sun.security.tools.keytool.Main.run(Main.java:421)
        at java.base/sun.security.tools.keytool.Main.main(Main.java:414)



FREQUENCY : always