ClassCastException from java.io.ObjectInputStream(setObjectFieldValue) - BEA WL

Customer Problem description:

> ]"I am calculating that we see ~5000 ClassCast exceptions in 4
> nightly builds
> ]with ~300 ejbc runs each loading ~300 MBI files. This comes to a
> probability
> ]of 1%"
> ]
> ]So we would like more urgent reaction if possibly on this in
> light of recent
> ]events.

> ]> We have seen this on 131 b24 and 131_01 with -server hotspot.
> ]>
> ]> We only see it Solaris, and it seems to be a timing issue and
> ]> exacerbated by a multi CPU.
> ]>
> ]> --we see it more often on a multi-cpu box
> ]> --if we instrument ObjectInputStream (see attached) for every
> ]> call to the offending method than this changes the timing
> ]> sufficiently that the problem goes away (so I don't see running
> ]> the debug VM as an option)

Configuration:

Solaris 5.7 or 5.8 running on multi-cpu configuration (atleast dual cpu).
BEA Weblogic 6.10

Stack trace:

java.lang.ClassCastException: Assigning instance of class java.lang.String
to field javax.management.MBeanParameterInfo#type
         at
java.io.ObjectInputStream.inputClassFields(ObjectInputStream.java:2271)
         at
java.io.ObjectInputStream.defaultReadObject(ObjectInputStream.java:519)
         at java.io.ObjectInputStream.inputObject(ObjectInputStream.java:1412)
         at java.io.ObjectInputStream.readObject(ObjectInputStream.java:386)
         at java.io.ObjectInputStream.inputArray(ObjectInputStream.java:1142)
         at java.io.ObjectInputStream.readObject(ObjectInputStream.java:374)
         at
java.io.ObjectInputStream.inputClassFields(ObjectInputStream.java:2263)
         at
java.io.ObjectInputStream.defaultReadObject(ObjectInputStream.java:519)
         at java.io.ObjectInputStream.inputObject(ObjectInputStream.java:1412)
         at java.io.ObjectInputStream.readObject(ObjectInputStream.java:386)
         at java.io.ObjectInputStream.inputArray(ObjectInputStream.java:1142)
         at java.io.ObjectInputStream.readObject(ObjectInputStream.java:374)
         at
java.io.ObjectInputStream.inputClassFields(ObjectInputStream.java:2263)
         at
java.io.ObjectInputStream.defaultReadObject(ObjectInputStream.java:519)
         at java.io.ObjectInputStream.inputObject(ObjectInputStream.java:1412)
         at java.io.ObjectInputStream.readObject(ObjectInputStream.java:386)
         at java.io.ObjectInputStream.readObject(ObjectInputStream.java:236)
         at weblogic.management.internal.Helper.getMBeanInfo(Helper.java:205)
         at
weblogic.management.internal.Helper.preloadMBeanInfos(Helper.java:232)
         at weblogic.management.Admin.initialize(Admin.java:212)
         at weblogic.t3.srvr.T3Srvr.initialize(T3Srvr.java:362)
         at weblogic.t3.srvr.T3Srvr.run(T3Srvr.java:202)
         at weblogic.Server.main(Server.java:35)

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Please see the workaround section for more insights into the problem.
(where the trigger for this exception happens)

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Debug output:
*************

Starting WebLogic Server ....
<Oct 17, 2001 12:04:59 PM BST> <Notice> <Management> <Loading configuration file ./config/ccwls006/config.xml ...>
exception from setObjectFieldValue:
         val instanceof type : true
        field.getName(): "attributeType"
        field.getType().getName(): "java.lang.String"
        field.getDeclaringClass().getName(): "javax.management.MBeanAttributeInfo"
        currentClassDesc.forClass().getName(): "javax.management.MBeanAttributeInfo"
        currentClassDesc.getName(): "javax.management.MBeanAttributeInfo"
        currentClassDesc.getSerialVersionUID(): "7043855487133450673"
        cl.getName(): "javax.management.MBeanAttributeInfo"
        ClassLoader.getSystemClassLoader().toString(): "sun.misc.Launcher$AppClassLoader@3f52a5"

        this.getClass().getClassLoader() is null
        o.getClass().getName(): "weblogic.management.tools.AttributeInfo"
        o.toString(): "weblogic.management.tools.AttributeInfo@c5da6, name=/opt/bea/wls/6.1sp1/wlserver/lib/weblogic.jar, dynamic=false, encrypted=false, configurable=false,oldprop=null,defaultValue=[<null>], max: null, min: null, null ok: false"
        o.getClass().getClassLoader().toString(): "sun.misc.Launcher$AppClassLoader@3f52a5"
        currentClassDesc.objFieldIDs[i]: "26"
        currentClassDesc.objFieldTypes[i].getName(): "java.lang.String"
        val.getClass().getName(): "java.lang.String"
        val.toString(): "java.lang.String"
        val class loader: "null"

associated java code (ObjectInputStream.java):
************************************************

} catch (ClassCastException e) {
              //(is val and instance of type)
              //this is the bit in ObjectInputStream.c that is failing
              System.out.println( "exception from setObjectFieldValue:" );
              if(currentClassDesc.objFieldTypes[i] != null && val != null) {
                System.out.println( "\t val instanceof type : "
                                    + (currentClassDesc.objFieldTypes[i].isAssignableFrom(val.getClass())));
              }
              if( field != null ){
                System.out.println( "\tfield.getName(): \"" + field.getName() + "\"" );
                System.out.println( "\tfield.getType().getName(): \"" + field.getType().getName() + "\"" );
                System.out.println( "\tfield.getDeclaringClass().getName(): \""
                                    + field.getDeclaringClass().getName() + "\"" );
              }else{
                System.out.println( "\tfield is null" );
              }
              System.out.println( "\tcurrentClassDesc.forClass().getName(): \""
                                  + currentClassDesc.forClass().getName() + "\"" );
              System.out.println( "\tcurrentClassDesc.getName(): \"" + currentClassDesc.getName() + "\"" );
              System.out.println( "\tcurrentClassDesc.getSerialVersionUID(): \""
                                  + currentClassDesc.getSerialVersionUID() + "\"" );
              System.out.println( "\tcl.getName(): \"" + cl.getName() + "\"" );
              System.out.println( "\tClassLoader.getSystemClassLoader().toString(): \""
                                  + ClassLoader.getSystemClassLoader().toString() + "\"\n" );
              if( this.getClass().getClassLoader() != null ){
                System.out.println( "\tthis.getClass().getClassLoader().toString(): \""
                                    + this.getClass().getClassLoader().toString() + "\"\n" );
              }else{
                System.out.println( "\tthis.getClass().getClassLoader() is null" );
              }
              System.out.println( "\to.getClass().getName(): \"" + o.getClass().getName() + "\"" );
              System.out.println( "\to.toString(): \"" + o.toString() + "\"" );
              System.out.println( "\to.getClass().getClassLoader().toString(): \""
                                  + o.getClass().getClassLoader().toString() + "\"" );
              System.out.println( "\tcurrentClassDesc.objFieldIDs[i]: \""
                                  + currentClassDesc.objFieldIDs[i] + "\"" );
              System.out.println( "\tcurrentClassDesc.objFieldTypes[i].getName(): \""
                                  + currentClassDesc.objFieldTypes[i].getName() + "\"" );
              if( val == null ){
                System.out.println( "val is null" );
              }else{
                System.out.println( "\tval.getClass().getName(): \"" + val.getClass().getName() + "\"" );
                System.out.println( "\tval.toString(): \"" + val.toString() + "\"" );
                if( val.getClass().getClassLoader() != null ){
                  System.out.println( "\tval.getClass().getClassLoader().toString(): \""
                                      + val.getClass().getClassLoader().toString() + "\"" );
                }else{
                  System.out.println( "\tval class loader: \"null\"" );
                }
              }
// fill in error message
System.out.println( "\n\n\n" );
e.printStackTrace();
System.out.println( "message is: " + e.getMessage() + "\n\n\n" );
throw new ClassCastException("Assigning instance of class " +
val.getClass().getName() + " to field " +
cl.getName() + '#' + field.getName());
} catch (Exception e) {
throw new InvalidClassException(cl.getName(),
"Invalid field " + field.getName());
}


Testcase available with DTS. Problem reproduced in or lab.