J2SE JRE 1.4.1_xx and 1.4.2, -server (C2) compiler only:

   Pure java application appears to be miscompiled by HotSpot C2.
   A method experiences an unexpected java.lang.NullPointerException as
   soon as it has been compiled. Does not happen with -client (C1)
   compiler, does not happen with 1.4.0 and does not happen with
   -server -XX:-UseOnStackReplacement

   Customer Situation:
   Large portal application, using:
    TomCat4.0.3_LE-jdk14
    xerces 1.4.4
    castor 0.9.4
   Production environment still running on 1.4.0 and not affected by
   the problem. Affected: testbed environment for evaluating 1.4.1_01.
   As long as the bug exists, it means customer cannot migrate to 1.4.1.
   Problem is only reproduceable at the customers site, so we do not have
   a testcase.

   Problem has first manifested in the following output:

   running with -XX:+PrintCompile we get the following output:

   ===8<
 ---
   73% ! org.exolab.castor.xml.Validator::validate @ 123 (278 bytes)
   442 org.apache.xerces.utils.StringPool::addSymbol (334 bytes)
   443 ! org.apache.catalina.util.RequestUtil::URLDecode (123 bytes)
   444 java.lang.ref.WeakReference:: (6 bytes)
   445 ! org.apache.catalina.core.ApplicationHttpRequest::setRequest (162 bytes)
   446 java.util.Hashtable$Enumerator::hasMoreElements (53 bytes)
   447 org.apache.oro.text.regex.Perl5Matcher::__match (2671 bytes)
   [29.10.02 15:17:00] [Thread-16] [INFO ] [rontend.actions.IBBAction]
   [BF1F82F6E575A0786BCB114E86689F31] [Set referer to: /application_ibb_viewturnover_struts]
   448 java.lang.Class$1:: (15 bytes)
    74% ! org.exolab.castor.xml.UnmarshalHandler::processAttributes @ 320 (832 bytes)
   449 org.exolab.castor.xml.FieldValidator::validate (601 bytes)
   450 ! org.exolab.castor.mapping.loader.MappingLoader::createFieldDesc (1777 bytes)
   [29.10.02 15:17:04] [Thread-17] [INFO ] [rontend.actions.IBBAction]
   [BF1F82F6E575A0786BCB114E86689F31] [Set referer to:
   /application_ibb_view_periodicmoneytransfer_struts]
   [29.10.02 15:17:04] [Thread-17] [INFO ] [k.core.mock.MarshalFacade]
   [BF1F82F6E575A0786BCB114E86689F31] [before loadMapping
   java.io.ByteArrayInputStream@36b1bb]
   [29.10.02 15:17:05] [Thread-17] [ERROR] [k.core.mock.MarshalFacade]
   [BF1F82F6E575A0786BCB114E86689F31] [MappingException]
   java.lang.NullPointerException
           at org.exolab.castor.xml.Validator.validate(Validator.java:128)
           at org.exolab.castor.xml.FieldValidator.validate(FieldValidator.java:217)
           at org.exolab.castor.xml.Validator.validate(Validator.java:133)
           at org.exolab.castor.xml.FieldValidator.validate(FieldValidator.java:217)
           at org.exolab.castor.xml.Validator.validate(Validator.java:133)
           at org.exolab.castor.xml.UnmarshalHandler.endElement(UnmarshalHandler.java:675)
           at org.apache.xerces.parsers.SAXParser.endElement(SAXParser.java:1392)
           at org.apache.xerces.validators.common.XMLValidator.callEndElement(XMLValidator.java:1550)
           at org.apache.xerces.framework.XMLDocumentScanner$ContentDispatcher.dispatch(XMLDocumentScanner.java:1204)
           at org.apache.xerces.framework.XMLDocumentScanner.parseSome(XMLDocumentScanner.java:381)
           at org.apache.xerces.framework.XMLParser.parse(XMLParser.java:1098)
           at org.exolab.castor.xml.Unmarshaller.unmarshal(Unmarshaller.java:530)
           at org.exolab.castor.mapping.Mapping.loadMappingInternal(Mapping.java:507)
           at org.exolab.castor.mapping.Mapping.loadMapping(Mapping.java:433)
           at de.advancebank.core.mock.MarshalFacade.mapping(MarshalFacade.java:138)
   [...]
   --->
 8===

   When customer adds a .hotspot_compiler file prescribing
    exclude org/exolab/castor/xml/Validator validate
   the C2 compiler picks it up:
   ### Excluding compile: org.exolab.castor.xml.Validator::validate
   and the problem no longer occurs, suggesting that the miscompile
   affects this method itself, and not one of its callers in the
   above stack.
   

   Now we still get Null Pointer Exceptions. But now in Unmarshaller.unmarshal.
   We have a lot of traces. See comments where.

   traces look like this:

DEOPT UNPACKING thread 0x11ca988 vframeArray 0x12c9100
     {method} 'scanMatchingName' '(III)I' in 'org/apache/xerces/readers/UTF8Reader' - aastore @ bci 1083 sp = 0xe1
d7f5f8
!! !! SAXException in Unmarshaller.unmarshal
java.lang.NullPointerException
        at org.apache.xerces.framework.XMLParser.parse(XMLParser.java:1111)
        at org.exolab.castor.xml.Unmarshaller.unmarshal(Unmarshaller.java:533)
        at org.exolab.castor.mapping.Mapping.loadMappingInternal(Mapping.java:507)
        at org.exolab.castor.mapping.Mapping.loadMapping(Mapping.java:433)
        at de.advancebank.core.mock.MarshalFacade.mapping(MarshalFacade.java:139)
        at de.advancebank.core.mock.MarshalFacade.doUnmarshal(MarshalFacade.java:194)
        at de.advancebank.core.mock.MarshalFacade.unmarshal(MarshalFacade.java:172)
        at de.advancebank.business.services.banking.client.DemoBankingDelegate.getAssetsStatusInternal(Unknown Sou
rce)
        at de.advancebank.business.services.banking.client.BankingDelegate.getAssetsStatus(Unknown Source)
        at de.advancebank.portal.application.ibb.frontend.actions.StatusOverviewAction.doPerform(Unknown Source)
        at de.advancebank.portal.application.ibb.frontend.actions.IBBAction.doPerformAction(Unknown Source)
        at de.advancebank.portal.framework.webcomponents.frontend.ComponentAction.doPerform(Unknown Source)
        at de.advancebank.portal.framework.session.frontend.struts.PortalAction.perform(Unknown Source)
        at org.apache.struts.action.ActionServlet.processActionPerform(ActionServlet.java:1787)
        at org.apache.struts.action.ActionServlet.process(ActionServlet.java:1586)
        at org.apache.struts.action.ActionServlet.doGet(ActionServlet.java:492)
        at javax.servlet.http.HttpServlet.service(HttpServlet.java:740)
        at de.advancebank.portal.framework.session.frontend.PortalKeeper.service(Unknown Source)
        at javax.servlet.http.HttpServlet.service(HttpServlet.java:853)
        at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:247)
        at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:193)
        at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:243)
        at org.apache.catalina.core.StandardPipeline.invokeNext(StandardPipeline.java:566)
        at org.apache.catalina.core.StandardPipeline.invoke(StandardPipeline.java:472)
        at org.apache.catalina.core.ContainerBase.invoke(ContainerBase.java:943)
        at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:190)
        at org.apache.catalina.core.StandardPipeline.invokeNext(StandardPipeline.java:566)
        at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:475)
        at org.apache.catalina.core.StandardPipeline.invokeNext(StandardPipeline.java:564)
        at org.apache.catalina.valves.CertificatesValve.invoke(CertificatesValve.java:246)
        at org.apache.catalina.core.StandardPipeline.invokeNext(StandardPipeline.java:564)
        at org.apache.catalina.core.StandardPipeline.invoke(StandardPipeline.java:472)
        at org.apache.catalina.core.ContainerBase.invoke(ContainerBase.java:943)
        at org.apache.catalina.core.StandardContext.invoke(StandardContext.java:2343)
        at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:180)
        at org.apache.catalina.core.StandardPipeline.invokeNext(StandardPipeline.java:566)
        at org.apache.catalina.valves.ErrorDispatcherValve.invoke(ErrorDispatcherValve.java:170)
        at org.apache.catalina.core.StandardPipeline.invokeNext(StandardPipeline.java:564)
        at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:170)
        at org.apache.catalina.core.StandardPipeline.invokeNext(StandardPipeline.java:564)
        at org.apache.catalina.core.StandardPipeline.invoke(StandardPipeline.java:472)
        at org.apache.catalina.core.ContainerBase.invoke(ContainerBase.java:943)
        at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:174)
        at org.apache.catalina.core.StandardPipeline.invokeNext(StandardPipeline.java:566)
        at org.apache.catalina.core.StandardPipeline.invoke(StandardPipeline.java:472)
        at org.apache.catalina.core.ContainerBase.invoke(ContainerBase.java:943)
        at org.apache.catalina.connector.warp.WarpRequestHandler.handle(WarpRequestHandler.java:217)
        at org.apache.catalina.connector.warp.WarpConnection.run(WarpConnection.java:194)
        at java.lang.Thread.run(Thread.java:536)
Nested exception is:
java.lang.NullPointerException
[20.01.03 16:27:13] [Thread-23] [ERROR] [k.core.mock.MarshalFacade] [5B0F1791F680F15C0EE15DFC9645E69E] [!!!]
[20.01.03 16:27:13] [Thread-23] [ERROR] [k.core.mock.MarshalFacade] [5B0F1791F680F15C0EE15DFC9645E69E] [java.lang.
NullPointerException
]
[20.01.03 16:27:13] [Thread-23] [ERROR] [k.core.mock.MarshalFacade] [5B0F1791F680F15C0EE15DFC9645E69E] [MappingExc
eption]
java.lang.NullPointerException
[20.01.03 16:27:13] [Thread-23] [FATAL] [rontend.actions.IBBAction] [5B0F1791F680F15C0EE15DFC9645E69E] [exception
caught:]
de.advancebank.core.error.InternalException: Nested error: java.lang.NullPointerException
        at de.advancebank.core.mock.MarshalFacade.doUnmarshal(MarshalFacade.java:217)
        at de.advancebank.core.mock.MarshalFacade.unmarshal(MarshalFacade.java:172)
        at de.advancebank.business.services.banking.client.DemoBankingDelegate.getAssetsStatusInternal(Unknown Sou
rce)