Schema compiler ignores regular expresssions that have unmatched close parenthesis:
<xsd:restriction base="xsd:string">
  <xsd:pattern value="ab|(d))"/>
</xsd:restriction>

other wrong schemata that are correct from the schema compiler point of view are attached.

the following JCK tests fail:
xml_schema/msData/regex/jaxp/reD10.html#reD10
xml_schema/msData/regex/jaxp/reD11.html#reD11
xml_schema/msData/regex/jaxp/reD12.html#reD12
xml_schema/msData/regex/jaxp/reE9.html#reE9
xml_schema/msData/regex/jaxb/reD10.html#reD10
xml_schema/msData/regex/jaxb/reD11.html#reD11
xml_schema/msData/regex/jaxb/reD12.html#reD12
xml_schema/msData/regex/jaxb/reE9.html#reE9
The attached invalid schema RegexTest_1258.xsd is also correct for the schema compiler