ADDITIONAL SYSTEM INFORMATION :
Dell Precision 5770 laptop / Windows 10 / JDK 17 and JDK 23 (Eclipse 2024-03)

A DESCRIPTION OF THE PROBLEM :
An "Invalid UTF-16 surrogate detected" error is triggered when certain (valid) UTF-8 characters appear in sequence over a 1024-byte boundary in the string output of an XML to UTF-8 text transformation.

This bug is strikingly similar to https://bugs.openjdk.org/browse/JDK-8207760 except that dealt with a 1024 byte buffer in the parser that was fixed in JDK 12. I believe this bug also existed since pre-JDK 12 VMs but was indistinguishable from the earlier bug, and that the same problem exists in a completely different buffer in the XML transformer code.

STEPS TO FOLLOW TO REPRODUCE THE PROBLEM :
Three files (Java, XSL, XML) are provided in Test Case Code (TransformTest.java, xml-to-text-test.xml, PMC6002666.xml). These should be in the same directory. Compile and run TransformTest.java without any special parameters.

The XSL should simply extract the text content from all nodes in the XML document. However, it instead throws a com.sun.org.apache.xalan.internal.xsltc.TransletException in the GregorSamsa translet indicating "Invalid UTF-16 surrogate detected: df00 d835 ?".

If you modify the XML file by either adding or deleting a single 'x' character from the <a> element, the transformation works.

The error is triggered when certain (valid) UTF-8 characters appear in sequence over a 1024-byte boundary in the string. For instance, you can add any multiple of 1024 additional single-byte characters before the string and trigger the same exception.

The transformation is not affected by the addition of characters after the string.

EXPECTED VERSUS ACTUAL BEHAVIOR :
EXPECTED -
xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx𝜃𝜀𝜀𝜀𝜀

ACTUAL -
Exception in thread "main" javax.xml.transform.TransformerException: com.sun.org.apache.xalan.internal.xsltc.TransletException: org.xml.sax.SAXException: java.io.IOException: Invalid UTF-16 surrogate detected: df00 d835 ?
java.io.IOException: Invalid UTF-16 surrogate detected: df00 d835 ?
org.xml.sax.SAXException: java.io.IOException: Invalid UTF-16 surrogate detected: df00 d835 ?
java.io.IOException: Invalid UTF-16 surrogate detected: df00 d835 ?
at java.xml/com.sun.org.apache.xalan.internal.xsltc.trax.TransformerImpl.transform(TransformerImpl.java:786)
at java.xml/com.sun.org.apache.xalan.internal.xsltc.trax.TransformerImpl.transform(TransformerImpl.java:394)
at TransformTest.main(TransformTest.java:36)
Caused by: com.sun.org.apache.xalan.internal.xsltc.TransletException: org.xml.sax.SAXException: java.io.IOException: Invalid UTF-16 surrogate detected: df00 d835 ?
java.io.IOException: Invalid UTF-16 surrogate detected: df00 d835 ?
org.xml.sax.SAXException: java.io.IOException: Invalid UTF-16 surrogate detected: df00 d835 ?
java.io.IOException: Invalid UTF-16 surrogate detected: df00 d835 ?
at java.xml/com.sun.org.apache.xalan.internal.xsltc.dom.SAXImpl.characters(SAXImpl.java:1560)
at java.xml/com.sun.org.apache.xalan.internal.xsltc.dom.DOMAdapter.characters(DOMAdapter.java:326)
at jdk.translet/die.verwandlung.GregorSamsa.applyTemplates()
at jdk.translet/die.verwandlung.GregorSamsa.applyTemplates()
at jdk.translet/die.verwandlung.GregorSamsa.template$dot$0()
at jdk.translet/die.verwandlung.GregorSamsa.applyTemplates()
at jdk.translet/die.verwandlung.GregorSamsa.transform()
at java.xml/com.sun.org.apache.xalan.internal.xsltc.runtime.AbstractTranslet.transform(AbstractTranslet.java:627)
at java.xml/com.sun.org.apache.xalan.internal.xsltc.trax.TransformerImpl.transform(TransformerImpl.java:782)
... 2 more
Caused by: org.xml.sax.SAXException: java.io.IOException: Invalid UTF-16 surrogate detected: df00 d835 ?
java.io.IOException: Invalid UTF-16 surrogate detected: df00 d835 ?
at java.xml/com.sun.org.apache.xml.internal.serializer.ToTextStream.characters(ToTextStream.java:234)
at java.xml/com.sun.org.apache.xml.internal.utils.FastStringBuffer.sendSAXcharacters(FastStringBuffer.java:987)
at java.xml/com.sun.org.apache.xml.internal.dtm.ref.sax2dtm.SAX2DTM2.dispatchCharactersEvents(SAX2DTM2.java:3024)
at java.xml/com.sun.org.apache.xalan.internal.xsltc.dom.SAXImpl.characters(SAXImpl.java:1558)
... 10 more
Caused by: java.io.IOException: Invalid UTF-16 surrogate detected: df00 d835 ?
at java.xml/com.sun.org.apache.xml.internal.serializer.ToStream.throwIOE(ToStream.java:1808)
at java.xml/com.sun.org.apache.xml.internal.serializer.ToStream.writeUTF16Surrogate(ToStream.java:976)
at java.xml/com.sun.org.apache.xml.internal.serializer.ToTextStream.writeNormalizedChars(ToTextStream.java:306)
at java.xml/com.sun.org.apache.xml.internal.serializer.ToTextStream.characters(ToTextStream.java:226)
... 13 more
---------
com.sun.org.apache.xalan.internal.xsltc.TransletException: org.xml.sax.SAXException: java.io.IOException: Invalid UTF-16 surrogate detected: df00 d835 ?
java.io.IOException: Invalid UTF-16 surrogate detected: df00 d835 ?
org.xml.sax.SAXException: java.io.IOException: Invalid UTF-16 surrogate detected: df00 d835 ?
java.io.IOException: Invalid UTF-16 surrogate detected: df00 d835 ?
at java.xml/com.sun.org.apache.xalan.internal.xsltc.dom.SAXImpl.characters(SAXImpl.java:1560)
at java.xml/com.sun.org.apache.xalan.internal.xsltc.dom.DOMAdapter.characters(DOMAdapter.java:326)
at jdk.translet/die.verwandlung.GregorSamsa.applyTemplates()
at jdk.translet/die.verwandlung.GregorSamsa.applyTemplates()
at jdk.translet/die.verwandlung.GregorSamsa.template$dot$0()
at jdk.translet/die.verwandlung.GregorSamsa.applyTemplates()
at jdk.translet/die.verwandlung.GregorSamsa.transform()
at java.xml/com.sun.org.apache.xalan.internal.xsltc.runtime.AbstractTranslet.transform(AbstractTranslet.java:627)
at java.xml/com.sun.org.apache.xalan.internal.xsltc.trax.TransformerImpl.transform(TransformerImpl.java:782)
at java.xml/com.sun.org.apache.xalan.internal.xsltc.trax.TransformerImpl.transform(TransformerImpl.java:394)
at TransformTest.main(TransformTest.java:36)
Caused by: org.xml.sax.SAXException: java.io.IOException: Invalid UTF-16 surrogate detected: df00 d835 ?
java.io.IOException: Invalid UTF-16 surrogate detected: df00 d835 ?
at java.xml/com.sun.org.apache.xml.internal.serializer.ToTextStream.characters(ToTextStream.java:234)
at java.xml/com.sun.org.apache.xml.internal.utils.FastStringBuffer.sendSAXcharacters(FastStringBuffer.java:987)
at java.xml/com.sun.org.apache.xml.internal.dtm.ref.sax2dtm.SAX2DTM2.dispatchCharactersEvents(SAX2DTM2.java:3024)
at java.xml/com.sun.org.apache.xalan.internal.xsltc.dom.SAXImpl.characters(SAXImpl.java:1558)
... 10 more
Caused by: java.io.IOException: Invalid UTF-16 surrogate detected: df00 d835 ?
at java.xml/com.sun.org.apache.xml.internal.serializer.ToStream.throwIOE(ToStream.java:1808)
at java.xml/com.sun.org.apache.xml.internal.serializer.ToStream.writeUTF16Surrogate(ToStream.java:976)
at java.xml/com.sun.org.apache.xml.internal.serializer.ToTextStream.writeNormalizedChars(ToTextStream.java:306)
at java.xml/com.sun.org.apache.xml.internal.serializer.ToTextStream.characters(ToTextStream.java:226)
... 13 more
---------
org.xml.sax.SAXException: java.io.IOException: Invalid UTF-16 surrogate detected: df00 d835 ?
java.io.IOException: Invalid UTF-16 surrogate detected: df00 d835 ?
at java.xml/com.sun.org.apache.xml.internal.serializer.ToTextStream.characters(ToTextStream.java:234)
at java.xml/com.sun.org.apache.xml.internal.utils.FastStringBuffer.sendSAXcharacters(FastStringBuffer.java:987)
at java.xml/com.sun.org.apache.xml.internal.dtm.ref.sax2dtm.SAX2DTM2.dispatchCharactersEvents(SAX2DTM2.java:3024)
at java.xml/com.sun.org.apache.xalan.internal.xsltc.dom.SAXImpl.characters(SAXImpl.java:1558)
at java.xml/com.sun.org.apache.xalan.internal.xsltc.dom.DOMAdapter.characters(DOMAdapter.java:326)
at jdk.translet/die.verwandlung.GregorSamsa.applyTemplates()
at jdk.translet/die.verwandlung.GregorSamsa.applyTemplates()
at jdk.translet/die.verwandlung.GregorSamsa.template$dot$0()
at jdk.translet/die.verwandlung.GregorSamsa.applyTemplates()
at jdk.translet/die.verwandlung.GregorSamsa.transform()
at java.xml/com.sun.org.apache.xalan.internal.xsltc.runtime.AbstractTranslet.transform(AbstractTranslet.java:627)
at java.xml/com.sun.org.apache.xalan.internal.xsltc.trax.TransformerImpl.transform(TransformerImpl.java:782)
at java.xml/com.sun.org.apache.xalan.internal.xsltc.trax.TransformerImpl.transform(TransformerImpl.java:394)
at TransformTest.main(TransformTest.java:36)
Caused by: java.io.IOException: Invalid UTF-16 surrogate detected: df00 d835 ?
at java.xml/com.sun.org.apache.xml.internal.serializer.ToStream.throwIOE(ToStream.java:1808)
at java.xml/com.sun.org.apache.xml.internal.serializer.ToStream.writeUTF16Surrogate(ToStream.java:976)
at java.xml/com.sun.org.apache.xml.internal.serializer.ToTextStream.writeNormalizedChars(ToTextStream.java:306)
at java.xml/com.sun.org.apache.xml.internal.serializer.ToTextStream.characters(ToTextStream.java:226)
... 13 more
---------
java.io.IOException: Invalid UTF-16 surrogate detected: df00 d835 ?
at java.xml/com.sun.org.apache.xml.internal.serializer.ToStream.throwIOE(ToStream.java:1808)
at java.xml/com.sun.org.apache.xml.internal.serializer.ToStream.writeUTF16Surrogate(ToStream.java:976)
at java.xml/com.sun.org.apache.xml.internal.serializer.ToTextStream.writeNormalizedChars(ToTextStream.java:306)
at java.xml/com.sun.org.apache.xml.internal.serializer.ToTextStream.characters(ToTextStream.java:226)
at java.xml/com.sun.org.apache.xml.internal.utils.FastStringBuffer.sendSAXcharacters(FastStringBuffer.java:987)
at java.xml/com.sun.org.apache.xml.internal.dtm.ref.sax2dtm.SAX2DTM2.dispatchCharactersEvents(SAX2DTM2.java:3024)
at java.xml/com.sun.org.apache.xalan.internal.xsltc.dom.SAXImpl.characters(SAXImpl.java:1558)
at java.xml/com.sun.org.apache.xalan.internal.xsltc.dom.DOMAdapter.characters(DOMAdapter.java:326)
at jdk.translet/die.verwandlung.GregorSamsa.applyTemplates()
at jdk.translet/die.verwandlung.GregorSamsa.applyTemplates()
at jdk.translet/die.verwandlung.GregorSamsa.template$dot$0()
at jdk.translet/die.verwandlung.GregorSamsa.applyTemplates()
at jdk.translet/die.verwandlung.GregorSamsa.transform()
at java.xml/com.sun.org.apache.xalan.internal.xsltc.runtime.AbstractTranslet.transform(AbstractTranslet.java:627)
at java.xml/com.sun.org.apache.xalan.internal.xsltc.trax.TransformerImpl.transform(TransformerImpl.java:782)
at java.xml/com.sun.org.apache.xalan.internal.xsltc.trax.TransformerImpl.transform(TransformerImpl.java:394)
at TransformTest.main(TransformTest.java:36)


---------- BEGIN SOURCE ----------
TransformTest.java

import java.io.ByteArrayOutputStream;
import java.io.FileInputStream;

import javax.xml.parsers.DocumentBuilder;
import javax.xml.parsers.DocumentBuilderFactory;
import javax.xml.transform.Result;
import javax.xml.transform.Source;
import javax.xml.transform.Transformer;
import javax.xml.transform.TransformerFactory;
import javax.xml.transform.dom.DOMSource;
import javax.xml.transform.stream.StreamResult;

import org.w3c.dom.Document;

public class TransformTest
{
    public static void main(String[] args) throws Exception
    {
        DocumentBuilderFactory factory = DocumentBuilderFactory.newInstance();
        factory.setNamespaceAware(true);

        DocumentBuilder builder = factory.newDocumentBuilder();

        Document xml = builder.parse(new FileInputStream("MC6002666.xml"));
        Document xsl = builder.parse(new FileInputStream("xml-to-text-test.xsl"));

        Source xmlSource = new DOMSource(xml);
        Source xslSource = new DOMSource(xsl);

        Transformer transformer = TransformerFactory.newInstance().newTransformer(xslSource);

        ByteArrayOutputStream baos = new ByteArrayOutputStream();

        Result result = new StreamResult(baos);

        transformer.transform(xmlSource, result);

        String output = new String(baos.toByteArray());
        System.out.println(output);
    }
}

xml-to-text-test.xml

<?xml version="1.0" encoding="UTF-8"?>
<xsl:stylesheet version="1.0" xmlns:xsl="http://www.w3.org/1999/XSL/Transform">
  <xsl:output encoding="UTF-8" method="text" />
  <xsl:template match="/"><xsl:apply-templates select="node()" /></xsl:template>
</xsl:stylesheet>

PMC6002666.xml

<?xml version="1.0" ?><a>xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx𝜃𝜀𝜀<b>𝜀</b>𝜀</a>

---------- END SOURCE ----------