FULL PRODUCT VERSION :
1.8.0_161
(and 162)

ADDITIONAL OS VERSION INFORMATION :
Windows 7 and 10, but problably Linux as well.

EXTRA RELEVANT SYSTEM CONFIGURATION :
Using
    System.setProperty("com.sun.xml.internal.ws.transport.http.client.HttpTransportPipe.dump", "true");

will reveal enconding of '\n' as "
" in body of SOAP-envelope in outgoing requests.

A DESCRIPTION OF THE PROBLEM :
We use an implementation of SOAPHander to inject a SOAP header into a outgoing SOAP-envelope of a SOAP-client of ours.
The "body" of the envelope at that point has been created using the WSDL-imported API of our SOAP-service - thus, it is performed using JAXB. When our payload of Strings contains newline chars, they are escaped by the framework (JRE) from '\n' to "
".
When this occurs, the SOAPMessageContext.getMessage() method is fooled to believe we have "creation of external entity reference", causing an exception like the following to be thrown:

jan. 29, 2018 6:43:45 PM com.sun.xml.internal.messaging.saaj.soap.SOAPDocumentImpl createEntityReference
SEVERE: SAAJ0543: Entity References are not allowed in SOAP documents
java.lang.UnsupportedOperationException: Entity References are not allowed in SOAP documents
at com.sun.xml.internal.messaging.saaj.soap.SOAPDocumentImpl.createEntityReference(SOAPDocumentImpl.java:148)
at com.sun.xml.internal.messaging.saaj.soap.SOAPPartImpl.createEntityReference(SOAPPartImpl.java:437)
at com.sun.xml.internal.ws.api.message.saaj.SaajStaxWriter.writeEntityRef(SaajStaxWriter.java:245)

This works OK unless a SOAP-header is added to the outgoing SOAP-envelope by means of a SOAPHandler<SOAPMessageContext> implementation.
In the implementation of SOAPHandler<SOAPMessageContext>.handleMessage( SOAPMessageContext smc ) we perform:

      SOAPMessage message = smc.getMessage();
      SOAPPart soapPart = message.getSOAPPart();
      SOAPEnvelope soapEnvelope = soapPart.getEnvelope();
      ..etc..

Of these first lines, the smc.getMessage() invocation will cause an exception to be thrown when the body of the already formatted envelope contains "&#xa;" encodings:

SEVERE: SAAJ0543: Entity References are not allowed in SOAP documents
java.lang.UnsupportedOperationException: Entity References are not allowed in SOAP documents
at com.sun.xml.internal.messaging.saaj.soap.SOAPDocumentImpl.createEntityReference(SOAPDocumentImpl.java:148)
at com.sun.xml.internal.messaging.saaj.soap.SOAPPartImpl.createEntityReference(SOAPPartImpl.java:437)

This is a very severe problem, as it only allows for security-headers to be added, when the body-payload does not contain newlines.


REGRESSION. Last worked in version 8u151

ADDITIONAL REGRESSION INFORMATION:
1.8.0_151

STEPS TO FOLLOW TO REPRODUCE THE PROBLEM :
Do WSDL-import of some HelloWorld() SOAP-service, having the HelloWordl() method receive a String.
Establish end-point and add SOAP-handler that will perform getMessage() on its context.
Make a String in the client as: "<hello>Hello\nclient<hello>\n<hello>Hello\nclient<hello>\n"
Invoke HelleWorld of SOAP-end-point using the defined string as input.
Experience exception to be thrown by the framework.


EXPECTED VERSUS ACTUAL BEHAVIOR :
EXPECTED -
Either newlines are not to be escaped (prior to SOAPMessageContext.getMessage()) or newlines are not to be escaped at all.
ACTUAL -
Our security token cannot be added as a SOAP-header withing a SOAP-envelope whose body contains hex-escaped newline characters.



ERROR MESSAGES/STACK TRACES THAT OCCUR :
SEVERE: SAAJ0543: Entity References are not allowed in SOAP documents
java.lang.UnsupportedOperationException: Entity References are not allowed in SOAP documents
at com.sun.xml.internal.messaging.saaj.soap.SOAPDocumentImpl.createEntityReference(SOAPDocumentImpl.java:148)
at com.sun.xml.internal.messaging.saaj.soap.SOAPPartImpl.createEntityReference(SOAPPartImpl.java:437)
...

REPRODUCIBILITY :
This bug can be reproduced always.

---------- BEGIN SOURCE ----------
It should be fairly easy to reproduce the problem having a HelloWorld end-point and associated WSDL-imported client created. Interface for the end-point could be:

@WebService(name="HelloWorld")
public interface
{
  String helloWorld( String payLoad );
}


---------- END SOURCE ----------

CUSTOMER SUBMITTED WORKAROUND :
Downgrade to Java 1.8_151.

It is impossible to force every SOAP-client not to deliver '\n' chars in their payloads to some SOAP-end-point.