Memory Leak in XSL Transform leading to OutOfMemory Exception

FULL PRODUCT VERSION :
java version "1.5.0_06"
Java(TM) 2 Runtime Environment, Standard Edition (build 1.5.0_06-b05)
Java HotSpot(TM) Client VM (build 1.5.0_06-b05, mixed mode, sharing)


ADDITIONAL OS VERSION INFORMATION :
Microsoft Windows XP
Professional Version 2002
Service Pack 2


A DESCRIPTION OF THE PROBLEM :
The XSL Transformation code in the standard JDK has a severe memory leak, which leads to an OutOfMemory exception when it is executed multiple times in a program. To replicate, run the test program.

When the start button is pressed in the test program, it creates and runs a new thread which performs an XSL transformation. This transformation reads an input XML file, and then generates an output HTML file, based on the rules XSLT file which is also read in. Looking at the test program, you will notice that there are no dangling references to any of the transformation objects. Start the test program under a profiler, and then monitor the memory usage. For each successive click on the start button, the memory profile of the program exhibits a class memory leak i.e. memory usage grows until the JVM runs out of memory, whereby an OutOfMemory Exception is thrown.

I have included a sample input XML file and rules XSLT file, however any valid XML and appropriate XSLT document will do. You may wish to use a larger XML file, and run the test program with a small JVM memory size, to more easily show the memory leak.

I have looked at the JVM heap under a profiler, and I believe that the bug ultimately lies in com.sun.org.apache.xml.internal.utils.XMLReaderManager. This class is a factory, which creates instances of XMLReader when getXMLReader() is called. The class also caches created XMLReaders, on a per-thread basis. When a thread is finished with an XMLReader, it calls releaseXMLReader() - XMLReaderManager marks its cached instance of XMLReader as being free, but hangs onto the instance. This is the problem, as the XMLReader has a reference to the complete source XML document that it has just transformed, as well as some other data structures.

The problem gets worse when the thread that performed the XSL transformation dies. The cached instance of XMLReader in XMLReaderManager is automatically freed, as per the semantics of the ThreadLocal var that is fine. But the Hashtable in XMLReaderManager still hangs onto a ref to the XMLReader instance, because the instance is a key to an entry in that table. So, if you start up a background thread to perform the transformation, and you do this multiple times as in the test program, you have multiple instances of XMLReader and the complete XML document being hung onto, and never freed. Hence the OutOfMemory exception.

As a comparison, you can comment out the creation of the background thread in the test program, and just do the transformation in the Swing thread, when the start button is pressed. You will notice in the profiler that only one instance of XMLReader is being held onto by XMLReaderManager, and successive clicks on the start button does not lead to an OutOfMemory exception; the XMLReader though still holds onto the last XML document it transformed, when really that memory should be released post the transformation.

I could be wrong about the ultimate source of the bug being XMLReaderManager i.e. the bug instead could be that reset() in not being called appropriately on the cached XMLReaders, which is meant to empty its state. I am not wrong though about a memory leak in the XSL Transformation.


STEPS TO FOLLOW TO REPRODUCE THE PROBLEM :
See above notes.


EXPECTED VERSUS ACTUAL BEHAVIOR :
EXPECTED -
See above notes.
ACTUAL -
See above notes.

REPRODUCIBILITY :
This bug can be reproduced always.

---------- BEGIN SOURCE ----------
The test program is:

import java.awt.*;
import java.awt.event.*;
import java.io.*;
import javax.swing.*;
import javax.xml.transform.Transformer;
import javax.xml.transform.TransformerFactory;
import javax.xml.transform.stream.StreamResult;
import javax.xml.transform.stream.StreamSource;

public class TestXSLTransform extends JFrame {

  public TestXSLTransform() {
    init();
    validate();
    setVisible(true);
  }

  public static void main(String[] args) {
    SwingUtilities.invokeLater(new Runnable() {
      public void run() {
        new TestXSLTransform();
      }
    });
  }

  private void init() {
    addWindowListener(new WindowAdapter() {
      public void windowClosing(WindowEvent evt) {
        System.exit(0);
      }
    });

    getContentPane().setLayout(new BorderLayout());

    JButton btn = new JButton("Start");
    btn.addActionListener(new ActionListener() {
      public void actionPerformed(ActionEvent e) {
        Thread t = new Thread(new Runnable() {
          public void run() {
            transform();
          }
        });

        t.start();
      }
    });


    getContentPane().add(btn, BorderLayout.CENTER);
    setSize(new Dimension(200, 100));
  }

  private void transform() {
    System.out.println("start transform");

    try {
      InputStream in = new BufferedInputStream(new FileInputStream("input.xml"));
      OutputStream out = new BufferedOutputStream(new FileOutputStream("output.html"));
      InputStream rulesIn = new BufferedInputStream(new FileInputStream("rules.xsl"));

      TransformerFactory tFactory = TransformerFactory.newInstance();
      Transformer transformer = tFactory.newTransformer(new StreamSource(rulesIn));

      transformer.transform(new StreamSource(in), new StreamResult(out));

      in.close();
      out.close();
      rulesIn.close();

    } catch (Exception e) {
      System.out.println("exception = " + e);
    }

    System.out.println("end transform");
  }

}


-----------------------------------

The input.xml file is:

<?xml version='1.0' encoding='utf-8'?>
<report>
<body>
<table>

<table_row>
<table_cell><cell_value><![CDATA[In progress]]></cell_value><cell_type>string</cell_type></table_cell>
<table_cell><cell_value><![CDATA[Unit Pricing]]></cell_value><cell_type>string</cell_type></table_cell>
<table_cell><cell_value><![CDATA[TEST_PAUL]]></cell_value><cell_type>string</cell_type></table_cell>
<table_cell><cell_value><![CDATA[30-Dec-2005]]></cell_value><cell_type>string</cell_type></table_cell>
<table_cell><cell_value><![CDATA[1]]></cell_value><cell_type>string</cell_type></table_cell>
<table_cell><cell_value><![CDATA[System]]></cell_value><cell_type>string</cell_type></table_cell>
<table_cell><cell_value><![CDATA[366162]]></cell_value><cell_type>string</cell_type></table_cell>
<table_cell><cell_value><![CDATA[05-May-2006 10:52:39]]></cell_value><cell_type>string</cell_type></table_cell>
<table_cell><cell_value><![CDATA[0]]></cell_value><cell_type>string</cell_type></table_cell>
<table_cell><cell_value><![CDATA[19305345]]></cell_value><cell_type>string</cell_type></table_cell>
<table_cell><cell_value><![CDATA[0]]></cell_value><cell_type>string</cell_type></table_cell>
<table_cell><cell_value><![CDATA[0]]></cell_value><cell_type>string</cell_type></table_cell>
<table_cell><cell_value><![CDATA[server]]></cell_value><cell_type>string</cell_type></table_cell>
</table_row>

</table>
</body>
</report>

-----------------------------------

The rules.xsl file is:

<?xml version="1.0"?>
<xsl:stylesheet xmlns:xsl="http://www.w3.org/1999/XSL/Transform" version="1.0">
  <xsl:output method="html" indent="yes"/>

  <xsl:template match="report">
    <html>
      <xsl:apply-templates/>
    </html>
  </xsl:template>

  <xsl:template match="body">
    <body>
      <xsl:apply-templates/>
    </body>
  </xsl:template>

  <xsl:template match="table">
    <table border="1">
      <xsl:apply-templates/>
    </table>
  </xsl:template>

  <xsl:template match="table_cell">
    <xsl:choose>
      <xsl:when test="cell_value != ''">
        <td>
          <xsl:value-of select="cell_value"/>
        </td>
      </xsl:when>
      <xsl:otherwise>
        <td>
          -
        </td>
      </xsl:otherwise>
    </xsl:choose>
  </xsl:template>

  <xsl:template match="table_row">
    <tr>
      <xsl:apply-templates/>
    </tr>
  </xsl:template>

</xsl:stylesheet>

---------- END SOURCE ----------

CUSTOMER SUBMITTED WORKAROUND :
I do not have a work-around.