Uploaded image for project: 'JDK'
  1. JDK
  2. JDK-8258956

Memory Leak in StringCoding on ThreadLocal resultCached StringCoding.Result

XMLWordPrintable

    • b06
    • generic
    • generic
    • Not verified

        ADDITIONAL SYSTEM INFORMATION :
        Running on Window 10 - 19019 but happens also on Linux distributions
        Tests using openjdk version "11.0.9.1" 2020-11-04


        A DESCRIPTION OF THE PROBLEM :
        In JDK 11 the class StringCoding has been change and has introduced a new ThreadLocal resultCached showing StringCoding.Result

        Line 512
             /* The cached Result for each thread */
            private static final ThreadLocal<StringCoding.Result>
                resultCached = new ThreadLocal<>() {
                    protected StringCoding.Result initialValue() {
                        return new StringCoding.Result();
                    }};

        Since the StringCoding.Result() is not encapsulated in a SoftReference the Garbage Collector cannot cleanup the internal byte[] even if choosing a very small value for -XX:SoftRefLRUPolicyMSPerMB=<ms>

        I have done a simple test to reproduce the error. Therfore I write a simple Spring Boot Web Application providng a Rest interface. Within the rest method I created a simple huge byte array and have called the String constructure which does a encoding.

            byte[] testArray = //some huge byte array
            new String(testArray, 0, testArray.length, "UTF-8");

        Internally java.lang.String calls StringCoding.decode
            
             StringCoding.Result ret = StringCoding.decode(charset, bytes, offset, length);

        Within StringCoding the StringCoding.Result class helds an byte[] of the given data.

        Now call the Rest interface most of the time the caller get a new http-nio-8080-exec-n thread which is processing the request.
        Now, after the StringCoding.decode method has been called the Result is cached in the ThreadLocal variable and not cleaned up after processing.
        This results in a memory leak if Thread Pools are used which is very usual in Spring Boot or Applications Server in general. If the values of the testArray are very huge (e.g. XML or JSON messages) while different Thread are taken from a Thread Pool it will result in a Out Of Memory Error after a view requests.

        Profilers Heap Walker says:
        org.apache.tomcat.util.threads.TaskThread 195 MB (48 %)
         195 MB (100.0%) threadLocals (declared by java.lang.Thread) java.lang.ThreadLocal$ThreadLocalMap
         195 MB (100.0%) table java.lang.ThreadLocal$ThreadLocalMap$Entry[ ]
         195 MB (100.0%) element java.lang.ThreadLocal$ThreadLocalMap$Entry
         195 MB (100.0%) value java.lang.StringCoding$Result
         195 MB (100.0%) value byte[ ]
         Another 8 instances with a total retained size of 784 bytes and a maximum single retained size of 200 bytes
         Another 4 instances with a total retained size of 256 bytes and a maximum single retained size of 104 bytes
        org.apache.tomcat.util.threads.TaskThread 195 MB (48 %)
         195 MB (100.0%) threadLocals (declared by java.lang.Thread) java.lang.ThreadLocal$ThreadLocalMap
         195 MB (100.0%) table java.lang.ThreadLocal$ThreadLocalMap$Entry[ ]
         195 MB (100.0%) element java.lang.ThreadLocal$ThreadLocalMap$Entry
         195 MB (100.0%) value java.lang.StringCoding$Result
         195 MB (100.0%) value byte[ ]
         Another 8 instances with a total retained size of 784 bytes and a maximum single retained size of 200 bytes
         Another 4 instances with a total retained size of 256 bytes and a maximum single retained size of 104 bytes

        ThreadLocal for StringCoding.Result has been introduced in JDK 11
        All version since JDK 11 are effected, also have took a look on JDK 17 and tested with JDK 17


        REGRESSION : Last worked in version 8

        STEPS TO FOLLOW TO REPRODUCE THE PROBLEM :
        I have done a simple test to reproduce the error. Therfore I write a simple Spring Boot Web Application providng a Rest interface. Within the rest method I created a simple huge byte array and have called the String constructure which does a encoding.

            byte[] testArray = //some huge byte array
            new String(testArray, 0, testArray.length, "UTF-8");

        Internally java.lang.String calls StringCoding.decode
            
             StringCoding.Result ret = StringCoding.decode(charset, bytes, offset, length);

        Within StringCoding the StringCoding.Result class helds an byte[] of the given data.

        Now call the Rest interface most of the time the caller get a new http-nio-8080-exec-n thread which is processing the request.
        Now, after the StringCoding.decode method has been called the Result is cached in the ThreadLocal variable and not cleaned up after processing.
        This results in a memory leak if Thread Pools are used which is very usual in Spring Boot or Applications Server in general. If the values of the testArray are very huge (e.g. XML or JSON messages) while different Thread are taken from a Thread Pool it will result in a Out Of Memory Error after a view requests.

        Profilers Heap Walker says:
        org.apache.tomcat.util.threads.TaskThread 195 MB (48 %)
         195 MB (100.0%) threadLocals (declared by java.lang.Thread) java.lang.ThreadLocal$ThreadLocalMap
         195 MB (100.0%) table java.lang.ThreadLocal$ThreadLocalMap$Entry[ ]
         195 MB (100.0%) element java.lang.ThreadLocal$ThreadLocalMap$Entry
         195 MB (100.0%) value java.lang.StringCoding$Result
         195 MB (100.0%) value byte[ ]
         Another 8 instances with a total retained size of 784 bytes and a maximum single retained size of 200 bytes
         Another 4 instances with a total retained size of 256 bytes and a maximum single retained size of 104 bytes
        org.apache.tomcat.util.threads.TaskThread 195 MB (48 %)
         195 MB (100.0%) threadLocals (declared by java.lang.Thread) java.lang.ThreadLocal$ThreadLocalMap
         195 MB (100.0%) table java.lang.ThreadLocal$ThreadLocalMap$Entry[ ]
         195 MB (100.0%) element java.lang.ThreadLocal$ThreadLocalMap$Entry
         195 MB (100.0%) value java.lang.StringCoding$Result
         195 MB (100.0%) value byte[ ]
         Another 8 instances with a total retained size of 784 bytes and a maximum single retained size of 200 bytes
         Another 4 instances with a total retained size of 256 bytes and a maximum single retained size of 104 bytes

        EXPECTED VERSUS ACTUAL BEHAVIOR :
        EXPECTED -
        StringCoding.Result is cleaned up after usage or encapsulated in a SoftReference to avoid OOM
        ACTUAL -
        Memory Leak on each new Thread

        FREQUENCY : always


              naoto Naoto Sato
              webbuggrp Webbug Group
              Votes:
              0 Vote for this issue
              Watchers:
              7 Start watching this issue

                Created:
                Updated:
                Resolved: