Uploaded image for project: 'JDK'
  1. JDK
  2. JDK-8160638

Solaris JVM unable to allocate more than 2GB of direct byte buffers when max heap is <= 2GB

    XMLWordPrintable

    Details

    • Subcomponent:
    • CPU:
      x86
    • OS:
      solaris_11

      Description

      FULL PRODUCT VERSION :
      java version "1.8.0_91"
      Java(TM) SE Runtime Environment (build 1.8.0_91-b14)
      Java HotSpot(TM) 64-Bit Server VM (build 25.91-b14, mixed mode)

      ADDITIONAL OS VERSION INFORMATION :
      SunOS tc-perf-026 5.11 11.2 i86pc i386 i86pc

      A DESCRIPTION OF THE PROBLEM :
      When I start the jvm with the following flags:

      -XX:MaxDirectMemorySize=100g -Xmx2048m

      I should be able to call java.nio.ByteBuffer.allocateDirect(32 * 1024 * 1024) 512 times to allocate a total of 8GB - but I get an OutOfMemoryError a bit before 2GB have been allocated.

      This works fine on OS X, Windows and Linux but not on Solaris. But If I do start the solaris JVM with the following flags:

      -XX:MaxDirectMemorySize=100g -Xmx2049m

      then I'm able to allocate the 8GB of direct byte buffers.

      STEPS TO FOLLOW TO REPRODUCE THE PROBLEM :
      I've included a simple test class (Buffers.java) that reproduces the problem. You just have to make sure:
       - that you're running it on a machine with enough free ram
       - that you add -XX:MaxDirectMemorySize=100g and -Xmx2g on the command line


      EXPECTED VERSUS ACTUAL BEHAVIOR :
      EXPECTED -
      The above Buffers class should report that it successfully allocated the 8GB.
      ACTUAL -
      The above Buffers class catches OutOfMemoryError when a little below 2GB have been allocated.

      REPRODUCIBILITY :
      This bug can be reproduced always.

      ---------- BEGIN SOURCE ----------
      import java.nio.*;
      import java.util.*;

      public class Buffers {
        // 32MB * 256 == 8GB
        private static final int BUFFER_SIZE = 32 * 1024 * 1024;
        private static final int BUFFER_COUNT = 256;

        public static void main(String[] args) throws Exception {
          long counter = 0L;
          try {
              List<ByteBuffer> buffers = new ArrayList<ByteBuffer>();
              for (int i=0 ; i<BUFFER_COUNT ; i++) {
                      ByteBuffer b = ByteBuffer.allocateDirect(BUFFER_SIZE);
                      buffers.add(b);
                      counter += BUFFER_SIZE;
              }
          } catch(Throwable t) {
              t.printStackTrace();
          }
          System.out.println("allocated : " + (counter / 1024 / 1024 + " MB"));
        }

      }
      ---------- END SOURCE ----------

      CUSTOMER SUBMITTED WORKAROUND :
      I found two:

      1) tell the JVM that its max heap size is over 2GB (ie: -Xmx2049m)
      2) use libumem's mmap backend to start the JVM:
      LD_PRELOAD=libumem.so UMEM_OPTIONS=backend=mmap java ...

        Attachments

          Issue Links

            Activity

              People

              Assignee:
              dbuck David Buck
              Reporter:
              webbuggrp Webbug Group
              Votes:
              0 Vote for this issue
              Watchers:
              7 Start watching this issue

                Dates

                Created:
                Updated: