java.lang.OutOfMemoryError: Java heap space happened during deployment toolkit back buffer allocation, when there are plenty of heap space left.

Log started: Mon, 29 Feb 2016 11:00:25 -0500
Java Plug-in 11.74.2.31
Using JRE version 1.8.0_74-b31 Java HotSpot(TM) Client VM
...
security: The CRL support is enabled
Exception in thread "Thread-16" java.lang.OutOfMemoryError: Java heap space
        at java.awt.image.DataBufferInt.<init>(Unknown Source)
        at java.awt.image.Raster.createPackedRaster(Unknown Source)
        at java.awt.image.DirectColorModel.createCompatibleWritableRaster(Unknown Source)
        at sun.awt.Win32GraphicsConfig.createAcceleratedImage(Unknown Source)
        at sun.awt.windows.WComponentPeer.createImage(Unknown Source)
        at java.awt.Component.createImage(Unknown Source)
        at com.sun.deploy.uitoolkit.impl.awt.AWTAnimationPanel2.allocateBackBuffer(Unknown Source)
        at com.sun.deploy.uitoolkit.impl.awt.AWTAnimationPanel2.doPaint(Unknown Source)
        at com.sun.deploy.uitoolkit.impl.awt.AWTAnimationPanel2.run(Unknown Source)
        at java.lang.Thread.run(Unknown Source)

GC log:
Memory: 4k page, physical 4193784k(2607232k free), swap 8385708k(6988108k free)
CommandLine flags: -XX:InitialHeapSize=805306368 -XX:MaxHeapSize=805306368 -XX:+PrintGC -XX:+PrintGCDetails -XX:+PrintGCTimeStamps -XX:-UseLargePagesIndividualAllocation

2.031: [GC (Allocation Failure) 2.031: [DefNew: 109094K->3526K(235968K), 0.0116185 secs]2.042:
[Tenured: 0K->3243K(524288K), 0.0110016 secs] 109094K->3243K(760256K),
[Metaspace: 8579K->8579K(9600K)], 0.0228790 secs] [Times: user=0.01 sys=0.00, real=0.02 secs]

2.054: [Full GC (Allocation Failure) 2.054:
[Tenured: 3243K->3017K(524288K), 0.0089003 secs] 3243K->3017K(760256K),
[Metaspace: 8560K->8560K(9600K)], 0.0090180 secs] [Times: user=0.02 sys=0.00, real=0.01 secs]