ADDITIONAL SYSTEM INFORMATION :
guest OS: macOS 10.15
host OS: Windows 10
Hypervisors: - VMWare Player 15.5
                    - VirtualBox 6.0

A DESCRIPTION OF THE PROBLEM :
When running a JavaFX application in macOS guest VM, the main stage is completely blank, with the following errors: CGLChoosePixelFormat error: 10002, CGLCreateContext error: 10002
This behavior was observed with a macOS 10.15 guest OS on both VMWare player and VirtualBox, on a Windows 10 host.

This is an old issue that has already been reported several times (notably as JDK-8154852) but is claimed to have been fixed by JDK-8154148.
However the fix provided by JDK-8154148 is incomplete; while it does fix a JVM crash, it doesn't prevent an incorrect pixel format to be retrieved, which is the root cause for the stage being empty.

The problematic code is located at #96 in GlassView3D.m:

095: CGLError err = CGLChoosePixelFormat(attributes, &pix, &npix);
096: if ((err == kCGLNoError) && (npix == 0))
097: {
098: const CGLPixelFormatAttribute attributes2[] =
099: {
100: kCGLPFAAllowOfflineRenderers,
101: (CGLPixelFormatAttribute)0
102: };
103: err = CGLChoosePixelFormat(attributes2, &pix, &npix);
104: }


In a comment in JDK-8154148, the following claim is made: "[...] an error happens is when something bad or invalid has occurred. An unsatisfied pixel format request is indicated by npix is 0 and pix is NULL. Hence I feel it is a cleaner and clearer logic to request a different format when npix is 0 and err is kCGLNoError." (https://bugs.openjdk.java.net/browse/JDK-8154148?focusedCommentId=13980465&page=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-13980465)

From what I could observe, this is not necessarily true; in this case where a pixel format satisfying the initial conditions could not be found, the return code is kCGLBadPixelFormat (10002); which means we don't try to get a pixel format with less restrictive condition.
This suggests that a better condition for the second call to CGLChoosePixelFormat (line #103) should be "if (pix == NULL)" instead of "if ((err == kCGLNoError) && (npix == 0))"
This is further supported by a sample in Apple developer's documentation on how to choose a pixel format: https://developer.apple.com/library/archive/documentation/GraphicsImaging/Conceptual/OpenGL-MacProgGuide/opengl_pixelformats/opengl_pixelformats.html



FREQUENCY : always