JFrame is created with JButton and JCheckBox. setBackground(new Color(0, 0, 0, 0)) is invoked for JCheckBox. When mouse cursor is over the JCheckBox, JButton component is painted below it.

Try attached code.

Having transparent color for JCheckBox is necessary for Translucency effects as by default it is painted with rectangular background. See also CR 6987838.
JComponent.setBackground javadoc doesn't state that color must be opaque in order for component to paint the background correctly.

This is differ from JWindow, JFrame and JDialog behavior where setting the translucent background color switches the window into translucency mode.

In order to resolve ambiguity the javadoc needs to be updated.