ADDITIONAL SYSTEM INFORMATION :
Reproduced using JDK versions: 1.8u102, 11.0.2, 15.0.2

This only appears to happen on macOS and not on Windows. Reproduced on macOS Catalina 10.15.7. This also occurs regardless of the following system setting: System Preferences > Mission Control > Displays have separate Spaces, which will cause all monitors/screens to have the top toolbar if turned on.

Displays: 2 displays with native resolution of 3840x2160, though in System Preferences > Displays I run both with the option to Scale to "Looks like 2560x1440". I can also reproduce this issue while running at the 3840x2160 resolution.

A DESCRIPTION OF THE PROBLEM :
On macOS if a JFrame is currently located in the top-left of the non-default screen and the takes up the full width/height of that screen, calling JFrame.setExtendedState(MAXIMIZED_BOTH) will cause the window to move partially to the primary screen. This is unexpected as the frame exists entirely within the bounds of the second screen.

STEPS TO FOLLOW TO REPRODUCE THE PROBLEM :
This bug requires at least two monitors.
1. Create a JFrame with the default/normal/unset extended state.
2. Position the JFrame to be in the top-left corner of the secondary monitor. On my system the secondary monitor on macOS does not have a toolbar at the top of the screen.
3. Size the frame to be the full width/height of the screen.
4. Set the extended state via: jFrame.setExtendedState(JFrame.MAXIMUM_BOTH);

EXPECTED VERSUS ACTUAL BEHAVIOR :
EXPECTED -
The window's should remain positioned in the secondary screen and its dimensions should be maximized if not already.
ACTUAL -
The window shifts partway between the two monitors. On my system it shifts the majority of its dimensions to the primary monitor with around ~300px overlap to the second monitor.

---------- BEGIN SOURCE ----------
import java.awt.Dimension;
import java.awt.GraphicsConfiguration;
import java.awt.GraphicsDevice;
import java.awt.GraphicsEnvironment;
import java.awt.Insets;
import java.awt.Point;
import java.awt.Rectangle;
import java.awt.Toolkit;

import javax.swing.JFrame;
import javax.swing.SwingUtilities;

public class TestMain {
public static void main(String[] args) {
SwingUtilities.invokeLater(() -> {
GraphicsEnvironment genv = GraphicsEnvironment.getLocalGraphicsEnvironment();

// This is the main screen which has the global mac toolbar.
GraphicsDevice defaultScreenDevice = genv.getDefaultScreenDevice();

// Set the size/location of the window to be on a screen which is not the main screen.
// On mac systems this is a screen which does not have the global mac toolbar.
Rectangle alternateWindowBounds = null;
for (GraphicsDevice device : genv.getScreenDevices()) {
if (device != defaultScreenDevice) {
alternateWindowBounds = device.getDefaultConfiguration().getBounds();
break;
}
}

Dimension startingBounds = alternateWindowBounds.getSize();
Point startingLocation = alternateWindowBounds.getLocation();

// Uncommenting the following lines appears to resolve the issue. On my screens (2 monitors 2560x1440)
// this will remove 23 pixels from the height, should be the height of to global mac toolbar. However
// after testing the magic number appears to be removing 11 pixels from height -- removing 10 pixels
// does not fix the behavior.
/*
GraphicsConfiguration defaultScreenConf = defaultScreenDevice.getDefaultConfiguration();
Insets screenInsets = Toolkit.getDefaultToolkit().getScreenInsets(defaultScreenConf);
startingBounds.width -= (screenInsets.left + screenInsets.right);
startingBounds.height -= (screenInsets.top + screenInsets.bottom);
*/

JFrame frame = new JFrame("Test");
frame.setSize(startingBounds);
frame.setLocation(startingLocation);
frame.setVisible(true);

// Setting the extended state to MAXIMIZED_BOTH results in moving the window to be primarily
// on the main screen, but offset by some consistent but seemingly-random offset.
// Note that if we do not call setSize() above then this also behaves as expected.
// Setting either VERT or HORIZ separately does not cause this issue.
frame.setExtendedState(JFrame.MAXIMIZED_BOTH);
});
}
}
---------- END SOURCE ----------

CUSTOMER SUBMITTED WORKAROUND :
Prior to calling setExtendedState(MAXIMUM_BOTH), reduce the height of the frame by some arbitrary size. On my system the window height must be reduced by at least 11 pixels, anything less will still result in the same repositioning behavior. In the test case I included I have commented that removing default screen's insets works in that case though it's unclear if that would work in every scenario.

FREQUENCY : always