-
Bug
-
Resolution: Unresolved
-
P3
-
15, 16
-
generic
-
windows
In some multiscreen configurations, the robot cannot move the mouse to the correct location.
1. Configuration to reproduce the bug:
Left monitor: main monitor, horizontal, 3840x2160, the top/left corner is 0,0, scale 100%(or 200%)
Right monitor: vertical, 2160x3840, the top/left corner is 3840, y=-1671, scale 100%
See display_configuration.png image for the exact layout.
I think that the important part of the bug is the negative Y of the top corner of the right monitor.
2. Run the ReproduceRobotBug.java using jdk15 or jdk16.
$.../javac ReproduceRobotBug.java
$.../java ReproduceRobotBug
3. It is expected that the mouse will be moved to the right monitor shifted by 100 pixels from the top/left corner:
int x = rightMonitor.x + 100;
int y = rightMonitor.y + 100;
robot.mouseMove(x, y);
4. But the mouse will be shifted from the top only, and placed at the left border of the right monitor.
To work around the bug uncomment this line:
//robot.mouseMove(x, y); // Uncomment to work around the bug
After additional research, I have found that the bug is in win32. It could be reproduced using "::SendInput":
https://docs.microsoft.com/en-us/windows/win32/api/winuser/nf-winuser-sendinput
I have attached the native bug reproducer, it has the same logic as java example. and have some additional comments.
To work around the bug uncomment this line:
//MouseMove(x, y); // Uncomment to work around the bug
Notes:
1. For the configuration described above it does not necessary to have HiDPI scale on any monitors, and the code for "setDPIAwareness()" can be removed from the native test, but I leave it to mimic how the jdk works, so it will be possible to verify the possible fix.
2. The bug initially was found in the HiDPI environment, looks like it is easy to reproduce in that case, but later I narrow it to the config above.
3. Both tests(java and native) tried to validate the location of the mouse after movement, and both sometimes passed! But visually the cursor is always in the wrong position
4. Yes to work around the bug we need to call mouseMove/MouseMove one more time using the same x/y.
1. Configuration to reproduce the bug:
Left monitor: main monitor, horizontal, 3840x2160, the top/left corner is 0,0, scale 100%(or 200%)
Right monitor: vertical, 2160x3840, the top/left corner is 3840, y=-1671, scale 100%
See display_configuration.png image for the exact layout.
I think that the important part of the bug is the negative Y of the top corner of the right monitor.
2. Run the ReproduceRobotBug.java using jdk15 or jdk16.
$.../javac ReproduceRobotBug.java
$.../java ReproduceRobotBug
3. It is expected that the mouse will be moved to the right monitor shifted by 100 pixels from the top/left corner:
int x = rightMonitor.x + 100;
int y = rightMonitor.y + 100;
robot.mouseMove(x, y);
4. But the mouse will be shifted from the top only, and placed at the left border of the right monitor.
To work around the bug uncomment this line:
//robot.mouseMove(x, y); // Uncomment to work around the bug
After additional research, I have found that the bug is in win32. It could be reproduced using "::SendInput":
https://docs.microsoft.com/en-us/windows/win32/api/winuser/nf-winuser-sendinput
I have attached the native bug reproducer, it has the same logic as java example. and have some additional comments.
To work around the bug uncomment this line:
//MouseMove(x, y); // Uncomment to work around the bug
Notes:
1. For the configuration described above it does not necessary to have HiDPI scale on any monitors, and the code for "setDPIAwareness()" can be removed from the native test, but I leave it to mimic how the jdk works, so it will be possible to verify the possible fix.
2. The bug initially was found in the HiDPI environment, looks like it is easy to reproduce in that case, but later I narrow it to the config above.
3. Both tests(java and native) tried to validate the location of the mouse after movement, and both sometimes passed! But visually the cursor is always in the wrong position
4. Yes to work around the bug we need to call mouseMove/MouseMove one more time using the same x/y.