It looks like the FX has the same issue as JDK-8351907, so we may directly borrow a code from it.
Basically it uses Remote Desktop XDG portal to control mouse and keyboard

https://flatpak.github.io/xdg-desktop-portal/docs/doc-org.freedesktop.portal.RemoteDesktop.html


n our robot implementation, we emulate mouse/keyboard actions using the XTEST extension protocol.

This issue is related to the libEI support is now wired to the XTEST, and we have two parts of the issue:

1.

XTEST is now wired to libEI, it now asks for a user confirmation to allow interaction(see the attached confirmation.png), so we should now be able to click on native applications, window titles, etc.

However, our robot implementation is mostly uses XTEST, but there is an exception:
for mouse move we use XWarpPointer, which is not part of the XTEST, and it is not propagated outside of Xwayland server.

So we have a situation where mouse is moving only inside Xwayland server, and we see the mouse enter/exit events,
on mouse press we call XTestFakeButtonEvent and it is handled by libei,
so the click happens where is the actual mouse cursor is in the system. That is why the test fails.

To avoid this situation we should use XTestFakeMotionEvent for mouseMove in Xwayland case.

I did a quick test with the XTestFakeMotionEvent, and it works with the provided test, and it allows to click on the window title as well as expected.

I also attached a native reproducer to play with XWarpPointer vs XTestFakeMotionEvent behavior.



2.

The second part of this issue is the user confirmation itself:
It doesn't persist across reboots, it may timeout after a period of inactivity.

It still seems to conform the spec, but we will not be able to run automated tests in this case.