Uploaded image for project: 'JDK'
  1. JDK
  2. JDK-8302618

[macos] Problem typing uppercase letters with java.awt.Robot when moving mouse

XMLWordPrintable

    • generic
    • os_x

      Reporting on behalf of a a11y user. Originally reported at https://github.com/adoptium/adoptium-support/issues/710

      ### Please provide a brief summary of the bug

      **Summary**
      java.awt.Robot is erroneously typing lowercase letters instead of capital letters on MacOS Ventura under certain conditions described herein. I wrote an example java program to isolate the problem. This issue occurs when this example program is run on macOS with apple silicon using the temurin-17 JDK, but not when run using temurin-8. The problem only occurs when the user is simultaneously moving their mouse pointer while the Robot class is typing text. If the mouse pointer is in motion, the example program will type 100% lowercase letters (for me). If the mouse pointer is holding perfectly still, the program will produce 100% uppercase letters. I know this description sounds somewhat strange. This took a while to troubleshoot and isolate.

      **Impact for us**
      I'm a developer of an on-screen keyboard accessibility application. For us, this behavior breaks the usability of the on screen keyboard program and the robot class under Temurin 17. There is no known workaround for our use case except reverting to Java 8.


      ### Please provide steps to reproduce where possible

      **Environment:**
      Here is the environment I was using. I have not tried it on other platforms. A MacOS laptop computer, updated to MacOS Ventura, with an M1 chip (apple silicon). Running NetBeans 12.2, with two JDKs installed, Temurin 17 and Temurin 8.

      **Steps to reproduce:**
      - Open TextEdit. Run the example program (in Netbeans or another IDE), using Temurin JDK 17. After you start the program, you have seven seconds to put the text cursor into TextEdit. Then the program will type characters for about 15 seconds, and then it will exit. You can repeat running the program as needed while testing. Move the mouse while the typing occurs. (Don't touch the keyboard or the mouse buttons, just move the mouse around. I was using the built in trackpad.). **While the mouse is moving, the letters will be erroneously lowercase**. While the mouse is holding still, uppercase.
      - Repeat these steps while running the program in Temurin JDK 8, and the problem will no longer exist.

      **Things that did not help. (For me.)**
      None of these items helped to solve or improve the issue.
      - Setting robot.setAutoWaitForIdle() to true or false.
      - Adjusting or removing robot.setAutoDelay();
      - Using thread.sleep() or robot.delay() to insert arbitrarily high delays between the keystrokes of the key combination.
      - Placing the entire key combination into a synchronized (atomic) function.


      **=== Example Program ===**

      ```
      package somepackage;

      import java.awt.Robot;
      import java.awt.AWTException;
      import java.awt.Toolkit;
      import java.awt.datatransfer.Clipboard;
      import java.awt.datatransfer.StringSelection;
      import java.awt.event.KeyEvent;

      public class Start {

          private static Robot robot;

          public static void main(String[] arguments)
                  throws AWTException, InterruptedException {

              // Create a robot instance, with an autodelay between keystrokes.
              robot = new Robot();
              robot.setAutoWaitForIdle(true);
              robot.setAutoDelay(100);

              // Wait 7 second for the user to open a text editor, to catch the text.
              Thread.sleep(7000);

              // Paste the java version. (optional)
              String javaVersion = System.getProperty("java.vendor") + " "
                      + System.getProperty("java.version") + " ";
              StringSelection stringSelection = new StringSelection(javaVersion);
              Clipboard clipboard = Toolkit.getDefaultToolkit().getSystemClipboard();
              clipboard.setContents(stringSelection, stringSelection);
              robot.keyPress(KeyEvent.VK_META);
              robot.keyPress(KeyEvent.VK_V);
              robot.keyRelease(KeyEvent.VK_V);
              robot.keyRelease(KeyEvent.VK_META);

              // Run the typing loop.
              for (int i = 0; i < 10; ++i) {

                  // Press shift-a.
                  robot.keyPress(KeyEvent.VK_SHIFT);
                  robot.keyPress(KeyEvent.VK_A);
                  robot.keyRelease(KeyEvent.VK_A);
                  robot.keyRelease(KeyEvent.VK_SHIFT);

                  // Wait 1 second between loops.
                  Thread.sleep(1000);
              }
              
              // Press enter.
              robot.keyPress(KeyEvent.VK_ENTER);
              robot.keyRelease(KeyEvent.VK_ENTER);
              
              // Exit the program.
          }
      }

      ```


      I ran the test with several different JDK vendors and versions. The vendors were Amazon Corretto, Eclipse Temurin, and Azul Systems Zulu. The JDK versions were Java 8, 11, 17, and 19. The problem does not exist in all vendors in Java 8. The problem does exist for all vendors for all Java versions 11 through 19. Below are the test results.

      Java 1.8
      Amazon.com Inc. 1.8.0_362 AAAAAAA (GOOD)
      Temurin 1.8.0_362 AAAAAAA (GOOD)
      Azul Systems, Inc. 1.8.0_362 AAAAAAA (GOOD)

      Java 11
      Amazon.com Inc. 11.0.18 AaaaaAA (BUGGED)
      Eclipse Adoptium 11.0.18 AaaaAAA (BUGGED)
      Azul Systems, Inc. 11.0.18 AaaAAAA (BUGGED)

      Java 17
      Amazon.com Inc. 17.0.6 AaaaaAA (BUGGED)
      Eclipse Adoptium 17.0.6 AaaAaAA (BUGGED)
      Azul Systems, Inc. 17.0.6 AaAaAaA (BUGGED)

      Java 19
      Amazon.com Inc. 19.0.2 AaaaAAa (BUGGED)
      Eclipse Adoptium 19.0.2 AaaAaAA (BUGGED)
      Azul Systems, Inc. 19.0.2 aAAaAaA (BUGGED)

            honkar Harshitha Onkar
            karianna Martijn Verburg
            Votes:
            0 Vote for this issue
            Watchers:
            5 Start watching this issue

              Created:
              Updated: