Uploaded image for project: 'JDK'
  1. JDK
  2. JDK-8330603

Relax the java.awt.Robot specification

XMLWordPrintable

    • behavioral
    • minimal
    • Updated documentation only.
    • Java API
    • SE

      Summary

      Update the specification of java.awt.Robot to describe how it can operate on some modern desktop environments (The backport is identical to the original fix).

      Problem

      Windowing systems have historically given applications with permission to run on the desktop with automatic permission for full access to the screen. Today desktops may require that permission, and others, to be explicitly given by a user. For example macOS requires the user to grant permissions for screen capture and event synthesis. The Wayland display server for Linux imposes similar restrictions, with variations depending on whether an application is running as an X11 application using Wayland's X11 compatibility mode, or as a native Wayland application.

      Currently the Robot API specification makes no provision for describing these limitations. Additionally, the Robot specification asserts that a screen capture will not include the mouse pointer. It is unclear how the Robot can assert this, since the windowing system is in control of that.

      Solution

      Update the documentation accordingly.

      Specification

      + * <p>
      + * Platforms and desktop environments may impose restrictions or limitations
      + * on the access required to implement all functionality in the Robot class.
      + * For example:
      + * <ul>
      + * <li> preventing access to the contents of any part of a desktop
      + * or Window on the desktop that is not owned by the running application.</li>
      + * <li> treating window decorations as non-owned content.</li>
      + * <li> ignoring or limiting specific requests to manipulate windows.</li>
      + * <li> ignoring or limiting specific requests for Robot generated (synthesized)
      + * events related to keyboard and mouse etc.</li>
      + * <li> requiring specific or global permissions to any access to window
      + * contents, even application owned content,or to perform even limited
      + * synthesizing of events.</li>
      + * </ul>
      + *
      + * The Robot API specification requires that approvals for these be granted
      + * for full operation.
      + * If they are not granted, the API will be degraded as discussed here.
      + * Relevant specific API methods may document more specific limitations
      + * and requirements.
      + * Depending on the policies of the desktop environment,
      + * the approvals mentioned above may:
      + * <ul>
      + * <li>be required every time</li>
      + * <li>or persistent for the lifetime of an application,</li>
      + * <li>or persistent across multiple user desktop sessions</li>
      + * <li>be fine-grained permissions</li>
      + * <li>be associated with a specific binary application,
      + * or a class of binary applications.</li>
      + * </ul>
      + *
      + * When such approvals need to given interactively, it may impede the normal
      + * operation of the application until approved, and if approval is denied
      + * or not possible, or cannot be made persistent then it will degrade
      + * the functionality of this class and in turn any part of the operation
      + * of the application which is dependent on it.
        *
        * @author      Robi Khan
        * @since       1.3
        */
      public class Robot {
      
      
           /**
            * Moves mouse pointer to given screen coordinates.
      +     * <p>
      +     * The mouse pointer may not visually move on some platforms,
      +     * while the subsequent mousePress and mouseRelease can be
      +     * delivered to the correct location
      +     *
            * @param x         X position
            * @param y         Y position
            */
      public synchronized void mouseMove(int x, int y) {
      
           /**
            * Returns the color of a pixel at the given screen coordinates.
      +     * <p>
      +     * If the desktop environment requires that permissions be granted
      +     * to capture screen content, and the required permissions are not granted,
      +     * then a {@code SecurityException} may be thrown,
      +     * or the content of the returned {@code Color} is undefined.
      +     * </p>
      +     * @apiNote It is recommended to avoid calling this method on
      +     * the AWT Event Dispatch Thread since screen capture may be a lengthy
      +     * operation, particularly if acquiring permissions is needed and involves
      +     * user interaction.
      +     *
            * @param   x       X position of pixel
            * @param   y       Y position of pixel
      +     * @throws  SecurityException if {@code readDisplayPixels} permission
      +     *          is not granted, or access to the screen is denied
      +     *          by the desktop environment
            * @return  Color of the pixel
            */
           public synchronized Color getPixelColor(int x, int y) {
      
           /**
      -     * Creates an image containing pixels read from the screen.  This image does
      -     * not include the mouse cursor.
      +     * Creates an image containing pixels read from the screen.
      +     * <p>
      +     * If the desktop environment requires that permissions be granted
      +     * to capture screen content, and the required permissions are not granted,
      +     * then a {@code SecurityException} may be thrown,
      +     * or the contents of the returned {@code BufferedImage} are undefined.
      +     * </p>
      +     * @apiNote It is recommended to avoid calling this method on
      +     * the AWT Event Dispatch Thread since screen capture may be a lengthy
      +     * operation, particularly if acquiring permissions is needed and involves
      +     * user interaction.
      +     *
            * @param   screenRect      Rect to capture in screen coordinates
            * @return  The captured image
      -     * @throws  IllegalArgumentException if {@code screenRect} width and height are not greater than zero
      -     * @throws  SecurityException if {@code readDisplayPixels} permission is not granted
      +     * @throws  IllegalArgumentException if {@code screenRect} width and height
      +     *          are not greater than zero
      +     * @throws  SecurityException if {@code readDisplayPixels} permission
      +     *          is not granted, or access to the screen is denied
      +     *          by the desktop environment
            * @see     SecurityManager#checkPermission
            * @see     AWTPermission
            */
           public synchronized BufferedImage createScreenCapture(Rectangle screenRect) {
      
           /**
            * Creates an image containing pixels read from the screen.
      -     * This image does not include the mouse cursor.
            * This method can be used in case there is a scaling tran
      ...
      -     * @throws  SecurityException if {@code readDisplayPixels} permission is not granted
      +     * @throws  SecurityException if {@code readDisplayPixels} permission
      +     *          is not granted, or access to the screen is denied
      +     *          by the desktop environment
      ...
          public synchronized MultiResolutionImage
                  createMultiResolutionScreenCapture(Rectangle screenRect) {

            azvegint Alexander Zvegintsev
            azvegint Alexander Zvegintsev
            Philip Race
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

              Created:
              Updated:
              Resolved: