Uploaded image for project: 'JDK'
  1. JDK
  2. JDK-8181305

[macos] Robot.getPixelColor and Robot.createScreenCapture returns wrong color values

XMLWordPrintable

    • x86_64
    • os_x

      FULL PRODUCT VERSION :
      Tested on JavaSE 7, 8 & 9

      java version "1.8.0_131"
      Java(TM) SE Runtime Environment (build 1.8.0_131-b11)
      Java HotSpot(TM) 64-Bit Server VM (build 25.131-b11, mixed mode)

      java version "9-ea"
      Java(TM) SE Runtime Environment (build 9-ea+166)
      Java HotSpot(TM) 64-Bit Server VM (build 9-ea+166, mixed mode)

      java version "1.7.0_79"
      Java(TM) SE Runtime Environment (build 1.7.0_79-b15)
      Java HotSpot(TM) 64-Bit Server VM (build 24.79-b02, mixed mode)

      ADDITIONAL OS VERSION INFORMATION :
      MacOS Sierra
      version 10.12.5

      EXTRA RELEVANT SYSTEM CONFIGURATION :
      Intel HD Graphics 4000 1536 MB
      Samsung S27B970 Display
      MacOS display profile S27B970


      A DESCRIPTION OF THE PROBLEM :
      Robot reports the wrong pixel color for any MacOS Display Profile except "Generic RGB Profile". Many monitors will come with their own display profile and is selected by MacOS by default.

      For example Robot reports a color values of (0, 11, 255) for a blue pixel (0, 0, 255)

      Note: MacOS application "Digital Color Meter" displays the proper color value when using the default color profile "Display Native Values" or matches the color profile assigned to the monitor (e.g. a generic profile not a monitor specific profile). It reports the same value as Robot if selecting the "Display in Generic RGB" color profile (which is generally not the case for a typical MacOS user configuration).

      STEPS TO FOLLOW TO REPRODUCE THE PROBLEM :
      Select a "Display Profile" other than "Generic RGB Profile" under "System Preferences..." / "Displays" / "Color".

      Run the sample code.

      EXPECTED VERSUS ACTUAL BEHAVIOR :
      EXPECTED -
      The Pixel color returned by Robot should be the same pixel color rendered.

      Without this consistency reliable graphic testing using Robot will fail across different user configurations. Additionally it is impossible to reproduce images correctly via Robot screen capture since the screen capture does not contain the color information to render the image back to the screen.
      ACTUAL -
      Captured pixel colors do not always match the rendered pixel colors (depending on color / display profile).

      REPRODUCIBILITY :
      This bug can be reproduced always.

      ---------- BEGIN SOURCE ----------
      package bugreportrobot;

      import java.awt.*;
      import java.awt.image.*;
      import java.lang.reflect.InvocationTargetException;
      import javax.swing.*;

      /**
       * Demonstrates pixel color returned by Robot does not always correspond to
       * pixel color rendered on MacOS. Robot returns pixel color values after the
       * display profile color transformation has been applied.
       *
       *
       *
       * <P><hr> Running the Test
       *
       * <P> Select a display profile other than "Generic RGB Profile" in MacOS
       * "System Preferences..." / "Displays" / "Color" before running the test. Many
       * monitors will have their own profile already selected in which case changing
       * display profile is not required to show a discrepancy between captured and
       * rendered pixel color values.
       *
       * <P><b>Programmatic Check:</b>
       * The sample code will programmatically check pure blue / green colors for
       * discrepancies and report the number of colors that do not match. The number
       * of mismatches may range from 0 to 256². The result is written to the console.
       *
       * <P><b>Visual Check:</b>
       * The sample code will visually demonstrate discrepancies by showing two color
       * channel matrixes side by side. The left matrix is rendered as is. The right
       * matrix shows the screen capture applied recursively, magnifying any
       * differences between rendered and capture pixel color. This is done for 3
       * different channel matrixes (green/blue, red/green, and blue/red).
       *
       * <P> If the capture pixel color is the same and the rendered pixel color then
       * both color channel matrixes will look the same.
       *
       *
       *
       * <P><hr> Expected Results
       *
       * <P> The color reported by Robot should be the same as the color rendered
       * otherwise: <br>
       * 1) Testing UI with Robot is problematic over different HW / OS setup <br>
       * 2) Any type of screen capture or color grabber functionality is impossible to
       * use since rendering with the information would result in a double application
       * of the OS color profile transformation.
       *
       * <P> Note: MacOS "Digital Color Meter" can retrieve the rendered color values
       * when selecting "Display native values" or if the color profile selected is
       * the same as the color profile used for the display (which is generally not
       * the case in a user environment).
       *
       */
      public class BugReportRobot {
      static volatile BufferedImage captured;
      static volatile int colorChannelPair; // 0=(G,B), 1=(R,G), 2=(B,R)

      public static void main(String[] args) throws AWTException, InterruptedException, InvocationTargetException {
      Rectangle bounds = GraphicsEnvironment.getLocalGraphicsEnvironment().getMaximumWindowBounds();

      JFrame window = new JFrame("Robot Bug Report");
      window.setDefaultCloseOperation(JFrame.EXIT_ON_CLOSE);
      window.setUndecorated(true);
      window.setAlwaysOnTop(true); // avoid shadows from other windows in MacOS
      window.setBounds(bounds.x, bounds.y, 512, 256); // left half is original image, right half recursively paints the captured screen image

      window.add(new JComponent(){
      private BufferedImage original;
      private int lastChannelPair = -1; // see colorChannelPair

      protected void paintComponent(Graphics g1) {
      super.paintComponent(g1);

      int tarBlock = colorChannelPair;
      if(lastChannelPair != tarBlock){
      lastChannelPair = tarBlock;
      int shiftMajor = ((tarBlock+1) % 3)*8;
      int shiftMinor = (tarBlock %3) * 8;
      original = new BufferedImage(256,256, BufferedImage.TYPE_INT_RGB);
      Graphics2D g = original.createGraphics();

      for(int start = 0; start < 256; start++){
      for(int i = 0; i < 256; i++){
      g.setPaint(new Color(start << shiftMajor | i << shiftMinor));
      g.fillRect(i, start, 1, 1);
      }
      }
      g.dispose();
      captured = copy(original);
      }
      Graphics2D g = (Graphics2D) g1;
      g.drawImage(original, null, 0, 0);
      g.drawImage(captured, null, 256, 0);
      }
      });

      window.setVisible(true);
      Thread.sleep(1000);

      Robot r = new Robot();
      bounds = window.getBounds();

      /***********************************************************************
      * Programmatic test of rendered vs captured pixels for green blue matrix
      ***********************************************************************/
      BufferedImage capture = r.createScreenCapture(bounds);
      int delta = 0;
      for(int y = 0; y < 256; y++){
      int target = y << 8;
      for(int x = 0; x < 256; x++){
      if((capture.getRGB(x,y) & 0x00_FFFFFF) != target) delta++;
      target++;
      }
      }

      System.out.println("Number rendered vs captured pixel color that are different: " + delta);
      //if(delta != 0) System.out.println("Pixels captured ≠ Pixels rendered "+delta);


      /***********************************************************************
      * Visual test for rendered vs captured pixels. If captured pixels
      * return the same color values as the rendered pixels than the right
      * side will be identical to the left size. If captured pixels are
      * different then
      ***********************************************************************/
      Rectangle captureBounds = new Rectangle(bounds.x+256,bounds.y, 256, 256);
      Rectangle lastRowBounds = new Rectangle(bounds.x, bounds.y + 255, bounds.width, 1);

      int lastColor = -1;
      for(int block = 0; block < 7; block++){
      final int setBlock = block;
      colorChannelPair = setBlock;
      window.repaint();

      int rgb1, rgb2;
      do{
      BufferedImage lastRow = r.createScreenCapture(lastRowBounds);
      rgb1 = lastRow.getRGB(255, 0);
      rgb2 = lastRow.getRGB(511, 0);
      }while(rgb1 == lastColor || rgb2 != rgb1);
      lastColor = rgb1;

      for(int i = 0; i < 100; i++){
      captured = r.createScreenCapture(captureBounds);
      window.repaint();
      }
      }

      Thread.sleep(1000);
      System.exit(0);
      }

      public static BufferedImage copy(BufferedImage bi) {
      ColorModel cm = bi.getColorModel();
      boolean isAlphaPremultiplied = cm.isAlphaPremultiplied();
      WritableRaster raster = bi.copyData(bi.getRaster().createCompatibleWritableRaster());
      return new BufferedImage(cm, raster, isAlphaPremultiplied, null);
      }
      }

      ---------- END SOURCE ----------

            serb Sergey Bylokhov
            webbuggrp Webbug Group
            Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

              Created:
              Updated:
              Resolved: