-
Enhancement
-
Resolution: Unresolved
-
P4
-
21
java.awt.Robot provides APIs to capture a single pixel or a rectangular area of the desktop contents.
Typically an application - or test program - is using this to capture some portion of one of its own windows - not some random portion of the desktop.
However some desktop environments (eg Wayland on Linux) not only deny access to the desktop contents, but also do not report window position on the desktop - applications can only tell the size of their window and the position is always reported as (0,0)
But applications should have access to the buffer that holds the contents of their window - possibly including window decoration.
So we can create a new API that specifies the window to be captured.
This should avoid the problems with window location, and also should mean no permission is needed. Although we may need to document a permission is required as some implementations may not have a separate buffer to capture and need to capture the screen.
We also need to note that when capturing a buffer, then there is no possibility of the contents being occluded, but when the capture implementation goes to the desktop, then occlusion is possible and what is captured may include the occluding window, just as is the case with Robot.createScreenCapture() today.
Here is a rough draft of the API
/*
* Capture contents of the specified window.
* In a compositing desktop environment this API may return the contents of the buffer for the window.
* Otherwise it will fall back to capturing the screen contents at the location of the window, which may
* include other desktop content, similar to #createScreenCapture.
* For capturing window decoration, it is desktop dependent whether this is part of the application
* owned window buffer.
* The desktop environment may require that permissions be granted for any of these.
* w - window to capture
* r - rectangle to capture. If null, capture entire window
* includeDecoration - if true this means r top-left is relative to decorations, else its relative to contents.
*
* @throws SecurityException ...
* .. if .. the usual reasons .. including if decoration is requested but it can't be obtained.
*/
BufferedImage createWindowCapture(Window w, Rectangle r, boolean includeDecoration)
----
We'll also need the single pixel version which will I think need only support grabbing from the interior of the window - no decoration -
Color getPixelColor(Window w, int x, int y)
Typically an application - or test program - is using this to capture some portion of one of its own windows - not some random portion of the desktop.
However some desktop environments (eg Wayland on Linux) not only deny access to the desktop contents, but also do not report window position on the desktop - applications can only tell the size of their window and the position is always reported as (0,0)
But applications should have access to the buffer that holds the contents of their window - possibly including window decoration.
So we can create a new API that specifies the window to be captured.
This should avoid the problems with window location, and also should mean no permission is needed. Although we may need to document a permission is required as some implementations may not have a separate buffer to capture and need to capture the screen.
We also need to note that when capturing a buffer, then there is no possibility of the contents being occluded, but when the capture implementation goes to the desktop, then occlusion is possible and what is captured may include the occluding window, just as is the case with Robot.createScreenCapture() today.
Here is a rough draft of the API
/*
* Capture contents of the specified window.
* In a compositing desktop environment this API may return the contents of the buffer for the window.
* Otherwise it will fall back to capturing the screen contents at the location of the window, which may
* include other desktop content, similar to #createScreenCapture.
* For capturing window decoration, it is desktop dependent whether this is part of the application
* owned window buffer.
* The desktop environment may require that permissions be granted for any of these.
* w - window to capture
* r - rectangle to capture. If null, capture entire window
* includeDecoration - if true this means r top-left is relative to decorations, else its relative to contents.
*
* @throws SecurityException ...
* .. if .. the usual reasons .. including if decoration is requested but it can't be obtained.
*/
BufferedImage createWindowCapture(Window w, Rectangle r, boolean includeDecoration)
----
We'll also need the single pixel version which will I think need only support grabbing from the interior of the window - no decoration -
Color getPixelColor(Window w, int x, int y)