[macosx] Tab group has null children

UI Browser http://pfiddlesoft.com/uibrowser/ would be a very helpful tool for debugging Mac accessibility issues. However, due to a bug in Swing it does not work. The following is the text of a note from Bill Cheeseman, the author of UI Browser.

I have figured out why UI Browser is not working as you expected with SwingSet2. It is because of a bug in SwingSet2's or Java's implementation of the accessibility API. You therefore cannot use SwingSet2 as a testbed for UI Browser, GUI Scripting or VoiceOver. I will explain the SwingSet2 bug in this message. Then in a subsequent message I will address your issues with System Preferences. (I have the latest Jave JRE version 8 installed, but I'm not enough into Java to know whether that has anything to do with this issue.)

Apple's Activity Monitor shows that the SwingSet2 process has a parent process of Jar Launcher. That is different from the parent process of traditional Cocoa and Carbon Mac applications.

When SwingSet2 is running, it does not appear in UI Browser's Target pop-up menu as an application. If you use UI Browser's Preferences window's General pane to "Include background applications in Target menu," Jar Launcher appears in the UI Browser Target list as a running background application. That doesn't help you, however, because you don't want to control Jar Launcher.

What you do want to control, probably in any Java application that is launched by Jar Launcher, is the "java" application. It does appear in UI Browser's Target menu even when you don't have "Include background applications in Target menu" turned on. Select "java" in the Target menu and open the Attributes drawer.

What I see when I do that is a root application UI element named 'application "java"', which is initially selected, and two child UI elements named 'standard window "SwingSet2" (window 1)' and 'menu bar (menu bar 1)'. If I select the window element in UI Browser, I see all of its child UI elements. With one exception, if I select a child of the window I see that child's grandchild UI elements, and selecting any of the child elements allows me to see all of its attributes, actions and notifications in the appropriate UI Browser drawer.

The one exception is when I select the 'tab group "Internal Frames Demo" (tab group 1)'. Instead of showing me its child elements (which should be all of the UI elements in the selected SwingSet2 tab view item), I see a list of items saying '[MISMATCH-no parent[ (null) (UI element 1)' and '... (UI element 2)' and so on.

The "MISMATCH" legend is produced by a diagnostic tool included in UI Browser, which helps developers of applications to test their implementation of the accessibility API to make sure their applications will be usable by computer users with disabilities. Here, it demonstrates that the developers of SwingSet2, or the developers of the whole Jar Launcher ecosystem on Macs, made a mistake in their implementation of accessibility for the tab group. In UI Browser's main browser view, what you see is a rendition of SwingSet2's accessibility hierarchy that starts with the root application element and moves "up" the child tree from child element to child element toward the leaf element (such as a button). Along the way, it discovers that the children of the tab group UI Element in SwingSet2 are all null. It also discovers that each of those child elements does have a valid parent element. Since the parent element's children are null, this is called a "mismatch" between the "up" and the "down" hierarchies, which is a bug.

If you switch to UI Browser's Screen Reader window and hover the mouse over any of the UI elements in the open SwingSet2 window, you will see each of those elements as well as its element path and the attributes. The path in the Screen Reader is complete even though the path in the main browser view is not complete. In the case of the Screen Reader, you are looking "down" the accessibility hierarchy from the leaf element on the screen toward the root application element, and in this direction SwingSet2 implements the hierarchy correctly -- each element has a valid parent element. This is why Apple's Accessibility Inspector screen reader can see the elements, paths and attributes, too -- it is also a screen reader looking "down" the hierarchy from a leaf element on the screen. In UI Browser's Screen Reader, however, our special diagnostic tool sees the "MISMATCH" in the tab group and flags the fact that its parent has "different children."

Together, these two "MISMATCH" reports from UI Browser demonstrate that the SwingSet2 accessibility hierarchy is different depending on whether you are looking "up" or "down". To work correctly, any application's accessibility hierarchy must be identical no matter which way you are looking. In the case of SwingSet2, the mismatch yields "(null)" children when you look "up" from the root application element. Unfortunately, AppleScript's GUI Scripting always attempts to look "up" because AppleScript always starts with the root application element -- for example, 'tell application "SwingSet2" to get ....' -- so GUI Scripting doesn't work. VoiceOver looks "up", too, so it doesn't work with SwingSet2, either.

Until this bug in SwingSet2 or Java is fixed, SwingSet2 cannot be used with GUI Scripting or VO.