I discovered this problem while analyzing performance problems with Ensemble. We perform 4 steps related to occlusion culling & dirty region analysis:

1) Synchronize the scene graph state. At this time we set dirty flags
2) Walk the tree and accumulate the dirty bounds of everything that is dirty to form our dirty regions
3) For each dirty region, determine the dirty root
4) Start rendering from each dirty root

The problem is in Steps #2 and #3. Suppose I have a group comprised of a rectangle and 100K circles, where all 100K circles are completely contained within the bounds of the rectangle. The circles never change their color or position (they are always clean) but the rectangle is animating its fill color. Of course, because the rectangle is changing its fill, we have to redraw all 100K circles as well.

Now suppose that I have another Rectangle that is fully opaque and fully covers this group. Although the bottom-most rectangle is changing its fill, it is *never* going to be visible (or the 100K circles) because the rectangle on top is covering it all up.

In our current implementation, what happens is that in step #2 we determine the dirty region to be the size of the bottom-most rectangle, and then in step #3 we find the top-most rectangle and use it as the dirty root, and so we only paint the top-most rectangle. Performance is good. However, we never needed to paint anything in this scene. This can be hugely problematic.

Suppose that instead of a single Rectangle covering everything, I have that Rectangle covering everything and another 100K circles on top of the rectangle. None of them are changing in any way. Performance tanks, because the completely occluded rectangle at the bottom of everything is changing, causing the rectangle on top and all of the upper 100K circles to render. Bad news!

There are a couple ways to look at fixing this. One solution perhaps is that when we get to our dirty root, if we determine that neither the root nor any of the following nodes are dirty, then we can just bail and not draw anything. In this case, that would mean we visit 100K nodes (+the rectangle) and never draw anything. Another solution might be to integrate the dirty region code and occlusion culling code such that we can discover as we're building the dirty regions that a dirty region is completely occluded by something that is clean, and therefore we can throw away the entire dirty region. This is probably the more efficient solution.

Here is a test case:


public class Bug extends Application {

    @Override
    public void start(Stage stage) throws Exception {
        final double sceneWidth = 800;
        final double sceneHeight = 600;
        final double hiddenHeight = sceneHeight - 100;
        final double hiddenWidth = sceneWidth - 100;
        final double radius = 10;

        Group hidden = new Group();
        Rectangle background = new Rectangle(-radius, -radius, hiddenWidth+2*radius, hiddenHeight+2*radius);
        hidden.getChildren().add(background);
        FillTransition tx = new FillTransition(Duration.seconds(5), background, Color.BLACK, Color.RED);
        tx.setAutoReverse(true);
        tx.setCycleCount(FillTransition.INDEFINITE);
        tx.play();
        for (int i=0; i<100000; i++) {
            double x = Math.random() * hiddenWidth;
            double y = Math.random() * hiddenHeight;
            Circle circle = new Circle(x, y, radius, Color.color(Math.random(), Math.random(), Math.random()));
            hidden.getChildren().add(circle);
        }
        hidden.setTranslateX(50);
        hidden.setTranslateY(50);

        Rectangle overlay = new Rectangle(sceneWidth, sceneHeight, Color.color(1, 1, 1, 1));

        Group visible = new Group();
        visible.getChildren().add(overlay);
        for (int i=0; i<100000; i++) {
            double x = Math.random() * hiddenWidth;
            double y = Math.random() * hiddenHeight;
            Circle circle = new Circle(x, y, radius, Color.color(Math.random(), Math.random(), Math.random()));
            visible.getChildren().add(circle);
        }

        Scene scene = new Scene(new Group(hidden, visible), sceneWidth, sceneHeight);
        stage.setScene(scene);
        stage.show();

        final PerformanceTracker tracker = PerformanceTracker.getSceneTracker(scene);
        AnimationTimer trackerTimer = new AnimationTimer() {
            long ticks = 0;
            @Override
            public void handle(long now) {
                ticks++;
                if (ticks % 60 == 0) {
                    System.out.println(tracker.getInstantFPS());
                }
            }
        };
        trackerTimer.start();
    }

    public static void main(String[] args) {
        launch(args);
    }
}