FULL PRODUCT VERSION :
java version "9.0.4"
Java(TM) SE Runtime Environment (build 9.0.4+11)
Java HotSpot(TM) 64-Bit Server VM (build 9.0.4+11, mixed mode)



ADDITIONAL OS VERSION INFORMATION :
Microsoft Windows [version 10.0.16299.248]

A DESCRIPTION OF THE PROBLEM :
JavaFX does'nt scale up when too many levels of Javafx nodes.

Memory and layout time increase quadratically with number of nested JavaFX node.

The attached application shows the memory used by nested Javafx nodes and layout time increase quadratically, especially when pseudo-classes are used in style sheet.





STEPS TO FOLLOW TO REPRODUCE THE PROBLEM :
Run the atttached exemple with the Java profiler, we observe the following resulst:

10 nested HBox : Used Memory = 14Mo, 2 000 pseudoClassState object
Type "40" in text field, and click on button "click":
 Used Memory = 14Mo, 12 000 pseudoClassState object

move mouse over hbox to trigger "hover" events":
Used Memory = 14Mo, 86 000 pseudoClassState object

Type "80" in text field, and click on button "click":
Used Memory = 24Mo, 64 000 pseudoClassState object

move mouse over hboxes to trigger "hover" events" :
 Used Memory = 47Mo, 700 000 pseudoClassState object


Type "160" in text field, and click on button "click"
move mouse over hboxes to trigger "hover" events"
Used Memory = 170Mo, 3 700 000 pseudoClassState objects!

Type 500 : application is not responding anymore!!!


EXPECTED VERSUS ACTUAL BEHAVIOR :
EXPECTED -
Layout time and memory should be linear!

REPRODUCIBILITY :
This bug can be reproduced always.

---------- BEGIN SOURCE ----------
Java class :
package perfo;

import javafx.application.Application;
import javafx.scene.Scene;
import javafx.scene.control.Button;
import javafx.scene.control.TextField;
import javafx.scene.control.ToolBar;
import javafx.scene.layout.BorderPane;
import javafx.scene.layout.HBox;
import javafx.scene.text.Text;
import javafx.stage.Stage;

/*
 * To change this license header, choose License Headers in Project Properties.
 * To change this template file, choose Tools | Templates
 * and open the template in the editor.
 */
/**
 *
 * @author daniel
 */
public class Perfo extends Application {

    private BorderPane pane;

    @Override
    public void start(Stage primaryStage) throws Exception {
        String dialogStyleSheet = getClass().getResource("perfo.css").toExternalForm();
        pane = new BorderPane();
        final Button button = new Button("click");
        TextField textField = new TextField("10");
        pane.setTop(new ToolBar(button, textField));
        button.setOnAction(e -> {
            HBox hbox = new HBox();
            hbox.getStyleClass().add("myhbox");
            int nb = Integer.parseInt(textField.getText());
            for (int i = 0; i < nb; i++) {

                hbox = new HBox(new Text("y"), hbox);
                hbox.getStyleClass().add("myhbox");
            }
            pane.setCenter(hbox);

        });

        Scene scene = new Scene(pane);
        scene.getStylesheets().add(dialogStyleSheet);
        primaryStage.setScene(scene);
        primaryStage.sizeToScene();
        primaryStage.show();
    }

    public static void main(String[] args) {
        launch(args);
    }
}

style sheet perfo.css:
.myhbox:hover {
    -fx-border-color: red;
    -fx-border-width: 1;
    -fx-padding : 1 ;
}
---------- END SOURCE ----------