ADDITIONAL SYSTEM INFORMATION :
macOS 12 (although the issue should be OS agnostic, tested on Windows with the same issue), with Azul JDK 11.0.14, but using a Maven dependency on:

            <dependency>
                <groupId>org.openjfx</groupId>
                <artifactId>javafx-controls</artifactId>
                <version>18</version>
                <classifier>mac</classifier>
            </dependency>

(other JavaFX 18 dependencies included via Maven as well, not shown here)

A DESCRIPTION OF THE PROBLEM :
The root problem stems from the implementation of JDK-8256283.

When you have TreeTableView with all columns defined but no data yet, if you attempt to add a sort ordering through adding to the ObservableList<TreeTableColumn<S,?>> returned from getSortOrder(), the sort will fail and the sort order that you attempted to add will be removed. That last part is the important regression. Prior to the above ticket, you could add a sort order to an empty table and then it would respect that sort order as new data comes into the table. But now, since the sort order is removed upon the failure to sort, you will have an unsorted table as data starts flowing through.

This is partly because we attempt to do a sort when the sort order changes:

TreeTableView#401:

        getSortOrder().addListener((ListChangeListener.Change<? extends TreeTableColumn<S, ?>> c) -> {
            doSort(TableUtil.SortEventType.SORT_ORDER_CHANGE, c);
        });

The stack trace for where we determine an empty table can't be sorted:
call:584, TreeTableView$3 (javafx.scene.control)
call:580, TreeTableView$3 (javafx.scene.control)
sort:1862, TreeTableView (javafx.scene.control)
doSort:1951, TreeTableView (javafx.scene.control)
lambda$new$0:401, TreeTableView (javafx.scene.control)
onChanged:-1, 357819648 (javafx.scene.control.TreeTableView$$Lambda$1786)
fireValueChangedEvent:164, ListListenerHelper$SingleChange (com.sun.javafx.collections)
fireValueChangedEvent:73, ListListenerHelper (com.sun.javafx.collections)
fireChange:239, ObservableListBase (javafx.collections)
commit:482, ListChangeBuilder (javafx.collections)
endChange:541, ListChangeBuilder (javafx.collections)
endChange:211, ObservableListBase (javafx.collections)
addAll:109, ModifiableObservableListBase (javafx.collections)
(from an invocation of TreeTableView.getSortOrder().addAll(..))


The stack trace for where we remove the sort order if we determine we can't sort:
handleSortFailure:140, TableUtil (javafx.scene.control)
sort:1884, TreeTableView (javafx.scene.control)
doSort:1951, TreeTableView (javafx.scene.control)
lambda$new$0:401, TreeTableView (javafx.scene.control)
onChanged:-1, 357819648 (javafx.scene.control.TreeTableView$$Lambda$1786)
fireValueChangedEvent:164, ListListenerHelper$SingleChange (com.sun.javafx.collections)
fireValueChangedEvent:73, ListListenerHelper (com.sun.javafx.collections)
fireChange:239, ObservableListBase (javafx.collections)
commit:482, ListChangeBuilder (javafx.collections)
endChange:541, ListChangeBuilder (javafx.collections)
endChange:211, ObservableListBase (javafx.collections)
addAll:109, ModifiableObservableListBase (javafx.collections)

For our particular usecase, we allow users to save the sort order for their tables in our GUI configuration files. When they load that GUI again, we want to restore their sort order upon table creation (before the data flows in, which happens asynchronously elsewhere). This was possible when we were using JavaFX 11, but we ran into the above regression when upgrading to 17.0.2/18.

REGRESSION : Last worked in version Openjfx11.0.10

STEPS TO FOLLOW TO REPRODUCE THE PROBLEM :
* Create a new TreeTableView
* Define the columns of the table via the .getColumns().addAll(..) function
* Define a sort order via the .getSortOrder().addAll(..) function

EXPECTED VERSUS ACTUAL BEHAVIOR :
EXPECTED -
* The empty table will have a sort order defined by what was added to the .getSortOrder() when data comes in
* table.getSortOrder().size() > 0
ACTUAL -
* No sort order is specified in the table after being added
* table.getSortOrder() == 0

CUSTOMER SUBMITTED WORKAROUND :
You could, perhaps, wait to initialize the sort order until after the table is populated with data, but this shouldn't be a requirement (and wasn't a requirement before the above ticket).

FREQUENCY : always