See attached movie and code sample. This issue is right only for single selection model.

Movie: I try to press shift+pageDown and shift+pageUp alternately. You can see that one time I select the last element (that is shift+pageDown) and return to previously selected element (shift+pageUp). Sometimes I select other elements (using up or down or mouse), but the story is the same.

Code:
import javafx.application.Application;
import javafx.event.Event;
import javafx.event.EventHandler;
import javafx.scene.Scene;
import javafx.scene.control.*;
import javafx.scene.layout.HBox;
import javafx.scene.layout.Pane;
import javafx.scene.layout.VBox;
import javafx.stage.Stage;

/**
 * @author Alexander Kirov
 */
public class ListViewApp extends Application {

    public static void main(String[] args) {
        launch(args);
    }
    final ListView listView = new ListView();

    @Override
    public void start(Stage stage) throws Exception {
        Pane pane = new Pane();
        pane.setPrefHeight(200);
        pane.setPrefWidth(200);

        pane.getChildren().add(listView);
        listView.setPrefHeight(200);
        
        listView.getSelectionModel().setSelectionMode(SelectionMode.SINGLE);

        VBox vb = new VBox();
        vb.getChildren().addAll(pane, getAddItemHBox());
        Scene scene = new Scene(vb, 400, 400);
        stage.setScene(scene);
        stage.show();
    }

    private HBox getAddItemHBox() {
        HBox hb = new HBox();
        Label lb = new Label("Add item");
        final TextField tf = TextFieldBuilder.create().prefWidth(50).build();
        Label atLb = new Label("at pos");
        final TextField tfPos = TextFieldBuilder.create().prefWidth(50).build();
        Button bt = ButtonBuilder.create().text("Add!").build();
        bt.setOnAction(new EventHandler() {

            public void handle(Event t) {
                int index = Integer.parseInt(tfPos.getText());
                ((ListView) listView).getItems().add(index, tf.getText());
            }
        });
        hb.getChildren().addAll(lb, tf, atLb, tfPos, bt);
        return hb;
    }
}

To reproduce: add some amount of elements to list view and select some element in the center of the list. Try to press shift+pageDown and shift+pageUp.