This test doesn't work with Graal. The stack trace looks like this:

Uncaught exception while adjusting compilation level: java.lang.StackOverflowError
java.lang.StackOverflowError
at java.util.concurrent.locks.AbstractOwnableSynchronizer.setExclusiveOwnerThread(java.base@11-internal/AbstractOwnableSynchronizer.java:74)
at java.util.concurrent.locks.ReentrantLock$Sync.nonfairTryAcquire(java.base@11-internal/ReentrantLock.java:131)
at java.util.concurrent.locks.ReentrantLock$NonfairSync.tryAcquire(java.base@11-internal/ReentrantLock.java:199)
at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquire(java.base@11-internal/AbstractQueuedSynchronizer.java:1239)
at java.util.concurrent.locks.ReentrantLock.lock(java.base@11-internal/ReentrantLock.java:267)
at ReservedStackTest$ReentrantLockTest.lockAndCall(ReservedStackTest.java:173)
at ReservedStackTest$ReentrantLockTest.lockAndCall(ReservedStackTest.java:174)
at ReservedStackTest$ReentrantLockTest.lockAndCall(ReservedStackTest.java:174)

I am not exactly sure what the problem is but given the comment and @run command in the test:

 * @run main/othervm -XX:MaxInlineLevel=2 -XX:CompileCommand=exclude,java/util/concurrent/locks/AbstractOwnableSynchronizer.setExclusiveOwnerThread ReservedStackTest

/* The exclusion of java.util.concurrent.locks.AbstractOwnableSynchronizer.setExclusiveOwnerThread()
 * from the compilable methods is required to ensure that the test will be able
 * to trigger a StackOverflowError on the right method.
 */

it might have to do with that method.