C2: assert(get_loop(lca)->_nest < n_loop->_nest || lca->in(0)->is_NeverBranch()) failed: must not be moved into inner loop

Please credit me in your PR, I took quite a bit of time to extract a nice Test.java from a set of classfiles, and nicely reduced it.

[~thartmann] wrote this:
Reproduces since JDK 17, probably since the assert was introduced with JDK-8269088 (which fixed a regression from JDK-8252372 in JDK 17 which had a huge bug tail and apparently still has ...).

emanuel@emanuel-oracle:/oracle-work/triage/current$ /oracle-work/jdk-fork1/build/linux-x64-debug/jdk/bin/java -XX:CompileCommand=printcompilation,Test::* -XX:CompileCommand=compileonly,Test::* -Xcomp Test.java
CompileCommand: PrintCompilation Test.* bool PrintCompilation = true
CompileCommand: compileonly Test.* bool compileonly = true
4714 97 b 3 Test::main (9 bytes)
4726 99 b 4 Test::main (9 bytes)
4728 100 b 3 Test::test (28 bytes)
4729 101 b 4 Test::test (28 bytes)
#
# A fatal error has been detected by the Java Runtime Environment:
#
# Internal Error (/oracle-work/jdk-fork1/open/src/hotspot/share/opto/loopopts.cpp:1072), pid=370817, tid=370831
# assert(get_loop(lca)->_nest < n_loop->_nest || lca->in(0)->is_NeverBranch()) failed: must not be moved into inner loop
#
# JRE version: Java(TM) SE Runtime Environment (24.0) (fastdebug build 24-internal-2024-07-17-0708486.emanuel...)
# Java VM: Java HotSpot(TM) 64-Bit Server VM (fastdebug 24-internal-2024-07-17-0708486.emanuel..., compiled mode, tiered, compressed oops, compressed class ptrs, g1 gc, linux-amd64)
# Problematic frame:
# V [libjvm.so+0x12a6f6f] PhaseIdealLoop::try_move_store_after_loop(Node*) [clone .part.0]+0x7cf
#
# Core dump will be written. Default location: Core dumps may be processed with "/usr/share/apport/apport -p%p -s%s -c%c -d%d -P%P -u%u -g%g -- %E" (or dumping to /oracle-work/triage/current/core.370817)
#
# An error report file with more information is saved as:
# /oracle-work/triage/current/hs_err_pid370817.log
#
# Compiler replay data is saved as:
# /oracle-work/triage/current/replay_pid370817.log
#
# If you would like to submit a bug report, please visit:
# https://bugreport.java.com/bugreport/crash.jsp
#
Aborted (core dumped)



Looking into it with RR:

(rr) p lca->dump_bfs(100,lca,"#cA")
dist apd dump
---------------------------------------------
  21 30 255 IfTrue === 254 [[ 256 ]] #1
  21 23 337 IfFalse === 336 [[ 332 256 ]] #0 !orig=[328]
  20 26 251 IfTrue === 250 [[ 257 ]] #1 !orig=[158] !jvms: Test::test @ bci:9 (line 14)
  20 23 256 OuterStripMinedLoop === 256 337 255 [[ 256 257 ]]
  19 23 257 CountedLoop === 257 256 251 [[ 257 243 250 ]] inner stride: -1 main of N257 strip mined !orig=[230],[214] !jvms: Test::test @ bci:21 (line 15)
  18 23 250 CountedLoopEnd === 257 249 [[ 251 253 ]] [gt] P=0.900000, C=-1.000000 !orig=[156] !jvms: Test::test @ bci:9 (line 14)
  17 23 253 IfFalse === 250 [[ 258 259 ]] #0 !orig=[252],[233],[157] !jvms: Test::test @ bci:9 (line 14)
  16 23 258 SafePoint === 253 1 218 1 1 259 107 217 1 [[ 254 ]] SafePoint !orig=221,188 !jvms: Test::test @ bci:22 (line 17)
  15 23 254 OuterStripMinedLoopEnd === 258 109 [[ 255 314 ]] P=0.900000, C=-1.000000
  15 16 336 If === 323 335 [[ 337 338 ]] P=0.000001, C=-1.000000 !orig=[327]
  14 23 314 IfFalse === 254 [[ 313 ]] #0 !orig=[286]
  14 16 338 IfTrue === 336 [[ 313 ]] #1 !orig=[307],253,[252],[233],[157] !jvms: Test::test @ bci:9 (line 14)
  14 22 271 IfTrue === 270 [[ 267 ]] #1 !orig=251,[158] !jvms: Test::test @ bci:9 (line 14)
  14 19 341 IfFalse === 340 [[ 292 267 263 ]] #0 !orig=[291]
  13 16 313 Region === 313 338 314 [[ 313 340 315 320 ]] !orig=[286]
  13 19 267 CountedLoop === 267 341 271 [[ 270 267 273 ]] inner stride: -1 post of N257 !orig=257,[230],[214] !jvms: Test::test @ bci:21 (line 15)
  12 16 340 If === 313 339 [[ 341 342 ]] P=0.500000, C=-1.000000 !orig=[290]
  12 19 270 CountedLoopEnd === 267 260 [[ 271 274 ]] [gt] P=0.500000, C=-1.000000 !orig=250,[156] !jvms: Test::test @ bci:9 (line 14)
  11 16 342 IfTrue === 340 [[ 280 ]] #1 !orig=[281],[252],[233],[157] !jvms: Test::test @ bci:9 (line 14)
  11 19 274 IfFalse === 270 [[ 280 ]] #0 !orig=253,[252],[233],[157] !jvms: Test::test @ bci:9 (line 14)
  10 16 280 Region === 280 274 342 [[ 280 232 282 283 ]] !orig=[252],[233],[157] !jvms: Test::test @ bci:9 (line 14)
  10 23 231 IfTrue === 223 [[ 232 ]] #1 !orig=[157] !jvms: Test::test @ bci:9 (line 14)
   9 16 232 Region === 232 231 280 [[ 232 161 166 237 239 ]] !orig=[157] !jvms: Test::test @ bci:9 (line 14)
   8 16 161 SafePoint === 232 1 237 1 1 166 107 1 1 [[ 206 ]] SafePoint !jvms: Test::test @ bci:25 (line 18)
   7 16 206 Loop === 206 87 161 [[ 206 205 343 ]] !orig=[88] !jvms: Test::test @ bci:0 (line 10)
   6 16 343 NeverBranch === 206 [[ 344 345 ]]
   5 16 344 CProj === 343 [[ 223 ]] #0
   4 16 223 If === 344 198 [[ 224 231 ]] P=0.100000, C=-1.000000 !orig=[156] !jvms: Test::test @ bci:9 (line 14)
   3 19 304 IfTrue === 303 [[ 300 ]] #1 !orig=251,[158] !jvms: Test::test @ bci:9 (line 14)
   3 16 224 IfFalse === 223 [[ 300 ]] #0 !orig=[158] !jvms: Test::test @ bci:9 (line 14)
   2 16 300 CountedLoop === 300 224 304 [[ 303 300 296 305 306 348 ]] inner stride: -1 pre of N257 !orig=257,[230],[214] !jvms: Test::test @ bci:21 (line 15)
   1 16 303 CountedLoopEnd === 300 293 [[ 304 323 ]] [gt] P=0.500000, C=-1.000000 !orig=250,[156] !jvms: Test::test @ bci:9 (line 14)
   0 0 323 IfFalse === 303 [[ 336 ]] #0


I suspect this is another infinite-loop bug, as the NeverBranch in the IR and the assertion suggests. Maybe the assertion is not 100% accurate, or something is handled wrong in the VM code.