I was getting ready to finish my statics fields in Class changes when I hit a failure with jbb and CMS. I've tracked it down to a race in the machinery for updating oop relocations and the logic for making sure that a scavengable nmethod is only scanned once. During a scavenge an nmethod can be reached for scanning in two different ways, either as a live activation on some thread stack or during the scan of scavengeable nmethods. The scan of scavengeable nmethods does two things though. It does the oops_do for the nmethod and then it calls fix_oop_relocations to update the generated code to match the new oop values. The problem is that the scan of the thread stacks and the scan of the scavengable nmethods are performed concurrently so the stack scanning thread might claim the nmethod first but actually scan the nmethod after the call to fix_oop_relocations in the other thread, leaving the oops valid but the code stale.

I think the logical place to move the fix_oop_relocations call is into nmethod::oops_do_marking_epilogue. Does this seem reasonable to anyone who understands the new nmethod scavenge code better than I do? It seems to work fine.