I'm seeing a case where ClassLoader.checkPackageAccess is invoked
multiple times for a single new (it is "called out" of the VM) and
this messes up single-stepping.

cd ~ivan/for/checkPackageAccess (or get the stuff from the bug attachments)
First check if you can issue:
java -classpath ".:unix.jar:term.jar" TermApp
(to make sure that LD_LIBRARY_PATH (for JNI) and other stuff all works).

Then go through the scenario described in "in" using jdb.
It demonstrates how for a particular 'new' ClassLoader.checkPackageAccess is
called many many times. It isn't stuck in an infinite loop;
eventually progress is made. But eventually jdb ends up
getting confused as described in "in".

Now one expects that when you single step over an opcode that if it's
an invoke one ends up at the invoked function and otherwise at the next opcode,
_not_ in the system internals.

Now this sort of situation happens in native applications as well, for
example when dispatching through a PLT, thunks, or an objective C method
dispatcher, so having a conceptually simple operation take a circuituous
route to get to the destination is not a new thing however ...

- in the native situation all the dispatching is done at the
  machine architecture level and there are no magical nested invokation
  from within the microcode into assembly inctructions.

- Often to get src-level single stepping to work right the runtime
  has to provide information to the debugger to help it skip over the
  dispatching code. For example, on Solaris, librtld_db.so has features
  that help dbx to skip over PLT's. But this kind of scaffolding isn't
  neccessary for assembly-level single-stepping, while, at least
  at first glance, it looks like some kind of feature will need to be added
  to JVMDI to help with this anomaly.

As it happens if a java opcode invokes only one callout things sort-of
work. Suppose the 'new' issued only one call to ClassLoader.checkPackageAccess.
In which case the debugger would detect a "call" into a "system" class
and set up a FRAME_POP event, which fires right before checkPackageAccess
is going to return. Then the debugger will turn on single stepping and take
one more step and be back after where the new was called. This
scenario is perhaps different because jdbx detects a call by seeing if
the PC offset is 0. I expect other debuggers might end up stepping for
a very long time or something like that. For this to work also requires that
a class can be classified as a system class but that is a very subjective
definition.

This scenario also works in the case of invocations, where I believe a callout
is made for loading a class or something. After doing the FRAME_POP thing,
the "one more step" gets up to the invoked destination.

But, I hope you agree, that this is all serendipity as opposed to design.
And the above luck won't hold if you have multiple callouts per opcode,
since the debugger will now have to decide how many callouts to ignore.

I"m also a bit worried that since there is a trend in the VM to move more semantics out of the native VM code and into java that the frequency
of such callouts will increase.

Right now this makes for very unreliable single-stepping.
I doubt that there is a succinct way to characterise and describe to
a user under what conditions a callout will take place (which explains why my reproducible testcase is so complex). And even if
there was a way it would be pointless since debugger users step reflexively
and having to stop and think about exceptions when stepping doesn't work
will surely drive them to S y s t e m . o u t . p r i n t l n ( ).

   
Having written all of this I went to check whether the JVMDI spec says anything
about what a "single step" means, and it means "The VM has reached a new
location". According to this, there is no bug. However the fact that the
spec provides no example or mention of the subtleties involved leads me to
believe that the defenition cannot be overgeneralized, never mind that
there is a practical issue to be dealt with here.

According to that definition, the VM will reach a new location either by:
- advancing the PC by one op
- setting the PC according to a branch
- setting the PC according to a "call"
- setting the pc according to a thrown exception
- setting the PC accoding to a "callout"
- other surprises I haven't run into?
The first three jive with traditional understanding of single-stepping.
The exception case is treated properly it seems (but then I haven't
gotten pas simple stuff).

In the 5th item, debuggers usually require an event or an inference rule
defining when a "call" has happenned. And they require an algorithm for
capturing the return from the said "call". (Note the additional
complication that frame_pop by itself doesn't define this "return" and
in turn (recursively) requires reliance on this IMO shaky definition of step)

In the case of callouts,
a) they are indistinguishable from "call"s.
b) there is no concrete way to capture a return from them.
c) If the callout happens as a side-effect of a "call" then there is
   no return even.