From ###@###.### Thu Jun 24 19:11:22 1999
Date: Wed, 3 Feb 1999 15:32:05 +0600
From: "Leonid M. Arbouzov" <###@###.###>
To: ###@###.###
Cc: ###@###.###, serguei.chukhontsev@Eng, ###@###.###,
    ###@###.###
Subject: Re: jcov support in javac?

Hello Iris,

We are glad to hear you are in javac team!
Thank you for your attention to issues of jcov support in javac.

| I have found the documentation in jcovguide.html to be most helpful in trying to
| ascertain the current behaviour of jcov, however any historical information
| about the development of jcov would also be very useful.

Konstantin maintains jcov only few months so I'll try to tell shortly
the whole history of jcov and describe what this jcov code in javac
is necessary for.

Well, NSK team began developing Java certification tests
for compiler and runtime Jan, 96. There was no Java specification
at that time yet except 40-pages (or so) document. Carla and Headly
(who left for Netscape few later) proposed to cover javac sources as one of
the ways to build a representative test suite. In a few monthes we developed
the first variant of Jcov that required support from javac and runtime.
(This variant was integrated later into jdk1.1.)
Important requirement from Tim L. and Frank Y. was not to change bytecode
produced by javac. So the only things jcov adds are
few additional attributes required to precisely map bytecode addresses into
original source location (file, line, position in line).
All data gathering is done in interpreter and the interpreter produces
simple ASCII jcov file containing gathered data.
(I should note that tcov design - Sun coverage tool for C and other
compilers - had afftected jcov design and even some of runtime code
was reused, this is why this runtime code still looks a little ugly)

Now few words about jcov support in compiler.
When javac is run under -prof=jcov (1.1.x) or -Xjcov (1.2.x)
option it adds the following attributes to class file:

"TimeStamp"
This attribute is used to differentiate versions of the same class
in different test runs. Consider an example. We compile class "A",
then use it and save gathered jcov data in file "a.jcov". Then we change
some method "A.f()" and recompile the class. The set of lines, blocks, branches
in the "A.f()" may be changed so now when we run modified version of "A" we can't
merge (sum) newly gathered jcov data with data saved in "a.jcov" previously.
What Jcov can do is to treat this new version of "A" as a different class in the
jcov file. "TimeStamp" is used exactly for this purpose - to differ one
version of class file from other.

"AbsoluteSourcePath"
In order for Jcov to be able to show sources of covered program
(in GUI or as annotated listing) it needs to have access to original
program sources. Unfortunately "SourceName" attribute contains only
short name of file not a full path name. "AbsoluteSourcePath" contains
full path name of source file from which a class was produced.
This attribute is then output into jcov data file by runtime.

"CoverageTable"
This table is created for each method of the class.
This table contains a triple for each bytecode that begin some new
coverage item (method, linear block, branch):
(item kind, byte address/number in method code, position in source file)
Runtime uses this table to count how many times each of covered items
has been executed. This data along with the counter then are passed into jcov
data file.

"CoverageTable" is central part of that Jcov implementation because it:
- lists what byte code addresses are interesting to count in interpreter
- provides backward mapping to source code

Any code optimizations and transformations (like inlining for example)
make it more difficult to map bytecode back to source so under -Xjcov
most of optimizations in javac are disabled. However some transformations
still remain like conversion of predicates in if() statement. It's important
for Jcov to correctly count how many times then-clause was executed and how many
times else-clause was executed. To keep this in mind special flag is added to tree nodes
and parameter to some of methods to save information about such inversions.

Of course we considered possibility to use "LineNumberTable" attribute
instead of "CoverageTable". However "LineNumberTable" provides mapping
to source for too small number of byte addresses in code and doesn't differentiate
code in the same source line which is critical when several
covered items (blocks, branches, conditions) are all placed in one source line.

Here is a list of javac files touched with Jcov changes.
We tried to minimize the changes and carefully mark them with //JCOV ... // end JCOV
comments:

jdk_1.2/src/share/classes/sun/tools grep -c JCOV `grep -l JCOV */*java`
asm/Assembler.java:7
asm/Assembler.n.java:7
asm/Instruction.java:6
asm/SwitchData.java:4
java/ClassFile.java:2
java/Constants.java:6
java/Environment.java:4
javac/BatchEnvironment.java:4
javac/Main.java:10
javac/SourceClass.java:14
tree/SwitchStatement.java:9

This first version of Jcov required to recompile covered
program with special option. It creates some difficulties
for test engineers because JDK build process was rather
complicated. Meanwhile most of them need estimate only draft coverage
of some API, namely public method coverage (if all public method
have been tested). There are no need in "CoverageTable"
for measuring method coverage. And to we have implemented
restricted mode -Xjcov:type=m for java_g that can use
any regular class files for method coverage estimation.

At the same time we worked on instrumented version of Jcov
that would allow to use it under any JVM and particularly
under HotSpot VM. Jcov instrumenter gets regular class file
and inserts bytecode calls of Jcov runtime classes methods
that increment counters and save them into jcov file when necessary.
One minor problem is where instrumenter can get mapping
from bytecode to source. Regular class file just doesn't
contain such fine-grained mapping. When possible the instrumenter
uses "LineNumberTable" but it doesn't give position in a source line.
The only way currently to provide this information to the instrumenter
is compiling classes with -Xjcov option of javac.
Of course if "-g" or some other debugging option of javac would add fine-grained
mapping from bytecode to source, the Jcov instrumenter could use it as well.
But for now only -Xjcov option provides such data.
This is why we would prefer to keep it in a new javac for a while
or invent some new way of saving this source mapping (it is obviously
necessary for debuggers and other runtime tools).

This is almost the whole story. I skip probably only history of evolution
of option names and jcov data file format.

Below is answers to your questions:

| At one time, jcov required support from javac in the form of code added directly
| to the javac codebase (this includes the addition of the -Xjcov option). Now
| that jcov has been written to do static instrumentation of class files or
| dynamic instrumentation via class loader, is this javac code still required?
| I'm guessing that the only time these changes are still used would be in the
| event -Xjcov is used in conjunction with java_g. Is that correct?

This is not fully correct.
This option is required for jcov support in java_g.
But it is used also with instrumenter version of Jcov
because fine-grained source mapping is taken by instrumenter
from "CoverageTable" attribute created by javac under -Xjcov.

| I'd like to avoid the situation where jcov suddenly gets broken when we switch
| over to the new compiler. Is there any support that you added to javac that I
| should be aware of?

All support added to javac by us is marked in javac sources
with //JCOV ... // end JCOV comments.
We would prefer to keep this support in new javac version too
or to substitute by some similar feature. Let us know and we
would be happy to give any necessary help to do that.

| Are there any plans for jcov development that rely on the current jcov-specific
| changes to javac or that may require the addition of new code to the existing
| javac codebase?

There are no new plans. But existing Jcov versions (both in java_g
and instrumenter version) rely on existing jcov support in javac.

Thank you once again for asking and let us know if we
can explain anything else or help somehow.
-leonid


| From konst@novo0 Wed Feb 3 10:39 NST 1999
| Date: Wed, 3 Feb 1999 10:38:21 +0600
| From: konst@novo0 (Konstantin S. Bobrovsky)
| To: leo@novo0
| Subject: jcov support in javac?
|
|
| ----- Begin Included Message -----
|
| Date: Tue, 2 Feb 1999 18:28:04 -0800
| From: "Iris Garcia [TEMP]" <###@###.###>
| To: ###@###.###
| CC: ###@###.###, Iris <###@###.###>
| Subject: jcov support in javac?
|
|
| Hi, Konstantine!
|
| Jon Gibbons gave me your name as the contact for jcov. I hope you're the right
| person, if not, please let me know.
|
| I have recently started to work on javac. We are looking at "end of life-ing"
| the current implementation of the compiler sometime in the future (the exact
| time is as yet undecided, perhaps in JDK1.3?). We are currently investigating
| the set of tools which have some sort of dependency on the compiler so that we
| may anticipate what the impact of this change will be and what sort of support
| those tools will require.
|
| At one time, jcov required support from javac in the form of code added directly
| to the javac codebase (this includes the addition of the -Xjcov option). Now
| that jcov has been written to do static instrumentation of class files or
| dynamic instrumentation via class loader, is this javac code still required?
| I'm guessing that the only time these changes are still used would be in the
| event -Xjcov is used in conjunction with java_g. Is that correct?
|
| I'd like to avoid the situation where jcov suddenly gets broken when we switch
| over to the new compiler. Is there any support that you added to javac that I
| should be aware of?
|
| Are there any plans for jcov development that rely on the current jcov-specific
| changes to javac or that may require the addition of new code to the existing
| javac codebase?
|
| I have found the documentation in jcovguide.html to be most helpful in trying to
| ascertain the current behaviour of jcov, however any historical information
| about the development of jcov would also be very useful.
|
| Please feel free to contact me if you have any further questions or require
| further clarification.
|
| Thanks,
| iris
|
|
| ----- End Included Message -----
|
|