Details
P3 Description
While working on my Java memory analysis tool, JOverflow, I found that heap dumps generated by HotSpot JVM lack some important information, and that may seriously affect the accuracy of calculating space taken by objects. That missing info is:
1. Object header size in memory
2. Pointer (object reference) size in memory
3. Object alignment in memory.
Currently the only information relevant for the above, that's stored in a heap dump, is "size of identifiers". It's supposed to be generally equal to the pointer size (item 2 above), but when HotSpot uses narrow references in 64-bit mode, that may not be the case (see below for more info). There is no way to find out object header size other than by guessing; same thing about object alignment. Guessing these values wrongly may lead to serious errors when calculating how much memory is occupied by objects of one class or another.
Note that the heap dump file format is extensible, so it's easy enough to store the above information in backward-compatible way by introducing a new record type (a new tag, etc. - see src/share/demo/jvmti/hprof/hprof_io.c)
Another important piece of information that is missing are types of reference data fields. That is, for primitive fields the type is stored properly in heap dumps, but for reference fields we only know whether it's an object or an instance reference; no further details are stored.
For a reference, here is the summary of the current situation with pointer, object header and alignment sizes in HotSpot, written in January 2012 by Paul Hohensee, that make guessing their values so complicated:
"The 64-bit jvm runs in one of two modes, narrow reference mode and wide reference mode. For heaps less than ~26gb, narrow reference mode is automatically used. Narrow references are 32-bit offsets from the Java heap base rather than 64-bit direct
addresses.
Object alignment defaults to 8 bytes, but can be changed via -XX:ObjectAlignmentInBytes=<n>, where <n> is 8, 16, 32, etc. If the object alignment is, say, 16 bytes, then it's possible to go up to ~60gb heaps using narrow references. And so on.
In wide reference mode, object reference fields are 64 bit direct addresses, object headers are two 64-bit words, and array object headers use the low 32-bits of a third 64-bit word to hold the array length (in elements, not bytes). If the array element size is 4 bytes or less, the first array element comes immediately after the length so that the high half of the third 64-bit word isn't wasted.
In narrow reference mode, object reference fields are 32-bit offsets from the Java heap base. Object headers are 12 bytes, and array object headers are 16. As with wide reference array object headers, the first non-array object field comes immediately after the 12-byte header if it's 4 bytes or less."
1. Object header size in memory
2. Pointer (object reference) size in memory
3. Object alignment in memory.
Currently the only information relevant for the above, that's stored in a heap dump, is "size of identifiers". It's supposed to be generally equal to the pointer size (item 2 above), but when HotSpot uses narrow references in 64-bit mode, that may not be the case (see below for more info). There is no way to find out object header size other than by guessing; same thing about object alignment. Guessing these values wrongly may lead to serious errors when calculating how much memory is occupied by objects of one class or another.
Note that the heap dump file format is extensible, so it's easy enough to store the above information in backward-compatible way by introducing a new record type (a new tag, etc. - see src/share/demo/jvmti/hprof/hprof_io.c)
Another important piece of information that is missing are types of reference data fields. That is, for primitive fields the type is stored properly in heap dumps, but for reference fields we only know whether it's an object or an instance reference; no further details are stored.
For a reference, here is the summary of the current situation with pointer, object header and alignment sizes in HotSpot, written in January 2012 by Paul Hohensee, that make guessing their values so complicated:
"The 64-bit jvm runs in one of two modes, narrow reference mode and wide reference mode. For heaps less than ~26gb, narrow reference mode is automatically used. Narrow references are 32-bit offsets from the Java heap base rather than 64-bit direct
addresses.
Object alignment defaults to 8 bytes, but can be changed via -XX:ObjectAlignmentInBytes=<n>, where <n> is 8, 16, 32, etc. If the object alignment is, say, 16 bytes, then it's possible to go up to ~60gb heaps using narrow references. And so on.
In wide reference mode, object reference fields are 64 bit direct addresses, object headers are two 64-bit words, and array object headers use the low 32-bits of a third 64-bit word to hold the array length (in elements, not bytes). If the array element size is 4 bytes or less, the first array element comes immediately after the length so that the high half of the third 64-bit word isn't wasted.
In narrow reference mode, object reference fields are 32-bit offsets from the Java heap base. Object headers are 12 bytes, and array object headers are 16. As with wide reference array object headers, the first non-array object field comes immediately after the 12-byte header if it's 4 bytes or less."