Uploaded image for project: 'JDK'
  1. JDK
  2. JDK-8218628

Add detailed message to NullPointerException describing what is null.



    • b20


      This Enhancement implements the algorithm to compute the null-detail message specified in JEP 358, "Helpful NullPointerExceptions" (https://openjdk.java.net/jeps/358).

      The messages printed are described in the JEP. A list of examples is attached to the JEP and can be found here:

      - Messages if classfiles contain debug info: http://cr.openjdk.java.net/~goetz/wr19/8218628-exMsg-NPE/21/output_with_debug_info.txt

      - Messages if classfiles contain no debug info: http://cr.openjdk.java.net/~goetz/wr19/8218628-exMsg-NPE/21/output_no_debug_info.txt

      This RFE gives some details on the implementation on top of what is described in the JEP.

      ### Modifications to class NullPointerException

      The JEP states that the null-detail message is computed delayed and on-demand only.

      This is implemented by adding a getMessage() method to NullPointerException. If the field Throwable.detailMessage is empty, it calls native method `getExtendedNPEMessage()` that is implemented in the virtual machine. This method returns a String Object containing the null-detail message or null if it failed to compute the message.

      ### Basic algorithm to compute the message

      This algorithm is passed the bytecodes of a method and the bytecode index where the exception occurred.

      First, the information to step backwards over the bytecodes is computed by a dataflow analysis. This is implemented in the constructor of class `ExceptionMessageBuilder`.

      For each bytecode, it builds a stack of `StackSlotAnalysisData` called `SimulatedOperandStack` describing each operand stack slot. The `StackSlotAnalysisData` is the analysis information for a stack slot and contains the bytecode index of the producer and the Java type of the value on the stack. It maintains an array indexed by bytecode indexes containing these stacks.

      The analysis walks forward over the bytecodes simulating the effects of the bytecode. Working on a certain bytecode, the simulation first duplicates the stack assigned to this bytecode. Then it pops as many `StackSlotAnalysisData` entries from the copied `SimulatedOperandStack` as the bytecode would pop stack slots. It then pushes, according to the semantics of the bytecode, new `StackSlotAnalysisData` entries on the stack and assigns the copy to the next bytecode. Let's look at a possible bytecode `8: getfield #13` at bytecode index 8. Let's assume there are already 5 slots on the stack for bytecode 8. The analysis then copies the stack of depth 5. The getfield bytecode pops an object reference, thus the entry at the top of the `SimulatedOperandStack` must be a reference. This entry is removed. If the constant pool entry #13 describes a reference field, a new entry <reference, 8> is pushed. The next bytecode is at index 11 and gets the new stack assigned.

      If bytecode 11 is another getfield and causes a NullPointerException, the operand slot 5 contained null. The `StackSlotAnalysisData` of the `SimulatedOperandStack` is <reference, 8> and tells that bytecode 8 pushed the null. This way we can step backwards over the bytecodes using the analysis information and find out where the null comes from.

      The stack for the first bytecode is initialized to be empty. At bytecodes altering the control flow the stack is just copied to the target. In case of several control flow targets it is copied to all. At a merge point of control flow, the stacks of the predecessors must be merged. All stack slots that are not identical are set to be undefined. Thus, walking backwards past control flow merges is not always possible.

      The access paths computed by this algorithm usually stem from expressions in Java code. Expressions usually do not contain control flow. (An exemption is the ? operator.) Therefore the algorithm rarely tries to walk back past control flow merges.

      Assembling the first part of the message is implemented in `print_NPE_failed_action()`. It takes the bytecode at the index passed to the algorithm. A switch over the bytecodes listed in Table 1 of the JEP prints this part of the message.

      Assembling the second part of the message is implemented in `print_NPE_cause()`. `print_NPE_cause()` is called with a bytecode index `bci` and an operand stack slot `slot`. It first steps back to the bytecode that pushed this operand stack slot. The index of this bytecode is taken from the corresponding `StackSlotAnalysisData` at `bci` and `slot`.

      When `print_NPE_cause()` is called from the main algorithm, it steps back to the
      bytecode that pushed the null value. The slot needed for this stepping is computed by `get_NPE_null_slot()` which is, similar to `print_NPE_cause()`, a switch over the bytecodes listed in Table 1 of the JEP.

      The message part of `print_NPE_cause()` is generated by a switch over the bytecodes listed in Table 2 of the JEP. It calls itself recursively to compute access paths covering several bytecodes. For array loads, it even does two recursive calls: to the bytecode that pushed the array reference and to the bytecode that pushed the array index.

      `print_NPE_cause()` keeps a counter of the steps walked back to limit its complexity and to limit the size of the access path printed. In the current implementation the counter limit is set to 6, so that the algorithm walks back at most 5 steps.

      `print_NPE_cause()` does cover all the bytecodes listed in Table 2 of the JEP. Arithmetic bytecodes as `iadd` and casts as `l2i` can be encountered in array index expressions but are not implemented. As stated in the JEP, '...' is printed instead.

      All message text is printed directly to a StringStream passed to the algorithm.

      ### Naming locals

      Local variables are managed in a section on the call stack that is split into 'slots'.
      If a local variable needs to be named in the message, we print 'local<slot nr>'.
      Longs and Doubles span two slots.

      This is not very helpful to the user, as this can not be mapped to the source code easily.
      We improve in several means:

      Instead of 'local0' we print 'this' in dynamic methods.

      If the method has parameters, these are mapped trivially to the local
      slots, parameter 0 is held in local slot 0 etc. For parameters, we print 'parameter<param nr>'.
      We parse the signature of the method to find parameters that
      span two slots and adapt the count properly.

      The slots used for parameters can be reused for other values once a parameter is not used
      any more by the method. Code generated by javac of OpenJDK does not
      do so, but other tools generating bytecode optimize it so that this happens.
      If the slot is used for another value, the first thing to happen is a store to the local slot.
      Analysing the bytecode it can not be decided whether stores to local slots are assignments
      to a parameter in the source code, or assignemnts to other locals mapped to this slot.
      The dataflow analysis described above implements a simple additional analysis determining
      whether a local slot has been assigned. If this is true, we do not print 'parameter<param nr>', but 'local<slot>'. Further, this analysis only covers up to 64 slots, i.e., for slots > 64 we don't print 'parameter<param nr>' either. This is a simple approximation to the source code.

      If the class file contains debug information, we can print more useful information. The debug information tells us the name of the variable or parameter of the source code that is accessed. We print this name.

      ### Computing the message delayed

      The algorithm to compute the message needs the bytecodes and the bytecode index of where the exception occurred. Obviously, both is available in internal data structures of the virtual machine when the exception is thrown.

      As stated in the JEP, the message is computed delayed. Thus, the bytecodes and the bytecode index must be preserved until the message is computed.

      Instead of preserving this information for the exception, we rely on an internal data structure of Throwable, the backtrace. A Throwable contains an array of StackTraceElements representing the stack when the exception occurred. This array is also computed delayed and on demand only. The information needed to compute this delayed is preserved in the backtrace data structure that contains references to the internal representation of methods and bytecode indexes.

      The top frame representation of this backtrace data structure contains just what we need to compute the exception delayed.

      The method `java_lang_Throwable::get_method_and_bci()` accesses this data structure and returns a pointer to the method containing the bytecodes we need and the bci.

      As we compute the message delayed, the method can be unloaded or rewritten in the meantime. If so, the bytecodes are not found and `get_method_and_bci()` fails. No message is returned. Similarly, flag OmitStackTraceInFastThrow is on per default. If the stack trace is omitted for an NPE, the backtrace data structure is missing and no message is returned.

      The internal backtrace data structure cannot be serialized. Therefore, if a Throwable is serialized, the Java-level StackTraceElements[] is computed. As specified in the JEP, similar does not happen for the null-detail message of NPE. Thus, the message of the NPE will be empty after deserialization, and the algorithm to compute it will be called if the message is
      accessed. `get_method_and_bci()` will fail as the backtrace data structure is not there and no message is returned.

      ### Detecting cases where NPE is thrown explicitly

      The null-detail message is not printed if the exception was thrown by the user.

      If this is the case, the bytecode at the bytecode index passed to the algorithm is an invoke of the NullPointerException::<init> method. This is recognized in `get_NPE_null_slot()` when we first look at the bytecode and null is returned for the message.

      Alternatively, the NullPointerException can be constructed via JNI. If so, the holder of the method is NativeConstructorAccessorImpl. This is checked after obtaining the method with the bytecodes.

      ### Handling hidden frames

      Exceptions print the stack trace of when the exception occurred.
      There can be methods on the stack of code that has been generated by
      the runtime. To not confuse the reader of the stackTrace, who only expects
      frames of methods visible in the code,
      the VM runtime omits these frames from the stack trace (if -XX:-ShowHiddenFrames, which is default.)

      The frames are already dropped when the backtrace data structure is

      If a NullPointerException is raised in a method whose stack
      frame is hidden, a message based on the wrong method
      will be printed. This is because the backtrace lacks the real top frame.

      We can not compute the proper NullPointerException string and thus must
      recognize this situation and omit the message.

      For this, this change adds a java.lang.Boolean(true) to the backtrace
      in case the real top frame is a hidden one and is dropped.

      When the NullPointerException message text is generated, this Boolean is
      checked and the message is skipped if the proper frame is not

      ### A flag to switch off the feature.

      This change adds a manageable flag called -XX:ShowCodeDetailsInExceptionMessages to
      configure the content of exception messages as above new message.
      The default value is "false", i.e., the message is off per default.
      This was requested by Oracle.


        Issue Links



              goetz Goetz Lindenmaier
              goetz Goetz Lindenmaier
              2 Vote for this issue
              9 Start watching this issue