In HotSpot, C++ functions that can potentially throw exceptions are declared with the TRAPS macro in their arguments. Usually, these functions are called with the CHECK (or CHECK_XXX) macros, so the caller will immediately return when an exception is thrown by the callee.

However, sometimes the caller of such a function needs to intercept the exception and perform additional processing. Currently, such interception are done hapazardly. This RFE tries to make such interception code safer, more consistent and more readable.

There are currently two types of interception:

(1) Statically assert that the callee will never throw an exception. This is currently done using the CATCH macro:

    int callee(TRAPS);
    int caller() {
        int x = callee(CATCH);
        // If an exception has happened, unconditionally clear it.
        // Debug only: print exception and abort VM.
        return x + 1;
    }

(2) General purpose interception. This is done by making the call with THREAD instead of CHECK_XXX. After the callee returns, the caller can perform various operations. Example:

    int caller(int n) {
        int x = callee(THREAD);
        if (HAS_PENDING_EXCEPTION) {
            if (n == 0) {
                // Clear the exception and proceed
                CLEAR_PENDING_EXCEPTION;
            } else if (n == 1) {
                // Propagate the same exception to my caller
                return 0;
            } else {
                // Throw a different exception to my caller
                CLEAR_PENDING_EXCEPTION;
                THROW_0(vmSymbols::java_lang_ClassNotFoundException());
            }
        }

        do_more_stuff();
        return 1;
    }

Problems with the current approach
===========================

In case (1), the meaning of "CATCH" is unclear. Also, usually there is no indication why exceptions should never happen.

In case (2), we should ensure that the caller actually checks HAS_PENDING_EXCEPTION. Otherwise, the caller could behave unexpectedly when an exception was thrown.

Also, in case (2), it's unclear if the callee is a TRAPS function, or is simply a function that takes a Thread* argument but will never thrown. In the latter case, it's perfectly OK to not check for HAS_PENDING_EXCEPTION. This is one major issue with the HotSpot code. We have lots of calls where the last argument is THREAD, and we have no idea whether we are intercepting an exception or not.

Proposal
========

Extend the CATCH macro to cover both cases to have a consistent interface. Also, interceptions can be clearly distinguished with the THREAD cases.

Static assert:

    int caller() {
        int x = callee(CATCH(t));
        assert(t.must_succeed(), "Some explanation why callee will never throw");
        return x + 1;
    }


General interception:

    int caller(int n) {
        int x = callee(CATCH(t));
        if (t.has_pending_exception()) {
            if (n == 0) {
                // Clear the exception and proceed
                t.clear_pending_exception();
            } else if (n == 1) {
                // Propagate the same exception to my caller
                THROW_0(t.propagate_exception());
            } else {
                // Throw a different exception to my caller
                 t.clear_pending_exception();
                oop ex = vmSymbols::java_lang_ClassNotFoundException();
                THROW_0(t.replace_exception(ex));
            }
        }

        do_more_stuff();
        return 1;
    }

For safety, in the C++ destructor of the "t" object, we do the following checks to ensure that he caller must perform these required steps:

+ has_pending_exception() or must_succeed() has been called. I.e., the caller remembers to check the exception status.
+ if the callee had throw an exception, the caller must have:
  - cleared the exception, or
  - propagated the exception, or
  - thrown a different exception

Prototype:
https://github.com/openjdk/jdk/compare/master...iklam:8262907-improve-exception-interception?diff=split&expand=1