Summary

This CSR refers to second iteration of the Foreign Memory Access (an incubating Java API) originally targeted for Java 14, with the goal of refining some of the API rough edges, as well as addressing the feedback received so far from developers.

Problem

Real-world use of the Foreign Memory Access API revealed some usability issues, listed below:

• converting MemoryAddress to/from long values is too hard
• The API's bias towards confinement would benefit from an API point to cooperatively change thread owership of a given segment (serial confinement)
• The MemorySegment::acquire is too complex, as it leads to trees of segments where the enclosing segment cannot be closed before the nested ones are; at the same time, this API fails to provide enough support for clients that wish for a truly unconfined memory segment
• While the API supports mapped memory segments via its MemorySegment::mapFromPath factory, the support is partial, since there's no way to force the mapped segment contents back into the mapped file.
• When implementing frameworks on top of memory access var handle, it is often handy to be able to perform deep adaptation of the var handles so that e.g. additional access coordinates can be inserted, dropped, or bound
• An unsafe API point is required to create a native segment out of an existing MemoryAddress instance; this is the moral equivalent of the JNI NewDirectByteBuffer function.

Solution

Here we describe the main ideas behind the API changes brought forward in this CSR:

• not all MemoryAddress instances are created equal; some addresses are checked --- that is, they are associated with an underlying MemorySegment, some are unchecked e.g. they are just wrappers around some native address. Dereference on a MemoryAddress instance can only be considered safe if the address being dereferenced is checked - since the associated segment would provide enough context (spatial, temporal, thread confinement bounds) to validate the access. With this distinction in mind, it is then possible to add API points to e.g. safely create a NULL address (which would be unchecked) or an address wrapping a given long value.
• Remove the MemorySegment::acquire method, whose complexity was ultimately not helping the use cases for which it was created. In its place, to allow for parallel processing of a segment content, we instead offer the ability to slice and dice a memory segment using a segment Spliterator.
• To make serial thread confinement more useful in producer/consumer use cases, a new API point MemorySegment::withOwnerThread is added so that one thread can give up ownership on a given segment and transfer ownership to a second thread.
• A new interface, namely MappedMemorySegment (which extends from MemorySegment) is provided, which adds functionalities equivalent to MappedByteBuffer::force and MappedByteBuffer::load.
• The new API also provides a number of new VarHandle adapters; while these are temporarily defined in the MemoryHandles class, in reality these adapters are general (pretty much like method handle adapters in the MethodHandles class) and will be moved into the MethodHandles class when the API exits incubation.
• A new memory segment factory has been added, namely MemorySegment::ofNativeRestricted which allows developers to take an existing memory address and create a segment out of it. Since an address is not guaranteed to have any associated spatial and temporal bounds (as is the case for unchecked addresses) such an operation is inherently unsafe. To limit the exposure of this API point, we have opted to guard calls to this method with a read-only JDK property, namely -Dforeign.restricted; this property can assume several values - the default value is deny, which will trigger an hard exception each time such a method is called. Developers can override this property value from the command line, to e.g. permit, which will allow calls to this method to succeed. Note: this way of accessing restricted foreign functionalities through a runtime property is a pragmatic compromise, which will be replaced by a more robust mechanism (perhaps based on the module system) by the time the API exits the incubation stage.

Specification

Here are some useful links which should help in navigating through the changes in the API.

Webrev:

http://cr.openjdk.java.net/~mcimadamore/8243491_v3/webrev

Javadoc:

http://cr.openjdk.java.net/~mcimadamore/8243491_v3/javadoc

Specdiff:

http://cr.openjdk.java.net/~mcimadamore/8243491_v3/specdiff/overview-summary.html

In addition, a specdiff of the changes as of May 1st 2020 has been attached to this CSR.