Summary
This CSR refers to the second preview iteration of the Foreign Function & Memory (FFM) API originally targeted for Java 19, with the goal of further refining and consolidating the FFM API.
Problem
Feedback on the incubating API revealed the following issues:
- The separation between
MemorySegment
andMemoryAddress
can be confusing, as aMemoryAddress
effectively acts as a zero-length memory segment; - The
MemorySession
interface has some issues:- Not all sessions are closeable, despite the interface implements
AutoCloseable
; - APIs sometimes need to use non-closeable session views to protect against clients closing sessions using the
MemorySegment::session
accessor; - Some of the predicates in
MemorySession
are leaking too much information (e.g.MemorySession::ownerThread
).
- Not all sessions are closeable, despite the interface implements
- The memory layout hierarchy is not exhaustive, which leads to problem when used in combination with pattern matching;
- The naming of the method
SymbolLookup::lookup
is unfortunate, as the same wordlookup
is used both in the noun form (class name) and the verb form (method name). - There is no way to allow restricted operations on a module in a custom layer, as the command line option only works for modules in the boot layer;
- The Linker API needs ways for clients to customize the properties of the downcall method handles. Some examples of this:
- Mark a downcall method handle as "trivial" - that is, a method handle that performs little computation, does not block and does not upcall to Java (e.g.
getpid
); - Request that the Linker saves the state of certain native variables, such as
errno
orLastError
(as these might get overwritten during normal JVM operation); - Request that one or more by-reference parameters in a downcall might be pinned heap segments. That is, memory segments backed by on-heap region of memory which should not be moved for the duration of the call;
- Specify custom calling conventions (e.g.
fastcall
); - Tweak the safety parameters of the API, to achieve different safety vs. performance trade-offs (e.g. avoid keeping alive by-reference parameters).
- Mark a downcall method handle as "trivial" - that is, a method handle that performs little computation, does not block and does not upcall to Java (e.g.
Solution
Here we describe the main changes brought forward in this CSR:
MemoryAddress
andAddressable
are removed from the FFM API. Instead, zero-length memory segments are used whenever the FFM API needs to model "raw" addresses coming from native code. This simplifies the FFM API, removing an ambiguous abstraction as well as some duplication in the API (see accessor methods inMemoryAddress
). The rationale behind these changes is described in [1].MemorySession
has been replaced withSegmentScope
, a pure lifetime abstraction which does not implementAutoCloseable
andSegmentAllocator
. ASegmentScope
controls which threads have access to a memory segment and when. A scope that can be used to allocate segments that are never allocated is the global scope (SegmentScope::global
). Or, clients can obtain a scope, the automatic scope (SegmentScope::auto
) that can be used to allocate segments that are deallocated automatically, by the garbage collector. Clients requiring deterministic deallocation must use the newArena
abstraction. AnArena
provides a scope - the arena scope (Arena::scope
). When the arena is closed, the arena scope becomes not alive, thus invalidating all the segments associated with the arena scope. This allows us to remove support for non-closeable memory session views (MemorySession::asNonCloseable
,MemorySession::isCloseable
,MemorySession::equals
,MemorySession::hashCode
). The rationale behind these changes is described in [2].The
MemorySession::ownerThread()
method has been replaced with a more focussed predicateSegmentScope:isOwnedBy(Thread)
The
SegmentAllocator::newNativeArena
factory methods have been replaced with a more basicSegmentAllocator::slicingAllocator(MemorySegment)
factory. The new slicing allocator is less powerful than the allocator it replaces, but also simpler to understand, and also much easier to build upon.To allow for "unsafe" access of zero-length memory segments, a new method has been added to
ValueLayout.OfAddress
, namelyasUnbounded
. This new restricted method takes an address layout and creates a new unbounded address layout. When using an unbounded layout to dereference memory, or construct downcall method handles, the FFM API will create memory segments with maximal length (i.e.Long.MAX_VALUE
, rather than zero-length memory segments, which can therefore be accessed.Several changes to the
MemoryLayout
hierarchy:- The hierarchy is now defined in terms of sealed interfaces
- Three new types have been added:
PaddingLayout
,StructLayout
andUnionLayout
(the latter two are a subtype ofGroupLayout
); - Several predicate methods (
isPadding
,isStruct
,isUnion
) have been dropped in favor of pattern matching (which now works as expected).
- The
SymbolLookup::lookup
method has been renamed toSymbolLookup::find
; - A new method, on
ModuleLayer.Controller
has been added to enable native access on a module in a custom layer; - The new interface
Linker.Option
has been introduced. This is a tag interface accepted inLinker::downcallHandle
. At the moment, only a single option is provided, to specify variadic function calls (because of this, theFunctionDescriptor
interface has been simplified, and is now a simple carrier of arguments/return layouts). More linker options will follow.
Specification
A specdiff and javadoc of the changes as of November 30th, 2022 are available below:
- http://cr.openjdk.java.net/~mcimadamore/jdk/8295044/v4/javadoc/java.base/module-summary.html
- http://cr.openjdk.java.net/~mcimadamore/jdk/8295044/v4/specdiff_out/overview-summary.html
A delta of the specdiff between what has been previously submitted is available here:
- (v2 vs. v1) http://cr.openjdk.java.net/~mcimadamore/jdk/8295044/v2/specdiff_out_delta/overview-summary.html
- (v3 vs. v2) http://cr.openjdk.java.net/~mcimadamore/jdk/8295044/v3/specdiff_out_delta/overview-summary.html
- (v4 vs. v3) http://cr.openjdk.java.net/~mcimadamore/jdk/8295044/v4/specdiff_out_delta/overview-summary.html
References
- csr of
-
JDK-8295044 Implementation of Foreign Function and Memory API (Second Preview)
-
- Resolved
-
- relates to
-
JDK-8282192 Implementation of Foreign Function & Memory API (Preview)
-
- Closed
-
-
JDK-8298095 Refine implSpec for SegmentAllocator
-
- Closed
-
-
JDK-8298096 Refine javadoc for Linker
-
- Resolved
-
-
JDK-8312523 Implementation of Foreign Function & Memory API
-
- Closed
-