Uploaded image for project: 'JDK'
  1. JDK
  2. JDK-8303240

Implementation of Foreign Function and Memory API (Third Preview)



    • CSR
    • Status: Closed
    • P3
    • Resolution: Approved
    • 21
    • core-libs
    • None
    • source, binary, behavioral
    • medium
    • Hide
      Some changes described in this CSR (e.g. renaming of `Arena` factories) might affect source compatibility when the FFM API is used directly. Code generated by the jextract tool is, on the other hand, only marginally affected, as jextract already wraps many of the API points discussed in this CSR.
      Some changes described in this CSR (e.g. renaming of `Arena` factories) might affect source compatibility when the FFM API is used directly. Code generated by the jextract tool is, on the other hand, only marginally affected, as jextract already wraps many of the API points discussed in this CSR.
    • Java API
    • SE



      This CSR refers to the third preview iteration of the Foreign Function & Memory (FFM) API originally targeted for Java 19, with the goal of further refining and consolidating the FFM API.


      Feedback on the previously incubating API revealed the following issues:

      • There are two ways to allocate memory that are equivalent via either the overloads of MemorySegment::allocateNative or via the Arena::allocate overloads which can be confusing.
      • The duality between Arena and SegmentScope is a source of confusion: in almost all cases, client code creates arenas, but then has to retrieve the arena scope to interact with other APIs (e.g. Linker::upcallStub).
      • The Linker::nativeLinker method throws UnsupportedOperationException for unsupported platforms.
      • The FFM API allows creation of some memory layouts (e.g. padding layouts) whose size is not a multiple of 8 bits, even though such layouts are not well supported.
      • The restricted methods for adapting the size and scope of a (zero-length) memory segment are hard to use, especially to resize existing segments (which is a fairly common operation when interacting with native libraries).
      • The VaList construct is neither used in the JDK nor in third-party libraries.
      • The mechanism for creating unbounded address layouts is overly unrestricted.
      • It is not possible to use linker options for upcalls. This might be useful in the future, e.g. to specify custom policies to handle uncaught exceptions.
      • The handling of CaptureCallState (e.g. for "errno") does not take into account unsupported runtime values (e.g. the same capture layout is exposed for both Linux and Windows).
      • The performance of very short-lived native calls (like clockgettime) could be improved.
      • Checking sequence and group layout constraints can be made upon layout construction instead of throwing an exception later when a VarHandle is constructed.


      Here we describe the main changes brought forward in this CSR:

      • The static factory MemorySegment::allocateNative overloads have been removed. Instead, allocation is made via the Arena::allocate overloads. See this document.
      • SegmentScope was dropped in favor of a much smaller construct MemorySegment.Scope which only has one predicate isAlive() and where the equals() method tests if two scopes model the same lifetime (and is alive). See this document.
      • The factories Arena::(openConfined, openShared) were renamed to Arena::(ofConfined, ofShared). The SegmentScope.global() factory was replaced by Arena.global(), while the SegmentScope.auto() factory was replaced by Arena.ofAuto(). Picking a more neutral name (of instead of open) was deemed necessary given that automatic arenas cannot be closed.
      • A fall-back linker based on libffi was added to simplify porting (see https://github.com/openjdk/panama-foreign/pull/770 for details).
      • The memory layout API now uniformly enforces that bit-size and bit-alignment constraints of all memory layouts (including padding layouts) must be a multiple of 8.
      • A new restricted instance method, namely MemorySegment::reinterpret, was added. This allows to create a new native segment, with new size and segment scope with the same base address as some existing segment. Some overloads are also provided, for usability. The MemorySegment::ofAddress method is kept, as a way to create a zero-length native memory segment from a raw long address. As such, this method is no longer restricted.
      • The VaList interface was dropped.
      • The ValueLayout.OfAddress::asUnbounded method was replaced with the new method ValueLayout.OfAddress::withTargetLayout, which allows to associate an address layout with a "target" layout (the layout of the memory region pointed to by a given address). This allows the API to expose a new kind of "dereference" PathElement, which can be used to create access var handles for expression such as *(a.b)[10]. Also, given the importance of address layouts, ValueLayout.OfAddress was moved to a toplevel class, namely AddressLayout.
      • Linker options can now be used when creating upcall stubs. (Note: at the time of writing there are no supported linker options for upcall stubs).
      • The CapturedCallState interface was removed. A new static method was added to query the layout of the runtime values to be preserved. This layout is now platform specific.
      • A new linker option was added to mark calls to native functions that are extremely short-lived (comparable to a no-op) and do not call back into Java (we call such functions "trivial functions"). This option might be used by the linker to speed up a native call (e.g. by removing thread state transitions).
      • Up-front checks are added in the sequence and group constructors as well as in the associated withBitAlignment() methods making sure the layouts are well formed.


      A specdiff and javadoc of the changes as of 2023-04-13 are available below:


      1. https://cr.openjdk.org/~mcimadamore/panama/why_lifetimes.html
      2. http://cr.openjdk.java.net/~mcimadamore/panama/scoped_arenas.html
      3. https://github.com/libffi/libffi
      4. https://github.com/openjdk/panama-foreign/pull/824


        Issue Links



              pminborg Per-Ake Minborg
              pminborg Per-Ake Minborg
              Joe Darcy
              0 Vote for this issue
              3 Start watching this issue