ADDITIONAL SYSTEM INFORMATION :
Tested with JDK 17 and 19 on macOS and Linux.

A DESCRIPTION OF THE PROBLEM :
JNI function NewDirectByteBuffer accepts a jlong argument for the capacity of the returned buffer.
However, If I pass a value larger than Integer.MAX_VALUE, it behaves unexpectedly, and the behavior is undocumented.
If I pass a value that will cast to a negative int, it will throw an IllegalArgumentException.
If I pass a value that will cast to a positive int, Both Java method ByteBuffer.capacity() and JNI function GetDirectBufferCapacity return a value that is not equal to the correct capacity.

STEPS TO FOLLOW TO REPRODUCE THE PROBLEM :
Set ByteBufferTest.SIZE to 3_000_000_000L, compile and run the code; then set ByteBufferTest.SIZE to 5_000_000_000L, compile and run the code.

ACTUAL -
If SIZE = 3_000_000_000L, it throws an IllegalArgumentException with the following stack trace:
Exception in thread "main" java.lang.IllegalArgumentException: capacity < 0: (-1294967296 < 0)
at java.base/java.nio.Buffer.createCapacityException(Buffer.java:282)
at java.base/java.nio.Buffer.<init>(Buffer.java:245)
at java.base/java.nio.ByteBuffer.<init>(ByteBuffer.java:298)
at java.base/java.nio.ByteBuffer.<init>(ByteBuffer.java:306)
at java.base/java.nio.MappedByteBuffer.<init>(MappedByteBuffer.java:113)
at java.base/java.nio.DirectByteBuffer.<init>(DirectByteBuffer.java:177)
at ByteBufferTest.allocBigBuffer(Native Method)
at ByteBufferTest.main(ByteBufferTest.java:11)

If SIZE = 5_000_000_000L, it prints 705032704 twice, which is not expected.

---------- BEGIN SOURCE ----------
// --- ByteBufferTest.java ---
import java.nio.ByteBuffer;

public class ByteBufferTest {
    static {
        System.loadLibrary("bytebuffertest");
    }

    private static final long SIZE = 3_000_000_000L;
    //private static final long SIZE = 5_000_000_000L;

    public static void main(String[] args) {
        var buf = allocBigBuffer(SIZE);
        System.out.println(buf.capacity());
        System.out.println(getLongCapacity(buf));
    }

    // See ByteBufferTest.c for implementations.
    private static native ByteBuffer allocBigBuffer(long size);
    private static native long getLongCapacity(ByteBuffer buf);
}

// --- ByteBufferTest.c ---
#include "ByteBufferTest.h"
#include <stdlib.h>

// private static native ByteBuffer allocBigBuffer(long size)
JNIEXPORT jobject JNICALL Java_ByteBufferTest_allocBigBuffer(JNIEnv *env, jclass cls, jlong size)
{
    return (*env)->NewDirectByteBuffer(env, malloc(size), size);
}

// private static native long getLongCapacity(ByteBuffer buf)
JNIEXPORT jlong JNICALL Java_ByteBufferTest_getLongCapacity(JNIEnv *env, jclass cls, jobject buf)
{
    return (*env)->GetDirectBufferCapacity(env, buf);
}

---------- END SOURCE ----------

FREQUENCY : always