The test failure due to this is as follows (TestDockerMemoryMetrics.java):

========== NEW TEST CASE: testMemoryFailCount20m
[COMMAND]
docker run --tty=true --rm --volume /disk/openjdk/upstream-sources/openjdk-head-2/JTwork/test_jdk/classes/jdk/internal/platform/docker/TestDockerMemoryMetrics.d:/test-classes/ --memory=20m jdk-internal:test-metrics-memory /jdk/bin/java -cp /test-classes/ --add-exports java.base/jdk.internal.platform=ALL-UNNAMED MetricsMemoryTester failcount
[ELAPSED: 2 ms]
[STDERR]

[STDOUT]
[failcount]
Exception in thread "main" java.lang.RuntimeException: Memory fail count : new : [0], old : [0]
at MetricsMemoryTester.testMemoryFailCount(MetricsMemoryTester.java:77)
at MetricsMemoryTester.main(MetricsMemoryTester.java:44)

What's happening?

MetricsMemoryTester's failcount test looks like this:

    private static void testMemoryFailCount() {
        long count = Metrics.systemMetrics().getMemoryFailCount();

        // Allocate 512M of data
        long[][] longs = new long[64][];
        for (int i = 1; i <= 64; i++) {
            try {
                longs[i] = new long[8 * 1024 * 1024];
            } catch (Error e) { // OOM error
                break;
            }
        }
        if (Metrics.systemMetrics().getMemoryFailCount() <= count) {
            throw new RuntimeException("Memory fail count : new : ["
                    + Metrics.systemMetrics().getMemoryFailCount() + "]"
                    + ", old : [" + count + "]");
        }
        System.out.println("TEST PASSED!!!");
    }

Two issues:

1. The test assumes this piece of code increases memory.failcnt:
     long[][] longs = new long[64][];
        for (int i = 1; i <= 64; i++) {
            try {
                longs[i] = new long[8 * 1024 * 1024];
            } catch (Error e) { // OOM error
                break;
            }
       }

It doesn't, though. At least for me on Fedora 29. Building a container as below does not increase the memory.failcnt counter unless the JDK under test is a fastdebug or debug JVM:

$ cat Dockerfile
FROM fedora:29
RUN dnf install -y java-1.8.0-openjdk-devel && \
    dnf -y clean all
ADD fastdebug-jdk /fastdebug-jdk
ADD release-jdk /release-jdk
ADD Memory.class /opt
CMD /bin/bash
$ cat Memory.java
public class Memory {

public static void main(String[] args) {
        long[][] longs = new long[64][];
        for (int i = 1; i <= 64; i++) {
            try {
                longs[i] = new long[8 * 1024 * 1024];
            } catch (Error e) { // OOM error
e.printStackTrace();
                break;
            }
        }

}
}
$ javac Memory.java
$ sudo docker build -t fedora-memory-test:v3 .
$ sudo docker run -ti --memory 20m --memory-swap 0 fedora-memory-test:v3 /bin/bash
[root@63f527777700 /]# cat /sys/fs/cgroup/memory/memory.failcnt
0
[root@63f527777700 /]# cd opt
[root@63f527777700 opt]# /release-jdk/bin/java Memory
java.lang.OutOfMemoryError: Java heap space
at Memory.main(Memory.java:7)
[root@63f527777700 opt]# cat /sys/fs/cgroup/memory/memory.failcnt
0
[root@63f527777700 opt]# /fastdebug-jdk/bin/java Memory
java.lang.OutOfMemoryError: Java heap space
at Memory.main(Memory.java:7)
[root@63f527777700 opt]# cat /sys/fs/cgroup/memory/memory.failcnt
1257

2. The loop counter is wrong. Goes from 1 to 64. Should be 0 to 63. ArrayIndexOutOfBoundsException is thrown otherwise. Not really related to this problem, though.