Thomas Stufe found this bug during following discussion:
http://mail.openjdk.java.net/pipermail/hotspot-runtime-dev/2018-June/028419.html

And then, I think I spot the following issues in os::committed_in_range:

1)

3129 // During shutdown, some memory goes away without properly
notifying NMT,
3130 // E.g. ConcurrentGCThread/WatcherThread can exit without
deleting thread object.
3131 // Bailout and return as not committed for now.
3132 if (mincore_return_value == -1 && errno == ENOMEM) {
3133 return false;
3134 }

here, if the reserved region spans multiple stripes, we would cancel
the mincore loop the moment we encounter a completely empty
(non-residental) page stripe? Is this intented? I think that would
lead to partly committed regions to be reported errornously as not
having any committed parts.

2)
3137 // Process this stripe
3138 for (int vecIdx = 0; vecIdx < pages_to_query; vecIdx ++) {
3139 if ((vec[vecIdx] & 0x01) == 0) { // not committed
3140 // End of current contiguous region
3141 if (committed_start != NULL) {
3142 break;
3143 }
3144 } else { // committed

Here, at line 3142, to my understanding we have found a complete
contiguous range of residential pages. That we should report, right?
So we should return.

However we dont, since the break at 3142 will just break out of the
inner for loop at 3138, not the outer mincore-stripe-loop at 3121.
Which means for multiple stripes, we would now continue with the next
page stripe and loose the information about the contiguous memory
range encountered.