Uploaded image for project: 'JDK'
  1. JDK
  2. JDK-8162520

(fs) FileStore should support file stores with > Long.MAX_VALUE capacity

XMLWordPrintable

    • b22
    • generic
    • generic

      FULL PRODUCT VERSION :
      All java versions.

      ADDITIONAL OS VERSION INFORMATION :
      Problem exists on all platforms.

      EXTRA RELEVANT SYSTEM CONFIGURATION :
      Amazon EFS mount point. This is an NFS filesystem where any user can mount a problematic 8 exabyte filesystem.

      A DESCRIPTION OF THE PROBLEM :
      Java's FileStore methods (getTotalSpace/getUnallocatedSpace/getUsableSpace) are limited to reporting bytes as signed long (2^63-1), if the filesystem is any bigger, then the size overflows.

      For example, any user of the Amazon EFS filesystem can easily mount 2^63 bytes, which is exactly one byte too big, and all methods return negative numbers (Long.MIN_VALUE). This is the problem, because its no longer a theoretical issue, but a practical one for anyone who uses that platform.

      On Unix systems, the underlying values are represented as two 64-bit unsigned integers: blocksize and block counts. This is enough to support enormous filesystems. The problem is that FileStore multiplies these together into one 64-bit signed long, which overflows.

      On Windows systems, the method being called actually returns a 64-bit unsigned long, but its treated by java as if it were signed, so it is currently not quite right either. This seems to be the maximum size supported by windows.

      See email discussion on nio-dev@
      http://mail.openjdk.java.net/pipermail/nio-dev/2016-July/003784.html

      STEPS TO FOLLOW TO REPRODUCE THE PROBLEM :
      Mount a filesystem larger than 2^63-1 bytes and inspect values reported by FileStore.

      EXPECTED VERSUS ACTUAL BEHAVIOR :
      EXPECTED -
      FileStore should have new methods that support large filesystems. For example ZFS can support up to 2^78 bytes.

      ACTUAL -
      Disk usage goes negative.

      ERROR MESSAGES/STACK TRACES THAT OCCUR :
      This breaks our application (Elasticsearch), because it is a distributed system, it needs some basic knowledge of disk usage in order to allocate data...

      REPRODUCIBILITY :
      This bug can be reproduced always.

            bpb Brian Burkhalter
            webbuggrp Webbug Group
            Votes:
            0 Vote for this issue
            Watchers:
            6 Start watching this issue

              Created:
              Updated:
              Resolved: