Uploaded image for project: 'JDK'
  1. JDK
  2. JDK-8246129

ZIP entries created for DOS epoch include local timezone metadata

XMLWordPrintable

    • b27
    • Verified

        ADDITIONAL SYSTEM INFORMATION :
        All OSes; the bug is platform-independent.

        A DESCRIPTION OF THE PROBLEM :
        (This is regression from JDK7, but the above form doesn't allow specifying that as an option.)

        Build tools needing deterministic timestamps use January 1, 1980 (the MSDOS epoch) as a "zero" value. Unfortunately, using midnight (00:00:00) of that day causes extra metadata (e.g. local timezone information) to be embedded so that the ZIP files are no longer generated hermetically.

        The root cause is that java.util.zip.ZipEntry. DOSTIME_BEFORE_1980 is actually a timestamp in 1980 (exactly midnight).

        The bug can be seen in this commit: http://hg.openjdk.java.net/jdk8u/jdk8u/jdk/rev/9a3a791cd28b (The commit message doesn't indicate where it was backported from.)

        STEPS TO FOLLOW TO REPRODUCE THE PROBLEM :
        1) Run MakeZips (source code attached) which creates four ZIP files. This sets the local timezone to two different values, creating a pair of ZIP files for each.
        2) Compare the sizes of the files
        3) Compare the contents of pairs out1{a,b}.zip and out2{a,b}.zip

        EXPECTED VERSUS ACTUAL BEHAVIOR :
        EXPECTED -
        1) all created ZIP files should have the same size (138 bytes)
        2) out1a.zip should be identical to out1b.zip
        3) out2a.zip should be identical to out2b.zip
        ACTUAL -
        1) files are not all the same size:
        $ wc -c *.zip
             156 out1a.zip
             156 out1b.zip
             138 out2a.zip
             138 out2b.zip

        2) out1{a,b} differ:
        $ cmp out1{a,b}.zip
        out1a.zip out1b.zip differ: char 45, line 1

        3) (sanity check) out2{a,b} are identical:
        $ cmp out2{a,b}.zip ; echo $?
        0

        ---------- BEGIN SOURCE ----------
        import java.io.File;
        import java.io.FileOutputStream;
        import java.util.Calendar;
        import java.util.GregorianCalendar;
        import java.util.TimeZone;
        import java.util.zip.*;

        public class MakeZips {
          public static void main(String[] args) throws Exception {
            TimeZone.setDefault(TimeZone.getTimeZone("America/Los_Angeles"));

            // java.util.zip.ZipEntry.DOSTIME_BEFORE_1980 is actually 1980-01-01 00:00:00,
            // meaning that the beginning of the DOS epoch causes extra metadata to be written
            makeZip(
                new File("out1a.zip"),
                new GregorianCalendar(1980, Calendar.JANUARY, 01, 0, 0, 0).getTimeInMillis());

            // ZIP files use 2-second precision, so the next possible timestamp is 1980-01-01 00:00:02;
            // this doesn't have extra metadata.
            makeZip(
                new File("out2a.zip"),
                new GregorianCalendar(1980, Calendar.JANUARY, 01, 0, 0, 2).getTimeInMillis());

            TimeZone.setDefault(TimeZone.getTimeZone("GMT"));

            // java.util.zip.ZipEntry.DOSTIME_BEFORE_1980 is actually 1980-01-01 00:00:00,
            // meaning that the beginning of the DOS epoch causes extra metadata to be written
            // --- this file will differ from out1a.zip
            makeZip(
                new File("out1b.zip"),
                new GregorianCalendar(1980, Calendar.JANUARY, 01, 0, 0, 0).getTimeInMillis());

            // ZIP files use 2-second precision, so the next possible timestamp is 1980-01-01 00:00:02;
            // this doesn't have extra metadata.
            // --- this file will be identical to out1a.zip
            makeZip(
                new File("out2b.zip"),
                new GregorianCalendar(1980, Calendar.JANUARY, 01, 0, 0, 2).getTimeInMillis());
          }

          private static void makeZip(File f, long time) throws Exception {
            try (ZipOutputStream out = new ZipOutputStream(new FileOutputStream(f))) {
              ZipEntry e = new ZipEntry("entry.txt");
              e.setTime(time);
              out.putNextEntry(e);
              out.write(new byte[] {0, 1, 2, 3});
              out.closeEntry();
            }
          }
        }
        ---------- END SOURCE ----------

        CUSTOMER SUBMITTED WORKAROUND :
        Build tools which need to clear timestamp metadata must be updated to use 1980-01-01 00:00:02, or any time after that instead of midnight.

        See https://github.com/bazelbuild/bazel/pull/10976 for example.

        FREQUENCY : always


          1. MakeZips.java
            2 kB
          2. out1a.zip
            0.2 kB
          3. out1b.zip
            0.2 kB
          4. out2a.zip
            0.1 kB
          5. out2b.zip
            0.1 kB

              redestad Claes Redestad
              webbuggrp Webbug Group
              Votes:
              0 Vote for this issue
              Watchers:
              8 Start watching this issue

                Created:
                Updated:
                Resolved: