-
Bug
-
Resolution: Fixed
-
P4
-
11, 15
-
b27
-
Verified
Issue | Fix Version | Assignee | Priority | Status | Resolution | Resolved In Build |
---|---|---|---|---|---|---|
JDK-8248087 | 16 | Claes Redestad | P4 | Resolved | Fixed | team |
ADDITIONAL SYSTEM INFORMATION :
All OSes; the bug is platform-independent.
A DESCRIPTION OF THE PROBLEM :
(This is regression from JDK7, but the above form doesn't allow specifying that as an option.)
Build tools needing deterministic timestamps use January 1, 1980 (the MSDOS epoch) as a "zero" value. Unfortunately, using midnight (00:00:00) of that day causes extra metadata (e.g. local timezone information) to be embedded so that the ZIP files are no longer generated hermetically.
The root cause is that java.util.zip.ZipEntry. DOSTIME_BEFORE_1980 is actually a timestamp in 1980 (exactly midnight).
The bug can be seen in this commit: http://hg.openjdk.java.net/jdk8u/jdk8u/jdk/rev/9a3a791cd28b (The commit message doesn't indicate where it was backported from.)
STEPS TO FOLLOW TO REPRODUCE THE PROBLEM :
1) Run MakeZips (source code attached) which creates four ZIP files. This sets the local timezone to two different values, creating a pair of ZIP files for each.
2) Compare the sizes of the files
3) Compare the contents of pairs out1{a,b}.zip and out2{a,b}.zip
EXPECTED VERSUS ACTUAL BEHAVIOR :
EXPECTED -
1) all created ZIP files should have the same size (138 bytes)
2) out1a.zip should be identical to out1b.zip
3) out2a.zip should be identical to out2b.zip
ACTUAL -
1) files are not all the same size:
$ wc -c *.zip
156 out1a.zip
156 out1b.zip
138 out2a.zip
138 out2b.zip
2) out1{a,b} differ:
$ cmp out1{a,b}.zip
out1a.zip out1b.zip differ: char 45, line 1
3) (sanity check) out2{a,b} are identical:
$ cmp out2{a,b}.zip ; echo $?
0
---------- BEGIN SOURCE ----------
import java.io.File;
import java.io.FileOutputStream;
import java.util.Calendar;
import java.util.GregorianCalendar;
import java.util.TimeZone;
import java.util.zip.*;
public class MakeZips {
public static void main(String[] args) throws Exception {
TimeZone.setDefault(TimeZone.getTimeZone("America/Los_Angeles"));
// java.util.zip.ZipEntry.DOSTIME_BEFORE_1980 is actually 1980-01-01 00:00:00,
// meaning that the beginning of the DOS epoch causes extra metadata to be written
makeZip(
new File("out1a.zip"),
new GregorianCalendar(1980, Calendar.JANUARY, 01, 0, 0, 0).getTimeInMillis());
// ZIP files use 2-second precision, so the next possible timestamp is 1980-01-01 00:00:02;
// this doesn't have extra metadata.
makeZip(
new File("out2a.zip"),
new GregorianCalendar(1980, Calendar.JANUARY, 01, 0, 0, 2).getTimeInMillis());
TimeZone.setDefault(TimeZone.getTimeZone("GMT"));
// java.util.zip.ZipEntry.DOSTIME_BEFORE_1980 is actually 1980-01-01 00:00:00,
// meaning that the beginning of the DOS epoch causes extra metadata to be written
// --- this file will differ from out1a.zip
makeZip(
new File("out1b.zip"),
new GregorianCalendar(1980, Calendar.JANUARY, 01, 0, 0, 0).getTimeInMillis());
// ZIP files use 2-second precision, so the next possible timestamp is 1980-01-01 00:00:02;
// this doesn't have extra metadata.
// --- this file will be identical to out1a.zip
makeZip(
new File("out2b.zip"),
new GregorianCalendar(1980, Calendar.JANUARY, 01, 0, 0, 2).getTimeInMillis());
}
private static void makeZip(File f, long time) throws Exception {
try (ZipOutputStream out = new ZipOutputStream(new FileOutputStream(f))) {
ZipEntry e = new ZipEntry("entry.txt");
e.setTime(time);
out.putNextEntry(e);
out.write(new byte[] {0, 1, 2, 3});
out.closeEntry();
}
}
}
---------- END SOURCE ----------
CUSTOMER SUBMITTED WORKAROUND :
Build tools which need to clear timestamp metadata must be updated to use 1980-01-01 00:00:02, or any time after that instead of midnight.
See https://github.com/bazelbuild/bazel/pull/10976 for example.
FREQUENCY : always
All OSes; the bug is platform-independent.
A DESCRIPTION OF THE PROBLEM :
(This is regression from JDK7, but the above form doesn't allow specifying that as an option.)
Build tools needing deterministic timestamps use January 1, 1980 (the MSDOS epoch) as a "zero" value. Unfortunately, using midnight (00:00:00) of that day causes extra metadata (e.g. local timezone information) to be embedded so that the ZIP files are no longer generated hermetically.
The root cause is that java.util.zip.ZipEntry. DOSTIME_BEFORE_1980 is actually a timestamp in 1980 (exactly midnight).
The bug can be seen in this commit: http://hg.openjdk.java.net/jdk8u/jdk8u/jdk/rev/9a3a791cd28b (The commit message doesn't indicate where it was backported from.)
STEPS TO FOLLOW TO REPRODUCE THE PROBLEM :
1) Run MakeZips (source code attached) which creates four ZIP files. This sets the local timezone to two different values, creating a pair of ZIP files for each.
2) Compare the sizes of the files
3) Compare the contents of pairs out1{a,b}.zip and out2{a,b}.zip
EXPECTED VERSUS ACTUAL BEHAVIOR :
EXPECTED -
1) all created ZIP files should have the same size (138 bytes)
2) out1a.zip should be identical to out1b.zip
3) out2a.zip should be identical to out2b.zip
ACTUAL -
1) files are not all the same size:
$ wc -c *.zip
156 out1a.zip
156 out1b.zip
138 out2a.zip
138 out2b.zip
2) out1{a,b} differ:
$ cmp out1{a,b}.zip
out1a.zip out1b.zip differ: char 45, line 1
3) (sanity check) out2{a,b} are identical:
$ cmp out2{a,b}.zip ; echo $?
0
---------- BEGIN SOURCE ----------
import java.io.File;
import java.io.FileOutputStream;
import java.util.Calendar;
import java.util.GregorianCalendar;
import java.util.TimeZone;
import java.util.zip.*;
public class MakeZips {
public static void main(String[] args) throws Exception {
TimeZone.setDefault(TimeZone.getTimeZone("America/Los_Angeles"));
// java.util.zip.ZipEntry.DOSTIME_BEFORE_1980 is actually 1980-01-01 00:00:00,
// meaning that the beginning of the DOS epoch causes extra metadata to be written
makeZip(
new File("out1a.zip"),
new GregorianCalendar(1980, Calendar.JANUARY, 01, 0, 0, 0).getTimeInMillis());
// ZIP files use 2-second precision, so the next possible timestamp is 1980-01-01 00:00:02;
// this doesn't have extra metadata.
makeZip(
new File("out2a.zip"),
new GregorianCalendar(1980, Calendar.JANUARY, 01, 0, 0, 2).getTimeInMillis());
TimeZone.setDefault(TimeZone.getTimeZone("GMT"));
// java.util.zip.ZipEntry.DOSTIME_BEFORE_1980 is actually 1980-01-01 00:00:00,
// meaning that the beginning of the DOS epoch causes extra metadata to be written
// --- this file will differ from out1a.zip
makeZip(
new File("out1b.zip"),
new GregorianCalendar(1980, Calendar.JANUARY, 01, 0, 0, 0).getTimeInMillis());
// ZIP files use 2-second precision, so the next possible timestamp is 1980-01-01 00:00:02;
// this doesn't have extra metadata.
// --- this file will be identical to out1a.zip
makeZip(
new File("out2b.zip"),
new GregorianCalendar(1980, Calendar.JANUARY, 01, 0, 0, 2).getTimeInMillis());
}
private static void makeZip(File f, long time) throws Exception {
try (ZipOutputStream out = new ZipOutputStream(new FileOutputStream(f))) {
ZipEntry e = new ZipEntry("entry.txt");
e.setTime(time);
out.putNextEntry(e);
out.write(new byte[] {0, 1, 2, 3});
out.closeEntry();
}
}
}
---------- END SOURCE ----------
CUSTOMER SUBMITTED WORKAROUND :
Build tools which need to clear timestamp metadata must be updated to use 1980-01-01 00:00:02, or any time after that instead of midnight.
See https://github.com/bazelbuild/bazel/pull/10976 for example.
FREQUENCY : always
- backported by
-
JDK-8248087 ZIP entries created for DOS epoch include local timezone metadata
- Resolved