-
Bug
-
Resolution: Fixed
-
P3
-
None
-
None
-
b12
The file compare_exceptions.sh.incl contains a lot of "known exceptions" to files that we have ignored various aspects of when comparing two builds using compare.sh.
Most of them should be removed. There are several reasons for this:
1) The script was developed when we compared two completely different build systems (old and new), and some leeway was needed in that comparison. Now we only make incremental changes to an existing system, and we need to sharpen the tool.
2) We now have continuous testing on build reproducability, where we build twice in a row and compare the two builds. Essentially, I believe only changes that are triggered under these circumstances should be allowed to be on this list. All other changes could not be just deemed as spurious anymore.
3) We have addressed multiple issues of non-reproducability in the build, but we have been bad at clearing out this file.
Finally, I believe the DIS_DIFF_FILTER does not belong in the exceptions file, since it's more about the core behavior of the compare script.
Most of them should be removed. There are several reasons for this:
1) The script was developed when we compared two completely different build systems (old and new), and some leeway was needed in that comparison. Now we only make incremental changes to an existing system, and we need to sharpen the tool.
2) We now have continuous testing on build reproducability, where we build twice in a row and compare the two builds. Essentially, I believe only changes that are triggered under these circumstances should be allowed to be on this list. All other changes could not be just deemed as spurious anymore.
3) We have addressed multiple issues of non-reproducability in the build, but we have been bad at clearing out this file.
Finally, I believe the DIS_DIFF_FILTER does not belong in the exceptions file, since it's more about the core behavior of the compare script.