Unfortunately JDK-8246484, which was supposed to make COMPARE_BUILD testing more robust, actually broke COMPARE_BUILD. :-( That patch tries to apply the patch with --dry-run at make startup, which was a good idea. Unfortunately, it tries to do so as well when running the second time in compare mode, and then the patch will fail to apply in --dry-run mode, since it is already applied.