-
Bug
-
Resolution: Fixed
-
P2
-
7
-
b73
-
generic
-
generic
Makefiles live to exec things, and normally on Solaris and Linux this kind of change would not be necessary, in normal situations. However, with Windows, and particularly CYGWIN /usr/bin/make usage, the Makefiles in the OpenJDK/JDK repositories seem to magnify the problem.
It is assumed that the key problem is the fork/exec costs on Windows, with CYGWIN, is at the root of this, but this is a hard thing to isolate. The build time comparison is up to 4X slower, so this is a significant issue. So what do we know:
Comparing MKS builds with CYGWIN builds:
1. langtools, jaxp, and jaxws building do not appear to be impacted, these are ant based primarily
2. hotspot building does not appear to be impacted, hotspot on Windows uses Visual Studio NMAKE primarily
3. corba and jdk repositories are impacted, both use GNU make heavily and use a similar style of 'include Defs-*.gmk' variable defining, and are heavily nested make systems (make running make, pretty deep).
This problem appears to manifest itself much more with Windows 64bit systems, not clear why, but we have some suspicions. With MKS we use a MKS-built GNU make, which is a native windows exe, with CYGWIN we use /usr/bin/make, or a GNU make built for the CYGWIN environment (my terminology might not be dead on here).
More data...
On an older Windows 2003 64bit machine that had both MKS and CYGWIN installed, I did builds of the jdk7 corba repository, using local jdk images as the boot jdk and import jdk, using Visual Studio 2003 in both cases.
MKS: 7min
CYGWIN: 20min
This is pretty old hardware, but this is the kind of build timing difference we are seeing, almost 3X slower for the corba repository. We suspect that the jdk repository is just as bad if not worse. The investigation continues....
More data
* cygwin find command is close to 2X slower on Windows 2003 X64 than MKS find
* rm seems about the same
* ls -alR is much faster with cygwin than MKS (5X?), go figure
* large file copies (240Mb file) are twice as slow with cygwin than MKS
* having a shell script run itself 100 times is 7X slower with cygwin
* but having it 'exec' itself 100 times was just 2X slower (no fork?)
* running expr 100 times in a loop is 2X slower with cygwin
* bash and sh behaved the same
* MSYS find tool was actually slower, find command was twice as slow as cygwin
Theory time...
* Native windows apps can write faster to the disk, so if the C++ compiler/linker is writing
out most of the data, performance will not be impacted as much (hotspot builds?)
* File system utilities like cp, tar, etc from cygwin could come with a 2X penalty???
* Reads and Writes of the file system all seem to be 2X
* MKS ls utility is really slow, I wonder why
Now what???
It is assumed that the key problem is the fork/exec costs on Windows, with CYGWIN, is at the root of this, but this is a hard thing to isolate. The build time comparison is up to 4X slower, so this is a significant issue. So what do we know:
Comparing MKS builds with CYGWIN builds:
1. langtools, jaxp, and jaxws building do not appear to be impacted, these are ant based primarily
2. hotspot building does not appear to be impacted, hotspot on Windows uses Visual Studio NMAKE primarily
3. corba and jdk repositories are impacted, both use GNU make heavily and use a similar style of 'include Defs-*.gmk' variable defining, and are heavily nested make systems (make running make, pretty deep).
This problem appears to manifest itself much more with Windows 64bit systems, not clear why, but we have some suspicions. With MKS we use a MKS-built GNU make, which is a native windows exe, with CYGWIN we use /usr/bin/make, or a GNU make built for the CYGWIN environment (my terminology might not be dead on here).
More data...
On an older Windows 2003 64bit machine that had both MKS and CYGWIN installed, I did builds of the jdk7 corba repository, using local jdk images as the boot jdk and import jdk, using Visual Studio 2003 in both cases.
MKS: 7min
CYGWIN: 20min
This is pretty old hardware, but this is the kind of build timing difference we are seeing, almost 3X slower for the corba repository. We suspect that the jdk repository is just as bad if not worse. The investigation continues....
More data
* cygwin find command is close to 2X slower on Windows 2003 X64 than MKS find
* rm seems about the same
* ls -alR is much faster with cygwin than MKS (5X?), go figure
* large file copies (240Mb file) are twice as slow with cygwin than MKS
* having a shell script run itself 100 times is 7X slower with cygwin
* but having it 'exec' itself 100 times was just 2X slower (no fork?)
* running expr 100 times in a loop is 2X slower with cygwin
* bash and sh behaved the same
* MSYS find tool was actually slower, find command was twice as slow as cygwin
Theory time...
* Native windows apps can write faster to the disk, so if the C++ compiler/linker is writing
out most of the data, performance will not be impacted as much (hotspot builds?)
* File system utilities like cp, tar, etc from cygwin could come with a 2X penalty???
* Reads and Writes of the file system all seem to be 2X
* MKS ls utility is really slow, I wonder why
Now what???
- relates to
-
JDK-6883790 corba build problem related to wildcard and vpath, regression
-
- Resolved
-
-
JDK-6892741 Improve make build performance
-
- Closed
-