-
Enhancement
-
Resolution: Fixed
-
P4
-
11, 14, 15
-
b10
Issue | Fix Version | Assignee | Priority | Status | Resolution | Resolved In Build |
---|---|---|---|---|---|---|
JDK-8240765 | 14.0.2 | Aleksey Shipilev | P4 | Resolved | Fixed | b01 |
JDK-8246667 | 13.0.4 | Aleksey Shipilev | P4 | Resolved | Fixed | b04 |
JDK-8239237 | 11.0.8-oracle | Aleksey Shipilev | P4 | Resolved | Fixed | b01 |
JDK-8239059 | 11.0.7 | Aleksey Shipilev | P4 | Resolved | Fixed | b04 |
This is very visible when compiling the JAR with many files. For example, compiling this one:
$ wget https://repo1.maven.org/maven2/org/apache/servicemix/bundles/org.apache.servicemix.bundles.quickfix/1.6.3_1/org.apache.servicemix.bundles.quickfix-1.6.3_1.jar
$ ./ctw.sh org.apache.servicemix.bundles.quickfix-1.6.3_1.jar
It contains about 0.5M methods, which ends up doing 2M compilations on all tiers, and CTW run on it takes multiple days -- I basically terminated the run after 40 hours, with about half done.
Profiling with this patch: http://cr.openjdk.java.net/~shade/8238247/8238247-tracing.patch -- shows that compilation and deoptimization time grows nearly-linearly with the number of nmethods in the system, and CTW stabilizes at about 110K nmethods present. This is, alas, too much for the stable point when aggressive compilation and deoptimization is needed.
Making the sweeper more aggressive with e.g. -XX:NmethodSweepActivity=2000 keeps nmethod population very low. The CTW now finishes in about 2 hours, which is at least 20x improvement. See the example chart from my experiments below. Default run was terminated early, because it was evident it goes nowhere.
https://cr.openjdk.java.net/~shade/8238247/plot.png
So, my suggestion would be to put -XX:NmethodSweepActivity in here:
diff -r c3c9f45a18cf test/hotspot/jtreg/testlibrary/ctw/Makefile
--- a/test/hotspot/jtreg/testlibrary/ctw/Makefile Thu Jan 30 13:31:24 2020 +0100
+++ b/test/hotspot/jtreg/testlibrary/ctw/Makefile Thu Jan 30 20:41:36 2020 +0100
@@ -67,11 +67,11 @@
$(DST_DIR):
@mkdir -p $@
$(DST_DIR)/ctw.sh: $(DST_DIR)
- echo '$${JAVA_HOME}/bin/java $${JAVA_OPTIONS} $(EXPORTS) -XX:-UseCounterDecay -Xbatch "-XX:CompileCommand=exclude,java/lang/invoke/MethodHandle.*" -XX:+UnlockDiagnosticVMOptions -XX:+WhiteBoxAPI -Xbootclasspath/a:wb.jar -jar ctw.jar $$@' > $@
+ echo '$${JAVA_HOME}/bin/java $${JAVA_OPTIONS} $(EXPORTS) -XX:-UseCounterDecay -Xbatch -XX:NmethodSweepActivity=2000 "-XX:CompileCommand=exclude,java/lang/invoke/MethodHandle.*" -XX:+UnlockDiagnosticVMOptions -XX:+WhiteBoxAPI -Xbootclasspath/a:wb.jar -jar ctw.jar $$@' > $@
chmod a+x $@
$(DST_DIR)/ctw.jar: filelist $(DST_DIR)/wb.jar
@mkdir -p $(OUTPUT_DIR)
$(JAVAC) $(EXPORTS) -sourcepath $(SRC_DIR) -d $(OUTPUT_DIR) -cp $(DST_DIR)/wb.jar @filelist
$ wget https://repo1.maven.org/maven2/org/apache/servicemix/bundles/org.apache.servicemix.bundles.quickfix/1.6.3_1/org.apache.servicemix.bundles.quickfix-1.6.3_1.jar
$ ./ctw.sh org.apache.servicemix.bundles.quickfix-1.6.3_1.jar
It contains about 0.5M methods, which ends up doing 2M compilations on all tiers, and CTW run on it takes multiple days -- I basically terminated the run after 40 hours, with about half done.
Profiling with this patch: http://cr.openjdk.java.net/~shade/8238247/8238247-tracing.patch -- shows that compilation and deoptimization time grows nearly-linearly with the number of nmethods in the system, and CTW stabilizes at about 110K nmethods present. This is, alas, too much for the stable point when aggressive compilation and deoptimization is needed.
Making the sweeper more aggressive with e.g. -XX:NmethodSweepActivity=2000 keeps nmethod population very low. The CTW now finishes in about 2 hours, which is at least 20x improvement. See the example chart from my experiments below. Default run was terminated early, because it was evident it goes nowhere.
https://cr.openjdk.java.net/~shade/8238247/plot.png
So, my suggestion would be to put -XX:NmethodSweepActivity in here:
diff -r c3c9f45a18cf test/hotspot/jtreg/testlibrary/ctw/Makefile
--- a/test/hotspot/jtreg/testlibrary/ctw/Makefile Thu Jan 30 13:31:24 2020 +0100
+++ b/test/hotspot/jtreg/testlibrary/ctw/Makefile Thu Jan 30 20:41:36 2020 +0100
@@ -67,11 +67,11 @@
$(DST_DIR):
@mkdir -p $@
$(DST_DIR)/ctw.sh: $(DST_DIR)
- echo '$${JAVA_HOME}/bin/java $${JAVA_OPTIONS} $(EXPORTS) -XX:-UseCounterDecay -Xbatch "-XX:CompileCommand=exclude,java/lang/invoke/MethodHandle.*" -XX:+UnlockDiagnosticVMOptions -XX:+WhiteBoxAPI -Xbootclasspath/a:wb.jar -jar ctw.jar $$@' > $@
+ echo '$${JAVA_HOME}/bin/java $${JAVA_OPTIONS} $(EXPORTS) -XX:-UseCounterDecay -Xbatch -XX:NmethodSweepActivity=2000 "-XX:CompileCommand=exclude,java/lang/invoke/MethodHandle.*" -XX:+UnlockDiagnosticVMOptions -XX:+WhiteBoxAPI -Xbootclasspath/a:wb.jar -jar ctw.jar $$@' > $@
chmod a+x $@
$(DST_DIR)/ctw.jar: filelist $(DST_DIR)/wb.jar
@mkdir -p $(OUTPUT_DIR)
$(JAVAC) $(EXPORTS) -sourcepath $(SRC_DIR) -d $(OUTPUT_DIR) -cp $(DST_DIR)/wb.jar @filelist
- backported by
-
JDK-8239059 CTW runner should sweep nmethods more aggressively
- Resolved
-
JDK-8239237 CTW runner should sweep nmethods more aggressively
- Resolved
-
JDK-8240765 CTW runner should sweep nmethods more aggressively
- Resolved
-
JDK-8246667 CTW runner should sweep nmethods more aggressively
- Resolved
- relates to
-
JDK-8325613 CTW: Stale method cleanup requires GC after Sweeper removal
- Resolved
-
JDK-8238737 remove DeoptimizeAllClassesRate from CTW library
- Resolved
(1 relates to)