Details
-
Bug
-
Resolution: Fixed
-
P3
-
10
-
None
-
b29
Description
Here at Google we've been seeing build failures like this:
/<cloudy-filesystem>/build/linux-x86_64-normal-server-release/make-support/module-deps.gmk:6: *** missing separator. Stop.
make/Main.gmk:143: recipe for target 'java.base-gensrc-moduleinfo' failed
After investigation, they appear only if:
- we're using at least GNU make 3.82
- we're using a fancy cloud filesystem that keeps reporting values from directory traversals in different orders (as filesystems are generally permitted to do)
- multiple build jobs are running (JOBS > 1)
I discovered this change in the make 3.82 NEWS:
* WARNING: Backward-incompatibility!
Wildcards were not documented as returning sorted values, but the results
have been sorted up until this release.. If your makefiles require sorted
results from wildcard expansions, use the $(sort ...) function to request
it explicitly.
The problem is that module-deps.gmk keeps getting regenerated, but only when it is already in use by multiple jobs, causing other jobs to read a half-constructed or corrupted makefile!
The "obvious" workaround is to re-add calls to $(sort ...)
+++ /...make/common/Modules.gmk
@@ -239,20 +239,20 @@
# configuration.
# Param 1 - Module to find for, set to * for finding all
FindAllModuleInfos = \
- $(wildcard \
+ $(sort $(wildcard \
$(foreach sub, $(SRC_SUBDIRS), \
$(patsubst %,%/$(strip $1)/$(sub)/module-info.java, $(TOP_SRC_DIRS))) \
- $(patsubst %,%/$(strip $1)/module-info.java, $(IMPORT_MODULES_SRC)))
+ $(patsubst %,%/$(strip $1)/module-info.java, $(IMPORT_MODULES_SRC))))
# Find module-info.java files in the specific source dir
# Param 1 - Src dir to find module-info.java files in
BUT the deeper problem is that the makefile system pretends that it can handle updates to the set of modules discovered during the build, when it cannot, due to build concurrency. Perhaps this code should be rewritten to compute module-deps.gmk exactly once? If you really want to support the set of modules changing dynamically during the build, you will probably need to write a separate module-info.gmk file for each JOB, which sounds difficult (not worth doing?).
/<cloudy-filesystem>/build/linux-x86_64-normal-server-release/make-support/module-deps.gmk:6: *** missing separator. Stop.
make/Main.gmk:143: recipe for target 'java.base-gensrc-moduleinfo' failed
After investigation, they appear only if:
- we're using at least GNU make 3.82
- we're using a fancy cloud filesystem that keeps reporting values from directory traversals in different orders (as filesystems are generally permitted to do)
- multiple build jobs are running (JOBS > 1)
I discovered this change in the make 3.82 NEWS:
* WARNING: Backward-incompatibility!
Wildcards were not documented as returning sorted values, but the results
have been sorted up until this release.. If your makefiles require sorted
results from wildcard expansions, use the $(sort ...) function to request
it explicitly.
The problem is that module-deps.gmk keeps getting regenerated, but only when it is already in use by multiple jobs, causing other jobs to read a half-constructed or corrupted makefile!
The "obvious" workaround is to re-add calls to $(sort ...)
+++ /...make/common/Modules.gmk
@@ -239,20 +239,20 @@
# configuration.
# Param 1 - Module to find for, set to * for finding all
FindAllModuleInfos = \
- $(wildcard \
+ $(sort $(wildcard \
$(foreach sub, $(SRC_SUBDIRS), \
$(patsubst %,%/$(strip $1)/$(sub)/module-info.java, $(TOP_SRC_DIRS))) \
- $(patsubst %,%/$(strip $1)/module-info.java, $(IMPORT_MODULES_SRC)))
+ $(patsubst %,%/$(strip $1)/module-info.java, $(IMPORT_MODULES_SRC))))
# Find module-info.java files in the specific source dir
# Param 1 - Src dir to find module-info.java files in
BUT the deeper problem is that the makefile system pretends that it can handle updates to the set of modules discovered during the build, when it cannot, due to build concurrency. Perhaps this code should be rewritten to compute module-deps.gmk exactly once? If you really want to support the set of modules changing dynamically during the build, you will probably need to write a separate module-info.gmk file for each JOB, which sounds difficult (not worth doing?).