-
Enhancement
-
Resolution: Unresolved
-
P4
-
24
### Motivation
The primary motivation is to overcome some limitations of the Java Fuzzer. Its randomized nature makes it difficult to steer the generator to trigger specific optimizations, like loop optimizations. The generated code often compiles to the same C2 Intermediate Representation (IR), resulting in inefficient test execution. Furthermore, JavaFuzzer's implementation in Ruby makes it hard for us to modify and extend it (we are Java/C++ experts). Most important, JavaFuzzer lacks support for testing new Java features like the Vector API or Value Classes.
The proposed Template-Based Test Generator aims to address these limitations by allowing more controlled, targeted test generation, particularly for testing specific C2 optimizations. It is intended to integrate with the JTreg framework, enabling a mix of templates and IR Framework tests, with a focus on a narrow, manageable syntax scope that allows fast test generation and execution.
### Goals
The key goal is quick adoption by OpenJDK contributors. JIT compiler changes should then be accompanied by a template-based test that exercises the affected code and guarantees more coverage than a non-template-based test. Compatibility with the IR Test Framework and JTreg testing framework should be ensured but with minimal coupling. Syntax changes should be minimized to avoid disruptions as contributors begin writing test. The templates should make it easy to write tests with some level of randomization and a focus on extreme values like Integer.MAX_VALUE or otherwise unlikely code paths. Narrowing the generation scope by the test writer should result in more targeted tests. Combined with the goal of fast execution and generation of tests, this should make testing much more efficient.
### Template Syntax
The syntax for the templates should allow for easy integration and interoperability with IR Framework and JTreg tests. This has to take into consideration using variables across scopes. The concept includes using collection of Compiler Regression Test Templates as a basis for a library of common templates and possibly predefining tests for basic operations.
### Implementation
The initial implementation has started with a Minimum Viable Product (MVP) prototype, Generator.java, by Tobias Hartmann.
The implementation considers using the string template JEP (preview) and possibly the Class-File API. For efficiency, the implementation strategy includes parallel test generation and execution threads (one thread generates tests, other threads execute them).
Integration with the IR framework and JTreg is envisioned with minimal coupling, focusing on composable templates that can easily nest within each other.
The primary motivation is to overcome some limitations of the Java Fuzzer. Its randomized nature makes it difficult to steer the generator to trigger specific optimizations, like loop optimizations. The generated code often compiles to the same C2 Intermediate Representation (IR), resulting in inefficient test execution. Furthermore, JavaFuzzer's implementation in Ruby makes it hard for us to modify and extend it (we are Java/C++ experts). Most important, JavaFuzzer lacks support for testing new Java features like the Vector API or Value Classes.
The proposed Template-Based Test Generator aims to address these limitations by allowing more controlled, targeted test generation, particularly for testing specific C2 optimizations. It is intended to integrate with the JTreg framework, enabling a mix of templates and IR Framework tests, with a focus on a narrow, manageable syntax scope that allows fast test generation and execution.
### Goals
The key goal is quick adoption by OpenJDK contributors. JIT compiler changes should then be accompanied by a template-based test that exercises the affected code and guarantees more coverage than a non-template-based test. Compatibility with the IR Test Framework and JTreg testing framework should be ensured but with minimal coupling. Syntax changes should be minimized to avoid disruptions as contributors begin writing test. The templates should make it easy to write tests with some level of randomization and a focus on extreme values like Integer.MAX_VALUE or otherwise unlikely code paths. Narrowing the generation scope by the test writer should result in more targeted tests. Combined with the goal of fast execution and generation of tests, this should make testing much more efficient.
### Template Syntax
The syntax for the templates should allow for easy integration and interoperability with IR Framework and JTreg tests. This has to take into consideration using variables across scopes. The concept includes using collection of Compiler Regression Test Templates as a basis for a library of common templates and possibly predefining tests for basic operations.
### Implementation
The initial implementation has started with a Minimum Viable Product (MVP) prototype, Generator.java, by Tobias Hartmann.
The implementation considers using the string template JEP (preview) and possibly the Class-File API. For efficiency, the implementation strategy includes parallel test generation and execution threads (one thread generates tests, other threads execute them).
Integration with the IR framework and JTreg is envisioned with minimal coupling, focusing on composable templates that can easily nest within each other.
- blocks
-
JDK-8352861 Template-Framework Library
-
- Open
-
- relates to
-
JDK-8337221 CompileFramework: test library to conveniently compile java and jasm sources for fuzzing
-
- Resolved
-
- links to
-
Review(master) openjdk/jdk/24217