JFR: Scrub recording data

Summary

Add programmatic and command-line support to scrub a recording file. To scrub a recording file means to remove sensitive or uninteresting data to reduce the file size.

Problem

JFR records data into files, typically with the default JDK configuration. Events may contain sensitive information, such as usernames and passwords stored in environment variables. Including this information can be problematic when sharing recording files with other parties, for example, software support. JDK Mission Control correctly warns if it detects suspicious-looking names but provides no means to remove the information. It is possible to create a custom configuration so as not to record sensitive data the next time, but users may not be aware of this possibility and may want to analyze data with complete information themselves.

If something unexpected happens in a Java application, for example, when an SLA is breached, a monitoring system may want to dump the last 30 seconds for later analysis. The dumped file is often sent upstream for additional processing and troubleshooting. Since a recording file includes unrelated information to the SLA breach, unnecessary data, classified as overhead, is sent and perhaps later archived.

Solution

Add a new 'scrub' command to the 'jfr'-tool that allows users to filter and remove events from a recording file. The following example produces a new file called recording-scrubbed.jfr without environment variables or system properties events:

$ jfr scrub 
   --exclude-events jdk.EnvironmentVariable,jdk.InitialSystemProperty
   --output public.jfr
   recording.jfr

To only keep events of a certain type, the option --include-events can be used, for example:

$ jfr scrub 
      --include-events jdk.ThreadStart 
      --output thread-starts.jfr
      recording.jfr

Events can be specified by category, using the options --exclude-categories and --include-categories, for example:

$ jfr scrub --include-categories GC recording.jfr
$ jfr scrub --exclude-categories Compiler recording.jfr

Events can also be specified by thread name, using options --exclude-threads and --include-threads that matches against a glob pattern, for example:

$ jfr scrub --include-threads 'main*' recording.jfr

Filters are applied in the order specified. For example, if the events are named A, B, C, D, the following command will only keep events named A and D.

$ jfr scrub --include-events A,B,C,D 
                 --exclude-events B,C
                 recording.jfr

A new API method, RecordingFile::write, will let users remove events programmatically for more advanced use cases.

The following example illustrates how a new recording file, now reduced in size, can be produced by only keeping events that happen 10 seconds before a CPU spike of 95% on the machine:

record Range(Instant begin, Instant end) {
    boolean contains(Instant time) {
        return time.isAfter(begin) && time.isBefore(end);
    }
};

public static void main(String... args) throws IOException {
    Duration window = Duration.ofSeconds(10);
    try (var rf = new RecordingFile(Path.of(args[0]))) {
        List<Range> ranges = new ArrayList<>();
        while (rf.hasMoreEvents()) {
            RecordedEvent e = rf.readEvent();
            String name = e.getEventType().getName();
            if (name.equals("jdk.CPULoad")) {
                if (e.getFloat("machineTotal") > 0.95) {
                    Instant begin = e.getStartTime().minus(window);
                    Instant end = e.getEndTime();
                    ranges.add(new Range(begin, end));
                }
            }
        }
        rf.write(Path.of("high-cpu-situations.jfr"), e -> {
            for (Range r : ranges) {
                if (r.contains(e.getStartTime())) {
                    return true;
                }
            }
            return false;
        });
    }
}

Specification

jdk.jfr/jdk.jfr.consumer.RecordingFile

/**
 * Filter out events and write them to a new file.
 *
 * @param destination path where the new file should be written, not
 *                    {@code null}
 *
 * @param filter filter that determines if an event should be included, not
 *                    {@code null}
 * @throws IOException if an I/O error occurred, it's not a Flight
 *                           Recorder file or a version of a JFR file that can't
 *                           be parsed
 *
 * @throws SecurityException if a security manager exists and its
 *                           {@code checkWrite} method denies write access to the
 *                           file.
 */
public void write(Path destination, Predicate<RecordedEvent> filter) throws IOException {

jfr help scrub

Scrub contents of a recording file

 jfr scrub [--include-events <filter>]
           [--exclude-events <filter>]
           [--include-categories <filter>]
           [--exclude-categories <filter>]
           [--include-threads <filter>]
           [--exclude-threads <filter>]
           <input-file>
           [<output-file>]

  --include-events <filter>       Select events matching an event name

  --exclude-events <filter>       Exclude events matching an event name

  --include-categories <filter>   Select events matching a category name

  --exclude-categories <filter>   Exclude events matching a category name

  --include-threads <filter>      Select events matching a thread name

  --exclude-threads <filter>      Exclude events matching a thread name

  <input-file>                    The input file to read events from

  <output-file>                   The output file to write filtered events to. 
                                  If no file is specified, it will be written to
                                   the same path as the input file, but with
                                  "-scrubbed" appended to the filename

  The filter is a comma-separated list of names, simple and/or qualified,
  and/or quoted glob patterns. If multiple filters are used, they 
  are applied in the specified order.

Example usage:

 jfr scrub --include-events 'jdk.Socket*' recording.jfr socket-only.jfr
 jfr scrub --exclude-events EnvironmentVariable recording.jfr no-psw.jfr
 jfr scrub --include-threads main recording.jfr
 jfr scrub --exclude-threads 'Foo*' recording.jfr
 jfr scrub --include-categories 'My App' recording.jfr
 jfr scrub --exclude-categories JVM,OS recording.jfr