JFR: Remote Recording Stream

Summary

Add classes jdk.management.jfr.RemoteRecordingStream and jdk.jfr.consumer.MetadataEvent to allow streaming of events and their metadata from a remote host over JMX.

Problem

Many tools for application monitoring can fetch data continuously over the network using JMX. For example, the CPU Load can be obtained from OperatingSystemMXBean and visualized in JDK Mission Control. JFR provides richer data that is structured, including stack traces and timestamped values, but there is currently no way to transfer this information over the network as it occurs. Instead, a recording must be stopped before it can be read as a byte[] and transferred using the FlightRecorderMXBean. This means it is not currently possible to draw a chart to reflect data that is updated continuously, for example a chart describing the longest GC pause times or a list describing the hottest methods.

Solution

JEP 349: JFR Event Streaming added support for streaming events from a disk repository, which is typically located in the temp directory. The plan is to provide an implementation of the jdk.jfr.consumer.EventStream interface that can stream JFR events over JMX.

MBeanServerConnection conn = ...
try (var rs = new RemoteRecordingStream(conn)) {
  rs.enable("jdk.JavaMonitorEnter").withStackTrace().withoutThreshold();
  rs.onEvent(System.out::println);
  rs.start();
}

The RemoteRecordingStream class will communicate with the FlightRecorderMXBean to transfer event data from the server to the client. The event data will be continuously written to a disk located on the client, in a similar manner to how it is continuously written to disk on the server. Using the classes added with JEP 349, it will be possible to stream events originating on the server from the disk located on the client.

The method jdk.management.jfr.FlightRecorderMXBean::openStream(long id, Map options) today forbids opening a stream against a running, ongoing recording. This restriction will be lifted if a client passes the option "streamVersion" with the value "1.0". The option is added so the RemoteRecordingStream class can transfer data for an ongoing recording to a client using the FlightRecorderMXBean. Reading data from an ongoing recording requires deep knowledge of the file format (contrary to a stopped recording) and it is not expected that users will ever explicitly use this option.

A client consuming event data may want to know the field layout before receiving the first event of a specific type. To support this use case, the method onMetadata(Consumer) will be added to the jdk.jfr.consumer.EventStream interface.

Specification

jdk.jfr.EventStream:

/**
 * Registers an action to perform when new metadata arrives in the stream.
 * 
 * The event type of an event always arrives sometime before the actual event.
 * The action must be registered before the stream is started.
 * 
 * @implSpec the default implementation of this method is empty.
 * 
 * @param action to perform, not {@code null}
 * 
 * @throws IllegalStateException if an action is added after the stream has
 *                               started
 */
default void onMetadata(Consumer<MetadataEvent> action);

jdk.jfr.consumer.MetadataEvent:

/**
* Event that contains information about event types and configurations.
*/
public final class MetadataEvent {

  /**
  * Returns a list of known event types.
  *
  * @return an immutable list of event types, not {@code null}
  */
  public final List<EventType> getEventTypes();

  /**
  * Returns a list of event types added since the last metadata event.
  * <p>
  * The delta is from the last metadata event. If no metadata event has been
  * emitted earlier, all known event types are included in the list.
  * 
  * @return an immutable list of added event types, not {@code null}
  */
  public final List<EventType> getAddedEventTypes();

  /**
  * Returns a list of event types removed since the last metadata event.
  * <p>
  * The delta is from the last metadata event. If no metadata event has been
  * emitted earlier, the list is empty.
  * 
  * @return an immutable list of removed event types, not {@code null}
  */
  public final List<EventType> getRemovedEventTypes();

  /**
  * Returns a list of configurations.
  * 
  * @return an immutable list of configurations, not {@code null}
  */
  public List<Configuration> getConfigurations();

}

jdk.jfr.RemoteEventStream

/**
* An implementation of an {@link EventStream} that can serialize events over
* the network using an {@link MBeanServerConnection}.
* <p>
* The following example shows how to record garbage collection pauses and CPU
* usage on a remote host and print the events to standard out.
* 
* <pre>
* 
*   {@literal
*   String host = "com.example";
*   int port = 4711;
* 
*   String url = "service:jmx:rmi:///jndi/rmi://" + host + ":" + port + "/jmxrmi";
* 
*   JMXServiceURL u = new JMXServiceURL(url);
*   JMXConnector c = JMXConnectorFactory.connect(u);
*   MBeanServerConnection conn = c.getMBeanServerConnection();
* 
*   try (var rs = new RemoteRecordingStream(conn)) {
*       rs.enable("jdk.GCPhasePause").withoutThreshold();
*       rs.enable("jdk.CPULoad").withPeriod(Duration.ofSeconds(1));
*       rs.onEvent("jdk.CPULoad", System.out::println);
*       rs.onEvent("jdk.GCPhasePause", System.out::println);
*       rs.start();
*   }
* }
* </pre>
*
* @since 16
*/
public final class RemoteRecordingStream implements EventStream {

/**
 * Creates an event stream that operates against a {@link MBeanServerConnection}
 * with a registered {@link FlightRecorderMXBean}.
 * <p>
 * To configure event settings, use {@link #setSettings(Map)}.
 * 
 * @param connection the {@code MBeanServerConnection} where the
 *                   {@code FlightRecorderMXBean} is registered, not
 *                   {@code null}
 * 
 * @throws IOException       if a stream can't be opened, an I/O error occurs
 *                           when trying to access the repository or the
 *                           {@code FlightRecorderMXBean}
 *
 * @throws SecurityException if a security manager exists and its
 *                           {@code checkRead} method denies read access to the
 *                           directory, or files in the directory.
 */
public RemoteRecordingStream(MBeanServerConnection connection) throws IOException;

/**
 * Creates an event stream that operates against a {@link MBeanServerConnection}
 * with a registered {@link FlightRecorderMXBean}.
 * <p>
 * To configure event settings, use {@link #setSettings(Map)}.
 * 
 * @param connection the {@code MBeanServerConnection} where the
 *                   {@code FlightRecorderMXBean} is registered, not
 *                   {@code null}
 * 
 * @param directory  the directory to store downloaded event data, not
 *                   {@code null}
 * 
 * @throws IOException       if a stream can't be opened, an I/O error occurs
 *                           when trying to access the repository or the
 *                           {@code FlightRecorderMXBean}
 *
 * @throws SecurityException if a security manager exists and its
 *                           {@code checkRead} method denies read access to the
 *                           directory, or files in the directory.
 */
public RemoteRecordingStream(MBeanServerConnection connection, Path directory) throws IOException;

@Override
public void onMetadata(Consumer<MetadataEvent> action);

/**
 * Replaces all settings for this recording stream.
 * <p>
 * The following example connects to a remote host and stream events using
 * settings from the "default" configuration.
 * 
 * <pre>
 * {@literal
 * 
 *  String host = "com.example";
 *  int port = 4711;
 * 
 *  String url = "service:jmx:rmi:///jndi/rmi://" + host + ":" + port + "/jmxrmi";
 * 
 *  JMXServiceURL u = new JMXServiceURL(url);
 *  JMXConnector c = JMXConnectorFactory.connect(u);
 *  MBeanServerConnection conn = c.getMBeanServerConnection();
 * 
 *  try (final var rs = new RemoteRecordingStream(conn)) {
 *      rs.onMetadata(e -> {
 *          for (Configuration c : e.getConfigurations()) {
 *              if (c.getName().equals("default")) {
 *                  rs.setSettings(c.getSettings());
 *              }
 *          }
 *      });
 *      rs.onEvent(System.out::println);
 *      rs.start();
 *  }
 * 
 * }
 * </pre>
 *
 * @param settings the settings to set, not {@code null}
 *
 * @see Recording#setSettings(Map)
 */
public void setSettings(Map<String, String> settings);

/**
 * Disables the event with the specified name.
 * <p>
 * If multiple events have the same name (for example, the same class is loaded in
 * different class loaders), then all events that match are disabled.
 * 
 * @param name the name of the event, not {@code null}
 *
 * @return an event setting for further configuration, not {@code null}
 *
 */
public EventSettings disable(String name);

/**
 * Enables the event with the specified name.
 * <p>
 * If multiple events have the same name (for example, the same class is loaded
 * in different class loaders), then all events that match are enabled.
 *
 * @param name the name of the event, not {@code null}
 *
 * @return an event setting for further configuration, not {@code null}
 *
 * @see EventType
 */
public EventSettings enable(String name);

/**
 * Determines how far back data is kept for the stream.
 * <p>
 * To control the amount of recording data stored on disk, the maximum length of
 * time to keep the data can be specified. Data stored on disk that is older
 * than the specified length of time is removed by the Java Virtual Machine
 * (JVM).
 * <p>
 * If neither the maximum size nor the maximum age is set, the size of the recording
 * may grow indefinitely if events are not consumed.
 *
 * @param maxAge the length of time to keep data, or {@code null} if
 *               infinite
 *
 * @throws IllegalArgumentException if {@code maxAge} is negative
 *
 * @throws IllegalStateException    if the recording is in the {@code CLOSED}
 *                                  state
 */
public void setMaxAge(Duration maxAge);

/**
 * Determines how much data is kept for the stream.
 * <p>
 * To control the amount of recording data that is stored on disk, the maximum
 * amount of data to keep can be specified. When the maximum age is
 * exceeded, the Java Virtual Machine (JVM) removes the oldest chunk to make
 * room for a more recent chunk.
 * <p>
 * If neither maximum age nor the maximum age is set, the size of the recording
 * may grow indefinitely if events are not consumed.
 * <p>
 * The size is measured in bytes.
 *
 * @param maxSize the amount of data to keep, {@code 0} if infinite
 *
 * @throws IllegalArgumentException if {@code maxSize} is negative
 *
 * @throws IllegalStateException    if the recording is in {@code CLOSED} state
 */
public void setMaxSize(long maxSize);

@Override
public void onEvent(Consumer<RecordedEvent> action);

@Override
public void onEvent(String eventName, Consumer<RecordedEvent> action);

@Override
public void onFlush(Runnable action);

@Override
public void onError(Consumer<Throwable> action);

@Override
public void onClose(Runnable action);

@Override
public void close();

@Override
public boolean remove(Object action);

@Override
public void setReuse(boolean reuse);

@Override
public void setOrdered(boolean ordered);

@Override
public void setStartTime(Instant startTime);

@Override
public void setEndTime(Instant endTime);

@Override
public void start();

@Override
public void startAsync();

@Override
public void awaitTermination(Duration timeout) throws InterruptedException;

@Override
public void awaitTermination() throws InterruptedException;
}

jdk.management.jfr.FlightRecorderMXBean::openStream(long, Map) throws IOException;

 * <td>{@code "50000"},<br>
 * {@code "1000000"},<br>
 * </tr>
+* <tr>
+* <th scope="row">{@code streamVersion}</th>
+* <td>Specifies format to use when reading data from a running recording 
+* </td>
+* <td>{@code "1.0"}</td>
+* <td>A version number with a major and minor.<br>
+* <br>
+* To read from a running recording, the value must be set</td>
+* <td>{@code "1.0"}
+* </tr>
 * </tbody>
 * </table>
 * If an option is omitted from the map the default value is used.
 * <p>
 * The recording with the specified ID must be stopped before a stream can
-* be opened. This restriction might be lifted in future releases.
+* be opened, unless the option {@code "streamVersion"} is specified.
 *
 * @param recordingId ID of the recording to open the stream for
 *
 * @param streamOptions a map that contains the options that controls the amount of data
 *        and how it is read, or {@code null} to get all data for the
 *        recording with the default block size
 *
 * @return a unique ID for the stream.
 *
 * @throws IllegalArgumentException if a recording with the iD doesn't
 *         exist, or if {@code options} contains invalid values
 *
 * @throws IOException if the recording is closed, an I/O error occurs, or
 *         no data is available for the specified recording or
 *         interval
 *
 * @throws java.lang.SecurityException if a security manager exists and the
 *         caller does not have {@code ManagementPermission("control")}
 */