Uploaded image for project: 'JDK'
  1. JDK
  2. JDK-8271188

Add UL option to replace newline char

    XMLWordPrintable

    Details

    • Type: CSR
    • Status: Closed
    • Priority: P4
    • Resolution: Approved
    • Fix Version/s: 18
    • Component/s: hotspot
    • Labels:
      None
    • Subcomponent:
    • Compatibility Kind:
      behavioral
    • Compatibility Risk:
      minimal
    • Compatibility Risk Description:
      Add new UL option. It does not affect default behavior when it is not specified.
    • Interface Kind:
      add/remove/modify command line option
    • Scope:
      Implementation

      Description

      Summary

      Introduce a new UL output option, foldmultilines, to replace newline characters within a multiline log event with the character sequence '\' and 'n'

      Problem

      Most of the UL entries would print each log message as a single line. However, in some cases (e.g. exceptions), a single log message is printed as multiple lines:

      [0.157s][info][exceptions] Exception <a 'java/lang/NullPointerException'{0x000000008b918f70}: test>
       thrown in interpreter method <{method} {0x00007f8335000248} 'main' '([Ljava/lang/String;)V' in 'Test'>
       at bci 9 for thread 0x00007f8330017160 (main)

      It is easier to parse with log shippers (Fluent Bit, Logstash, etc) if each log message in UL is printed as a single line.

      Famous log shippers support multiline logs, but its configuration tends to be complex, and also some input plugins (e.g. TCP on Fluent Bit) do not support multiline logs.

      Solution

      Introduce new boolean UL option foldmultilines to escape newline (\n: 0x0a) and backslash (\: 0x5c) characters in the UL output. It is set false by default.

      When writing a character buffer into the output file, we treat the buffer as a stream of (8-bit) bytes and handle each byte individually:

      • If the byte is the newline character (0x0a), the following two bytes will be written into the output file: 0x5c (backslash), 0x6e (lowercase 'n')
      • If the byte is the backslash character (0x5c), the following two bytes will be written into the output file: 0x5c (backslash), 0x5c (backslash)
      • Otherwise the byte is written as is into the output file

      Note that foldmultilines=true should be used only with compatible character encodings. For example, this proposal may inadvertently convert 0x0a and 0x5c in Shift JIS and BIG5 because they can have 0x0a and/or 0x5c in multi-byte sequences. This proposal is compatible with UTF-8, where no multi-byte sequences will contain them.

      foldmultilines does not affect newline between log events. Each log event will be terminated by exactly one newline character.

      Log message in C

      char *message = "line 1\nline \\2\nline 3";

      foldmultilines=false

      line 1
      line \2
      line 3

      foldmultilines=true

      line 1\nline \\2\nline 3

      The original text with newline characters can be restored by replacing each pair of '\' characters with a single '\', and replacing each sequence "\n" with the actual newline character, whilst ensuring we scan to the next '\' after performing any such replacement. This is demonstrated by the following Java code:

          String decoded = "";
          int fromIndex = 0;
          int found = 0;
          // Find backslash from the log entry.
          while ((found = ENCODED_LOGLINE.indexOf('\\', fromIndex)) != -1) {
            if (found == ENCODED_LOGLINE.length() - 1) {
              throw new RuntimeException("Backslash shouldn't be located at the tail.");
            }
      
            decoded += ENCODED_LOGLINE.substring(fromIndex, found);
            char next = ENCODED_LOGLINE.charAt(found + 1);
            if (next == '\\') { // next char is backslash ("\\\\" in Java)
              decoded += '\\'; // Treat as single backslash
            } else if (next == 'n') { // next char is "n" ("\\n" in Java)
              decoded += '\n'; // Treat as newline
            }
            fromIndex = found + 2; // Forward index to next char
          }
          decoded += ENCODED_LOGLINE.substring(fromIndex); // Add remaining chars to decoded string

      Specification

      description for output options in man page of java

      output-options is
      
             filecount=file-count filesize=<file size with optional K, M or G suffix> foldmultilines=<true|false>

      command line

      $ java -Xlog:exceptions=info:file=npe.log:::foldmultilines=true Test

      npe.log

      [0.166s][info][exceptions] Exception <a 'java/lang/NullPointerException'{0x000000008b918f70}: test>\n thrown in interpreter method <{method} {0x00007fbc81000248} 'main' '([Ljava/lang/String;)V' in 'Test'>\n at bci 9 for thread 0x00007fbc9c0171a0 (main)

        Attachments

          Issue Links

            Activity

              People

              Assignee:
              ysuenaga Yasumasa Suenaga
              Reporter:
              ysuenaga Yasumasa Suenaga
              Reviewed By:
              David Holmes, Ioi Lam
              Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

                Dates

                Created:
                Updated:
                Resolved: