Name: saC57035 Date: 12/17/96


The java_io.StreamTokenizer.nextToken() handles EOLs as comment terminator
even when EOL is ordinary.

Here is the example demonstrating the bug:
---- Test.java ----------------------
import java.io.*;

public class Test {

public static void main( String argv[] ) {
 StreamTokenizer st = new StreamTokenizer
               (new StringBufferInputStream("/comment \n continued"));
 st.ordinaryChar('\n'); // now \n should not break comments
 try {
   while(st.nextToken() != st.TT_EOF) System.out.println(st);
 } catch (Throwable e) {
   System.out.println("Test failed: "+e+" is thrown");
 }
}

}
-- The output ----------------
Token[EOL], line 1
Token[continued], line 1
------------------------------
No output is expected is this case

NB: line counter is not incremented - that's right 'cos \n
is not line terminator now.
======================================================================

Name: saC57035 Date: 12/17/96


The same problem is discovered when testing quoted strings and // comments.
Please fix that code as well.

======================================================================