The StreamTokenizer class fails to properly handle the '.' character when you
have added it as a word character and it is the first thing in a token. Compile
and run the following code:

[3:22pm sdirose@shadowfax] ~/tmp/3164839/t2> cat example.java
import java.io.*;

public class example
 {
  
  public static void main(String[] args)
    {
      try
{
          File f = new File("input.txt");
          FileInputStream fis = new FileInputStream(f);
          InputStreamReader isr = new InputStreamReader(fis);
          StreamTokenizer st = new StreamTokenizer(isr);
          st.wordChars((int)'.', (int)'.');

          while(st.nextToken() != StreamTokenizer.TT_EOF)
{
String token = st.sval;
if(token == null) continue;
int index = token.indexOf("\"");
             while(index >= 0)
{
StringBuffer tokensb = new StringBuffer(token);
tokensb.insert(index,"\\");
token = tokensb.toString();
index = token.indexOf("\"", index+2);
               }
System.out.println(token);
            }
}
      catch(Exception e) { e.printStackTrace(); System.exit(0); }

    }

 }

[3:10pm sdirose@shadowfax] ~/tmp/3164839> cat input.txt
This is just a test
of a ...input file which
will be parsed okay?

However, if you add '"' to the list of word characters, and then you put
"...input in the input.txt file instead of ...input, the '.' is then parsed
as a word character.

This was first noticed by the customer on 1.1.4, and is still present in
1.2 Beta 2