String Tokenizer

Notes by Matt.

1. import java.util.StringTokenizer;  // (goes at the top of the program)
    
2. StringTokenizer st = new StringTokenizer(String sToTokenize, String delimiter);  // (goes in method that uses the string tokenizer)
   • String sToTokenize is the String that we want to tokenize
   • String delimiter is a String containing the character that we want to split up the String sToTokenize
      
3. StringTokenizer st = new StringTokenizer(String sToTokenize);  // (goes in method that uses the string tokenizer)
   • Works the same as above- but the difference is that we don't specify the delimiter.  If we don't specify the delimiter, the StringTokenizer will separate the String sToTokenize on spaces (" ")
      
4. A typical implementation after this is the following:

   while (st.hasMoreTokens()) {
     System.out.println(st.nextToken());
     System.out.println("Number of Tokens Left: " + st.countTokens());
   }

String Tokenizer Methods:

As shown above, the most common methods used on a String Tokenizer are:

• nextToken() - returns us a String, gets us the next token out of the String
• countTokens() - returns us an int, counts the number of tokens remaining
• hasMoreTokens() - returns a boolean (true or false) depending on whether there are tokens remaining

Code Examples:

StringTok1.java (simple String Tokenizer, separate w/ spaces)

StringTok2.java (simple String Tokenizer, separate w/ colon)

StringTok3.java (using a String Tokenizer to read tokens from lines of a file) using input.txt