se.l4.lect.tokens

Class TextTokenization

java.lang.Object

se.l4.lect.tokens.TextTokenization

public class TextTokenization
extends Object

Utilities to help with some text tokenization tasks.

Author:

Andreas Holstenson

Method Summary

Modifier and Type	Method and Description
static int	find(CharSequence text, int start, CharPredicate predicate) Find the index after or at the given start that matches the predicate.
static TokenType	findBestTokenType(CharSequence text) Find the best token type for the given text.
static int	findNext(CharSequence text, int start, CharPredicate predicate) Find the index after the given start that matches the predicate.
static int	findNextNonWhitespace(CharSequence text, int start) Find the index of the first non-whitespace character following the given start index.
static int	findNextWhitespace(CharSequence text, int start) Find the index of the first whitespace character following the given start index.
static int	findNonWhitespace(CharSequence text, int start) Find the index where the first non-whitespace character is available, either at the given start of after it.
static int	findPreviousNonWhitespace(CharSequence text, int start) Find the index of the first non-whitespace character preceding the given start index.
static int	findPreviousWhitespace(CharSequence text, int start) Find the index of the first whitespace character preceding the given start index.
static int	findWhitespace(CharSequence text, int start) Find the index where the first whitespace character is available, either at the given start of after it.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Method Detail

findBestTokenType

public static TokenType findBestTokenType(CharSequence text)

Find the best token type for the given text.

Parameters:

text -

Returns:

find

public static int find(CharSequence text,
                       int start,
                       CharPredicate predicate)

Find the index after or at the given start that matches the predicate.

Parameters:

text -

start -

predicate -

Returns:

findNext

public static int findNext(CharSequence text,
                           int start,
                           CharPredicate predicate)

Find the index after the given start that matches the predicate.

Parameters:

text -

start -

predicate -

Returns:

findNonWhitespace

public static int findNonWhitespace(CharSequence text,
                                    int start)

Find the index where the first non-whitespace character is available, either at the given start of after it.

Parameters:

text - text to search in

start - starting offset

Returns:

offset if found, or -1 if not found

findWhitespace

public static int findWhitespace(CharSequence text,
                                 int start)

Find the index where the first whitespace character is available, either at the given start of after it.

Parameters:

text - text to search in

start - starting offset

Returns:

offset if found, or -1 if not found

findNextNonWhitespace

public static int findNextNonWhitespace(CharSequence text,
                                        int start)

Find the index of the first non-whitespace character following the given start index.

Parameters:

text - text to search in

start - starting offset

Returns:

offset if found, or -1 if not found

findNextWhitespace

public static int findNextWhitespace(CharSequence text,
                                     int start)

Find the index of the first whitespace character following the given start index.

Parameters:

text - text to search in

start - starting offset

Returns:

offset if found, or -1 if not found

findPreviousNonWhitespace

public static int findPreviousNonWhitespace(CharSequence text,
                                            int start)

Find the index of the first non-whitespace character preceding the given start index.

Parameters:

text - text to search in

start - starting offset

Returns:

offset if found, or -1 if not found

findPreviousWhitespace

public static int findPreviousWhitespace(CharSequence text,
                                         int start)

Find the index of the first whitespace character preceding the given start index.

Parameters:

text - text to search in

start - starting offset

Returns:

offset if found, or -1 if not found