se.l4.lect.tokens

Interface TokenizerFactory

public interface TokenizerFactory

Factory for instances of Tokenizer working on CharSequences.

Author:

Andreas Holstenson

Method Summary

Modifier and Type	Method and Description
Tokenizer	create(CharSequence text) Create a new Tokenizer for the given input text.
default Tokenizer	create(Token token) Create a new Tokenizer for the given input token.

Method Detail

create

Tokenizer create(CharSequence text)

Create a new Tokenizer for the given input text.

Parameters:

text -

Returns:

create

default Tokenizer create(Token token)

Create a new Tokenizer for the given input token.

Parameters:

token -

Returns: