software.amazon.smithy.model.loader

Interface IdlTokenizer

All Superinterfaces:

java.util.Iterator<IdlToken>

public interface IdlTokenizer
extends java.util.Iterator<IdlToken>

Iterates over a Smithy IDL model as a series of tokens.

Method Summary

Modifier and Type	Method and Description
static IdlTokenizer	create(java.lang.CharSequence model) Create a tokenizer for the given model.
static IdlTokenizer	create(java.lang.String filename, java.lang.CharSequence model) Create a tokenizer for the given filename and model.
default IdlToken	expect(IdlToken... tokens) Assert that the current token is one of tokens.
default void	expect(IdlToken token) Assert that the current token is token.
int	getColumn() Get the current column number of the tokenizer, starting at 1.
IdlToken	getCurrentToken() Get the current IdlToken.
int	getCurrentTokenColumn() Get the column of the current token.
int	getCurrentTokenEnd() Get the end position of the curren token.
java.lang.String	getCurrentTokenError() If the current token is an error, get the error message associated with the token.
default java.lang.CharSequence	getCurrentTokenLexeme() Get the lexeme of the current token.
int	getCurrentTokenLine() Get the line of the current token.
default SourceLocation	getCurrentTokenLocation() Get the source location of the current token.
java.lang.Number	getCurrentTokenNumberValue() If the current token is a number, get the associated parsed number.
default int	getCurrentTokenSpan() Get the length of the current token.
int	getCurrentTokenStart() Get the start position of the current token.
java.lang.CharSequence	getCurrentTokenStringSlice() If the current token is a string or text block, get the parsed content as a CharSequence.
int	getLine() Get the current line number of the tokenizer, starting at 1.
java.lang.CharSequence	getModel() Get the model being tokenized.
default java.lang.CharSequence	getModel(int start, int end) Get a borrowed slice of the model being tokenized.
int	getPosition() Get the current position of the tokenizer.
java.lang.String	getSourceFilename() Get the filename of the content being tokenized.
default boolean	isCurrentLexeme(java.lang.CharSequence chars) Test if the current token lexeme is equal to the give chars.

Methods inherited from interface java.util.Iterator

forEachRemaining, hasNext, next, remove

Method Detail

create

static IdlTokenizer create(java.lang.CharSequence model)

Create a tokenizer for the given model.

Parameters:

model - IDL model contents to parse.

Returns:

Returns the tokenizer.

create

static IdlTokenizer create(java.lang.String filename,
                           java.lang.CharSequence model)

Create a tokenizer for the given filename and model.

Parameters:

filename - Filename being parsed.

model - IDL model contents to parse.

Returns:

Returns the tokenizer.

getSourceFilename

java.lang.String getSourceFilename()

Get the filename of the content being tokenized.

Returns:

Returns the filename used in source locations.

getModel

java.lang.CharSequence getModel()

Get the model being tokenized.

Returns:

Returns the model.

getModel

default java.lang.CharSequence getModel(int start,
                                        int end)

Get a borrowed slice of the model being tokenized.

Parameters:

start - Start position to get, inclusive.

end - End position to stop at, exclusive.

Returns:

Returns the slice.

getPosition

int getPosition()

Get the current position of the tokenizer.

Returns:

Returns the absolute position.

getLine

int getLine()

Get the current line number of the tokenizer, starting at 1.

Returns:

Get the current line number.

getColumn

int getColumn()

Get the current column number of the tokenizer, starting at 1.

Returns:

Get the current column number.

getCurrentToken

IdlToken getCurrentToken()

Get the current IdlToken.

Returns:

Return the current token type.

getCurrentTokenLine

int getCurrentTokenLine()

Get the line of the current token.

Returns:

Return the line of the current token.

getCurrentTokenColumn

int getCurrentTokenColumn()

Get the column of the current token.

Returns:

Return the column of the current token.

getCurrentTokenStart

int getCurrentTokenStart()

Get the start position of the current token.

Returns:

Return the 0-based start position of the current token.

getCurrentTokenEnd

int getCurrentTokenEnd()

Get the end position of the curren token.

Returns:

Return the 0-based end position of the current token.

getCurrentTokenSpan

default int getCurrentTokenSpan()

Get the length of the current token.

Returns:

Return the current token span.

getCurrentTokenLocation

default SourceLocation getCurrentTokenLocation()

Get the source location of the current token.

Returns:

Return the current token source location.

getCurrentTokenLexeme

default java.lang.CharSequence getCurrentTokenLexeme()

Get the lexeme of the current token.

Returns:

Returns the lexeme of the current token.

getCurrentTokenStringSlice

java.lang.CharSequence getCurrentTokenStringSlice()

If the current token is a string or text block, get the parsed content as a CharSequence. If the current token is an identifier, the lexeme of the identifier is returned.

Returns:

Returns the parsed string content associated with the current token.

Throws:

ModelSyntaxException - if the current token is not a string, text block, or identifier.

getCurrentTokenNumberValue

java.lang.Number getCurrentTokenNumberValue()

If the current token is a number, get the associated parsed number.

Returns:

Returns the parsed number associated with the current token.

Throws:

ModelSyntaxException - if the current token is not a number.

getCurrentTokenError

java.lang.String getCurrentTokenError()

If the current token is an error, get the error message associated with the token.

Returns:

Returns the associated error message.

Throws:

ModelSyntaxException - if the current token is not an error.

expect

default void expect(IdlToken token)

Assert that the current token is token.

The tokenizer is not advanced after validating the current token.

Parameters:

token - Token to expect.

Throws:

ModelSyntaxException - if the current token is unexpected.

expect

default IdlToken expect(IdlToken... tokens)

Assert that the current token is one of tokens.

The tokenizer is not advanced after validating the current token.

Parameters:

tokens - Assert that the current token is one of these tokens.

Returns:

Returns the current token.

Throws:

ModelSyntaxException - if the current token is unexpected.

isCurrentLexeme

default boolean isCurrentLexeme(java.lang.CharSequence chars)

Test if the current token lexeme is equal to the give chars.

Parameters:

chars - Characters to compare the current lexeme against.

Returns:

Returns true if the current lexeme is equal to chars.