Package com.intellij.lexer

Class Lexer

java.lang.Object

com.intellij.lexer.Lexer

Direct Known Subclasses:

LexerBase

public abstract class Lexer
extends java.lang.Object

Interface for breaking a file into a sequence of tokens.

See Also:

for certain methods' implementation

Constructor Summary

Constructors

Constructor and Description
Lexer()

Method Summary

Modifier and Type	Method and Description
abstract void	advance() Advances the lexer to the next token.
abstract int	getBufferEnd() Returns the offset at which the lexer will stop lexing.
abstract java.lang.CharSequence	getBufferSequence() Returns the buffer sequence over which the lexer is running.
abstract LexerPosition	getCurrentPosition() Returns the current position and state of the lexer.
abstract int	getState() Returns the current state of the lexer.
abstract int	getTokenEnd() Returns the end offset of the current token.
java.lang.CharSequence	getTokenSequence()
abstract int	getTokenStart() Returns the start offset of the current token.
java.lang.String	getTokenText()
abstract IElementType	getTokenType() Returns the token at the current position of the lexer or null if lexing is finished.
abstract void	restore(LexerPosition position) Restores the lexer to the specified state and position.
void	start(java.lang.CharSequence buf)
void	start(java.lang.CharSequence buf, int start, int end)
abstract void	start(java.lang.CharSequence buffer, int startOffset, int endOffset, int initialState) Prepare for lexing character data from buffer passed.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

Lexer

public Lexer()

Method Detail

start

public abstract void start(java.lang.CharSequence buffer,
                           int startOffset,
                           int endOffset,
                           int initialState)

Prepare for lexing character data from buffer passed. Internal lexer state is supposed to be initialState. It is guaranteed that the value of initialState is the same as returned by getState() method of this lexer at condition startOffset=getTokenStart(). This method is used to incrementally re-lex changed characters using lexing data acquired from this particular lexer sometime in the past.

Parameters:

buffer - character data for lexing.

startOffset - offset to start lexing from

endOffset - offset to stop lexing at

initialState - the initial state of the lexer.

start

public final void start(java.lang.CharSequence buf,
                        int start,
                        int end)

start

public final void start(java.lang.CharSequence buf)

getTokenSequence

public java.lang.CharSequence getTokenSequence()

getTokenText

public java.lang.String getTokenText()

getState

public abstract int getState()

Returns the current state of the lexer.

Returns:

the lexer state.

getTokenType

public abstract IElementType getTokenType()

Returns the token at the current position of the lexer or null if lexing is finished.

Returns:

the current token.

getTokenStart

public abstract int getTokenStart()

Returns the start offset of the current token.

Returns:

the current token start offset.

getTokenEnd

public abstract int getTokenEnd()

Returns the end offset of the current token.

Returns:

the current token end offset.

advance

public abstract void advance()

Advances the lexer to the next token.

getCurrentPosition

public abstract LexerPosition getCurrentPosition()

Returns the current position and state of the lexer.

Returns:

the lexer position and state.

restore

public abstract void restore(LexerPosition position)

Restores the lexer to the specified state and position.

Parameters:

position - the state and position to restore to.

getBufferSequence

public abstract java.lang.CharSequence getBufferSequence()

Returns the buffer sequence over which the lexer is running. This method should return the same buffer instance which was passed to the start() method.

Returns:

the lexer buffer.

getBufferEnd

public abstract int getBufferEnd()

Returns the offset at which the lexer will stop lexing. This method should return the length of the buffer or the value passed in the endOffset parameter to the start() method.

Returns:

the lexing end offset