Package org.antlr.v4.runtime
Class CommonTokenFactory
- java.lang.Object
-
- org.antlr.v4.runtime.CommonTokenFactory
-
- All Implemented Interfaces:
TokenFactory<CommonToken>
public class CommonTokenFactory extends Object implements TokenFactory<CommonToken>
This default implementation ofTokenFactory
createsCommonToken
objects.
-
-
Field Summary
Fields Modifier and Type Field Description protected boolean
copyText
Indicates whetherCommonToken.setText(java.lang.String)
should be called after constructing tokens to explicitly set the text.static TokenFactory<CommonToken>
DEFAULT
The defaultCommonTokenFactory
instance.
-
Constructor Summary
Constructors Constructor Description CommonTokenFactory()
CommonTokenFactory(boolean copyText)
Constructs aCommonTokenFactory
with the specified value forcopyText
.
-
Method Summary
All Methods Instance Methods Concrete Methods Modifier and Type Method Description CommonToken
create(int type, String text)
Generically usefulCommonToken
create(Pair<TokenSource,CharStream> source, int type, String text, int channel, int start, int stop, int line, int charPositionInLine)
This is the method used to create tokens in the lexer and in the error handling strategy.
-
-
-
Field Detail
-
DEFAULT
public static final TokenFactory<CommonToken> DEFAULT
The defaultCommonTokenFactory
instance.This token factory does not explicitly copy token text when constructing tokens.
-
copyText
protected final boolean copyText
Indicates whetherCommonToken.setText(java.lang.String)
should be called after constructing tokens to explicitly set the text. This is useful for cases where the input stream might not be able to provide arbitrary substrings of text from the input after the lexer creates a token (e.g. the implementation ofCharStream.getText(org.antlr.v4.runtime.misc.Interval)
inUnbufferedCharStream
throws anUnsupportedOperationException
). Explicitly setting the token text allowsToken.getText()
to be called at any time regardless of the input stream implementation.The default value is
false
to avoid the performance and memory overhead of copying text for every token unless explicitly requested.
-
-
Constructor Detail
-
CommonTokenFactory
public CommonTokenFactory(boolean copyText)
Constructs aCommonTokenFactory
with the specified value forcopyText
.When
copyText
isfalse
, theDEFAULT
instance should be used instead of constructing a new instance.- Parameters:
copyText
- The value forcopyText
.
-
CommonTokenFactory
public CommonTokenFactory()
Constructs aCommonTokenFactory
withcopyText
set tofalse
.The
DEFAULT
instance should be used instead of calling this directly.
-
-
Method Detail
-
create
public CommonToken create(Pair<TokenSource,CharStream> source, int type, String text, int channel, int start, int stop, int line, int charPositionInLine)
Description copied from interface:TokenFactory
This is the method used to create tokens in the lexer and in the error handling strategy. If text!=null, than the start and stop positions are wiped to -1 in the text override is set in the CommonToken.- Specified by:
create
in interfaceTokenFactory<CommonToken>
-
create
public CommonToken create(int type, String text)
Description copied from interface:TokenFactory
Generically useful- Specified by:
create
in interfaceTokenFactory<CommonToken>
-
-