Package org.antlr.v4.runtime
Class CommonToken
- java.lang.Object
-
- org.antlr.v4.runtime.CommonToken
-
- All Implemented Interfaces:
Serializable,Token,WritableToken
- Direct Known Subclasses:
TokenTagToken
public class CommonToken extends Object implements WritableToken, Serializable
- See Also:
- Serialized Form
-
-
Field Summary
Fields Modifier and Type Field Description protected intchannelThis is the backing field forgetChannel()andsetChannel(int).protected intcharPositionInLineThis is the backing field forgetCharPositionInLine()andsetCharPositionInLine(int).protected static Pair<TokenSource,CharStream>EMPTY_SOURCEprotected intindexThis is the backing field forgetTokenIndex()andsetTokenIndex(int).protected intlineThis is the backing field forgetLine()andsetLine(int).protected Pair<TokenSource,CharStream>sourceThis is the backing field forgetTokenSource()andgetInputStream().protected intstartThis is the backing field forgetStartIndex()andsetStartIndex(int).protected intstopThis is the backing field forgetStopIndex()andsetStopIndex(int).protected StringtextThis is the backing field forgetText()when the token text is explicitly set in the constructor or viasetText(java.lang.String).protected inttypeThis is the backing field forgetType()andsetType(int).-
Fields inherited from interface org.antlr.v4.runtime.Token
DEFAULT_CHANNEL, EOF, EPSILON, HIDDEN_CHANNEL, INVALID_TYPE, MIN_USER_CHANNEL_VALUE, MIN_USER_TOKEN_TYPE
-
-
Constructor Summary
Constructors Constructor Description CommonToken(int type)Constructs a newCommonTokenwith the specified token type.CommonToken(int type, String text)Constructs a newCommonTokenwith the specified token type and text.CommonToken(Pair<TokenSource,CharStream> source, int type, int channel, int start, int stop)CommonToken(Token oldToken)Constructs a newCommonTokenas a copy of anotherToken.
-
Method Summary
All Methods Instance Methods Concrete Methods Modifier and Type Method Description intgetChannel()Return the channel this token.intgetCharPositionInLine()The index of the first character of this token relative to the beginning of the line at which it occurs, 0..n-1CharStreamgetInputStream()Gets theCharStreamfrom which this token was derived.intgetLine()The line number on which the 1st character of this token was matched, line=1..nintgetStartIndex()The starting character index of the token This method is optional; return -1 if not implemented.intgetStopIndex()The last character index of the token.StringgetText()Get the text of the token.intgetTokenIndex()An index from 0..n-1 of the token object in the input stream.TokenSourcegetTokenSource()Gets theTokenSourcewhich created this token.intgetType()Get the token type of the tokenvoidsetChannel(int channel)voidsetCharPositionInLine(int charPositionInLine)voidsetLine(int line)voidsetStartIndex(int start)voidsetStopIndex(int stop)voidsetText(String text)Explicitly set the text for this token.voidsetTokenIndex(int index)voidsetType(int type)StringtoString()StringtoString(Recognizer<?,?> r)
-
-
-
Field Detail
-
EMPTY_SOURCE
protected static final Pair<TokenSource,CharStream> EMPTY_SOURCE
-
type
protected int type
This is the backing field forgetType()andsetType(int).
-
line
protected int line
This is the backing field forgetLine()andsetLine(int).
-
charPositionInLine
protected int charPositionInLine
This is the backing field forgetCharPositionInLine()andsetCharPositionInLine(int).
-
channel
protected int channel
This is the backing field forgetChannel()andsetChannel(int).
-
source
protected Pair<TokenSource,CharStream> source
This is the backing field forgetTokenSource()andgetInputStream().These properties share a field to reduce the memory footprint of
CommonToken. Tokens created by aCommonTokenFactoryfrom the same source and input stream share a reference to the samePaircontaining these values.
-
text
protected String text
This is the backing field forgetText()when the token text is explicitly set in the constructor or viasetText(java.lang.String).- See Also:
getText()
-
index
protected int index
This is the backing field forgetTokenIndex()andsetTokenIndex(int).
-
start
protected int start
This is the backing field forgetStartIndex()andsetStartIndex(int).
-
stop
protected int stop
This is the backing field forgetStopIndex()andsetStopIndex(int).
-
-
Constructor Detail
-
CommonToken
public CommonToken(int type)
Constructs a newCommonTokenwith the specified token type.- Parameters:
type- The token type.
-
CommonToken
public CommonToken(Pair<TokenSource,CharStream> source, int type, int channel, int start, int stop)
-
CommonToken
public CommonToken(int type, String text)Constructs a newCommonTokenwith the specified token type and text.- Parameters:
type- The token type.text- The text of the token.
-
CommonToken
public CommonToken(Token oldToken)
Constructs a newCommonTokenas a copy of anotherToken.If
oldTokenis also aCommonTokeninstance, the newly constructed token will share a reference to thetextfield and thePairstored insource. Otherwise,textwill be assigned the result of callinggetText(), andsourcewill be constructed from the result ofToken.getTokenSource()andToken.getInputStream().- Parameters:
oldToken- The token to copy.
-
-
Method Detail
-
getType
public int getType()
Description copied from interface:TokenGet the token type of the token
-
setLine
public void setLine(int line)
- Specified by:
setLinein interfaceWritableToken
-
setText
public void setText(String text)
Explicitly set the text for this token. If {code text} is notnull, thengetText()will return this value rather than extracting the text from the input.- Specified by:
setTextin interfaceWritableToken- Parameters:
text- The explicit text of the token, ornullif the text should be obtained from the input along with the start and stop indexes of the token.
-
getLine
public int getLine()
Description copied from interface:TokenThe line number on which the 1st character of this token was matched, line=1..n
-
getCharPositionInLine
public int getCharPositionInLine()
Description copied from interface:TokenThe index of the first character of this token relative to the beginning of the line at which it occurs, 0..n-1- Specified by:
getCharPositionInLinein interfaceToken
-
setCharPositionInLine
public void setCharPositionInLine(int charPositionInLine)
- Specified by:
setCharPositionInLinein interfaceWritableToken
-
getChannel
public int getChannel()
Description copied from interface:TokenReturn the channel this token. Each token can arrive at the parser on a different channel, but the parser only "tunes" to a single channel. The parser ignores everything not on DEFAULT_CHANNEL.- Specified by:
getChannelin interfaceToken
-
setChannel
public void setChannel(int channel)
- Specified by:
setChannelin interfaceWritableToken
-
setType
public void setType(int type)
- Specified by:
setTypein interfaceWritableToken
-
getStartIndex
public int getStartIndex()
Description copied from interface:TokenThe starting character index of the token This method is optional; return -1 if not implemented.- Specified by:
getStartIndexin interfaceToken
-
setStartIndex
public void setStartIndex(int start)
-
getStopIndex
public int getStopIndex()
Description copied from interface:TokenThe last character index of the token. This method is optional; return -1 if not implemented.- Specified by:
getStopIndexin interfaceToken
-
setStopIndex
public void setStopIndex(int stop)
-
getTokenIndex
public int getTokenIndex()
Description copied from interface:TokenAn index from 0..n-1 of the token object in the input stream. This must be valid in order to print token streams and use TokenRewriteStream. Return -1 to indicate that this token was conjured up since it doesn't have a valid index.- Specified by:
getTokenIndexin interfaceToken
-
setTokenIndex
public void setTokenIndex(int index)
- Specified by:
setTokenIndexin interfaceWritableToken
-
getTokenSource
public TokenSource getTokenSource()
Description copied from interface:TokenGets theTokenSourcewhich created this token.- Specified by:
getTokenSourcein interfaceToken
-
getInputStream
public CharStream getInputStream()
Description copied from interface:TokenGets theCharStreamfrom which this token was derived.- Specified by:
getInputStreamin interfaceToken
-
toString
public String toString(Recognizer<?,?> r)
-
-