org.antlr.runtime.CommonTokenStream Class Reference
Public Member Functions | |
| CommonTokenStream () | |
| CommonTokenStream (TokenSource tokenSource) | |
| CommonTokenStream (TokenSource tokenSource, int channel) | |
| void | consume () |
| Token | LT (int k) |
| int | getNumberOfOnChannelTokens () |
| void | setTokenSource (TokenSource tokenSource) |
Protected Member Functions | |
| Token | LB (int k) |
| int | skipOffTokenChannels (int i) |
| int | skipOffTokenChannelsReverse (int i) |
| void | setup () |
Protected Attributes | |
| int | channel = Token.DEFAULT_CHANNEL |
Detailed Description
The most common stream of tokens where every token is buffered up and tokens are filtered for a certain channel (the parser will only see these tokens).Even though it buffers all of the tokens, this token stream pulls tokens from the tokens source on demand. In other words, until you ask for a token using consume(), LT(), etc. the stream does not pull from the lexer.
The only difference between this stream and BufferedTokenStream superclass is that this stream knows how to ignore off channel tokens. There may be a performance advantage to using the superclass if you don't pass whitespace and comments etc. to the parser on a hidden channel (i.e., you set $channel instead of calling skip() in lexer rules.)
Definition at line 48 of file CommonTokenStream.java.
Constructor & Destructor Documentation
| org.antlr.runtime.CommonTokenStream.CommonTokenStream | ( | ) |
Definition at line 52 of file CommonTokenStream.java.
| org.antlr.runtime.CommonTokenStream.CommonTokenStream | ( | TokenSource | tokenSource | ) |
Definition at line 54 of file CommonTokenStream.java.
| org.antlr.runtime.CommonTokenStream.CommonTokenStream | ( | TokenSource | tokenSource, | |
| int | channel | |||
| ) |
Definition at line 58 of file CommonTokenStream.java.
Member Function Documentation
| void org.antlr.runtime.CommonTokenStream.consume | ( | ) |
Always leave p on an on-channel token.
Reimplemented from org.antlr.runtime.BufferedTokenStream.
Definition at line 64 of file CommonTokenStream.java.
| Token org.antlr.runtime.CommonTokenStream.LB | ( | int | k | ) | [protected] |
Reimplemented from org.antlr.runtime.BufferedTokenStream.
Definition at line 74 of file CommonTokenStream.java.
| Token org.antlr.runtime.CommonTokenStream.LT | ( | int | k | ) |
Get Token at current input pointer + i ahead where i=1 is next Token. i<0 indicates tokens in the past. So -1 is previous token and -2 is two tokens ago. LT(0) is undefined. For i>=n, return Token.EOFToken. Return null for LT(0) and any index that results in an absolute address that is negative.
Reimplemented from org.antlr.runtime.BufferedTokenStream.
Definition at line 89 of file CommonTokenStream.java.
| int org.antlr.runtime.CommonTokenStream.skipOffTokenChannels | ( | int | i | ) | [protected] |
Given a starting index, return the index of the first on-channel token.
Definition at line 109 of file CommonTokenStream.java.
| int org.antlr.runtime.CommonTokenStream.skipOffTokenChannelsReverse | ( | int | i | ) | [protected] |
Definition at line 118 of file CommonTokenStream.java.
| void org.antlr.runtime.CommonTokenStream.setup | ( | ) | [protected] |
Reimplemented from org.antlr.runtime.BufferedTokenStream.
Definition at line 125 of file CommonTokenStream.java.
| int org.antlr.runtime.CommonTokenStream.getNumberOfOnChannelTokens | ( | ) |
Count EOF just once.
Definition at line 137 of file CommonTokenStream.java.
| void org.antlr.runtime.CommonTokenStream.setTokenSource | ( | TokenSource | tokenSource | ) |
Reset this token stream by setting its token source.
Reimplemented from org.antlr.runtime.BufferedTokenStream.
Definition at line 149 of file CommonTokenStream.java.
Member Data Documentation
int org.antlr.runtime.CommonTokenStream.channel = Token.DEFAULT_CHANNEL [protected] |
Skip tokens on any channel but this one; this is how we skip whitespace...
Definition at line 50 of file CommonTokenStream.java.
The documentation for this class was generated from the following file:
- /Users/parrt/antlr/code/antlr/main/runtime/Java/src/main/java/org/antlr/runtime/CommonTokenStream.java
