Class UnbufferedCharStream
- java.lang.Object
-
- org.antlr.v4.runtime.UnbufferedCharStream
-
- All Implemented Interfaces:
CharStream,IntStream
public class UnbufferedCharStream extends Object implements CharStream
Do not buffer up the entire char stream. It does keep a small buffer for efficiency and also buffers while a mark exists (set by the lookahead prediction in parser). "Unbuffered" here refers to fact that it doesn't buffer all data, not that's it's on demand loading of char. Before 4.7, this class used the default environment encoding to convert bytes to UTF-16, and held the UTF-16 bytes in the buffer as chars. As of 4.7, the class uses UTF-8 by default, and the buffer holds Unicode code points in the buffer as ints.
-
-
Field Summary
Fields Modifier and Type Field Description protected intcurrentCharIndexAbsolute character index.protected int[]dataA moving window buffer of the data being scanned.protected Readerinputprotected intlastCharThis is theLA(-1)character for the current position.protected intlastCharBufferStartprotected intnThe number of characters currently indata.StringnameThe name or source of this char stream.protected intnumMarkersprotected intp0..n-1 index intodataof next character.-
Fields inherited from interface org.antlr.v4.runtime.IntStream
EOF, UNKNOWN_SOURCE_NAME
-
-
Constructor Summary
Constructors Constructor Description UnbufferedCharStream()Useful for subclasses that pull char from other than this.input.UnbufferedCharStream(int bufferSize)Useful for subclasses that pull char from other than this.input.UnbufferedCharStream(InputStream input)UnbufferedCharStream(InputStream input, int bufferSize)UnbufferedCharStream(InputStream input, int bufferSize, Charset charset)UnbufferedCharStream(Reader input)UnbufferedCharStream(Reader input, int bufferSize)
-
Method Summary
All Methods Instance Methods Concrete Methods Modifier and Type Method Description protected voidadd(int c)voidconsume()Consumes the current symbol in the stream.protected intfill(int n)Addncharacters to the buffer.protected intgetBufferStartIndex()StringgetSourceName()Gets the name of the underlying symbol source.StringgetText(Interval interval)This method returns the text for a range of characters within this input stream.intindex()Return the index into the stream of the input symbol referred to byLA(1).intLA(int i)Gets the value of the symbol at offsetifrom the current position.intmark()Return a marker that we can release later.protected intnextChar()Override to provide different source of characters thaninput.voidrelease(int marker)Decrement number of markers, resetting buffer if we hit 0.voidseek(int index)Seek to absolute character index, which might not be in the current sliding window.intsize()Returns the total number of symbols in the stream, including a single EOF symbol.protected voidsync(int want)Make sure we have 'need' elements from current positionp.
-
-
-
Field Detail
-
data
protected int[] data
A moving window buffer of the data being scanned. While there's a marker, we keep adding to buffer. Otherwise,consume()resets so we start filling at index 0 again.
-
n
protected int n
The number of characters currently indata.This is not the buffer capacity, that's
data.length.
-
p
protected int p
0..n-1 index intodataof next character.The
LA(1)character isdata[p]. Ifp == n, we are out of buffered characters.
-
numMarkers
protected int numMarkers
-
lastChar
protected int lastChar
This is theLA(-1)character for the current position.
-
lastCharBufferStart
protected int lastCharBufferStart
WhennumMarkers > 0, this is theLA(-1)character for the first character indata. Otherwise, this is unspecified.
-
currentCharIndex
protected int currentCharIndex
Absolute character index. It's the index of the character about to be read viaLA(1). Goes from 0 to the number of characters in the entire stream, although the stream size is unknown before the end is reached.
-
input
protected Reader input
-
name
public String name
The name or source of this char stream.
-
-
Constructor Detail
-
UnbufferedCharStream
public UnbufferedCharStream()
Useful for subclasses that pull char from other than this.input.
-
UnbufferedCharStream
public UnbufferedCharStream(int bufferSize)
Useful for subclasses that pull char from other than this.input.
-
UnbufferedCharStream
public UnbufferedCharStream(InputStream input)
-
UnbufferedCharStream
public UnbufferedCharStream(Reader input)
-
UnbufferedCharStream
public UnbufferedCharStream(InputStream input, int bufferSize)
-
UnbufferedCharStream
public UnbufferedCharStream(InputStream input, int bufferSize, Charset charset)
-
UnbufferedCharStream
public UnbufferedCharStream(Reader input, int bufferSize)
-
-
Method Detail
-
consume
public void consume()
Description copied from interface:IntStreamConsumes the current symbol in the stream. This method has the following effects:- Forward movement: The value of
index()before calling this method is less than the value ofindex()after calling this method. - Ordered lookahead: The value of
LA(1)before calling this method becomes the value ofLA(-1)after calling this method.
index()is incremented by exactly 1, as that would preclude the ability to implement filtering streams (e.g.CommonTokenStreamwhich distinguishes between "on-channel" and "off-channel" tokens). - Forward movement: The value of
-
sync
protected void sync(int want)
Make sure we have 'need' elements from current positionp. Last validpindex isdata.length-1.p+need-1is the char index 'need' elements ahead. If we need 1 element,(p+1-1)==pmust be less thandata.length.
-
fill
protected int fill(int n)
Addncharacters to the buffer. Returns the number of characters actually added to the buffer. If the return value is less thann, then EOF was reached beforencharacters could be added.
-
nextChar
protected int nextChar() throws IOExceptionOverride to provide different source of characters thaninput.- Throws:
IOException
-
add
protected void add(int c)
-
LA
public int LA(int i)
Description copied from interface:IntStreamGets the value of the symbol at offsetifrom the current position. Wheni==1, this method returns the value of the current symbol in the stream (which is the next symbol to be consumed). Wheni==-1, this method returns the value of the previously read symbol in the stream. It is not valid to call this method withi==0, but the specific behavior is unspecified because this method is frequently called from performance-critical code.This method is guaranteed to succeed if any of the following are true:
i>0i==-1andindex()returns a value greater than the value ofindex()after the stream was constructed andLA(1)was called in that order. Specifying the currentindex()relative to the index after the stream was created allows for filtering implementations that do not return every symbol from the underlying source. Specifying the call toLA(1)allows for lazily initialized streams.LA(i)refers to a symbol consumed within a marked region that has not yet been released.
If
irepresents a position at or beyond the end of the stream, this method returnsIntStream.EOF.The return value is unspecified if
i<0and fewer than-icalls toconsume()have occurred from the beginning of the stream before calling this method.
-
mark
public int mark()
Return a marker that we can release later.The specific marker value used for this class allows for some level of protection against misuse where
seek()is called on a mark orrelease()is called in the wrong order.
-
release
public void release(int marker)
Decrement number of markers, resetting buffer if we hit 0.- Specified by:
releasein interfaceIntStream- Parameters:
marker-- See Also:
IntStream.mark()
-
index
public int index()
Description copied from interface:IntStreamReturn the index into the stream of the input symbol referred to byLA(1).The behavior of this method is unspecified if no call to an
initializing methodhas occurred after this stream was constructed.
-
seek
public void seek(int index)
Seek to absolute character index, which might not be in the current sliding window. Moveptoindex-bufferStartIndex.
-
size
public int size()
Description copied from interface:IntStreamReturns the total number of symbols in the stream, including a single EOF symbol.
-
getSourceName
public String getSourceName()
Description copied from interface:IntStreamGets the name of the underlying symbol source. This method returns a non-null, non-empty string. If such a name is not known, this method returnsIntStream.UNKNOWN_SOURCE_NAME.- Specified by:
getSourceNamein interfaceIntStream
-
getText
public String getText(Interval interval)
Description copied from interface:CharStreamThis method returns the text for a range of characters within this input stream. This method is guaranteed to not throw an exception if the specifiedintervallies entirely within a marked range. For more information about marked ranges, seeIntStream.mark().- Specified by:
getTextin interfaceCharStream- Parameters:
interval- an interval within the stream- Returns:
- the text of the specified interval
-
getBufferStartIndex
protected final int getBufferStartIndex()
-
-