ANTLR v3.3

November 29, 2010

Terence Parr
ANTLR project lead and supreme dictator for life
University of San Francisco
Credits

ANTLR v3.3 is primarily a bug fix release but has 2 new targets.

New targets!

Kyle Yetter has built a new and complete Ruby target for 3.3. More info.
Ruby target examples.

Alan Condit has built an Objective-C target for 3.3 following on Kay Roepke's work.

Removed conversion timeout failsafe; no longer needed really. If the unlikely occurs and ANTLR spins forever, kill it and run again with -Xwatchconversion. You'll see things like:

If it gets stuck, look at the decision (line 109 here) and add k=1 and backtracking or something.
Stats/profiling updated to be correct for -report and -profile but the output is very different. -report:

And -profile:
Doesn't write profile data to file anymore; emits decision data to stderr
There are Java generics in ANTLR itself and runtime; shouldn't be a problem because we generate Java 1.4 compatible binaries.
The DebugEventListener interface changed [BREAKS BACKWARD COMPATIBILITY]

Added source name to syntax error msgs
added new method to get subset of tokens to buffered token streams:
public List get(int start, int stop);
Refs to other tokens in a lexer rule didn't get its line/charpos right. altered Java.stg.
Instead of sharing Token.EOF_TOKEN, I'm now creating EOF tokens so I can set the char position for better error messages.
added new buffered on-demand streams: BufferedTokenStream. Renamed CommonTokenStream to LegacyCommonTokenStream and made new one as subclass of BufferedTokenStream.
Added org.antlr.runtime.UnbufferedTokenStream. Was trivial and works!
added range to TokenStream and implementors:
Added MachineProbe class to make it easier to highlight ambig paths in
grammar. More accurate than DecisionProbe; retrofitted from v4.
lexerStringRef was missing elementIndex attribute. i='import' didn't work
in lexer. Altered all target stg files. Set in codegen.g
Added -Xsavelexer option
greedy=true option shuts off nondeterminism warning.
Tried to make output more deterministic:
- added toArray in OrderedHashSet to make addAll calls get same order for DFA edges and possibly code gen in some areas.
- Made OrderedHashSet have deterministic iteration

code gen for AST and -profile didn't compile. had useless line:
Missing -trace in help msg
Added boolean decisionCanBacktrack to Parser and enterDecision in dbg interface. Breaks AW
interface and other tools! [BREAKS BACKWARD COMPATIBILITY]

output=AST, rewrite=true for tree rewriters broken. nextNode for subtree
streams didn't dup node, it gave whole tree back.
Creating token from another token didn't copy input stream in CommonToken. makes sense to copy too; i don't think anybody relies on it being null after a copy. We might want to know where token came from.
TreeParser.getMissingSymbol() used CommonTree instead of using adaptor.create()
Fixed bug in TreeVisitor when rewrites altered number of children. Thanks to Chris DiGiano.
Couldn't properly reuse parser state; ctor reset the state; fixed.
Parser(TokenStream input, RecognizerSharedState state)
LookaheadStream<T> used some hardcoded Object return types for LT, etc...
uses T now.