|
All content with label faq_lexing.
Related Labels:
faq_translation, faq_parsing, c, faq_trees
How can I emit an error token upon lexical error?
(ANTLR 3)
v3.1 required (but change state to "this" and should be ok in 3.0.1) Sometimes you want the lexer to emit error tokens to the parser rather than consume the bad input and throw it away. To do this, we make a modification of FAQ entry How can ...
|
How can I make the lexer exit upon first lexical error?
(ANTLR 3)
v3.1 required (but change state to "this" and should be ok in 3.0.1) You need to override the default behavior which tries to recover and keep lexing. There are two approaches depending upon if you care about the lexer throwing a proper RecognitionException or not. If not, just override ...
|
How do I handle abbreviated keywords?
(ANTLR 3)
Given a language that accepts abbreviated forms of the following keywords: Keyword Abbreviations print {{pr}}, {{pri}}, {{prin}}, {{print}} read {{re}}, {{rea}}, {{read}} fold {{fo}}, {{fol}}, {{fold}} apply {{ap}}, {{app}}, {{appl}}, {{apply}} There are two basic strategies ...
|
How do I access hidden tokens from parser?
(ANTLR 3)
Rules in the lexer can send tokens to the parser on different "channels". See How do I track whitespace, comments, and other hidden channels during AST construction? For example, COMMENT : '/' . '/' ; But, how do you access them from the parser ...
Other labels:
faq_translation
|
How do I implement include files?
(ANTLR 3)
Stacked Input Streams "include" files Java Here is sample Javacode for implementation 'include' directive. Thanks Terence Parr for important notices and Tim Clark for improvements. @lexer::members { class SaveStruct { SaveStruct(CharStream input) public CharStream input ...
Other labels:
c
|
How do I use a custom token object?
(ANTLR 3)
All you have to do is override Lexer.emit() and define your own token class: grammar s; @lexer::members { public class MyToken extends CommonToken { int x; // custom field public MyToken(CharStream input, int type, int ...
|
How do I get case insensitivity?
(ANTLR 3)
Discussions There is some discussion about this subject (ANTLR v3) here: http://www.antlr.org:8080/pipermail/antlrinterest/2006May/016269.html In summary, there is no ANTLR option that enables case insensitivity, as this is hard or impossible to do completely correctly, taking into account all possible internationalization issues ...
|
How do I combine fuzzy parsing and stream rewriting?
(ANTLR 3)
you want to parse just pieces of, say, a Java file using filter mode (fuzzy parsing) and do some replacement like TokenRewriteStream does, how can you do it? TokenRewriteStream requires use of a parser/lexer combo whereas ...
Other labels:
faq_translation
|
How can I allow keywords as identifiers?
(ANTLR 3)
grammar allows "if if call call;" and "call if;". grammar Pred; prog: stat ; stat: keyIF expr stat keyCALL ID ';' ';' ; expr: ID ; keyIF : ? ID ; keyCALL : ? ID ; ID : 'a'..'z' ; WS : (' ' '\n') ; You can make those semantic ...
Other labels:
faq_parsing
|
How do I selectively ignore tokens depending on parser context?
(ANTLR 3)
Contributed by Edson Tirelli Edson told me that "for the most part of our source code file (the one we use antlr to parse), we want to ignore white space and line breaks, but for some special constructs, like java code ...
|
|
|
