<div dir="ltr">I am having similar issues - I'm having to encode the parser state into the lexer. This is because I have character sequences that are subsets of one another, etc. that should only match in certain places. The high level parser rules determine this very nicely - but I have to explicitly push down those rules into the lexer using predicates.<br>
<br>I was wondering myself if it would be possible to automate this process.<br><br>Matt.<br><br><div class="gmail_quote">On Sun, Aug 3, 2008 at 7:34 PM, Foust <span dir="ltr"><<a href="mailto:javafoust@gmail.com">javafoust@gmail.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;"><div class="Ih2E3d">> At 9:40pm, August 02, 2008 Gavin wrote:<br>
><br>
> At 11:06 3/08/2008, Foust wrote:<br>
</div><div class="Ih2E3d">> >Do lexer predicates work in v3?<br>
><br>
</div><div class="Ih2E3d">> That depends on what you mean. You can certainly use both<br>
> syntactic and semantic predicates within the lexer, but they can<br>
> only use lexer state.<br>
<br>
</div>That would explain why setting a static flag in the Lexer from the Parser<br>
has no effect -- the Lexer has already run to completion before the parser<br>
receives the first token.<br>
<div class="Ih2E3d"><br>
><br>
> Also, while I'm not entirely sure about this, I think predicates<br>
> in the lexer can only be used to decide between alts within a<br>
> single lexer rule. I vaguely recall some trouble when trying to<br>
> use them to decide between multiple lexer rules (at the top<br>
> level).<br>
<br>
</div>I'll keep that in mind. I've had nothing but trouble trying to get the Lexer<br>
to return tokens based on context (as best determined by the Parser).<br>
<div class="Ih2E3d"><br>
<br>
> Generally speaking, you should keep your lexer fairly<br>
> straightforward and unambiguous, and defer semantic decisions (and<br>
> ambiguity resolution) until the parsing phase.<br>
<br>
</div>Yes... it started out that way. But to allow spaces to be part of a config<br>
value (read up to EOL), the Lexer needs to honor state. (Place spaces in the<br>
HIDDEN channel for all other cases - outside of a special<br>
config/preprocessor rule).<br>
<br>
The parser shouldn't have to go through contortions because of lexer design.<br>
In fact, it seems as though the lexer itself is fine, if only it would get<br>
tokens as required, rather than all at once.<br>
<br>
Might I suggest the lexer be endowed with a state mechanism that can be<br>
controlled from the parser?<br>
<br>
<br>
</blockquote></div><br></div>