Dashboard > Labels > tokens
  Labels Log In | Sign Up   View a printable version of the current page.  
  Label: tokens
View: Popular Labels | All Labels

All content with label tokens.
Related Labels: buffering, channels, rewriting

How do I fetch tokens on demand not all at once up front? (ANTLR 3)
CommonTokenStream sucks in all tokens up front until EOF. This works fine for files, but does not work for interactive programs or programs sucking from a socket. I will add a class to handle this to v3.2 probably. In the meantime, here is one I'm using ...
Other labels: buffering
How can I emit more than a single token per lexer rule? (ANTLR 3)
Here is a rule to match floats: NUMFLOAT : DIGITS '.' (DIGITS)? (EXPONENTPART)? (FLOATTYPESUFFIX)? '.' DIGITS (EXPONENTPART)? (FLOATTYPESUFFIX)? DIGITS EXPONENTPART FLOATTYPESUFFIX DIGITS EXPONENTPART DIGITS FLOATTYPESUFFIX ; Now if you want to add '..' range operator so 1..10 makes sense ...
Token stream rewriting with rewrite rules (Terence Parr)
Let's talk tree and token stream rewriting. Starting with simple token rewriting. I've thought about this and I think we can fold rewriting patterns into grammar composition and token stream stuff. Here's a simple rule that just replaces ...
Other labels: rewriting, channels
Site powered by a free Open Source Project / Non-profit License (more) of Confluence - the Enterprise wiki.
Learn more or evaluate Confluence for your organisation.
Powered by Atlassian Confluence, the Enterprise Wiki. (Version: 2.5.1 Build:#806 May 06, 2007) - Bug/feature request - Contact Administrators