[#ANTLR-130] ANTLR doesn't finish

ANTLR v3

ANTLR doesn't finish

Created: 05/Jun/07 02:22 PM Updated: 27/Dec/07 12:33 PM

Component/s:

analysis

Affects Version/s:

3.0

Fix Version/s:

3.1

File Attachments:

RFC3986.g (4 kb)

Description

« Hide

Wincent Colaiuta says:

tp://pastie.textmate.org/68006/download

A bit more playing with the commandline I discovered the "-report", "-X" and "-Xdbgconversion" switches... Interestingly, if I pass the "-Xdbgconversion" switch, then ANTLR finishes in about 10 seconds and there is no out-of-memory exception. The generated lexer (in C) is huge though (nearly 300,000 lines). GCC compiles it without complaining.

But I think the size of the lexer indicates something bad is going on in the grammar. No idea why ANTLR would run out of memory without the...

I removed the DEC_OCTET and IPV6_ADDRESS, and it worked fine. With
either of them it ran into the same problems again. Maybe check these
two out.

All

Comments

Change History

Sort Order:

Ascending order - Click to sort in descending order

[ Permlink | « Hide ]

Terence Parr - 05/Jun/07 02:24 PM

Matt Diehl said:

I removed the DEC_OCTET and IPV6_ADDRESS, and it worked fine. With
either of them it ran into the same problems again. Maybe check these
two out.

[ Permlink | « Hide ]

Terence Parr - 06/Jun/07 11:50 AM

Ron Blaschke says:

Here are a few random observations that may help:

* The IPV6_ADDRESS and subrules are enough to make the lexer generator
choke really hard

* With enough heap memory (2GB+) there's some progress. Memory usage
goes up to about 1.8 GB, falls down to less than 100MB and creeps up to
1.8 GB again, and repeats.

* With heap limited to 100MB ANTLR dies with OutOfMemoryError. Looking
at the memory dump almost all heap memory is retained by 128,000+
StringTemplate objects. With 1GB heap there are 660,000+ StringTemplate
objects, retaining almost the entire heap.

[ Permlink | « Hide ]

Terence Parr - 27/Dec/07 12:33 PM

acyclic DFA generation inline can still explode because a single state may be reachable from multiple other states. Each pass to that state regenerates all the subsequent edges. There is still a very large expansion in the number of states generated. Gets huge. Solution was to introduce option:

-Xmaxinlinedfastates m max DFA states before table used rather than inlining

and set default to 10 states.