History | Log In     View a printable version of the current page.  
Issue Details (XML | Word | Printable)

Key: ANTLR-235
Type: Improvement Improvement
Status: Closed Closed
Resolution: Fixed
Priority: Major Major
Assignee: Terence Parr
Reporter: Terence Parr
Votes: 0
Watchers: 0
Operations

If you were logged in you would be able to see more operations.
ANTLR v3

12-31.14 release is 10x faster than 12-31.17

Created: 25/Mar/08 12:51 PM   Updated: 05/May/08 06:06 PM
Component/s: analysis
Affects Version/s: None
Fix Version/s: 3.1


 Description  « Hide
Mark Wright says:

My favourite pre-release is antlr-2007-12-31.14.
antlr-2007-12-31.14 is more than 10 times faster than the
antlr-2007-12-31.17 pre-release and all subsequent pre-releases up
to the latest I could download antlr-2008-02-05.15 (fisheye never
works for me).

antlr-2007-12-28.10 was the first pre-release that can compile my
grammar. I just tried ANTLR 3.0.1, it runs for 6 minutes before
running out of heap space with -Xmx512m.

So yes: antlr-2007-12-28.10 to antlr-2007-12-31.14 were really great
pre-releases, they compile my grammar faster that ANTLR 3.0.1 and
current pre-releases:

25416 ms - antlr-2007-12-31.14 - love this pre-release!
361301 ms - antlr-2007-12-31.17 - first release with the change.
405494 ms - antlr-2008-02-05.15 - its great that it compiles it.

There is a comment added to the CHANGES.TXT file between the
antlr-2007-12-31.14 and antlr-2007-12-31.17 pre-releases:

goanna% diff -wc antlr-2007-12-31.14/CHANGES.txt
antlr-2007-12-31.17/CHANGES.txt *** antlr-2007-12-31.14/CHANGES.txt
Tue Jan 1 09:03:10 2008 --- antlr-2007-12-31.17/CHANGES.txt Tue
Jan 1 12:10:04 2008 ***************
*** 10,15 ****
--- 10,24 ----

December 31, 2007

+ * Added the start of a semantic predicate computation for LL(1)
to solve
+ a problem with slow grammar analysis even with k=1 due to
predicates.
+ Then I realized the problem with that grammar was elsewhere.
+ Semantic context really shouldn't be used when preventing
closure
+ recomputation. The predicates became huge even though the
reduced
+ value would be no different. The analyzer seems faster now that
+ I am not testing predicate values all the time. Further it may
+ terminate sooner just due to reduced closure recursion.
+
* Moved FIRST/FOLLOW computations to a separate class LL1Analyzer
to tidy up.

goanna%

I was crazy enough to be working on my grammar on New Years Eve as
well.

The output for antlr-2007-12-31.17 pre-release is below, the outputs
for antlr-2007-12-31.14 and antlr-2008-02-05.15 were in my last
email.

Thanks, Mark

Compilation started at Tue Feb 12 02:42:17

antlr-2007-12-31.17
-------------------

gmake p
java -Xmx512m
-classpath /h/goanna/2/eng/dev/tntdbo/java_src:/h/goanna/2/eng/dev/
tntdbo:/h/goanna/2/ts/antlr/antlr-2007-12-31.17/lib/
antlr-2007-12-31.17.jar:/h/goanna/2/ts/antlr/antlr-2007-12-31.17/lib/
runtime-2007-12-31.17.jar:/h/goanna/2/ts/antlr/antlr-2007-12-31.17/
lib/stringtemplate-3.1b1.jar:/h/goanna/2/ts/antlr/
antlr-2007-12-31.17/lib/antlr-2.7.7.jar
org.antlr.Tool -Xconversiontimeout 600000 -report Tntdbo.g ANTLR
Parser
Generator Version 3.1b1 (??) 1989-2007 ANTLR Grammar Report; Stats
Version 4 Grammar: Tntdbo Type: combined
Target language: Java
Output: AST
Grammar option k: none
Grammar option backtrack: false
Rules: 193
Productions: 484
Decisions: 266
Cyclic DFA decisions: 0
LL(1) decisions: 167
Min fixed k: -1
Max fixed k: 5
Average fixed k: 0.462406015037594
Standard deviation of fixed k: 1.060658499894118
Min acyclic DFA states: 0
Max acyclic DFA states: 88
Average acyclic DFA states: 8.101503759398497
Standard deviation of acyclic DFA states: 15.430948979548202
Total acyclic DFA states: 2155
Min cyclic DFA states: 0
Max cyclic DFA states: 0
Average cyclic DFA states: 0.0
Standard deviation of cyclic DFA states: 0.0
Total cyclic DFA states: 0
Vocabulary size: 297
DFA creation time in ms: 361301
Number of semantic predicates found: 149
Number of manual fixed lookahead k=value options: 68
Number of nondeterministic decisions: 109
Number of nondeterministic decisions resolved with predicates: 109
Number of DFA conversions terminated early: 0
Number of errors: 0
Number of warnings: 0
Number of infos: 0
Number of syntactic predicates found: 0
Decisions with syntactic predicates: 0
Decision DFAs using syntactic predicates: 0
Decisions with semantic predicates: 69
Decision DFAs using semantic predicates: 109

Backtracking report:
Number of decisions that backtrack: 0

NFA conversion early termination report:
Number of NFA conversions that terminated early: 0


Compilation finished at Tue Feb 12 02:48:32



 All   Comments   Change History      Sort Order: Ascending order - Click to sort in descending order
Terence Parr - 02/May/08 05:09 PM
same as ANTLR-218

Terence Parr - 05/May/08 05:29 PM
I commented out pred ctx check on 12-31-07

// Check epsilon cycle (same state, same alt, same context)
for (int i = 0; i < numConfigs; i++) {
NFAConfiguration c = (NFAConfiguration) d.closureBusy.get(i);
if ( proposedNFAConfiguration.state==c.state &&
proposedNFAConfiguration.alt==c.alt &&
// proposedNFAConfiguration.semanticContext.equals(c.semanticContext) &&
proposedNFAConfiguration.context.suffix(c.context) )
{
return true;
}
}

probably bad idea.

Terence Parr - 05/May/08 05:33 PM
Added unit that fails until i uncomment that line:
 TestSemanticPredicates.testSemanticContextPreventsEarlyTerminationOfClosure()


Terence Parr - 05/May/08 06:06 PM
Changed DFAState.closureBusy back to Set from list and make closureBusy a hash lookup from linear search.