Dashboard > People > Terence Parr > 2007 > October > 11 > Mantra pipe speed improvement
  Terence Parr Log In | Sign Up   View a printable version of the current page.  
  Mantra pipe speed improvement
Added by Terence Parr, last edited by Terence Parr on Oct 11, 2007  (view change)
Labels: 
(None)

I replaced my blocking queue sitting between pipeline actors (threaded consumer/producers) and speed of my word freq program dropped from 5.2s to 3.5s on 5M file simply by replacing the blocking queue with my new ping-pong buffered version. That is close to the 3.0s achieved by the nonthreaded version.

// threaded, pipeline version; 3.5s (from 5.2s) on 5M file
f => Words() => { string w | ... }
// nonthreaded, nested map operation on big list; 3.0s
f.lines() => a;
a:{ string line | line.split():{ string w | ... }};

Buffer size is 1000. When I drop to 100, slows down by .2s. When I increase to 4000, no change. 40000, slows down by .5s. Size 400 seems to be slightly faster. Default queue size set to 400 now.

Thanks to Sriram Srinivasan for the idea of ping-ponging the buffers.

Ter,
No offence mate, but I'm wondering why we're getting so many Mantra postings on the ANTLR wiki? When it directly relates to ANTLR no problem, but otherwise it seems to be getting off-topic ... ?

Cheers,
Jonathan.

Jonathan,

That's just because Mantra is a big (probably the biggest) open source project that uses v3 at the moment. And besides, the postings are in the blog section of Ter, AFAICS.

cheers,
-k

October 2007
Sun Mon Tue Wed Thu Fri Sat
  1 2 3 4 5 6
7 8 9 10 11 12 13
14 15 16 17 18 19 20
21 22 23 24 25 26 27
28 29 30 31      

Faster expression parsing for ANTLR
Mixins part deux

Site powered by a free Open Source Project / Non-profit License (more) of Confluence - the Enterprise wiki.
Learn more or evaluate Confluence for your organisation.
Powered by Atlassian Confluence, the Enterprise Wiki. (Version: 2.5.1 Build:#806 May 06, 2007) - Bug/feature request - Contact Administrators