I replaced my blocking queue sitting between pipeline actors (threaded consumer/producers) and speed of my word freq program dropped from 5.2s to 3.5s on 5M file simply by replacing the blocking queue with my new ping-pong buffered version. That is close to the 3.0s achieved by the nonthreaded version.
Buffer size is 1000. When I drop to 100, slows down by .2s. When I increase to 4000, no change. 40000, slows down by .5s. Size 400 seems to be slightly faster. Default queue size set to 400 now.
Thanks to Sriram Srinivasan for the idea of ping-ponging the buffers.
Labels:
None
2 Comments
comments.show.hideOct 18, 2007
Jonathan Thomas
Ter,
No offence mate, but I'm wondering why we're getting so many Mantra postings on the ANTLR wiki? When it directly relates to ANTLR no problem, but otherwise it seems to be getting off-topic ... ?
Cheers,
Jonathan.
Oct 18, 2007
Kay Röpke
Jonathan,
That's just because Mantra is a big (probably the biggest) open source project that uses v3 at the moment. And besides, the postings are in the blog section of Ter, AFAICS.
cheers,
-k