Mantra pipe speed improvement

Skip to end of sidebar
Go to start of sidebar
Skip to end of metadata
Go to start of metadata

I replaced my blocking queue sitting between pipeline actors (threaded consumer/producers) and speed of my word freq program dropped from 5.2s to 3.5s on 5M file simply by replacing the blocking queue with my new ping-pong buffered version. That is close to the 3.0s achieved by the nonthreaded version.

Buffer size is 1000. When I drop to 100, slows down by .2s. When I increase to 4000, no change. 40000, slows down by .5s. Size 400 seems to be slightly faster. Default queue size set to 400 now.

Thanks to Sriram Srinivasan for the idea of ping-ponging the buffers.

Labels:
None
Enter labels to add to this page:
Please wait 
Looking for a label? Just start typing.
  1. Oct 18, 2007

    Ter,
    No offence mate, but I'm wondering why we're getting so many Mantra postings on the ANTLR wiki? When it directly relates to ANTLR no problem, but otherwise it seems to be getting off-topic ... ?

    Cheers,
    Jonathan.

  2. Oct 18, 2007

    Jonathan,

    That's just because Mantra is a big (probably the biggest) open source project that uses v3 at the moment. And besides, the postings are in the blog section of Ter, AFAICS.

    cheers,
    -k