[stringtemplate-interest] PyST: first() and rest() don’t work on iterable non-lists

[Benjamin Niemann]

Created ST-43, so it does get forgotten.

On Sat, Sep 13, 2008 at 1:05 PM, Benjamin Niemann <pink at odahoda.de> wrote:
> Hi Roman,
>
> On Mon, Sep 8, 2008 at 11:03 AM, Roman Odaisky <roma at qwertty.com> wrote:
>> Hi,
>>
>> Until someone points me to a PyST bugtracker, I'll spam this list ;-)
>
> http://www.antlr.org:8888/browse/ST
>
>> In PyST 3.1, $first(something)$ and $rest(something)$ only work on built-in
>> types. When I pass an iterable object, the functions seem to treat it as a
>> whole. However, the template application operator does recognize them, so the
>> result looks like this:
>>
>>>>> st = StringTemplate(r'''$first(list):{<$it$>}$; $rest(list):{($it$)}$''')
>>>>> st["list"] = xrange(0, 5)
>>>>> print unicode(st)
>> <0><1><2><3><4>;
>>
>> while I was expecting:
>>
>> <0>; (1)(2)(3)(4)
>>
>> Proper handling of iterable objects is very important.
>
> Yes, the way it works now is very unpythonic. The original Java
> implementation uses a lot a class checking to decide how to handle
> objects and the current Python implementation mostly mimics this
> behavior. This doesn't work properly in Python where the various
> iterables don't share a common base class, so you're constrained to
> use list()s to get the desired semantics.
>
>> For example, in my case
>> the objects are database result sets, and fetching them in bite-size portions
>> is much more efficient than getting all the data into a Python list and
>> passing it to ST, as I'm forced to do.
>>
>> Perhaps it's possible to return a generator from language.ASTExpr.rest?
>
> There was an attempt to use generators to make things more efficient,
> but it turned out to open a set of bugs (i.e. if you expand an
> iterator twice, the second invocation doesn't yield a result, because
> the iterator is already exhausted by the first invocation). Not sure
> if this can be fixed in a sane way.
>
> -Ben
>