[stringtemplate-interest] pystringtemplate 2.2 UnicodeDecodeError -- been fixed?

[Benjamin Niemann]

Hi Bob,

I'm currently working on getting ST ready for the next release.
Correct unicode handling one of the things, I want to get working.

On Sun, Aug 17, 2008 at 2:51 PM, Bob Foltrigg <bob.foltrigg at gmail.com> wrote:
> I'm resuming this thread because I'm still experiencing issues regarding
> unicode and PyStringTemplate -- however I have managed to observe some
> progress. As mentioned in my original mail below, I could go pretty far to
> have things look fine and dandy, but it turned out it's only as smooth as
> long I run my app in CPython. As soon as I switch over to Jython and present
> bindings for my templates in something other than ascii, things break down
> again, and I get the dreaded UnicodeDecodeError. The affecting spot is line
> 605 in stringtemplate3/language/AST3Expr.py:
>
> (
> 602:        if renderer is not None:
> 603:            v = renderer.toString(o, self.formatString)
> 604:        else:
> 605:            v = unicode(o)
> )
>
> I'm aware that this may well be labeled as a Jython issue, but still I think
> that PyStringTemplate is a bit too insisting on blindly converting
> everything to the type unicode, while Jython's str is unicode-aware by
> default, which means Jython strings suffer a double decoding inside PyST.
> This is just a weak assumption, maybe somebody with more experience in the
> PyST/encodings/Jython area could shed some light on the issue.

I have no experience with Jython, so I can't tell how hard it is to
get this working right. If I find some time, I may look into it, but
no promises.

> On a different note: the unit tests in the package seem to be broken for me
> (>20 error on both CPython and JPython). Am I missing something or is this
> the current state of affairs? If so, then is there any undergoing effort to
> fix them, or could I be of help and start to plow thru it?

Can you be a bit more specific? I usually make sure that the tests
pass 100% before submitting.

-Ben

>> -----Original Message-----
>> From: bob foltrigg [mailto:bob.foltrigg at gmail.com]
>> Sent: 2008. június 16. 20:39
>> To: Benjamin Niemann
>> Cc: stringtemplate-interest at antlr.org
>> Subject: Re: [stringtemplate-interest] pystringtemplate 2.2
>> UnicodeDecodeError -- been fixed?
>>
>> Hi,
>> thank you for the quick answer. I'm using V3.1 and have the
>> problems described. I can get around the bug right now,
>> because my templates are HTMLs and can substitute characters
>> for HTML entites, but in the future I'd like to see proper
>> Unicode support in PyST. Haven't investigated the problem
>> myself either but I'm ready to contribute whatever I can come
>> up with if it can help.
>> -Bob.
>>
>> On Mon, Jun 16, 2008 at 5:04 PM, Benjamin Niemann
>> <pink at odahoda.de> wrote:
>> > Hi Bob,
>> >
>> > I upgraded stringtemplate for Python to V3.1. I haven't looked too
>> > deep into unicode support though.
>> > Are you still having these problems with 3.1?
>> >
>> > -Ben
>> >
>> >
>> > On Mon, Jun 16, 2008 at 4:48 PM, bob foltrigg
>> <bob.foltrigg at gmail.com> wrote:
>> >> Hi all,
>> >> It seems like I'm affected by the same error as Bohumil
>> Simcik (was)
>> >> as mentioned by him in a posting sent to this list last
>> May (archived
>> >> under
>> >>
>> http://www.antlr.org:8080/pipermail/stringtemplate-interest/2007-May/
>> >> 001052.html) Terence's response was that the source of the bug is
>> >> caused by PyST lagging behind the other versions (see next
>> message by
>> >> thread after the previous one in the archives).
>> >> My question would be whether this failure has been appointed to
>> >> anybody since then, if not, then what steps should I take
>> be allowed
>> >> to submit my solution for it.
>> >> Thank You,
>> >> -Bob.
>> >> _______________________________________________
>> >> stringtemplate-interest mailing list
>> >> stringtemplate-interest at antlr.org
>> >> http://www.antlr.org:8080/mailman/listinfo/stringtemplate-interest
>> >>
>> >
>>
>
> _______________________________________________
> stringtemplate-interest mailing list
> stringtemplate-interest at antlr.org
> http://www.antlr.org:8080/mailman/listinfo/stringtemplate-interest
>