[stringtemplate-interest] pystringtemplate 2.2 UnicodeDecodeError -- been fixed?

[Bob Foltrigg]

I'm resuming this thread because I'm still experiencing issues regarding
unicode and PyStringTemplate -- however I have managed to observe some
progress. As mentioned in my original mail below, I could go pretty far to
have things look fine and dandy, but it turned out it's only as smooth as
long I run my app in CPython. As soon as I switch over to Jython and present
bindings for my templates in something other than ascii, things break down
again, and I get the dreaded UnicodeDecodeError. The affecting spot is line
605 in stringtemplate3/language/AST3Expr.py:

(
602:        if renderer is not None:
603:            v = renderer.toString(o, self.formatString)
604:        else:
605:            v = unicode(o)
)

I'm aware that this may well be labeled as a Jython issue, but still I think
that PyStringTemplate is a bit too insisting on blindly converting
everything to the type unicode, while Jython's str is unicode-aware by
default, which means Jython strings suffer a double decoding inside PyST.
This is just a weak assumption, maybe somebody with more experience in the
PyST/encodings/Jython area could shed some light on the issue.

On a different note: the unit tests in the package seem to be broken for me
(>20 error on both CPython and JPython). Am I missing something or is this
the current state of affairs? If so, then is there any undergoing effort to
fix them, or could I be of help and start to plow thru it?

Thank you for any help,
-Bob. 

> -----Original Message-----
> From: bob foltrigg [mailto:bob.foltrigg at gmail.com] 
> Sent: 2008. június 16. 20:39
> To: Benjamin Niemann
> Cc: stringtemplate-interest at antlr.org
> Subject: Re: [stringtemplate-interest] pystringtemplate 2.2 
> UnicodeDecodeError -- been fixed?
> 
> Hi,
> thank you for the quick answer. I'm using V3.1 and have the 
> problems described. I can get around the bug right now, 
> because my templates are HTMLs and can substitute characters 
> for HTML entites, but in the future I'd like to see proper 
> Unicode support in PyST. Haven't investigated the problem 
> myself either but I'm ready to contribute whatever I can come 
> up with if it can help.
> -Bob.
> 
> On Mon, Jun 16, 2008 at 5:04 PM, Benjamin Niemann 
> <pink at odahoda.de> wrote:
> > Hi Bob,
> >
> > I upgraded stringtemplate for Python to V3.1. I haven't looked too 
> > deep into unicode support though.
> > Are you still having these problems with 3.1?
> >
> > -Ben
> >
> >
> > On Mon, Jun 16, 2008 at 4:48 PM, bob foltrigg 
> <bob.foltrigg at gmail.com> wrote:
> >> Hi all,
> >> It seems like I'm affected by the same error as Bohumil 
> Simcik (was) 
> >> as mentioned by him in a posting sent to this list last 
> May (archived 
> >> under 
> >> 
> http://www.antlr.org:8080/pipermail/stringtemplate-interest/2007-May/
> >> 001052.html) Terence's response was that the source of the bug is 
> >> caused by PyST lagging behind the other versions (see next 
> message by 
> >> thread after the previous one in the archives).
> >> My question would be whether this failure has been appointed to 
> >> anybody since then, if not, then what steps should I take 
> be allowed 
> >> to submit my solution for it.
> >> Thank You,
> >> -Bob.
> >> _______________________________________________
> >> stringtemplate-interest mailing list
> >> stringtemplate-interest at antlr.org
> >> http://www.antlr.org:8080/mailman/listinfo/stringtemplate-interest
> >>
> >
>