[stringtemplate-interest] [FIX] ChunkTokens withoutLine/ColumnInformation? St#

[Terence Parr]

Hi. added a bug

http://www.antlr.org:8888/browse/ST-20

  thanks for the report.  And the fix! I will try to do a clean fix  
when I get back to ST.
Ter
On Mar 16, 2007, at 7:30 AM, Dreyer Ulf (CR/APA3) wrote:

>  Hello Terence, hello Kunle
>
>    whoops my Fix does not work as advertised:
>
>    The token   'abeg' is not always defined during ACTION.
>    This results in a NullPointerException if something like
>    if/else/endif or a region is parsed.
>
>   I made a new fix wich works but is kind of ugly:
>
> 1) Add a method to angle.bracket.template.g:
>
> protected IToken GetActionBeginToken(params IToken[] ptokens)
>         {
>             foreach (IToken token in ptokens)
>             {
>                 if (null != token) return token;
>             }
>             return null;
>         }
>
> 2) prefix the different occurences of "<"! in ACTION with
>    a1:"<"! through a5:"<"!
>
> 3) Change my first token-position fix to
>
> 	IToken aBegin = GetActionBeginToken(a1,a2,a3,a4,a5);
>       ctok.setLine(aBegin.getLine());
>       ctok.setColumn(aBegin.getColumn());
>
> Any Ideas to fix this that result in a cleaner grammar?
>
> -Ulf
>
>> -----Original Message-----
>> From: stringtemplate-interest-bounces at antlr.org
>> [mailto:stringtemplate-interest-bounces at antlr.org] On Behalf
>> Of Dreyer Ulf (CR/APA3)
>> Sent: Friday, March 16, 2007 1:49 PM
>> To: StringTemplate
>> Cc: Kunle_Odutola at hotmail.com
>> Subject: Re: [stringtemplate-interest] [FIX] ChunkTokens
>> withoutLine/ColumnInformation? St#
>>
>> Hello Terence, hello Kunle
>>
>>    it was a long day yesterday so I forgot to mention that only
>>    ChunkTokens (the ones that contain the actions) don't come with
>>    position information.
>>    Today I found out why and found a fix:
>>
>>    Why: ChunkTokens differ from the default tokens coming from the
>>         lexer and are created with new ChunkToken(....).
>>  	  Position information however is only set in
>> (lexer).makeToken(type).
>>
>> 	  I did not check whether the Java version or other
>> lexers are affected.
>>
>> Here is a sample fix for AngleBracketTemplateLexer. (Only the
>> ACTION Rule has to be changed)
>>
>> ACTION
>> options {
>>    		generateAmbigWarnings=false; // <EXPR> is ambig
>> with <!..!>
>> }
>> {
>>     int startCol = getColumn();
>> }
>> // --- [not listing rest of ACTION] ---
>> // ----------------------- original: -------------------------
>> 	|   '<'! EXPR '>'!
>>     	)
>>     	{
>>         ChunkToken ctok = new ChunkToken(_ttype, $getText,
>> currentIndent);
>>         $setToken(ctok);
>>     	}
>>     ;
>> 	
>> // -------------------------- corrected: ----------------------	
>> 	|   abeg:'<'! EXPR '>'!
>>     	)
>>     	{
>>         ChunkToken ctok = new ChunkToken(_ttype, $getText,
>> currentIndent);
>> 		//ctok is created without the use of
>> .makeToken(type) and therefore does not contain
>>         //position-Information. Create some:
>>         ctok.setLine(abeg.getLine());
>>         ctok.setColumn(abeg.getColumn());
>>         $setToken(ctok);
>>     	}
>>     ;
>>
>> -Ulf
>>
>> --
>> Dipl. Inf. Ulf Dreyer
>> Robert Bosch GmbH
>> Zentralbereich Forschung und Vorausentwicklung
>> Software und Systemengineering in der
>> Fertigungsautomatisierung CR/APA3
>> Postfach 30 02 40 D-70442 Stuttgart
>> Tel.: 0711/811- 34365
>> Fax: 0711/811-518 34365
>> eMail: ulf . dreyer at de . bosch . com
>>
>> Robert Bosch GmbH, Sitz: Stuttgart, Registergericht:
>> Amtsgericht Stuttgart HRB 14000 Aufsichtsratsvorsitzender:
>> Hermann Scholl; Geschäftsführung: Franz Fehrenbach, Siegfried
>> Dais; Bernd Bohr, Wolfgang Chur, Rudolf Colm, Gerhard Kümmel,
>> Wolfgang Malchow, Peter Marks; Volkmar Denner, Peter Tyroller.
>>
>>
>>> -----Original Message-----
>>> From: stringtemplate-interest-bounces at antlr.org
>>> [mailto:stringtemplate-interest-bounces at antlr.org] On Behalf
>>> Of Terence Parr
>>> Sent: Thursday, March 15, 2007 7:14 PM
>>> To: StringTemplate
>>> Subject: Re: [stringtemplate-interest] ChunkTokens without
>>> Line/ColumnInformation? St#
>>>
>>>
>>> On Mar 15, 2007, at 9:41 AM, Dreyer Ulf (CR/APA3) wrote:
>>>
>>>> Hello all!
>>>>
>>>>  I just stumbled upon a strange behaviour.
>>>>  The tokens coming from AngleBracketLexer (in ST#) do not contain
>>>>  a Line and Column - information  (getLine/getColumn yield 0/0)
>>>>
>>>>  Is this intentional? If so why?
>>>>
>>>>  How can I 'fix' this without rewriting the
>>> TokenTypes/Lexers in ST#?
>>>
>>> That's weird.  All tokens or just some?
>>>
>>> Ter
>>>
>>> _______________________________________________
>>> stringtemplate-interest mailing list
>>> stringtemplate-interest at antlr.org
>>> http://www.antlr.org:8080/mailman/listinfo/stringtemplate-interest
>>>
>> _______________________________________________
>> stringtemplate-interest mailing list
>> stringtemplate-interest at antlr.org
>> http://www.antlr.org:8080/mailman/listinfo/stringtemplate-interest
>>
> _______________________________________________
> stringtemplate-interest mailing list
> stringtemplate-interest at antlr.org
> http://www.antlr.org:8080/mailman/listinfo/stringtemplate-interest