[stringtemplate-interest] whitespace in IF

[Zenaan Harkness]

On Sat, Nov 07, 2009 at 11:38:57AM -0800, Terence Parr wrote:
> Hi. well, i'm am wondering if think of it as consistent with empty  
> expressions:
> 
> <emptyexpression>
> 
> on a line by itself gets no newline. so, neither would if, else,  
> endif.  Anything that evaluates to nothing on line by itself is a no  
> op in terms of whitespace.

What about just a blank line? That evaluates to a blank line right?

And if there is whitespace before the expression?
Of course - this is kind of what we are discussing here.


> I looked at my templates and I NEVER use multiline templates to get a  
> single line of output. I can image i would with a long chain of if- 
> elseif but my ANTLR templates don't have this trait.  That would be  
> lots of logic in the templates.

Here's an example from my st usage, just pretend it does _not_ have
the leading spaces on each line:

   get(att) ::= <<
   <if(att.hasBody)>
   <callGet(att)>
   <else><if(att.final)>
   <varGet(att)>
   <else><if(model.final)>
   <varGet(att)>
   <else>
   <callGet(att)>
   <endif><endif><endif>
   >>


You can see the inconsistencies that are evident already - <else> is
followed immediately (no space) by the next <if>, while <if> is not
immediately followed by anything; there is the usual consumption of
\n's and the finish is <endif><endif><endif> (which to be honest I
can't even remember if it was necessary or just to make it look tidier
in my opinion).

I guess my point is, at the moment we have inconsistency - sort of
consistent, but with an incomplete set of concessions to simplified
syntax layout.

I have a few thousand lines of ST templates across about 30 ST group
files, and without a backwards compatibility option I would be
required to carefully run through each file, test cases etc, to debug
all required changes. On the other hand I'm personally happy doing
that if it means consistency.


> Technically, we could think of IF clauses not as templates but just  
> more TEXT and EXPR chunks.  Only explicit templates like in
> 
> t() ::= <<
> blort
>  >>
> 
> would have whitespace removed.

The more consistent, in my book the better. Of what's important,
consistency is right up there just below correctness.


> With {x | foo} subtemplates, I'm ignoring whitespace after the pipe so  
> you can format like:
> 
> {x |
> foo
> }

Again, breaking consistency.

What about whitespace prefix operators :

   {; x |
   foo
   }

and

   {: x |
   foo
   }

??

These would produce "ignore all leading and trailing whitespace"
and "include all leading and trailing whitespace" templates.

Determinism, certainty, is really really valuable.
Providing deterministic output which has syntax of a low
cognitive-load level, is a good thing.

Sure ST is deterministic, but especially with these proposed changes,
only within one [major?] version.


> But, what about the \n after foo?  does that count?  if it's a  
> template the no.  I'm conflicted about this.

When the behaviour is conditional on the specific context/ location,
and possibly dependent on the specific syntax (language elements) in
use, you have a royal mess!

Why do it?

Just say no!

Or, at least provide a low cognitive overhead deterministic behaviour,
as in, such behaviour accessible with simple syntax extension.
Say ... whitespace prefix or infix operator perhaps?

:)


> In general, I can imagine not throwing out whitespace anymore.  It  
> arose from template files that had no surrounding <<...>> or "...".   
> That is no longer the case.  ALL templates are formal defs:
> 
> mytemplate(args) ::= <<
> even if in a file
>  >>
> 
> That way, a group file is literally the cat of template files in a dir  
> (well, minus the group header).

Oh, well that's a very nice trait.

Indeed it would also be a nice default!

And to solve the backwards compat. matter: the white-space specific
prefix or infix operator could be "to cause backwards compatibility"
in cases where that is really needed for some reason.

Either way, I'm certainly feeling more and more partial to bi-modal
space handling. And BTW, _not_ only as a command-line switch/ flag.
As in, yes at the pre/in-fix operator level.


> So, we if want no newline we do
> 
> mytemplate(args) ::= <<
> even if in a file>>
> 
> no starting newline:
> 
> mytemplate(args) ::= <<even if in a file
>  >>
> 
> or
> 
> mytemplate(args) ::=
> <<even if in a file
>  >>
> 
> no newlines:
> 
> mytemplate(args) ::= <<even if in a file>>
> 
> Hmm...i might have just convinced myself to not scarf any newlines at  
> all!  Only thing to scarf would be a single space char after pipe  
> maybe in a subtemplate.
> 
> Thoughts?

Wholeheartedly agree! Primary consideration/ operation mode ought to
be "_everything_ in a template is _part_of_ the template, and only
what's within the template is part of the template".

It's intuitive, deterministic, with a low cognitive overhead, and
accords with principle of least surprise. I have spent many
frustrating hours in total, fiddling with end-of-line rubbish,
in ST, when it should be simple and deterministic. I'd say this is
my greatest frustration with ST to date (non deterministic, at least
apparently so, whitespace behaviour in templates). This essentially
comes down to what is _for_the_user_ of ST, effectively a correctness
issue, more important than consistency of syntax. Ultimately I had to
"fine tune" my templates, often three or more times swapping various
templates between "one immediately after the other", including extra
blank lines at start/end of template", creating sub templates and
other similar 'hacks', all just to get correct output (eg Javadoc
output)! (My difficulty with precise output generation was my
combination of nested templates with ST conditionals... but
nevertheless, should not have been so trial-and-error like to
achieve.)

Any deviation (from simplest obviously correct mode of operation)
should be something that is overrided with explicit syntax.

Would "<>" (without the quotes) mean "absolutely nothing", or "empty
template" etc, and provide for desires to layout a little nicer, eg:

   mytemplate(args) ::= <<<
      >even if in a file<
   >>>

or

   <if(x)><
   >foo<
   ><else><
   >bar<
   ><endif>

Of course, such requirement would have strong deterrent factor in the
first place, which deterrent is not altogether undesirable :)


On the other hand, when I think back to my 3000+ lines of ST group
file code, I have dozens of templates defined which take advantage of
the current "ignore leading and trailing whitespace" behaviour.

ST4 is a break-backwards compatibility release though... so clean up
is first priority.


> On Nov 6, 2009, at 9:42 PM, Zenaan Harkness wrote:
> 
> > On Fri, Nov 06, 2009 at 06:11:33PM -0800, Terence Parr wrote:
> >> ok, a related problem.   Is
> >>
> >> <if(x)>
> >> foo
> >> <else>
> >> bar
> >> <endif>
> >>
> >> same as
> >>
> >> <if(x)>foo<else>bar<endif>
> >>
> >> It's same in v3. I'm wondering if we want the \n after foo or bar.

Re-reading your OP here, I think that in the sandbox of whitespace-
consumption, anything other than consume-all or include-all (first
and last that is) is a recipie for bitchingly frustrating
inconsistency! Unless you do it with specific operators.

Then the question arises consume-from-line-start ('leading' or
'line leading')?

We can treat white-space-in-templates removal in two sane ways as
I see it right now:

as bi-state:

   -  consume first+last   Y/N?
   -  consume line leading Y/N?


or as tri-state:

   -  consume first   Y/N?
   -  consume last    Y/N?
   -  consume leading Y/N?



If included as single-character syntactic operators (somehow)
in ST4, perhaps the following characters:

   -  for first,   use  :
   -  for leading, use  ,
   -  for last,    use  .


> > What will likely happen is complaints of inconsistency.

And I should have said "with the first complaints coming from me" :)

best
zen


-- 
Free Australia: www.UPMART.org
Please respect the confidentiality of this email as sensibly warranted.