Making Coffeescript’s Whitespace More Significant

https://github.com/raganwald/homoiconic/blob/master/2011/11/sans-titre.md#readme

by raganwald • 15 years ago

143 91 15 years ago

PLejeck3• 15 years ago

Nice title change, much better than the racism from before, but I'm afraid my spacebar, being white, is offended.

Additionally, my previous opinion (http://news.ycombinator.com/item?id=3296010) still stands, that whitespace-significance isn't such a good thing, and this whole "YAY TREES" stuff is overrated.

raganwald9• 15 years ago

I regularly use languages where whitespace is not significant. However, in those languages, whitespace is not significant. It isn’t significant some of the time and not significant some of the time. It is a separator all of the time.

Coffeescript is a language where whitespace is held out to be significant, so I’m simply saying “Great! Well in that case, let’s make it more significant."

I have no argument with the idea that perfectly good programming languages do not consider whitespace significant.

PLejeck3• 15 years ago

I don't think whitespace should EVER be significant, it seems like a very bad setup that's more prone to issues. I also have an unnatural hatred of all things Compile-to-JS.

dextorious2• 15 years ago

"""Coffeescript is a language where whitespace is held out to be significant, so I’m simply saying “Great! Well in that case, let’s make it more significant.""""

Ever heard of the concept: "too much of a good thing"?

scott_s• 15 years ago

Responding to what you said before: I also believe that representing everything as a tree (why are programmers so obsessed with trees!?) is ridiculous: code is, quite simply, code. Outline structure or not, what matters is the actual meaning of the code, and this is part of your mental model of the program, not the syntax.

Compilers represent your code as a tree - a parse tree first, then an abstract syntax tree. That is how code is understood. It's not an arbitrary decision on raganwald's part.

Groxx3• 15 years ago

I like it a lot. It would also simplify the ".end()" function in jQuery[1], because you could control it with indentation:

  $('.class')
    .find('.sub')
      .remove()
    .find('.stuff')
      .addClass('other')
      .find('.more_stuff')
        .removeClass('things')
    .filter('li')
      .appendTo('#my_list')

[1]: http://api.jquery.com/end/

raganwald9• 15 years ago

Funny you mention jQuery. I wrote “Significant Whitespace” in March of 2010, long before trying Coffeescript:

https://github.com/raganwald/homoiconic/blob/master/2010/03/...

dextorious2• 15 years ago

Yes, it could control it with indentation. "Simplify" though? Are you sure?

This whitespace zig-zaging is barely readable.

Groxx3• 15 years ago

Very much so. Otherwise you get this, at best:

  $('.class')
    .find('.sub')
      .remove()
      .end()
    .find('.stuff')
      .addClass('other')
      .find('.more_stuff')
        .removeClass('things')
        .end()
      .end()
    .filter('li')
      .appendTo('#my_list')

Or, to match jQuery's documentation style for that function, this:

  $('.class').find('.sub').remove()
    .end().find('.stuff').addClass('other').find('.more_stuff').removeClass('things')
    .end().end().filter('li').appendTo('#my_list')

Matching .end() with each selector change is equivalent to writing valid XML by hand, without the aid of an auto-tag-closer, and without a validator - you only see the error on run, and only if you hit that code path, and only if it does something you notice isn't correct.

None of this is meant to imply that chaining things like this is a Good Idea™, and I avoid .end() like the plague and use intermediate variables. But when you don't need the root or intermediate results for anything else, yes, this is more readable, more easily optimized (you can't get it wrong, every level is cached for you), less prone to error, and significantly fewer characters / lines of code. That's simplifying and improving.

-- late-edit:

Less whitespace zigzaggery is also possible (I agree, not easy to read such dense zigzagging), and similarly easier with significant whitespace. My example was essentially just a trivial one, I tend to see larger ones where I see that kind of indentation at all. Is this better?

  $('.class')
    .find('.sub').remove()
    .find('.stuff').addClass('other')
      .find('.more_stuff').removeClass('things')
    .filter('li').appendTo('#my_list')

raganwald9• 15 years ago

Great point:

    This whitespace zig-zaging is barely readable.

I considered an example like this in the post, but it confuses accidental with essential complexity. If you’re going to do all of that, then I suggest that significant indentation is a win over everything being a chain and using something like jQuery’s `.end()` to discriminate chaining from cascading.

It’s a lot easier to spot an error in indentation than a missing `.end()`. That being said... This could be a false dichotomy. Perhaps the right thing to do is use named temporary variables or structure the code another way.

Although I don’t see a ton of it in the wild, it’s 100% valid to write your own custom jQuery traversals, filters, and so forth. my own jQuery Combinators includes `.K` and `.T` combinators so that you can break code up into functions so you don’t need to write elaborate trees.

So... I agree that an elaborate tree is a difficult problem to handle, and perhaps neither significant whitespace nor `.end()` is the answer. But for the limited and possibly artificial choice of a fluent interface OR significant whitespace, I prefer significant whitespace.

iambot• 15 years ago

I actually agree with everything that is suggested in this submission. I wonder what the best way would be to get it implemented is. Perhaps as a form of feature request poll, to see what the suppor for it is. Or perhaps a call for people that agree to "watch" it on github.

clutchski• 15 years ago

It's being discussed here (along with some alternate syntax ideas):

https://github.com/jashkenas/coffee-script/issues/1889

raganwald9• 15 years ago

I think this is a little closer:

https://github.com/jashkenas/coffee-script/issues/1495

The above link is actually a discussion about Dart’s “Monocle Moustache,” so I console myself that when people say it’s ugly, they mean the moustache and not significant whitespace :-)

phzbOx• 15 years ago

There was a conversation on this recently on HN. IIRC jashkenas said he liked the idea but it would be better to encourage library author's to write a functional style enabling chaining rather than adding a new feature to the language.

Btw, I found that missing too in Python and created Moka (http://www.phzbox.com/moka/ It's still in heavy construction)

raganwald9• 15 years ago

I recall this discussion:

http://news.ycombinator.com/item?id=3174442

There may be others...

herge• 15 years ago

Saw your presentation at Montréal-Python, it was very good.

Have you played with pointfree (http://markshroyer.com/docs/pointfree/latest/module.html#mod...)? What do you think about it compared to Moka?

moomin2• 15 years ago

Actually, that was in response to a previous request for a dedicate chaining syntax. This syntax is actually more useful for dealing with when chaining is already implemented.

alexyoung2• 15 years ago

This is important to me:

s/Coffeescript/CoffeeScript/g

s/Javascript/JavaScript/g

mhartl• 15 years ago

I don't know why so many people are cavalier about this. Proper capitalization is part of proper spelling, which is important for clear communication. Whether it's 37Signals, Github, or Javascript, it irks me every time.

This cavalier attitude is so entrenched that attempts to correct it are sometimes even met with hostility, which on HN manifests itself as downvotes. Apparently there are those who feel that comments on this subject (e.g., this one or its parent) don't add to the discussion. And yet, I sometimes stop reading otherwise interesting articles simply because they exceed my misspelling or typo threshold—reading badly edited copy is unpleasant, and the lack of attention to detail undermines its credibility. I'd much prefer to avoid the problem altogether. In that spirit, I'd like to offer some surefire advice on how to prevent this kind of nitpicking: Get it right in the first place. Anyone who can develop awesome web apps or write an optimizing compiler can surely spell 37signals, GitHub, and JavaScript correctly as well.

subsection1h• 15 years ago

    I don't know why so many people are cavalier about this.
    Proper capitalization is part of proper spelling [...]

My favorite example of this behavior is when copywriters, graphic designers, etc. are inconsistent regarding the capitalization of the name of their own organization. I can't count all the times I couldn't figure out how best to bookmark an organization's website because the copywriters, etc. referred to it as Organization, ORGANIZATION, OrganiZation, and Organi Zation.

JadeNB• 15 years ago

> I don't know why so many people are cavalier about this. Proper capitalization is part of proper spelling, which is important for clear communication. Whether it's 37Signals, Github, or Javascript, it irks me every time.

So, presumably, is the proper use of articles, but how many people do you know who insert 'The' before all and only weak proper nouns? (For example: Is your alma mater University of Blah or The University of Blah? I'll bet you don't know; I didn't.) I'm on your side, and I like to make this distinction; but it's obvious that we have to draw the line somewhere or spend so much time on pedantry that we have no time left for meaning, and I don't have a problem with people who choose to draw the line before worrying about internal capitalisation.

nkohari• 15 years ago

Sometimes punctuation and capitalization is important. sometimes its not

morsch• 15 years ago

And s/Nvidia/NVIDIA/g? Ugh.

English has a normal way to deal with proper nouns, the first letter are capitalised, the others aren't. CamelCase is an exceptional phenomenon in written English. It's not unusual for exceptional stuff to get regularised, particularly (relatively) high-frequency words like Javascript. Especially since there is zero loss of information involved; there is no added ambiguity.

alexyoung2• 15 years ago

"writing functions to return a certain thing just to cater to how you like to write programs is hacking around a missing language feature"

To me chaining demonstrates just how flexible JavaScript is, rather than pointing out a fundamental missing language feature. In fact, by avoiding adding language features like this, I feel like the language is simpler and allows me to be more creative within its constraints.

raganwald9• 15 years ago

    Java(S|s)cript != Coffee(S|s)cript

:-)

Lisp does not consider indentation significant, Python does. Smalltalk has cascading messages, Ruby doesn’t. I think these are simply design choices, and the goal is to find a set of choices that work together harmoniously.

Note that my proposed syntax still allows you all the chaining you want.

perfunctory• 15 years ago

I like it

lisper6• 15 years ago

The problem with significant whitespace is that you can't count on whitespace to be preserved across many common protocols. Text editors will convert spaces to tabs and vice-versa. HTML rendering eats whitespace. Cut-and-paste may or may not preserve whitespace.

Python has had this problem since its inception. If you're editing a Python program in emacs python mode and you hit TAB at the wrong time you can inadvertently change the semantics of your code. And that's just the tip of the iceberg. I'm a big Python fan, but significant whitespace is a bad idea.

masklinn2• 15 years ago

> Text editors will convert spaces to tabs and vice-versa.

Get a good text editor?

> HTML rendering eats whitespace.

Except when you tell it not to, of course.

> Python has had this problem since its inception.

problem being mostly encountered by those who never use it, interestingly.

> If you're editing a Python program in emacs python mode and you hit TAB at the wrong time you can inadvertently change the semantics of your code.

So can you if you hit "}" or ";" at the wrong time in a braceful language...

The primary (and as far as I'm concerned the only significant) issue of significant indentation (it's not even significant whitespace) is auto-generated code (which is why Haskell has a braceful syntax and an indentation-based transformation of it), as giving the right contextual indentation to a piece of code may make the code generator much more complex (codegen targetting Python should probably generate python bytecode, rather than generating code).

And to support my claim that significant indentation is not effectively an issue, I will use Haskell: Haskell can be written using both a brace-and-semicolon syntax and an indentation-based one. Both forms are perfectly equivalent and can be translated into one another without loss of information.

I do not remember ever seeing a Haskell piece of code, article, demonstration or example which used braces except when the article was about the braceful syntax or about auto-generated code.

If significant indentation was such a crippling problem, would Haskell users not have coalesced around the "less problematic" braceful syntax?

lisper6• 15 years ago

I use Python a lot. And I encounter these problems often enough for them to be very annoying.

> So can you if you hit "}" or ";" at the wrong time in a braceful language...

The difference is that when you hit "}" or ";" the effect is always the same, it's always visible, and it's always possible to undo by hitting DELETE. If you hit either of those characters N times, you can always undo that by hitting delete N times.

This is not true for the tab key. The effect of hitting TAB depends on the context. Determining whether your last press of the TAB key had an effect or not requires that you remember the previous state, and so undoing the effect (or lack thereof) of hitting TAB requires that you remember the previous state. And if you ever do a block auto-indent at the wrong time you are pretty much hosed.

thedufer11• 15 years ago

Presumably if its a problem, the accidental change in indentation changed the code to a wrong form, rather than a form that can't be parsed (i.e. unindenting in the middle of a block, which will just cause easy-to-identify indentation errors).

Think about the same situation in, say, C. Now, your program still compiles correctly - but people read it as doing something differently. I'd argue that its worse to accidentally disconnect a reader's interpretation from a compiler's than to change both of them to something that's logically incorrect.

lisper6• 15 years ago

I agree. But those are not our only choices.

The ironic thing about Python is that it actually does have an open-brace. It's the colon. The compiler can tell that this:

    def foo():
    baz()

is syntactically incorrect. And if you tab the second line, auto-indent can do the Right Thing. The screw case is this:

    def foo():
      if baz:
        bar()
        bing()
      boff()

If you auto-indent the last line, it will quietly change the semantics of your program. That's bad.

I always end my blocks with a PASS statement (or a return), i.e.:

    def foo():
      if baz:
        bar()
        bing()
        pass
      boff()
      return

If you do this, then auto-indent will always do the Right Thing. This is particularly beneficial if you want to take a big block of code and wrap in an an outer block. I can add two lines to the above code:

    def foo():
      while snoz:   <--
      if baz:
        bar()
        bing()
        pass
      boff()
      pass       <--
      return

Then I can just auto-indent the whole thing and be confident that the result will be correct.

Contrast this with the traditional method where you have to manually re-indent your code. If you accidentally select the wrong region to re-indent you can change the semantics of the code in a way that loses the information about what the semantics should have been. The only way to recover from this is to manually reconstruct the correct semantics. It may not happen very often, but when it does it's a colossal PITA. (Entering those examples was a colossal PITA too.)

scotty79• 15 years ago

Where do I sign?

Do you think it would be hard to introduce such feature to CoffeScript on your own?

jashkenas2• 15 years ago

No, it wouldn't be terribly hard. The source code is all annotated to make it easier for folks to get started trying out their own flavors:

http://coffeescript.org/documentation/docs/grammar.html

You'd probably want to start by altering the lexer to stop considering ...

    a
      .b
      .c
      .d

... as effectively a single line, and turn it into some sort of "chain" node. Then, the value of the expression "a" can be cached at the beginning, and all further operations in the chain can be performed against the original value.

thedufer11• 15 years ago

I like the idea, but the lack of backwards-compatibility is less than ideal. Some people who update CoffeeScript compilers will suddenly find their code mysteriously doing the wrong thing (when they wrote what this suggestion considers "cascading messages", but expect them to not cascade). I have worked on multiple projects that would fall prey to this issue.

Semiapies• 15 years ago

Perhaps a "legacy" option to disable cascading?

thedufer11• 15 years ago

The problem is the time it will take people to realize that they need to turn on the "legacy" option. The failures this change could cause could be very difficult to track down.

quitedisgusted• 15 years ago

For posterity, the original title of this blog post was "White Power":

https://github.com/raganwald/homoiconic/commit/bd55e8ad731cc...

Reg Braithwaite doing the Clayton Bigsby. Stay classy.

jewel• 15 years ago

I've thought about something similar, but for a different reason. I'd like to be able to omit the parenthesis on multiline, chained statements, like this:

  $ 'class'
    .addClass 'babies'
    .removeClass 'kids'

jashkenas2• 15 years ago

@raganwald -- Lovely post, as always. One (somewhat important) question: If same-level continued calls mean chaining, and indented continued calls mean to use the value of the previous line, then what do outdented continued calls mean?

      first
    .second

Or heading 'offsides', in this fashion:

      first
        .second
    .third

Are those both syntax errors at compile time?

Also, how does this play with things that are not necessarily side-effect-ful? For example:

    object.property
    .value

Currently means `object.property.value`. Under your rubric, would it evaluate to `object.value`? Making any trailing access effectively a no-op?

eric-hu• 15 years ago

What about maintaining backwards compatibility while adding an operator specifically for this purpose. Example

     # same as first.second.third
       first
         .second
       .third

     # same as first.second, first.third
       first
         .second
       ^third

     # same as first.second.third
       first
         .second
     .third

     # syntax error
       first
         .second
     ^third

satyr2• 15 years ago

There's no outdented continued call in current CoffeeScript; an outdented line always makes OUTDENT.

(Or are you suggesting to allow such code? I personally see no value in it.)

raganwald9• 15 years ago

Outdented calls use the last value in the column to their left. So:

      first
    .second

Never means `first.second`, either `.second` is an error, or it is applied to something above this, such as:

    someValue
      .methodName ->
        first
      .second

...which becomes:

    someValue.methodName(-> first)
    someValue.second

If such a value can't be found, it's an error. Of interest is the meaning of `.propertyName` in the leftmost column, as in:

    object
    .property

I suggest it's an error. As long as you can write:

    object.property # or...
    object
      .property

I don't see the value in

    object
    .property

This:

    object.property
      .value

Means `object.property.value`.

sausagefeet• 15 years ago

IMHO, Smalltalk got this right using ; rather than whitespace. And it's easier to read.

satyr2• 15 years ago

> I suggest it's an error.

So we'd have to give up:

    array
    .map (x) ->
      f x
    .reduce (a, b) ->
      g a, b

groby_b2• 15 years ago

I read the article, and I simultaneously liked the idea and was fundamentally disturbed by it. At the same time, Smalltalks cascading messages never disturbed me.

So after going through the initial two reactions I have to every thought-provoking blog posts (#1: OMG - raganwald is insane!, followed by #2: OMG - raganwald is brilliant!), here's what actually doesn't work for me:

It is overloading whitespace with both control flow and data flow. Smalltalks cascade neatly introduce a different symbol to sidestep that.

So, ultimately, I'd rather see cascades introduced into CS than overloading the meaning of whitespace.

wwweston2• 15 years ago

Amen. The part about data flow is insightful. The idea of overloading it into whitespace is insane. There has to be a better way, and SmallTalk's might be it.

Blocks delimited by whitespace work because they rely on a familiar set of conventions that have already evolved among most programmers over 30 years or more (and interestingly, a lot of programmers still really hate the idea of having it enforced). Moving what is essentially the alleged JavaScript BadPart(TM) "with" into an overloaded whitespace/dot combo is going to throw at least as many programmers for a loop as "with" has.

I also think there's something smells wrong about the examples. For all the dataflow insight -- did I miss the part where he talks about how exactly we're keeping track of the destination of the return values? And the problem with the ".pop" example specifically might be less that you can't call it fluently three times than that you have to call it three times to get three items off the collection instead of ".pop 3".

groby_b2• 15 years ago

There's no destination, exactly.(I think). A "data scope" for lack of a better term just implies that all functions are called on the object returned by the statement that introduced the scope.

It is _exactly_ like JS's 'with' statement, just that we let the language automatically infer that we probably meant "with".

Uncompetative• 15 years ago

@raganwald -- fascinating ideas

Whilst the 'staircase' form forces each message to await the return from the reciever. The 'cascade' form could be used to post commands to a concurrent process into a separate recieving processor's message queue with no need to await a reply - as in Eiffel's Command/Query Separation Principle.

Also, 'futures' could be used to decouple queries from having to await replies from the reciever's of their messages. All that is needed is for variables defined through assignment to a query to remain potentially undefined until needed by some command. At this point all of the command's arguments would need to be defined and it would either have to await a reply from the queried process, or await some globally visible but yet to be defined thread to bind a value to the variable i.e. dataflow.

All of this hinges on using a language that doesn't freak out when processing undefined variables, but regards them as their symbolic names, reducing complex expressions with a collection of rewrite rules.

I'd be interested to know what you think about my proposal for these richer concurrent semantics.

-- Uncompetative

raganwald9• 15 years ago

I like it! I’ve had some similar thoughts along slightly different lines recently.

Making Coffeescript’s Whitespace More Significant

Related Stories