Syntax Design (2014)

https://cs.lmu.edu/~ray/notes/syntaxdesign/

by memorable • 4 years ago

220 72 4 years ago

cs.lmu.edu

I find the section on "syntactic salt" interesting:

The opposite of syntactic sugar, a feature designed to make it harder to write bad code. Specifically, syntactic salt is a hoop the programmer must jump through just to prove that he knows what’s going on, rather than to express a program action.

This is perhaps an uncharitable way to describe it, but the concept does ring a bell. Rust's unsafe {}, C++'s reinterpret_cast<>(), etc - all slightly verbose. More important than jumping through hoops, the verbosity helps when reading code to know that something out of the ordinary is going on.

DonHopkins• 4 years ago

And then there's Perl's "syntactic syrup of ipecac".

https://en.wikipedia.org/wiki/Syrup_of_ipecac

zppln• 4 years ago

syntactic salt

I feel like this describes Rust's lifetime annotations pretty well too.

epage• 4 years ago

Not just lifetimes but types used to do more complex lifetimes that are normalized by other languages like RefCell, Arc, etc.

Karliss• 4 years ago

I disagree about lifetime annotations being syntax salt. At least with my interpretation of what syntax salt.

From syntax perspective lifetime annotations are almost as short as possible assuming you want to explicitly convey this information at all ('a is just two symbols and one of them is identifier). The alternative of not specifying it at all comes with major tradeoffs of either in memory safety (like in C) or runtime performance (like most programming languages with dynamic memory management). In theory there is third option of compiler fully deducing lifetimes, but that's far from trivial, has it's own costs and realistically even further narrow down what programs the compiler considers valid and increasing compilation time.

There are strong similarities with typing strategies. Just because there are programming languages with dynamic typing doesn't mean that explicit static typing is salt. Dynamic typing has performance cost, and static inferred typing has worse self documentation properties and slightly bigger compilation time cost since you can't process each function independently.

On the other hand reinterpret_cast<Foo*>(expr) doesn't provide to compiler any extra information that couldn't be conveyed with simpler less verbose syntax like R(Foo*)expr or (expr as Foo*). Same with unsafe{} blocks, compiler already know which operations are unsafe.

nyanpasu645• 4 years ago

What I can't stand about Rust is that the language developers think they know better than language users developing software. They stack on syntactic salt to make it more unpleasant to write the equivalent of correct C++ programs with aliased mutation or manual freeing, in situations where the idiomatic Rust ways are also unpleasant to write (Cell, RefCell, raw pointers), have runtime overhead (RefCell), are busywork to implement in working programs (restructuring your entire program around an ownership tree with only ephemeral stack-allocated cross-linking &mut, which is only sometimes possible without reducing performance or increasing memory use), or are easy, tempting, and undefined behavior in Rust but not C++ (casting *mut to &mut in an unsafe block in a safe function).

athrowaway3z• 4 years ago

"Man tries to hammer in screw. Angry at hammer manufactures for not being screwdriver manufactures"

----

I'll just edit this comment to clarify: Rust does a lot of thing really well. Non-RC manual memory management for a tree/graph structure is absolutely not it.

As for the verbose 'unsafe' pointer/memory manipulations, i really don't see the issue. I've written my share and I think its fine to add a roadblock if you want to shoulder the ability to add segfaults and other issues into a codebase. Additionally, it usually helps that you decide to encapsulate it into the least number of unsafe functions, instead of 'doing it manually' all over a codebase.

Rusky2• 4 years ago

This is not an accurate characterization of the Rust language developers. Neither of these features were designed as "syntactic salt!" They are compromises, made on a time budget to achieve goals which were higher priority for the project- but the door is still open to improve them. This is a far cry from "knowing better than language users," which implies that they could have simply left that syntax out while still achieving their goals. (Or worse, that their goal was specifically to annoy people...?)

For instance, they are not satisfied with the current raw pointer syntax either, as it interacts poorly with lvalue/place syntax in ways that make unsafe code harder to audit. There are regular proposals for how to improve things like `(*ptr).field` or the use of raw pointers as method receivers.

The situation with interior mutability is similar: compile-time memory safety inherently requires some limitations on programming style, but I regularly see proposals for how to improve "field projection" syntax.

undefined behavior in Rust but not C++ (casting *mut to &mut in an unsafe block in a safe function).

The question of "syntactic salt" aside, this is simply false.

cxr3• 4 years ago

More important [...] the verbosity helps when reading code to know that something out of the ordinary is going on

This applies to JS as well with its strict equality check (triple equals). Bad practices within the NodeJS ecosystem, however, have led to circumstances where triple equals has been cargo culted as the "right" thing to do for any equality comparison. The consequences of this include code that is more verbose, is no more type safe (and often doesn't do the right thing for some inputs—whereas with double equals, in contrast, it would...), and that the appearance of triple equals is no longer a strong signal that there's something happening that's worth paying attention to.

adamddev12• 4 years ago

Can you give some examples of inputs where === doesn't do the right thing?

mvf4z7• 4 years ago

This reminds me of Reacts "dangerouslySetInnerHTML" prop.

https://reactjs.org/docs/dom-elements.html#dangerouslysetinn...

tmtvl• 4 years ago

I've heard it called "syntactic vinegar" instead.

I wonder if there's a term for syntactic useless stuff, like commas in Clojure quasiquoted lists.

lioeters• 4 years ago

I propose "syntactic konnyaku".

Konnyaku is a rubbery, flavorless zero-calorie food made of yams that are high in fiber and added to various Japanese foods for its squishy texture.

Existenceblinks3• 4 years ago

Fun read. I would love to read what if it's not text based, is it going to be different? Visual programming seems to suffer from composability and it's also bounded to be using human language as well, box with border is hard to comprehend, can get messy easily.

I mean, visual but text-like theme. It seems to be in sweat spot. Only fix some downside/limit of text.

csmeyer2• 4 years ago

Shameless plug for my hybrid visual/text pl, Pickcode, which matches what you're talking about

Demo programs: https://app.pickcode.io/playground

Vermeulen• 4 years ago

Wow this is awesome, love this style. I did something really similar with our game's scripting language called MBScript: https://docs.modboxgame.com/docs/mbscript Same kind of line setup, visual add button, etc.

Whats Pickcode made for? Web programming?

Existenceblinks3• 4 years ago

Hey, yes! Nice, I'm thinking more serious and ambitious. You could go mass by adding "module" and ways to compose. The keyboard navigation is ok-ish (honest opinion) because this is the hardest ones which is design for every day task in long run, at minimum should be as fast as text based programming. At least 3-4 devs are comfortable to work on this codebase (non-realtime, just normal version branching flow)

I really really want this decades old idea to take off. We should have grammar files in .. json is fine (have to start somewhere), and have spec for editor implementor to spread it across platforms. Ideally languages creators only have to customize "view" to decide how their lang would look like. Probably configure keymaps if they think their lang can be developed fast with certain keystroke (akin to emacs but more friendly because medium is not text anymore)

masklinn2• 4 years ago

You could check out Self. It’s image-based so the objects are “live”, and can be interacted with directly via the UI.

Existenceblinks3• 4 years ago

Is it https://www.youtube.com/watch?v=CCx6Nj_Hr1g ? I've seen quite many live programming languages. Though not a single one seems to want to go mass. Like able to have at least a 3-4 devs team work on it.

JohnDeHope• 4 years ago

I enjoy this sort of "one example, multiple different lenses" style of discussion. It reminds me of this book... Exercises in Programming Style by Cristina Videira Lopes.

rramadass• 4 years ago

Exercises in Programming Style by Cristina Videira Lopes

I didn't know of this book (have been looking for something like this for a while) and hence thanks for the pointer.

wodow• 4 years ago

I really like the term "Sugary Functional Style" -- sweetening pure functional programming with faux procedurality.

Looks like it's a (three word) Google Whack at the time of writing: https://www.google.com/search?hl=en&q=%22Sugary%20Functional...

bhauer• 4 years ago

It's awesome to see an article from Dr. Ray Toal on HN this morning! In my biased opinion, the excellent tenured professors at LMU's CS program make it a stand out for its size.

tabtab• 4 years ago

I actually like the VB-style, but VB did it mostly wrong: if you start the block with X, you should always end it with "End X". Thus, you'd have While ... End While instead of crap like "Wend" and "Next" (in For...Next).

It's more legible to know what block is being ended. C-style continually frustrates me that area. The End-X style just never found a nice way to wrap text for longer statements.

C-style also has a problem in that there is no way to define arbitrary blocks: it relies too much on key-words. I'm trying to remedy this with "Moth" syntax:

https://www.reddit.com/r/ProgrammingLanguages/comments/ky22d...

It's LINQ-esque but without the bloated Lambda conventions, and influenced by XML in that you have a simple syntax pattern that can "implement" many domains' needs. It started with an attempt to merge the best of Lisp and C-style. (Whether it succeeded or not is hotly contested. I welcome other attempts.)

Jtsummers5• 4 years ago

I actually like the VB-style, but VB did it mostly wrong: if you start the block with X, you should always end it with "End X". Thus, you'd have While ... End While instead of crap like "Wend" and "Next" (For...).

That's covered in their syntactic salt section, but Ada does, mostly, what you describe.

  procedure Proc(...) is
    -- vars, types, and subprograms defined here
  begin
    ...
  end Proc;

  for I in Some_Array'Range loop
    ...
  end loop;

  if condition then
    ...
  end if;

f1shy• 4 years ago

Note that in the for case, you can have many "next" and having many "end" would be silly.

mojifwisi• 4 years ago

C-style also has a problem in that there is no way to define arbitrary blocks

I might have misunderstood what you mean by "arbitrary blocks", but you can definitely do this in C:

  int main()
  {
    {
      /* arbitrary block */
      return 0;
    }
  }

Jtsummers5• 4 years ago

They seem to mean a computational block (or a closure/lambda) that can be passed on to other functions. Try to do this in C (it's invalid as presented, but this is the concept):

  int main() {
    int* collection = ...
    int* filtered = filter(collection, int func(int item) { return item > 10; });
    ...
  }

You have to actually define a function at the top level in order to pass that in and there is no notion of closures so you can't do the more useful thing that you might have in even C++ these days:

  int main() {
    int* collection = ...
    int limit = ...
    auto filter = [&limit](auto item) { return item > limit; };
    int* filtered = filter(collection, filter); // assuming `filter` is defined
    ...
  }

You can come close, but you create a lot of extra bookkeeping in your C program to pull it off, and the functions are still only defined at the top-level.

adamnemecek• 4 years ago

Seems down https://web.archive.org/web/20221018135106/https://cs.lmu.ed...

mncharity• 4 years ago

Another exploration of syntax: http://rigaux.org/language-study/syntax-across-languages.htm...

xaedes• 4 years ago

Wow, amazing! This really is a comprehensive overview. Basically a syntax Tafelwerk.

djedr9• 4 years ago

Very nice little article! Learned some new terms.

To anybody dabbling as I do in syntax design, who may be looking for an extremely minimal representation for trees (even more minimal than S-exprs!) I would like to introduce my little project called Jevko: https://djedr.github.io/posts/jevko-2022-02-22.html

It is pure distilled treeness. Its grammar fits into one line, if compressed well:

  Jevko = *("[" Jevko "]" / "`" ("`" / "[" / "]") / %x0-5a / %x5c / %x5e-5f / %x61-10ffff)

This took me years of syntax golfing to figure out. I think it's turned out pretty nice. It's complete, formally defined, with a few simple parsers written, except it has no users. ;D

To relate back to the article, an interesting and AFAIK original feature of this syntax is that newlines or other whitespace are neither significant nor insignificant nor "possibly significant" in Jevko. I'd call it whitespace-agnostic. Various whitespace rules can be laid on top of it, producing for example a Lisp-like language with native multiword identifiers with spaces, e.g.:

  define [sum primes [[a][b]]
    accumulate [
      [+]
      [0]
      filter [
        [prime?]
        enumerate interval [[a][b]]
      ]
    ]
  ]

here "sum primes" and "enumerate interval" are two double-word identifiers. It's the only right_solution to the identifierWars, I-tell-you!

nathell• 4 years ago

I thought “what a weird name”, then silently pronounced it and my Polish ear heard “drzewko”, meaning “little tree”. What a fitting name. :)

djedr9• 4 years ago

For a long time I couldn't find the right name that would express the generic nature of it.

An earlier prototype was called TAO, as an acronym for Tree Annotation Operator (it had an extra feature called operators), and as a reference to the ancient Chinese concept, in essence nameless and by design hard to pin down -- this seemed to fit perfectly.

However there is about 2^42 cowznofski potrzebie things called TAO (kind of ironic, as the original idea was that the Tao would be distinct from the countless named things), so it turned out to be a bad name after all. So I decided to find a more unique one and here we are.

The amount of time spent thinking about this and the lengths I went to are better left untold.

In other words, naming is hard.

znkr• 4 years ago

This is very beautiful, nice work. I wonder if I should use it for something...

djedr9• 4 years ago

Thank you. :)

I wonder if I should use it for something...

I'd be honored!

A couple of ideas:

How about a simple configuration format? https://gist.github.com/djedr/681e0199859874b3324eaa84192c42... (I should make a library out of this)

Or you can put it in your query strings to make them more humane: https://github.com/jevko/queryjevko.js

Or make up a markup DSL: https://github.com/jevko/markup-experiments#asttohtmltable

Or serialize game objects in your indie game. Or make it the interface of your experimental app. Or use it to shave off a few unnecessary characters off your data: https://jevko.github.io/compactness.html

No parser in your favorite language? A basic one should be only a couple dozen lines! https://github.com/jevko/parsejevko.js / https://github.com/jevko/specifications/blob/master/spec-sta...

abathur2• 4 years ago

This reminds me of Breck Yunits' Tree Notation (https://treenotation.org/). Both seem to have a ~totalizing energy. Maybe some common cause. :)

djedr9• 4 years ago

Indeed, it's close. Obviously mine and Breck's levels of appreciation for indentation/brackets are very different. ;) Although independent, the paths we have taken to arrive at these are somewhat similar (somewhere early in there are experiments with visual programming). As are the tools of thought (minimalism). We were thus taken to similar places.

Before I was aware of the existence of Tree Notation I put my syntax online at tree-annotation.org (now defunct), so even naming converged. I was initially very confused myself. :D

Ultimately I think that the existence of multiple incarnations of this idea suggests that there is (perhaps a very niche) need for a minimal syntax like this. Something like S-exps, but general-purpose. Trying to satisfy that need is the common cause.

The way I imagine it is that it would be supported across programming languages, like JSON. It could be an universal format for (tree) structured data. There is this piece of the Unix philosophy which says that text streams are the universal interface. That's true on a certain level. On another level not far below binary streams are the universal interface. On another level not far above... there was nothing universal until XML. But that was overkill, so JSON displaced it. But that's still overkill, so...

justinpombrio• 4 years ago

Could you write that example as an s-expression? I can't quite parse it.

djedr9• 4 years ago

Sure. This is translated straight out of SICP 3.5.1[0]:

  (define (sum-primes a b)
    (accumulate +
                0
                (filter prime? (enumerate-interval a b))))

If you have an S-exp parser built into your optical nerve (to paraphrase L. Torvalds), it will be indeed hard to parse.

The translation rules for an S-exp are roughly:

1. switch to square brackets

2. surround all symbols except the first in brackets

3. move the first symbol outside the opening bracket

4. replace dashes in symbols with spaces

5. leading and trailing whitespace doesn't count into a symbol.

That's basically it.

Besides that indentation and closing brackets are written C-style rather than Lisp-style.

Now the real confusion might begin if you have an S-exp with an S-exp as the first element like:

  ((fnr) 1 2)

This would translate to:

  fnr[][[1][2]]

which is... interesting.

Anyway, this is just a theoretical example to demonstrate the whitespace-agnosticism. I have not implemented this particular language. But it's one way to arrange these trees into a Lisp-like language, if one was so inclined.

The cool thing is the simplicity and identifiers with spaces. Not cool, besides unfamiliarity, is the verbosity compared to regular S-exps (more brackets).

[0] https://mitp-content-server.mit.edu/books/content/sectbyfn/b...

emmelaich2• 4 years ago

I like the idea of [] rather than {} or ().

Saves using the shift key!

djedr9• 4 years ago

Glad you like it. That's the intention. ;)

kenshoen• 4 years ago

So is it a sintax for n-ary trees? Nice!

  newtype Jevko = Jevko ([(String, Jevko)], String)

djedr9• 4 years ago

Indeed that's a useful way to look at it and a type definition that'll do the job of storing Jevko parse trees.

mcluck• 4 years ago

I just want to tell you that Jevko is quite a fun little language you've built. I just threw together a parser for it in TypeScript (I'll upload it to Github eventually haha). It was super easy. I'm going to recommend it to my friends who aren't as excited about parsers and things as me for an introduction to parsing programming languages.

djedr9• 4 years ago

Thank you, I'm very happy to hear that. :)

Ease of implementing is a goal, so that there is minimal friction when porting into any language.

This reminds me that I should finish this tutorial: https://github.com/jevko/tutorials/blob/master/parser.md

I hope you can get your friends excited! ;)

I'd really love to see folks put Jevko to use!

samsquire2• 4 years ago

I began designing a language that handled recursion and iteration as relations between variables which are topologically sorted to determine control flow.

Each function is a toplogical graph of stream functions so it is similar to a data flow language or reactive programming language. The goal is that you should express the critical insight of the algorithm to work out what to write and the code is not nested so there is very little tree structure.

Algebralang is rough notes on how it would appear.

https://github.com/samsquire/algebralang

Example programs in the repository are binary search, btrees, a* algorithm.

I wrote a multithreaded parallel actor interpreter in Java and it uses an invented assembly language which doesn't have a bytecode, it's just text.

https://github.com/samsquire/multiversion-concurrency-contro...

I like the ideas behind ani language https://github.com/waves281/anic

Jtsummers5• 4 years ago

If possible, I'd add indentation to your examples to make them much more readable. As it stands, it's like reading one of my math prof's C code (he was an old FORTRAN when it was shouting coder and never learned to indent):

  insert t node = recursive_deepest_first(items=t.children,item=t,
  lastRecursion=l)(
  if len(t.children) == 0 t.activate()
  location = reversed(t.children).find(item=i, item.value >= node.value)
  output = insert t node
  if len(t.children) > 3 {
  replace(t, Node(value=t.children÷(point=middle=m) 
  = m.value,children.sort(item=i,sortKey=i.value)=t) 
  output = l.t }
  else deepest t.children.append(node)
  )

Assuming this doesn't invalidate the program it reads more clearly and only takes one more line:

  insert t node = recursive_deepest_first(items=t.children, item=t, lastRecursion=l)(
    if len(t.children) == 0
      t.activate()
    location = reversed(t.children).find(item=i, item.value >= node.value)
    output = insert t node
    if len(t.children) > 3 {
      replace(t, Node(value=t.children÷(point=middle=m) = m.value, children.sort(item=i, sortKey=i.value)=t) 
      output = l.t
    }
    else
      deepest t.children.append(node)
    )

That original program was hard to decipher for both lack of indentation and the odd line breaks, and inconsistent choice of space or no space after commas. Another question, why do you use `if ... then...` in some examples and not in others? Is that a user choice?

samsquire2• 4 years ago

Wow thanks for reading my page and looking at the examples. I appreciate your time.

There's a few bugs in that code. Sorry for presenting something that obviously wasn't ready. I didn't use location when I insert into that position in the btree. And the node spliting code has an error.

And thanks for reformatting the code.

It's a very rough design. The critical insight over many algorithms is hidden in one character or line. Such as a strategic -1 or +1 or pattern of recursion that means it becomes understandable.

I find when writing code the structure of the code is more important than the calculation or addition or subtraction. Which is surprising because computers are calculators. The structure of traversal, laying out data in memory and structure of the jumping around instructions in memory is harder than the core insight of a division, or subtraction or addition or append or +1 or if statement here or there.

When I write recursive code I often want to refer to outer context of an outer recursion. So that's the meaning of the "deepest"

hzhou321• 4 years ago

The `infix` syntax is missing from the major items. Without infix syntax, all languages are just variations of LISP -- I guess that was all the article is about.

Arch-TK• 4 years ago

Because C does not have real arrays

C does have real arrays, they just get implicitly converted to pointers to their first element in a lot of cases (for a multitude of reasons in part having to do with simplifying the language), A[B] is defined as such so it works with normal pointers and arrays-converted-to-pointers in the same manner.

Try using an array with sizeof, unary &, or in the form of a string literal used to initialise an array. In those situations it suddenly stops behaving like a pointer to its first element and definitely behaves like something which is unlike anything else in C (hint: it's an array).

kragen• 4 years ago

Another interesting weird syntax I ran across a few years ago is OGDL: https://ogdl.org/

It's sort of an alternative to S-expressions with much less punctuation, but the data model is slightly different — in S-expressions you label the leaves, and in OGDL you label the nodes. In other contexts these node-labeled trees are sometimes called "rose trees"; they are the basic data model of, for example, Prolog. Labeling nodes is almost equivalent to labeling arcs, but OGDL does support multiple references, so not quite.

The OGDL proposal was intended for data, like XML, not programs. They started out by trying to simplify YAML, which has arrays and dicts, and they simplified it by unifying them into a single structure.

Here's one of their examples:

    network
      eth0
        ip   192.168.0.10
        mask 255.255.255.0
        gw   192.168.0.1

    hostname crispin

This is not quite just an edge-labeled digraph because, as in S-expressions, the order of arcs within a node is significant; you can have multiple edges with the same label in the same node, and you can select edges by ordinal rather than, or in addition to, label.

This is of course amenable to use as a programming syntax.

Archelaos• 4 years ago

I wonder what would be possible if we could move away from the text editor and code with an e-pen on a tablet, the way mathematicians use a black board, where the parsing is not from left to right, but follows specific paths in two dimensions (like the notations for fractions, square roots, sums, integrals, etc. we learned at high school). -- Has this been tried out recently?

emmelaich2• 4 years ago

[mark the end by] .. spelling the initial word backwards

This reminds of the quirk in shell. if/fi, case/esac but ... do/done?

It might be because `od` was taken for octal dump.

Syntax Design (2014)

Related Stories