Typed, modular macros for OCaml

https://oliviernicole.github.io/about_macros.html

by otini • 10 years ago

134 44 10 years ago

oliviernicole.github.io

wcrichton• 10 years ago

This is awesome, and a big step forward for OCaml. Only criticism is that it seems difficult/awkward that you need different functions/values for each phase. In other staged macro systems (Scala/LMS, Rust/Compiler Plugins) you can refer to the same functions in any phase, and you can also splice values from the staging phase into the runtime phase without explicit use of a function like Expr.of_int.

a0• 10 years ago

I also agree that there seems to be too much separation between the phases. I wonder if this is a design choice or a design limitation. MetaOCaml also requires explicit lifting/unlifting of values.

otini7• 10 years ago

There are certainly some design choices here. I want to emphasize that in the few projects where I've used macros (e.g. https://github.com/OlivierNicole/macros-examples), I found that these translating functions were scarcely needed and I never had to pass a non-standard type between phases.

You may want to read this comment which I find relevant: https://www.reddit.com/r/ocaml/comments/5e8slg/bringing_type...

otini7• 10 years ago

This sounds very handy and I definitely need to learn more about LMS and Compiler Plugins. I wonder then what are the limitations on the values you can pass automatically across phases, since there must be (you cannot transfer e.g. a file handle from compile-time to run-time).

kagebe• 10 years ago

As far as I understand, for LMS: Those types T for which one has implemented needed operations for the Rep[T] type.

Shameless plug: For another comparison of MetaOCaml, LMS, Terra and AnyDSL (our approach) under the lens of staging/partial evaluation see our paper from GPCE'15: http://www.cdl.uni-saarland.de/papers/gpce15.pdf

yallop2• 10 years ago

There are some difficulties with moving arbitrary values between phases: it's easy to move an int or a list, but what about a mutable reference or a closure?

However, I don't think this will ultimately be a problem in practice, for two reasons. First, global values can be used in different phases via "module lifting". Second, there's a separate proposal for adding overloading to OCaml in the form of modular implicits:

https://www.cl.cam.ac.uk/~jdy22/papers/modular-implicits.pdf

Modular implicits will make it possible to use a single overload function ('lift', say) in place of a family of functions 'Expr.of_int', 'Expr.of_float', etc., which will make things much less awkward. And it's only a small step from there to having 'lift' called implicitly/automatically at appropriate points. Here's a message from an earlier discussion with a few more details:

https://sympa.inria.fr/sympa/arc/caml-list/2015-05/msg00032....

naasking2• 10 years ago

There are some difficulties with moving arbitrary values between phases: it's easy to move an int or a list, but what about a mutable reference or a closure?

Wasn't this already answered by "Closing the Stage" [1]?

[1] http://lambda-the-ultimate.org/node/2575

nemoniac• 10 years ago

Hasn't this matter been addressed years ago?

http://www.cs.utah.edu/plt/publications/macromod.pdf

Ericson23142• 10 years ago

This seems well done, the sort of design I'm pushing for Rust and Haskell.

The money question is: does it work with cross compiling? This is a proper phased design so it should, but that doesn't mean it doesn't.

otini7• 10 years ago

Well what macros only ever do is generate OCaml ASTs, so the should work whatever the compilation target is.

Ericson23142• 10 years ago

Things like the ^ quoting get more complex. You need to quite the target's version of the module.

coldcode• 10 years ago

Having never tried OCaml, what advantages does it have over other languages? Two languages I don't know seem more interesting, Clojure and Rust. Generally I work in Swift these days.

the_af• 10 years ago

Over other languages in general? That's hard to say about any language :)

OCaml is a language in the ML family, is backed by INRIA and is the main language of Jane Street Capital (so it definitely has "real world" use!), which also contribute a lot to it. OCaml is pretty good for functional programming, and unlike Clojure, it's statically typed. If you like static typing -- which I personally think is the way to go -- that's a huge reason to prefer it over Clojure. You may also check out Haskell if you go that way.

An interesting case study about going to OCaml from another language (Python, in this case) was posted to HN many times already: http://roscidus.com/blog/blog/2014/06/06/python-to-ocaml-ret.... I recommend you read it if you want to learn more about why would one choose this language.

Why not choose a language in the ML family? Well, if not having C-like syntax is a deal breaker for you, then OCaml is going to frustrate you. A real programmer shouldn't let syntax stop them from learning new languages, though :)

Ultimately one has to try doing something with this kind of language, even if it's just a toy project. Just reading about it won't enlighten you much, and you will end up focusing on its syntax instead and deciding it looks too unfamiliar.

owl57• 10 years ago

Well, if not having C-like syntax is a deal breaker for you, then OCaml is going to frustrate you.

There is a new alternative Reason syntax (https://facebook.github.io/reason/mlCompared.html) which replaces some of the unusual syntax with braces and semicolons. Though it still probably looks less C-like than Swift. And I haven't tried writing anything in it, so can't say for sure if it really feels less weird than standard syntax on a "real program".

int_19h7• 10 years ago

IMO, the single most interesting thing about OCaml is its object model. On one hand, you have a fairly conventional setup with classes, methods, inheritance, generics etc. On the other hand, the entire system is structurally typed, with pervasive type inference. Recycling bits from my earlier comment on the subject...

    # let f obj x y = (obj#m x)#n y;;
    val f : < m : 'a -> < n : 'b -> 'c; .. >; .. > -> 'a -> 'b -> 'c = <fun>

Here I defined a function taking 3 arguments, called method m on obj, passing x as argument, then called method n on the result of that, passing y as argument.

Note the inferred type signature for this: the first argument is of type <m : 'a -> <n : 'b -> 'c; ..>; ..> - i.e. any object that has a method named m (and possibly some other unspecified members - that's what ".." means), with said method m having a signature that allows us to pass some 'a, and returning another object of type <n : 'b -> 'c; ..> - i.e. something with a method named n that accepts 'b and returns 'c. 'a and 'b' are then used as types of x and y, respectively, and 'c is the the result of f.

nv-vn• 10 years ago

I would say that the most interesting feature (for me) is the module system (although with structural typing as a close second) :)

groovy2shoes4• 10 years ago

This is actually row polymorphism rather than structural subtyping. The difference is rather small, but important:

Under structural subtyping (or any kind of subtyping, for that matter), the subtyping relation admits a rule called "subsumption", which can lead to information loss. Consider this behavior, under structural subtyping:

    fun f {x : real, y : real} = {x = 2 * x, y = 2 * y};
    (* f : {x : real, y : real} -> {x : real, y : real} *)

    f {x = 2.0, y = 3.0, z = 4.0};
    (* it = {x = 4.0, y = 6.0}; *)

Above, when `f` is applied to a subrecord with an additional `z : real` field, the result is missing the extra field! This is because the subrecord was subsumed by the (super-) record type taken by `f`, leading to information loss. This is the reason why you often see people using up-casts in languages like Java (especially before generics were introduced) and Go (which is lacking generics for some befuddling reason): up-casts allow you to circumvent the type system and force a particular value to take a particular type, thus recovering the lost information, but can be unsafe.

Under row polymorphism, however, you can achieve basically the same effect, but you get to save any unused/irrelevant type information inside of a row variable (the `..` in your example) and include it in the result. Then, no information is lost, and no safety is compromised:

    fun f {x : real, y : real, R} = {x = 2 * x, y = 2 * y, R};
    (* f : {x : real, y : real, _ : ..a} -> {x : real, y : real, _ : ..a} *)

    f {x = 2.0, y = 3.0, z = 4.0};
    (* it = {x = 4.0, y = 6.0, z = 4.0}; *)

Users of languages like Lua or Python might be reminded of "duck typing", and not without reason: row polymorphism enables many of the same programming patterns/techniques/abstractions in typed languages that are enabled by duck "typing" in untyped ones. It's a shame that more typed languages don't incorporate the idea, because it's a really wonderful one, and it can lighten the relative "burden" of writing typed code in some cases :)

The folks at INRIA has come up with some really neat stuff, and, if you haven't already, it's definitely worth it to check out their publications, too!

int_19h7• 10 years ago

Oh, and the other neat thing is that OCaml cleanly separates subtyping and inheritance. In other words, in OCaml, class inheritance is for implementation reuse only, and is completely orthogonal to subtyping. In many common cases (similar to Java etc) inheritance does produce a subtyping relationship in practice; but because of this decoupling, you don't get one shoehorned onto your classes when you just want the other.

PeCaN• 10 years ago

Honestly for me, it's just that it's damn useful. It's not especially new and shiny and hip, but it's easy to think in and easy to reason about. OCaml is a nice tool for writing software. That's about it. It's pretty simple functional programming that can switch to imperative and object-oriented programming where appropriate.

patrec4• 10 years ago

Ocaml is concise yet comparatively simple (despite various warts), offers good javascript interop and (quickly!) compiles to fast code and has a good type system. It's extensively used for language prototyping and tooling (e.g. at facebook). Rust's compiler was originally written in Ocaml: https://github.com/rust-lang/rust/tree/ef75860a0a72f79f97216....

Compared to Rust it compiles much faster, offers an interactive shell and is more expressive and mature, but doesn't offer similar low-level and parallelism capabilities and eco-system coherence (e.g. there are half a dozen competing build systems). I also believe that Rust, with it's clearly defined niche will win a much bigger total market share.

Compared to Clojure it offers static typing, much better startup times and native code compilation, but doesn't have similarly modern functional datatypes and the standard library is less well thought out.

There are a couple of other things which are rare or nonexistent in other languages:

- Polymorphic variants, the article linked downthread has some use cases

- extensible variants

- Functors (see https://realworldocaml.org/v1/en/html/functors.html)

- a structurally typed OO system, as someone has already mentioned

It also has a few interesting syntactic ideas that I wish more languages would steal:

- Nice sugar for passing keyword arguments: `f ~somearg` is the same as `f ~some_arg: some_arg`. I wish I could do `f(=somearg, y)` instead of `f(somearg=somearg, y)` in python

- It supports new user-defined operators in a way that doesn't render your code unreadable, because the precedence is derived from existing operators.

- A nice syntax for "importing" from a namespace for an expression. Assuming Mod defines `foo` and you can write Mod.(foo x (bar f x)) instead of `Mod.foo x (Mod.bar f x)`

int_19h7• 10 years ago

Love the design! Maybe not the simplest to use, and fairly verbose at times... but it's very clear what's going on at any given point - and given how hairy macros can get in production code, I think emphasizing clarity is the right approach.

patrec4• 10 years ago

What's the story for error messages? I read it's inspired by racket; the big thing about racket is that from what I hear it seems to be the only language that has good support for syntactic extension without sacrificing error reporting.

otini7• 10 years ago

Since macros aren't just a plugin but are directly baked in the compiler code, error messages can be anything we want… what are you suggesting?

patrec4• 10 years ago

Let's take your static printf as a concrete example. Here's what a not so great error message looks like (ocaml's built in Printf):

  utop # Printf.sprintf "Some int: %d another int: %d" 1 "2";;
  Error: This expression has type string but an expression was expected of type int

Here's what a good error message looks like:

  clang -Wall printf.c
  printf.c:6:52: warning: format specifies type 'int' but the argument has type 'char *' [-Wformat]
     printf ("Some int: %d some other int: %d\n", 1, "2");
                                           ~~        ^~~
                                           %s

Notice how it also highlights the mismatched part in the string-based DSL.

lacampbell2• 10 years ago

Can someone tell me why I would want statically typed macros? Why not just simply type check after macro expansion phase? Seems a lot simpler and more intuitive.

It strongly feels like a case of everything looks like a nail when you really like hammers (static typing), but I am willing to be proven wrong.

sordina2• 10 years ago

There may be a bunch of other reasons, but the one that springs to mind is localisation of errors. You'd ideally like to know about any mistakes you've made in your macro before you run your macro. And when you do learn about a mistake you'd like the error to describe where the mistake is in macro-land rather than generated-code land.

junke• 10 years ago

Why just type errors? You might want to trace any error that occurs at runtime back to the original source code anyway.

adamnemecek• 10 years ago

Look at the C++ templates, I think that some of the clusterfuck that is C++ template compiler errors would go away if templates were type aware.

otini7• 10 years ago

Firstly, if your macro does not typecheck, it means that executing it might result in a segfault or something similar, since macros are regular OCaml functions. Secondly, if macro typechecks, then it is guaranteed to expand without errors, and I think that's a nice guarantee to have.

lacampbell2• 10 years ago

Firstly, if your macro does not typecheck, it means that executing it might result in a segfault or something similar, since macros are regular OCaml functions.

I see. The only time I've extensively used (real) macros has been with racket, where they aren't regular functions at all. I suppose my ignorance of Ocaml macros is showing here.

Typed, modular macros for OCaml

Related Stories