HN user
Understandable how you'd get that impression, but it's off the mark. We do lots of "ordinary" programming that looks nothing like waiting a compiler. Data analysis, high performance packet processing, log analysis, UI work, you name it.
We use OCaml as a general purpose tool, and it really excels in that role.
But we did, though we used the term "algebraic effects". And while it's being actively worked on, it hasn't landed yet, so I don't know that I'd call it recent...
These are a bit out of date, but they give you some flavor of the library, in an interactive form:
If you're interested in understanding more about what Hardcaml is trying to achieve, you might enjoy this interview with Andy Ray, the primary author of the library:
https://signalsandthreads.com/programmable-hardware/
It's a podcast, but you can read the transcript too, if you're not into that format.
One thing that really struck me about the conversation is how much of the benefit that Andy sees coming from Hardcaml is from the testing story.
Hilariously, yes.
Right now, we're looking to hire an FPGA engineer in London. Our hardware team is still small, but I expect we'll be doing more hiring as time goes on.
https://www.janestreet.com/join-jane-street/apply/ldn/full-t...
And, shameless plug, if you're interested in seeing what it's like using a functional language at scale for solving real problems, well, you can apply...
Hey, author of the linked post here.
A few thoughts, since some things have changed since that post was written:
First, the tooling limitations that I mentioned in the article have gotten a lot better. In particular:
Merlin now provides IDE-like functionality for your editor of choice (including code, vim, and emacs).
Also, Dune is an excellent build system for OCaml that does an enormous amount to simplify the build process, and tie a bunch of different tools in the ecosystem together. One great thing about Dune is it does a lot to unify the experience we've long had inside of Jane Street with the open-source OCaml experience. It's really a big upgrade.
We've also made some progress on debugging tools, like the spacetime allocation profiler. There's also active work on making GDB/LLDB debugging in OCaml really first class.
Also, OCaml has had some major industrial uptake. Notably, Facebook has several major projects built in OCaml (Hack, Flow, Infer) as well as their own syntactic-skin-plus-tooling on top of OCaml, in the form of Reason. Reason has gotten a lot of traction in the webdev world, which is awesome. Bloomberg, and Docker are some other big names that have real dependencies on OCaml, along with some more names you probably don't know like Ahrefs, LexiFi, and SimCorp.
People sometimes feel like Jane Street is the only real user of OCaml, so they imagine that Jane Street's needs are the ones that drive the language priorities. So, the thinking goes, if you're not a trading firm, you should look elsewhere. But this is the wrong picture. First, there are other serious users, as discussed above. Besides, the community doesn't just roll over and do what we say. If you don't believe it, go and see how often our PRs to OCaml get rejected.
And even our interests in the language have grown beyond what you might imagine a trading firm would care about. We use OCaml for building traditional UNIX system software, like MTAs, for designing hardware (via HardCaml), and for building dynamic browser-based applications (via Incr_dom).
For sure, there are still challenges of being a minority language (and there's still no multicore GC, despite some exciting progress). But I believe OCaml is a yet better choice than it was in 2011 when I wrote the article.
OCaml isn't without its warts (like any language), but I don't quite think you've nailed them.
For one, the different stdlibs are in fact highly compatible. Basic types (option, result, string, int, array, float) are all the same, so code using different stdlibs works together seamlessly most of the time.
Lwt and Async are a different story, and there is a real incompatibility problem there.
The syntax extension story is pretty clear and simple: PPX rules the roost, and the tools for building PPXs are quickly getting better and more unified. Reason is an interesting variant in the ecosystem, but its existence doesn't amount to a wart in my eyes. It's an alternative syntax that you can use interoperably with the rest of the OCaml ecosystem (and Dune makes that awfully easy.)
That's not the intended implication, and Jane Street has in no way taken over the show. Our PRs are discussed vigorously, just like everyone else's, and they're definitely not always accepted!
Indeed, one of the great things about the OCaml compiler development process is that the core team is highly skeptical, and does a good job of rejecting marginal changes.
We also just released improved docs for Base. Still more to do, but much better than before:
I'm not sure I agree with the "don't do it often". Having jane-script (which is our OCaml scripting system) has allowed us to greatly reduce our dependence on Bash.
That said, there are still little things we use Bash for. But our tolerance for large bash scripts has diminished greatly over the years.
Indeed, js_of_ocaml is kind of shocking in its fidelity. It can compile even highly complex libraries that do lots of runtime tricks. Async_kernel and Incremental_kernel both compiled and worked without any issues whatsoever.
We've even built some support for making incrementally rendered web-apps in OCaml, using Async and Incremental. Here's a link:
https://github.com/janestreet/incr_dom
It sounds like Bucklescript is doing something quite different, which is to aim for pretty JavaScript output, while compromising and maintaining semantic consistency with OCaml. I don't fully understand the use-case, but for us, js-of-ocaml is clearly the thing we want.
These days, it's pretty good. opam is the tool of choice:
I've found it to be surprisingly reliable and easy to use.
Actually, we've funded lots of things that we don't directly use. For example, we funded the development of OPAM, which we don't use internally at all. Moreover, the funding of OCaml Labs was aimed broadly at improving the OCaml ecosystem, rather than just addressing our narrow internal needs. Our work and funding on compiler improvements like Flambda is aimed at things we want, but they're also of broad utility to the users of the language. Merlin is another example of something we've supported that is useful to us internally, but also useful more broadly.
This is of course self serving in the sense that we think the OCaml ecosystem is important to our future, and so we want to help it flourish. But it's a relatively enlightened form of self interest...
Here's an example of someone from Linked-In describing a "low latency" setup with a 6GiB heap.
https://engineering.linkedin.com/garbage-collection/garbage-...
I've heard similar things from folks at Twitter, IIRC. But I do find the whole thing kind of mysterious, I have to admit. I'd love to learn that I was wrong.
Async and LWT give you lightweight concurrency mechanisms. Libraries like Async.RPC give you lightweight remote invocation, and Async_parallel gives you simple mechanisms for spinning up multiple physical processes and communicating between them.
Don't get me wrong: OTP by all accounts has richer support for this kind of stuff. I think OCaml is a better language for many purposes, but OTP is a great runtime and set of libraries whose equal is not yet found in any other language as far as I can tell.
For an HTTP server you should use Async or LWT, which are lightweight concurrency libraries. You can handle quite high levels of concurrency with either one. My impression is that people building highly parallel HTTP servers do quite well with collections of processes with a load-balancer in front, but it's not my area of expertise.
For what it's worth, the world has changed since then: Core_kernel (which is the highly portable bits of Core, which is most of it) works on Windows just fine. Indeed, WODI, which is the best windows package manager for OCaml right now, ships with it.
Real World OCaml does indeed cover Core and related libraries, but you can also just read the API documentation:
Unix support is definitely better than Windows. That said, the core language and base libraries are pretty portable. Here's one nice retrospective on a move from Python to OCaml for a program that needed portability to Windows as a fundamental concern.
http://roscidus.com/blog/blog/2014/06/06/python-to-ocaml-ret...
Probably the biggest problem on Windows is that OPAM, the package manager, doesn't work there. That will come eventually, though.
My experience has been building parallel and distributed programs using multiple runtime instances that communicate via message passing. There are libraries that help automate this, link our own Async_parallel (https://blogs.janestreet.com/async-parallel/). One advantage of this approach is that it scales beyond a single box.
There is work going on at OCaml Labs on a parallel runtime. I suspect it will be useful, but in the end, message passing is I think a better idiom than shared memory threads for parallel programming. When the true parallel runtime lands, I'm not sure that we'll actually use it much for running truly parallel threads.
It depends on what CS1 really means. I do think cs51 is comparable to CIS120 at Penn, and I think the later is pretty clearly CS1. That said, I find it hard to really figure out what the precise criteria are. It's pretty clear that Harvard encourages concentrators to take cs50, but I think the same course at other institutions would often be skipped by concentrators.
The biggest change is OPAM, http://opam.ocaml.org/, a top-notch package management system that makes installing OCaml packages much easier.
Shameless plug: there's also a newish O'Reilly book, Real World OCaml http://realworldocaml.org, which I think is a big help in learning the language.
Oh, and there's OCaml Labs, a new lab at Cambridge University that's dedicated to improving the language.
And of course the compiler is constantly making progress. The upcoming 4.02 release is a pretty fun one, which I documented here: https://blogs.janestreet.com/ocaml-4-02-everything-else/ And before that, changes like GADTs and first-class modules landed, which have been quite useful extensions.
Really, it's a very active and fun community these days. The language is getting better quite quickly, but mostly in conservative and tasteful ways. The people in charge of the core language have been doing a great job, and the community infrastructure (things like OPAM) have been making big strides as well.
Jane Street makes use of OCaml's built-in metaprogramming facility, camlp4. While these are in some sense "our own extensions", they're wholly shareable without using a hacked version of the compiler, and just because we build our code using these syntax extensions, doesn't mean that users of our libraries need to do the same. You can use Core_kernel, for example, without using our syntax extensions.
The French thing is a non-issue. The compiler is written and documented in English, and all the main contributors are fluent English speakers, and the mailing lists are almost entirely in English.
I would have thought cs50 is a cs0 course, and cs51 is the cs1. cs50 is the concentrators-and-non-concentrators course that's all flash and fun. What do you think is the cs0 at Harvard?
As for your point about Penn's CS120, I guess I don't fully understand the criterion. From the way you described it, CS120 seems like a CS1 course: not for someone who has had zero programming, but the very first course taken by most CS concentrators.
FWIW, I spoke to Benjamin Pierce about the structure of the course, and he said he didn't want to use "Real World OCaml" because it's too early of a course: he has students who he thinks don't yet understand things like what a scope is. From the sense I got from him, it's quite early in the curriculum.
That said, this is mostly minor quibbling. I think there's little doubt that "very early" courses, for some reasonable definition of "very early", lean towards Python. And reasonably so.
(Author of the article here)
The stdlib that ships with the compiler is indeed minimal, though it is used for things other than the compiler. It can be used for other projects, but you probably want something more full-featured. Core, and the more minimal and portable Core_kernel (https://github.com/janestreet/core_kernel) is a full-featured alternative that is growing in popularity, and is what the book I worked on (Real World OCaml, http://realworldocaml.org) is based on.
OCaml (and most of both the stdlib and Core) default to immutable data structures, but OCaml has good support for programming imperatively, to its credit, in my view.
ML (OCaml and SML) seems to get short shrift in your analysis. CS51 at Harvard is in OCaml. Brown's cs017 is OCaml, Scheme, Scala and Java. Penn's CS120 is OCaml (I think. It definitely used to be, but I can't find the course online anymore. You can reach out to Benjamin Pierce to ask, though.) And CMU has an SML-based FP course as part of their intro sequence
None of this is to deny the fact that Python is highly popular as an intro language.
I think we're pretty clearly the most intense user (most code, most developers using it), but not technically the biggest company using it. Bloomberg and Facebook are examples of bigger companies using it in serious ways.
I don't think this is correct. All pure data structures are threadsafe. It's true that mutable datastructures like the built-in hashtable are not threadsafe. But isn't that the standard tradeoff with threads? Most things aren't threadsafe, but with mutexes and condition variables and the like, you can make them so. Surely one doesn't want to make the performance destroying mistake of making _everything_ threadsafe.
To be clear, it is true that the runtime has a single lock for the GC, so for now, it's not possible to gain physical parallelism from multiple threads. That's being worked on, but I have to say that I rather prefer message passing and I don't know that how we build programs will change much when shared memory threads are available.