Why Erlang matters

http://petrohi.me/post/35644212001/why-erlang-matters

by petrohi • 14 years ago

73 74 14 years ago

rustc• 14 years ago

On a related note, has anyone tried Elixir[1], which is a Ruby-like language that runs on the Erlang VM. If so, what project was that? how was your experience?

[1] http://elixir-lang.org/

rubyrescue• 14 years ago

Inaka does a lot of Erlang. We really like Elixir and we're excited about it. It's still early but it gives you the runtime environment of Erlang with the syntax of Ruby. It has a few very active, passionate folks behind it which will cause it to continue to grow.

ludicast• 14 years ago

Elixir is awesome, it's like Ruby and Clojure having sex on an Erlang waterbed. However it still is maturing at a quick clip so I haven't done any production work in it.

But since Jose Valim is probably my favorite developer alive, I have faith that Elixir will do for Erlang what Rails did for Ruby.

I know it sounds like I'm comparing Rails (a framework) to Elixir (a language), but what I am comparing is a way of bridging-in Joe/Jane Developer.

commentzorro2• 14 years ago

Erlang's been fine for showing us what's good and what's bad. But most of all, Erlang allows us to take the good parts and move them to a "friendlier" language when the time comes.

hvs• 14 years ago

What do you mean by "Erlang allows us to take the good parts and move them to a 'friendlier' language when the time comes"? Is there something I don't know about porting code from Erlang to other languages? (Not being snarky, genuinely curious.)

commentzorro2• 14 years ago

Keep the idea of lightweight processes, crash at will, message passing, etc. and move these techniques into other languages so we don't have the awkward syntax.

ericmoritz2• 14 years ago

Libraries like Akka <http://akka.io/>; or the Celluloid stack <http://celluloid.io/>; are nice and bring the good parts of OTP to more "friendlier" languages but you're still missing some fundamentals the Erlang VM which bring predictable performance such as distributed garbage collection.

jfaucett• 14 years ago

People are always asking what is a good HN article, I'd say this is, great links to research that backs up many of the assertions.

"Erlang matters today because it demonstrates how these semantics can be elegantly packaged in one language, execution model and virtual machine." I think yes, in a theoretical sense this is absolutely true. The question is why is it not a language in widespread practical use?

davidw3• 14 years ago

Here's something I wrote about Erlang a while back, and why I don't think it'll ever go mainstream:

http://journal.dedasys.com/2007/09/22/erlang

petrohi9• 14 years ago

One of the huge hurdles for me personally was Erlang's heritage of Prolog syntax. I actually gave up once, and only cleared it second time around. Now I can see certain elegance and succinctness in the syntax.

seiji2• 14 years ago

That is the only proper way to learn Erlang. Your first week or two becomes so frustrating, you just leave for a month. Then it starts to click when you come back.

Things are much easier now with line numbers in error messages though.

klibertp9• 14 years ago

> The question is why is it not a language in widespread practical use?

1. It kind of is, I think RabbitMQ, CouchDB and Riak are doing ok, for example.

2. Both syntax and semantics are sufficiently different from "industry standard" to be a hurdle for majority of people... I'm being told. I don't understand this; the more different the language is the more interesting it seems to me and the more happy I am to learn it[1], but I heard this argument enough times to accept it as (sad) reality.

[1] And there is still so many of them to learn! Take a look at any one RosettaCode page (http://rosettacode.org/wiki/Y_combinator) if you don't believe me :)

derdesign3• 14 years ago

Adding to your response: Big ones are using Erlang, such as Github, Heroku, Facebook and Amazon...

codewright5• 14 years ago

RabbitMQ has some pretty serious limitations related to it being implemented on Erlang that keep it pretty strictly in the realm of being a message queue rather than a generic queue.

CouchDB is more C++ than Erlang these days, bad example. It hasn't been mostly Erlang for a really long time. Erlang basically only does the clustering behavior, which is something Zookeeper et al. do already.

Riak is the only solid example out of those you listed, and its known for having poor raw performance, excellent availability, easy clustering, and awful usability.

dhucerbin• 14 years ago

>> It kind of is, I think RabbitMQ, CouchDB and Riak are doing ok, for example.

And scalaris [1]. For me it's killer app for Erlang.

[1] http://code.google.com/p/scalaris/

rossjudson• 14 years ago

What's missing from your article is a demonstration or theory that outlines how transmission over CPU-interconnects of immutable messages as blocks of memory outperforms the cache coherence algorithms already in place.

Immutable message transmission has no way to say "I already know about this message; don't send it again."

Most of the good characteristics of Erlang that you describe are achievable in other languages.

So what I'm asking is, by what concrete mechanism does immutable message passing outperform cache coherence?

seiji2• 14 years ago

It's not about raw performance. It's about getting shit done and keeping shit highly available. In large systems, shared data isn't mentally optimal to work with (see: every multithreaded program with shared data segfaulting for some unknown reason this very second).

We can always drop down to C or asm to make ultra-shared data structures with minimal overhead. We could always do that. But people end up wanting to spend their precious lives making a difference in the world and not checking the return value of malloc thirty times per day.

derdesign3• 14 years ago

> But people end up wanting to spend their precious lives making a difference in the world and not checking the return value of malloc thirty times per day

So true!

petrohi9• 14 years ago

Immutable message passing is an abstraction. In a optimized virtual machine it may well be implemented by leveraging cache coherence. The sender process writes memory block. This places it into its local cache. Then the receiver process reads this memory block. This uses cache coherence to sniff the block directly from the sender's cache. This is awesome way to leverage parts of cache coherence protocol that play well with immutable memory blocks. It can also scale well if sniffing is not a broadcast. Cache coherence can quickly get ugly when a cache line is contested for writes--be this false sharing or a spin lock. My point I guess is that immutability and message passing promotes awesome parts of cache coherence and limits the need for the ugly parts.

bradkittenbrink• 14 years ago

I don't think he's arguing that message passing will generally outperform shared state with cache coherence. Particularly not for applications which are designed well. The argument (as I understand it) is that since the hardware is fundamentally doing message passing, (and since that will become even more of a problem as hardware continues to scale) it will be easier to design an application that performs well (but maybe not optimally) if you use the message passing paradigm, and easier for your application to perform poorly if you don't. You're right though that this isn't exclusive to erlang, but applies to all languages which eschew shared state.

dschiptsov• 14 years ago

It is not just "matters". It is a deeply-researched and well-engineered explicitly concurrent programming language by smart and passionate people. It is a very rare example of first-class project, compared to the common nonsense, like Java or, god forbid, NodeJS.)

If it is not that popular (which is rather good) it is not a language's fault. It is due to inability to acquire and maintain an appropriate mindset.

btw, there is a nice lecture - http://www.youtube.com/results?search_query=Erlang%2BJoe%2BA...

klibertp9• 14 years ago

> language by smart and passionate people [...] compared to the common nonsense, like Java or, god forbid, NodeJS.

Are you implying that creators of Node and Java are stupid and lack passion? It seems like you are... And it's a) not relevant here at all; b) simply not true.

> It is due to inability to acquire and maintain an appropriate mindset.

It's debatable, but I tend to agree here.

shtylman• 14 years ago

This reminds me of a great quote.

"There are only two kinds of languages: the ones people complain about and the ones nobody uses" -stroustrup

If you have to write about how your language matters, you have already missed the point. If it mattered, people would be using it.

gamache• 14 years ago

You make phone calls, right?

derdesign3• 14 years ago

I bet you or a friend of yours chats on facebook[1] or Watsapp[2]...

[1] http://www.erlang-factory.com/upload/presentations/31/Eugene...

[2] http://blog.whatsapp.com/index.php/2012/01/1-million-is-so-2...

confluence4• 14 years ago

Guess you didn't get the memo that worse is better.

That "common nonsense" you speak of (aka Java) runs on billions of devices and handles trillions in revenue. Erlang keeps legacy phone switches running. It's like people who say X language is amazing and lament the crap that is English.

Who cares?

English runs the world economy.

klibertp9• 14 years ago

> Guess you didn't get the memo that worse is better.

In what sense? Why do I have to paste this link: http://www.paulgraham.com/avg.html here of all places?

davidw3• 14 years ago

> If it is not that popular (which is rather good)

Not popular often means you have to write libraries or code yourself that in a more popular language would already be sitting around ready to use. In other words, in a business context, it might cost you more money to use a less popular language.

> it is not a language's fault

I don't agree. I think for instance that Erlang could likely have a more C like syntax, and that would have bought it more users. There's plenty of other stuff that you can do to make a learning curve relatively easy, so that people can slowly start out and get to know a language.

Of course there are factors beyond the control of a language's designers and proponents, but looking at things like PHP and Ruby, you can certainly get popular without, say, having huge corporate backing.

klibertp9• 14 years ago

> Erlang could likely have a more C like syntax [...] so that people can slowly start out and get to know a language

This reminded me about famous Linus rant against C++ - in the same vein it can be argued that "cryptic" (actually very readable and consistent IMHO) syntax would be a win even if it did nothing more than keep C (style) programmers out of Erlang development :)

You're certainly right from business perspective, however.

pdog• 14 years ago

In what way is Java not a first-class project? That "common nonsense" had some of the greatest minds in computing behind it.

pla3rhat3r• 14 years ago

How come there are no Erlang articles start with, "this new technology is GREAT!" Oh yeah. Nevermind.

xnld• 14 years ago

But... Erlang isn't new technology.

rartichoke2• 14 years ago

Would it be safe to say Scala with the Akka lib offers the same robustness and distributed scalability as Erlang?

Erlang is using the actor model and so is Akka.

I really want to learn one or the other as a goto functional language.

I keep looking at sites like Netflix, Twitter and Foursquare. Those seem to be sites running on either Java or Scala.

Why would they pick that instead of Erlang?

SoftwareMaven• 14 years ago

Pragmatism. It is easier to find JVM-knowledgeable people than to find BEAM-knowledgeable people.

The actor model isn't the only aspect that gives Erlang its robustness. In fact, far more important is its supervisory model, which goes as deep as you want it and can supervise across system boundaries.

I keep waiting for the language that learns these lessons from Erlang with a happier syntax, but I have yet to find it.

codewright5• 14 years ago

Going back to the OP, does Akka provide that depth or not? Akka is a LOT more than just Actor-based concurrency.

ericmoritz2• 14 years ago

Akka does have supervisors. If you find Scala to be a happier syntax than Erlang (eye of the beholder), than Akka is worth looking into. It appears to be directly inspired by OTP and Erlang's "let it crash" ideology.

petrohi9• 14 years ago

It is interesting to look at virtual machines from the prospective of what they enforce. The JVM was a huge step forward when it allowed for safer memory management. Garbage collection is possible precisely because JVM enforces its memory model. You cannot use pointer arithmetic and arbitrary casts. This helps to ensure the level of ecosystem and make developers more productive. Erlang VM enforces immutability at its core. We can call it safer concurrency management. We can combine hundreds of libraries and still maintain this safety. I would argue that if something is not enforced in the VM, but rather a is "good guideline", the ecosystem descends to the lowest safety that is enforced.

lttlrck• 14 years ago

I wonder how JVM garbage collection impacts real-time systems vs. Erlang.

rdtsc2• 14 years ago

Usually badly. Erlang got one thing very right -- private heaps for each process. So when garbage collectors run they never have to stop all the processes and collect, and that is very important in types of system Erlang runs in.

I know Java has "pauseless" garbage collector but I think that is just marketing talk. Given how references to objects can be shared between threads I don't see a way for it to work.

petrohi9• 14 years ago

Erlang VM is much friendlier to garbage collection. For example immutability removes one of the biggest overheads of GC--tracking reference mutations. Process isolation allows to run garbage collection concurrently and without employing esoteric techniques. Many processes are short lived and never reach GC point, when they terminated memory is reclaimed wholesale.

rdtsc2• 14 years ago

> Would it be safe to say Scala with the Akka lib offers the same robustness and distributed scalability as Erlang?

Depends why you picked Erlang. Erlang has unsurpassed fault tolerance built in. This mean ability to crash and restart individual processes, ability to hot code reloading. Its standard library by default comes with such things as a distributed application controller that will make sure you application will immediately start running on another machine if it crashes locally.

I don't know of any other languages or platforms that provide that as a first class.

Then for things Erlang would be used you really would want something battle tested. Eventually Akka will be battle tested and this won't be an issue but I think it is not there yet.

rartichoke2• 14 years ago

I mainly wanted to use it to develop web apps. Everything from the server up to the business logic.

I picked Erlang or Scala/Akka because both of them seem to have reasonable support for web libs and are based around distributed concurrency.

Erlang was a choice because it seems to run really well on medium-tier hardware. What can be served with a single Erlang server might take 10 Python servers, etc..

I haven't seen any Akka benches but if huge crazy scale web sites are serving everything with Scala then I have to assume it's speedy.

I also want to dabble in the idea of creating 2D games and I know Erlang's math computation speed isn't very good but Java's (and that means Scala too) is very reasonable.

It would be nice to just learn Scala and use that for the core game engine (not 3D) logic / collision detection on the server side. I think with Erlang you would have to end up writing the game stuff in C to get reasonable performance?

rvirding3• 14 years ago

Every time I read something about the weird Erlang syntax I get so tired and try and tell myself not to get into the discussion, it's not worth the effort, some people will never just never learn, etc. But sometimes I can't help myself. So some of my thoughts on this:

- Syntax is the easiest part of learning a language. It's all written down and its just a RTFM. And yes, I have written a lot of C and lisp and prolog and erlang and ... and I never have problems with the syntax.

- If people worry so much about the syntax what happens when they get to the semantics?

- Seriously, why would you want to Erlang to have similar syntax to a language with different semantics? In my view that would be asking for trouble. I think one reason why I haven't had trouble with mixing languages is that both the syntax and the semantics are different. C looks like and behaves like C while Erlang looks like and behaves like Erlang.

- The erlang syntax is actually very simple, much simpler than the languages you would like it to look like.

I will now stop before I say something nasty.

I liked the blog.

petrohi9• 14 years ago

When semantics are very different and syntax tries to pretend they aren't to create false familiarity this will lead to some very painful brain-splits later on. I can assert from my own experience with alternating the use of = between Erlang and C++.

Glad you liked the blog.

rvirding3• 14 years ago

Yes, in retrospect using = was probably not a very good choice. I will admit some things in Erlang didn't receive much thought before being added while to some things we gave a lot thought; for example the language/systems basic properties and the semantics of concurrency, message passing and error handling.

hassy• 14 years ago

It's almost an acid test: the person complaining either has only a cursory familiarity with Erlang or is someone who prefers style over substance.

z3phyr3• 14 years ago

How does Erlang compare to Haskell?

petrohi9• 14 years ago

Two most important differences that come to mind are 1) Erlang is dynamically typed language vs statically typed Haskell; 2) Haskell is functional language with lazy evaluation where side effects are the primary concern, Erlang has eager evaluation.

z3phyr3• 14 years ago

But, they are equally deployable in similar circumstances. Right?

Fixnum• 14 years ago

Until recently, Haskell didn't have an equivalent to Erlang's built-in support for distributed computing, but the new Cloud Haskell library (http://www.haskell.org/haskellwiki/Cloud_Haskell) provides message-passing concurrency with a semantics very close to that of Erlang (but with more support for type-safety).

I'm not sure Cloud Haskell is ready to dethrone Erlang (and the OTP framework, and tooling for debugging distributed processes, and special VM features, ...) for high-reliability distributed computing today, but Haskell and its libraries are evolving with absurd speed and the Cloud Haskell ecosystem could certainly be competitive with Erlang in the near future. Haskell can certainly be used to write high-performance network applications (e.g., Mighttpd, Warp), so it's surely only a matter of time ...

z3phyr3• 14 years ago

+1 to Cloud Haskell.

But, they should change its name so that it sounds like a library and not another language dilect. :)

Judging by its name, I thought that its a new haskell dilect ;)

rvirding3• 14 years ago

I think one significant difference is in the actual implementation, or rather its intended use. The BEAM implementation is built to be used in products, it is very stable and bug-free and the OTP team are very cautious when making changes. Incompatible changes are introduced with early notices and warnings, usually 3 major releases before the change is made.

My impression is that Haskell implementations are more used as research tools and there is less control of what goes in and how it is to be supported.

In that sense it would be safer to build a product on top of Erlang.

That is just my impression and I could be wrong.

millerm• 14 years ago

I was just put on a new Erang/OTP project at work and will be working on it for the next 9 months or so. I'm currently on a "ramp up sprint" to get to learn the language, tools and libraries, then we will move on to development. I'm really enjoying the language. OTP is quite elegant and just awesome. Sure, there are a few annoying things in the language that I have already read about on the Net/here but, I can look past those paper cuts. I've been really diggin' Lisp (Clojure, Racket, CL and Scheme) and other functional languages and have been enjoying the concepts so much. I've really become enamored with functional languages and really don't want to write another lick of Java. I'll keep the Objective-C because I have to, you know why. I am not dissin' (crap, I used that word) Java. But, it is nice to be given the opportunity to think differently (damn, another unoriginal phrase).

Anyway, just giving a shout out to Erlang as I am finding the platform to be very interesting and that makes the day go by much better.

petrohi9• 14 years ago

Great to see more adoption for Erlang/OTP. The return on investment is clearly there for us at connect.me. We're implementing our backend and API in Erlang/OTP.

Can you share what type of application you will be working on?

dsiemon• 14 years ago

http://dl.acm.org/citation.cfm?id=2209269

petrohi9• 14 years ago

Great article. I've been inaccurate in the blog saying that cache coherence will or should go away. What I mean is that ugly (non-scalable) effects of cache coherence as implemented by today's vendors can be relieved by Erlang VM to some degree. In the comment above [1] I say "It can also scale well if sniffing is not a broadcast". Your article is to the point by addressing exactly this issue with tracking of sharers.

[1] http://news.ycombinator.com/item?id=4780706

chaitanyapandit• 14 years ago

I had the syntax fear when I started learning Erlang, but I think the syntax is tailored to the language itself: - Recursion - Pattern Matching

The problem occurs when you try to translate the Erlang syntax to the language of your choice, which isn't the right approach. C, and Java don't use pattern matching, so naturally their syntax is not designed for it.

Maybe one should write a small program to demonstrate the key functionalities of Erlang, like write a TCP packet decoder etc, then try doing the same in your prefereed language and then see what makes sense.

Why Erlang matters

Related Stories