Twenty Years of Open Source Erlang: A Retrospective from the trenches

https://www.erlang-solutions.com/blog/twenty-years-of-open-source-erlang.html

by francescoc • 8 years ago

287 66 8 years ago

mahesh_rm• 8 years ago

Kudos to Francesco and the guys at Erlang Solutions for their outstanding work in the open source high concurrency field!

francescoc3• 8 years ago

Thanks, but it is not only us at ESL. Thank Joe, Mike, Robert and Jane, the OTP team and Ericsson first and foremost, and the community as a whole. It has been an amazing journey, looking forward to the next 20 years :-)

xutopia• 8 years ago

I didn't hear about Erlang as much as I did since Elixir came out and showed me the power of the VM and its ecosystem. As a ruby developer it's easy to pick up Elixir and I'm learning so much more about functional programming and systems than I did ever before.

justbaker• 8 years ago

The Elixir core oddly was my first taste of an Erlang project. I'm amazed at how the ecosystem and language has expanded since 2011. Jose Valim is a very personable person and thriving code ecosystems are built around individuals like that.

binaryapparatus2• 8 years ago

What an amazing language. I am working on a Elixir project, with a lot of Erlang peaces used directly. Integration is flawless and beautiful to observe. I do have many languages in my toolbox but only few of them were truly exciting for me, Haskell six or so years ago and Elixir/Erlang/OTP this year.

Sileni• 8 years ago

I've really wanted to dig into functional languages like this, because I love the theoretical benefits they provide, but haven't really sunk my teeth into one in a way that would let me understand the flavor.

Let me ask, were you immediately drawn to Haskell and Erlang, or was it a feeling that grew while working with them? Would you recommend working with Haskell or Erlang directly before trying out a language like Elixir, or should I go high level down?

Thanks

binaryapparatus2• 8 years ago

I did initial plunge into Haskell when I started using xmonad, looking at config file was visibly 'something else'. I would probably recommend starting with Haskell first -- if you want to properly blow your mind that's the way to go. All the principles are common so diving into Elixir took very little time afterwards. Some principles are getting their way into imperative languages and it is much easier to grasp them on pure functional level first. Understanding and using map, filter and reduce, in JS project, was much easier to grasp immediately because they borrow that logic from functional languages.

Elixir is beautiful but probably few notches easier to understand and write than Haskell. Reason I would start with Haskell is that it doesn't hide functional logic with syntactic sugar. For example variables in Elixir are not mutable but since you can assign to the variable already used in function, it can prevent programmer from understanding what immutable really means.

pmarreck5• 8 years ago

In my case I found I was already starting to write Ruby in a very functional way (PORO which is like POJO, only setting properties at object instantiation, focusing on data expressed in built-in classes instead of custom classes, etc.) and that seemed to ease the transition quite a bit.

I've found that "immutability all the way down" has eliminated an entire class of bugs related to unexpected mutation, deadlocking/concurrency issues, etc.

Maultasche• 8 years ago

Before learning Elixir, I used functional-like functionality in C# (LINQ) and Javascript (particularly Rambda and Bacon.js) before learning Elixir, and I found that was plenty of preparation to understand Elixir. There's no need to start off with another functional language prior to learning Elixir.

I'm currently learning Elixir and writing about it so that anyone interested in Elixir can learn with me as I learn it. The goal is to help other learn Elixir as well.

I make some comparisons to features in C# and Javascript, and the occasional reference to other languages like Python, but you don't have to know any of those in order to benefit.

If you're interested, come check it out: https://inquisitivedeveloper.com/

losvedir• 8 years ago

Full time Elixir dev here. This was a very neat walk down memory lane. Thanks for sharing! Since Elixir is so new, and I wasn't that aware of Erlang prior to it, it's sometimes easy to forget just how long Erlang has been around!

andruby• 8 years ago

I used Erlang before Elixir and it was awesome, albeit with a syntax that required getting used too. I've often looked at bridging the gap between Ruby and Erlang, and closely watched and tested solutions like Reia [0], Erlectricity [1] and Ernie [2].

I was delighted when Jose started getting involved in the space and released Elixir. Thanks Jose!

[0] http://reia-lang.org

[1] https://gilesbowkett.blogspot.com/2007/05/erlectricity-erlan...

[2] https://github.com/mojombo/ernie

sb8244• 8 years ago

Ernie was my first foray into Erlang (run many production elixir apps now). It's still humming along in one of our legacy services. Never had an issue or downtime event

dudul3• 8 years ago

Elixir's syntax may look more aesthetically pleasing than Erlan's, but Erlang's is way more consistent and simpler once you get used to it/understand the rules.

BoorishBears• 8 years ago

I both admire Elixr for making Erlang “cool” and detest it for ruining my single most favorite part about the language, it’s simplicity and clarity.

There’s only one way to do a given thing and you can learn each of the things the syntax can do in a weekend.

It was my first non-C-like, non-assembler language and I loved it.

josevalim4• 8 years ago

What makes it interesting is that the syntaxes have (most likely) been designed with different goals. I can say that Elixir was designed with extensibility and meta-programming in mind. So there are some cases where I prefer the Erlang syntax (for example, I still find defining multiple function clauses each with a single expression more elegant and concise in Erlang) while there are other cases I prefer Elixir's syntax exactly because we were able to extend the language (such as the `with` special form for handling nested case expressions).

The funny thing is, if you try to add those features to the other language, they are usually refused. Erlang wants to avoid new syntactical expressions to keep it simple while Elixir's extensibility is bound to a set of limited AST rules (and we don't want to add new ones).

macintux3• 8 years ago

Thanks, somehow I’d missed Reia in my list of BEAM languages.

https://gist.github.com/macintux/6349828

macintux3• 8 years ago

(Incidentally, when did Github break gist editing? I can no longer paste using my soft keyboard on my iPad.)

the_clarence• 8 years ago

Is Elixir more than just a ruby syntax of erlang? Because I prefer Erlang’s syntax.

hellbanner2• 8 years ago

That's mostly it. Personally I find Elixir's syntax confusing. For example:

Losing the ; and . function endings from Erlang you can put same-named functions throughout your module. I tried doing

def create def handle(:create)

def update def handle(:update)

But the compiler warns. So that loss isn't helpful.

Atoms require a : because variable names are lowercase.

Uppercase variable names and lowercase module names is easier to read in Erlang.

The syntactic sugar is too clever for readability imo.

The package management through mix is decent. I used to use an erlang.mk file, looking at hex it looks like the Erlang ecosystem is quite evolved.

Phoenix + Ecto seem to be very actively maintained, useful if you're writing web apps.

When I tried Cowboy a few years ago some of the documentation was out of date, so that's a point in Elixir's favor.

schpaencoder• 8 years ago

You have macros in Elixir

josevalim4• 8 years ago

Elixir is more than syntax. Or rather, it is not even about the syntax :) This is a common question, so I even gave a talk about it some time ago: https://www.youtube.com/watch?v=V5fMQcSy3y8

Elixir features meta-programming, structs and protocols, first-class documentation, strong focus on the tooling, some abstractions that make concurrency more accessible (such as tasks and streams), etc.

However, Erlang and Elixir share a lot! They have the same data-types (with the same names and even the same syntax for almost all of them), the runtime is the same, and the same foundation about processes, fault-tolerance and distribution.

My suggestion is to go with whatever "touches your heart" because, even if Elixir has its own features, they are more alike than they are different and most concepts you learn for one will also apply to the other.

Jedi72• 8 years ago

Can we please have one BEAM thread where this isnt brought up? Its such a minor thing. I went from erlang to elixir and whilst at first I really missed atoms that start with a colon, you get used to it. Its 90% the same ideas, and Elixirs pipe operator is worth any small syntax niceties erlang has. No language is completely without its quirks even erlang (unless you LIKE endless repituons of the same variable into var_1, var_2, etc - and no, this doesnt reduce bugs http://blog.plataformatec.com.br/2016/01/comparing-elixir-an... )

innocentoldguy• 8 years ago

I've been engineering software for almost 30 years and no languages have ever been more enjoyable for me to work with than Erlang and now Elixir. I'd like to thank the teams behind both languages for making my career interesting again.

Specifically, I love working with pattern matching, OTP, and the functional nature of both languages. Thanks Erlang (and Elixir) guys!

dudul3• 8 years ago

I really like Erlang and Elixir, but I'm really worried when I see a tech stack that is still trying to figure out deployment and runtime configuration of systems.

I'm not trying to be inflammatory here. As per many resources such as "Phoenix in Action" deployment is still a major culprit. And Dockyard themselves (one of the major Elixir shops, where McCord works actually) have a full time guy on the payroll to try to solve the problem of runtime configuration.

It just sucks, because these are 2 major concerns of any production-ready stack.

brightball• 8 years ago

It’s not so much an issue as it is something they want to make cleaner for new users. If you are coming for a background in other languages, the process doesn’t behave like you’re already used to.

So much of the work that Jose and Chris put into Elixir and Phoenix is striking the balance between what’s possible and what the onboarding experience looks like for new users.

apabepa• 8 years ago

I am not sure what you mean with production-ready here.

Erlang has certainly been used in production, not only Whatsapp but also have a lot of production systems in telecom where requirements on up-times etc are pretty high.

If you mean that there are no ready made off-the-shelf standard way to handle runtime configuration and deployment I guess it is because the big telcom products usually roll their own per product so it haven't made it into the OTP backlog.

lemur-dev• 8 years ago

While I agree that deployment can still improve, it is mostly because it doesn’t have the same polish as everything else Elixir ships with. In any case, it has already improved considerably over the last year.

My first application was deployed to Heroku and it just worked. For AWS, we had to do some research but once we got Distillery running, everything worked without major concerns. And Distillery uses Erlang releases which have been around for decades so we can rely on its stability.

Btw, runtime configuration works like a breeze in Distillery 2.0, which is the result of Paul’s amazing work at Dockyard (among other things).

toast0• 8 years ago

Deployment for Erlang needs to be a lot more flexible than other systems, because you have the opportunity to hotload code as a core feature [1], having your deployment solution invoke code loads is totally scriptable, but seems more likely to want human eyes watching than a traditional push, stop, start cycle.

OTP does provide the application construct which is supposed to provide some amount of configuration and start/stop/update support; although my team doesn't use the update support, we just push code and hotload it or stop and start beam depending on the changes.

[1] of course, you can hotload code in most languages if you try hard enough, I've done it in C and almost did it in perl; as long as you can load code and get a function pointer at runtime, you should be able to make it work.

pmarreck5• 8 years ago

For managed deployment (with a free tier) check out https://www.gigalixir.com/

jesses• 8 years ago

Founder of gigalixir here in case you have any questions.

alxndr• 8 years ago

Satisfied occasional user of Gigalixir here! I have a side project chugging along on their free tier.

rkangel2• 8 years ago

For one datum, as an inexperienced Phoenix user, it took me 1 day to get a Phoenix app deployed and running on Google App Engine flex, including DB and some other details.

yawaramin• 8 years ago

Yeah but an Erlang deployment is roughly equivalent to a Kubernetes deploy (more powerful in some ways), are you similarly worried about the amount of effort it takes to set up a k8s cluster?

cdelsolar• 8 years ago

I forget where I read that there were estimated to be only about 1K professional Erlang coders in the world. That seems absurdly low. Does anyone believe this is actually true?

jimbokun3• 8 years ago

If true, it's astounding so few programmers wrote systems that handle such astounding amounts of data.

stcredzero• 8 years ago

That's what people used to think of Symbolics Lisp machines. Customers would assume Symbolics had a team of thousands of programmers, when really it was about a dozen.

francescoc3• 8 years ago

Incorrect, it probably got confused with the number of Erlang developers at Ericsson. Not all use it on a daily basis, but estimates on their developers, testers and system integrators is in that range.

jimbokun3• 8 years ago

I had never watched Erlang the Movie before, this is truly a gem:

https://www.youtube.com/watch?v=uKfKtXYLG78

This is a great way to explain the benefits of Erlang. Anyone who has tried to write robust, distributed systems should immediately see the benefits. Even all of these years later.

francescoc3• 8 years ago

I use Erlang the Movie and its Sequel https://www.youtube.com/watch?v=rRbY3TMUcgQ&t=9s whenever I teach my classes at Oxford Uni. #studentlife :-)

jimbokun3• 8 years ago

Genius.

pmarreck5• 8 years ago

You caught the advancing of the movie to the 9 second mark in your URL. (I can't stand when youtube does this automatically, such as when adding videos to playlists...)

davidw2• 8 years ago

I dearly wish he'd make more videos. I also loved 'MongoDB is web scale' - https://www.youtube.com/watch?v=b2F-DItXtZs

whizzkid• 8 years ago

Not specifically Erlang but I have used Elixir and it was one of the rare languages that blew my mind when it comes to new ways of thinking when implementing things.

How not to love some of these features!

   [first, second | remaining_of_people] = list_of_people


   greeting = fn
     %{name: name} -> "Hello, #{name}!"
     %{} -> "Hello, Anonymous Stranger!"
   end

hellbanner2• 8 years ago

Fantastic tutorial: http://learnyousomeerlang.com/book

dhab2• 8 years ago

My limited understanding of erlang/elixir is that it helps easily run actor model programs (fail-safe loosely defined as something with retries built-in) in a grid of long-running interconnected instances. With more and more focus on "serverless" development that I am being exposed to, I am curious to know if it is a good fit in this space. Say if you are constrained to use AWS serverless solutions - sqs, sns, dynamodb, lambda, cognito, api-gateway ... etc, with event-sourcing (for event-drivenness) with retries (automatic/configurable), (why) would using erlang/elixir vs say nodejs/typescript/go/haskell/java be a good choice (let's assume you equally love these languages for the sake of not getting in to 'but X is better than lang Y')?

busterarm• 8 years ago

From my perspective, I see it as a different way of achieving the same end goal. With "serverless", someone still has to maintain servers somewhere and if your company is big enough, AWS is actually the more expensive option.

So if you are running your own infrastructure on bare metal, application development in erlang/elixir lets you better take advantage of the operations staff that you already have because running that infrastructure is much less complicated than the alternatives.

lotyrin2• 8 years ago

The modern serverless paradigm is a way to give these more traditional computing runtimes/languages a lot of the features of Erlang/OTP.

The downside to having all of these things in services on the network and outside of the runtime is latency. Imagine a runtime where everything is a message, everything goes into inboxes/queues but passing many of those messages is zero-copy, doesn't hit a network interface, and going from the calling code to the called code isn't even a OS level context switch -- so the message is written to and read from a core/thread's local cache lines.

dhab2• 8 years ago

For fault-tolerance in case the box is unresponsive for whatever reason, I think you would have to have a network behind a load balancer. So the issue of network latency still exists, but probably to a lesser extent?

rkangel2• 8 years ago

To me, the bigger downside is system complexity. If everything is an BEAM process, then testing, debugging and generally understanding is much easier. Things like spinning up a dev 'deployment' become trivial and things are going to behave in production as I expect them to.

eeZah7Ux• 8 years ago

No. Kubernetes & friends implement most of the features of Erlang in a language-independent manner.

fmap• 8 years ago

The article is underselling Erlang's meteoric success. "Adoption was slow during the first few years." - After 5 years there was an international conference devoted to Erlang, a global community around it and the language enjoyed commercial success from the beginning.

It just goes to show that Erlang fills a real niche that is ill served by most other programming languages. Programming distributed systems remains painful in 2018 - not because there aren't any theoretical solutions to make it easier, but because there are astonishingly few practical systems that offer built-in support. Erlang is such a practical system and if you didn't already look into the language it is well worth your time to pick it up. :)

tormeh• 8 years ago

Erlang truly is something. It's amazing how much effort is expended trying to achieve even a semblance of its functionality in other languages/ecosystems.

zem• 8 years ago

had to smile at the nod to adrian mole :)

Twenty Years of Open Source Erlang: A Retrospective from the trenches

Related Stories