Adopting Erlang

by ProfDreamer • 7 years ago

366 68 7 years ago

mononcqc11• 7 years ago

One of the authors here. This website is still a work in progress.

It's an ongoing effort to gather all the resources that will help you use Erlang in a business. The booksite is divided in three sections focusing particularly on Erlang/OTP’s higher level concepts in the current open source ecosystem, how to use it in production (while setting up a pipeline for continuous development and delivery), and how to build a team when you’re starting from scratch.

We're working mostly linearly in each section, but they're not all progressing at the same pace. We have nothing yet on "building a team" although our outlines are all planned; the "hard things to get right" chapter was supposed to come in later but we saw enough questions about these topics to rush it out.

We might release things from each section as they are ready and we see fit. In the meanwhile, we're always appreciating feedback and people checking in to see what's new in terms of content.

Driky2• 7 years ago

Is it strictly Erlang only or does elixir has it's place in it as part of the ecosystem (no polemic intended here, hope for a direct yes or no)?

mononcqc11• 7 years ago

Erlang first.

There is a large overlap with Elixir stuff, mostly at a higher level; whatever covers OTP, handling supervision, releases, docker image practices, approaches to testing, production practices, etc. will very likely be useful to Elixir users.

But the lower-level details like handling dependencies, build tool commands, test framework specifics, the code snippets and samples, and so on are all going to be Erlang-specific.

I use both languages frequently for work and there's far more in common between them than there are differences. If that's not your cup of tea and Elixir is really all you're after, you can look for Ben Marx's Adopting Elixir (https://pragprog.com/book/tvmelixir/adopting-elixir) book. We are not affiliated with it and don't aim to cover the same content (we just have similar titles), but it's a good option that goes Elixir-first.

davidw6• 7 years ago

I see you mention circuit breakers.

I think they deserve more thorough coverage because 1) you really need them in a lot of applications, and 2) they're not in OTP, so picking one (especially if you're not really an Erlang expert) and employing it in the right way are a bit trickier.

A simple example of an application that probably needs a circuit breaker is a web application that can keep running even if the database is not available, even if all it does is return an error message to the browser.

yawaramin• 7 years ago

Don't circuit breakers arise naturally from configuring supervisors in a particular way? I.e. supervisor failover after a certain number/period of retries.

hinkley4• 7 years ago

One of the things I've learned from many attempts to document internal processes at organizations, is that often the authors are least equipped to explain things.

They know too much, they often don't recognize how many pieces of assumed information they're injecting into their so-called explanations. You often get circular explanations in addition to circular reasoning and it can be off-putting.

A better use of time is to get someone who is just started to exhibit competence to write the docs. They won't assume the jargon, they can't assume deep knowledge, and because they're living right at the pain point they can be especially sympathetic. And if what they explain is dead wrong, this is your chance to correct a horrible misunderstanding before they get too far along. Pointing out the 'false friends' in a system early on can be pretty powerful. Educationally or as an opportunity for improving the system.

In other words, I'd encourage you to invite new people to contribute. Just keep an eye on what they do and be ready to jump in with corrections.

mononcqc11• 7 years ago

Is this a general tip you're giving, or is there specifically pieces of information you feel are being too opaque in the text at this point?

dmix• 7 years ago

Can't people just remember what it was like to learn the things they know? They were all those newbies at one point.

K0SM0S3• 7 years ago

This is gospel for teachers / learners. Well put.

There's such as thing as too verbose, but it usually comes from writing stuff in the wrong kind of document: there is always a place for in-depth explanation, and there is a place for terseness to the point of familiarity.

One last thing: if your topic has a jargon of its own, a particular meaning to words, introduce these as early on as possible, and be excellent in consistency throughout docs (in addition, consider a definition link/info widget for each such term on its first occurence in any document, letting people 'jump right in' to any section/article). If you are going to force people to re-wire their brain to suit your particular vocabulary, you should maintain 'immersion' at all times.

If find this especially true in not-so-well-defined paradigms such as OOP when approached by various programming languages. You need a "map of terminology" more than anything else to quickly grasp how to use language XYZ.

verttii• 7 years ago

You could and probably should promote this to the Elixir community too. Especially the OTP parts are very relevant fundamental knowledge for any Elixir dev.

davidw6• 7 years ago

Another random thought:

A section on "stuff that's not in OTP, but many people end up using".

For instance, poolboy comes to mind, alongside the circuit breakers I mentioned in the other comment.

lager for logging.

At one point, there was a lot of stuff in the basho github repo that seemed pretty widely used.

It's been a bit since I've used Erlang so maybe there are some others that come to mind as "you're likely to see this included in a production Erlang system".

_asummers3• 7 years ago

Logging is included out of the box in recent OTP versions (21+ I think?) with no need to include lager!

cpeterso• 7 years ago

What are the arguments for adopting Erlang instead of Elixir for new projects in 2019? I understand there is a lot of existing Erlang code out there, but it has a reputation for being less approachable than something like Elixir (syntax and tooling).

mononcqc11• 7 years ago

Erlang's tooling has vastly improved in the last few years to the point it's not as much of a concern anymore (a formatter is the big missing thing), and I just happen to prefer its syntax to Elixir's in the first place.

While I can argue in favor of tooling being roughly on-par (sometimes better, sometimes worse), I can't really discuss syntax since this is often just related to taste more than anything.

My stronger argument would usually be in favor of the community you end up with. While both communities are starting to merge and grow closer over time, your average Erlang dev is likely to work on infrastructure components, and your average Elixir dev is likely to be working on web apps. I personally like working on the former better, so there's more limited interest for me in terms of most Elixir jobs anyway, even if it's not like no infra work exists in Elixir.

Both can have a great time working on APIs of any kind, and both can reuse most libraries of each other's community, so as I said, this is starting to become a moot point though. I personally have no problem with either.

macintux• 7 years ago

I would argue, probably without a great deal of agreement, that Erlang’s syntax is simpler than Elixir. It’s less familiar, but there’s practically nothing to it.

jlarocco• 7 years ago

This might be on your radar already, but the rendering is broken in Firefox. I'm using version 60.8.0esr, which is what I get with "apt-get install firefox".

When I open the link I see the table of contents on the left and a bunch of empty white space. Scrolling down, the content eventually comes into view, but scrolls on top of the table of contents, making it impossible to click.

I'm blocking the google fonts site, so maybe that's part of the problem.

fiter• 7 years ago

For reference, Firefox 60 was released May 2018.

rdtsc• 7 years ago

Just wanted to thank you and Tristan for the great resource. It's much appreciated!

The cheatsheet about what the processes do when they trap exits/are linked/are port of OTP is awesome. I never remember that very well and was looking for a handy reference to have. Now I have it.

Docker and Kubernetes references are great as those are often involved in today's development and operations, and everyone is probably wondering what's the best way to combine those with Erlang.

_randyr• 7 years ago

There seems to be a slight encoding issue with external URLs. In the dependencies section under "Private Hex Mirrors", there is a link to mini_repo which results in a 404. It seems like the underscore in mini_repo is encoded to %5f which github doesn't seem to like?

dynamite-ready2• 7 years ago

I've noticed there's a section in there about deploying with Docker. I know many other development communities swear by Docker, but a number of the OTP/BEAM devs I've spoken to seem unsure about it.

I also tend to believe Docker makes little sense, where OTP releases are the alternative.

What's the opinion here? This question only pertains to OTP development.

whalesalad• 7 years ago

It’s difficult to deploy Erlang apps in Erlang-style using Docker. The style I speak of is zero downtime deploys where the VM is never turned off, but is updated with new software as a living organism.

You can run BEAM apps in a container the same way you’d run any other app. It’s possible. But BEAM is less like a PHP/Python/Ruby etc interpreter and more like an abstraction for the OS (some might even consider it to be one). When you’re in Erlang you think in Erlang and the runtime abstracts things like processes, threads, and network partitions away... so you tend to want a big beast BEAM instance (or a few big beasts) rather than a lot of tiny ones (as one might scale up a horizontally scaled app like one on Heroku dynos).

It can be done. It’s just kinda going against the grain.

kungfooguru4• 7 years ago

It is not going against the grain. Using release upgrades (upgrading a release without restarting the VM -- unless the version of erts has changed) should only be used where necessary and they rarely are.

They are great to have when you need them but should not be chosen lightly.

Part of the point of this booksite is to show integrating Erlang into a modern environment and not being the oddball deploy at the company. It is the lack of libraries for services used for deployment/monitoring and lack of documentation that is the reason for this.

Also, there is no abstracting away network partitions.

dynamite-ready2• 7 years ago

This is what I've been thinking. At the very least, I know that Docker interferes with the BEAM VM's default behaviour for port management, and node discovery. Docker containers require developers to select which ports containers communicate through. Isn't that right?

samvher2• 7 years ago

I deploy the Erlang system I use at work using Docker and it's working very well for me. Like you say, it's not a zero-downtime relup but rather just replacing one app container by an updated one. It was a little bit tricky to get the image built properly (the right combination of dependencies, ERTS inclusion, etc) and things like remote_console are slightly more cumbersome than they would be otherwise (it becomes a docker exec command) but overall I feel like the advantages (mainly very smooth deployment and declarative configuration) outweigh the disadvantages.

kungfooguru4• 7 years ago

Hi, one of the authors here, the Docker chapter and soon to come Kubernetes chapter hopefully shed some light on this very question. They are not alternatives, docker and k8s complement Erlang.

Jose Valim, creator of Elixir, also wrote about this recently http://blog.plataformatec.com.br/2019/10/kubernetes-and-the-...?

StreamBright• 7 years ago

Do you have experience with performance optimization of Erlang code running in Docker? Is there a way to make the BEAM schedulers sticky? Does it make sense for latency reasons? I know that some products rely on core stickiness for maximum performance. Maybe Erlang/BEAM is not in that category.

https://www.scylladb.com/2018/08/09/cost-containerization-sc...

h91wka• 7 years ago

I also tend to believe Docker makes little sense, where OTP releases are the alternative.

Docker makes a lot of sense, actually, because it allows you to run integration tests easily and ship Erlang releases in the same way as everything else. There are exceptions, of course. As for OTP releases, they are a good way intermediate step before packaging code to RPMs, but upgrades should be done the usual way: via blue-green deployment. Don't even argue with me :-) I happened to support a huge system that uses OTP upgrades, and the amount of complexity and tricky bugs, that live code loading introduces, is immense. No one is smart enough to do it :-)

GordonS• 7 years ago

Was going to say something similar - having a consistent, predictable deployment environment is really valuable from a support perspective, especially if you are shipping a service to several end users (as opposed to deploying an in-house service).

conradfr• 7 years ago

Sometimes I wonder if Linux containers (e.g lxd) would not be more suited than docker for Beam apps, but nobody hardly ever mentions them.

yellowapple2• 7 years ago

It's just a matter of using the right tool for the right job. There's a lot of things Docker can do that an OTP application already handles (and in a way that's better-integrated with your application). Same with an orchestration system (e.g. Kubernetes).

You can use Docker to provide a common/consistent base on which you'd build the nodes running your OTP applications. If you have zero non-OTP dependencies, then this is overkill (you'd be better off deploying directly to some server, be it a VM or bare-metal, and managing configuration/deployment on that level), but most real-world applications do have other Dockerizable dependencies (databases come to mind, though there's ongoing debate on whether or not putting Postgres in a Docker container is actually smart). Also, using Docker to abstract the runtime environment for BEAM does help considerably with deployment to various PaaS platforms (e.g. Heroku, Bluemix, Elastic Beanstalk, etc.).

Personally, I lean toward a couple big servers dedicated to BEAM instances (running all the OTP apps), and a couple big servers dedicated to some SQL database (usually Postgres). Whether you maintain those servers as Docker containers or EC2 instances or DO droplets or physical servers is an implementation detail that doesn't really matter that much as long as it works and it's easy for you (or your DevOps / sysadmin teams) to maintain.

On that note: I don't think it makes sense to have a bunch of little containerized microservices each running a separate BEAM and a separate set of OTP applications. OTP already takes care of that sort of orchestration, and OTP applications already are (or can be) microservices. Just throw 'em all into a big server and let BEAM's scheduler and OTP's supervision model do their jobs.

kungfooguru4• 7 years ago

If you get to choose your infrastructure that may be fine. If you don't and you work in even a medium size company you likely work with what they have, which is increasingly kubernetes.

That said, they still do not do similar things. OTP does not take care of orchestration if you must run more than 1 node.

You are still benefiting from OTP's supervision and scheduling when using docker or k8s, the difference is in what other features and the size you need.

No one is saying always deploy to k8s, but they are not overlapping.

Nor does using BEAM make it uniquely fitted to running on physical machines compared to other languages like java, go, C++, Rust, ... If your application is suited to running on a few droplets the language you are using is not likely the deciding factor for that.

btbuildem• 7 years ago

I've always kludged my way through making Erlang applications. I still do, but having had a good read of this, I'm encouraged. Thanks!

dxhdr• 7 years ago

This is amazing, thanks for all of your hard work! It's very encouraging to see continual effort put into the Erlang ecosystem. I haven't used Erlang in around 5 years so this looks right up my alley for getting up to date with the latest best practices.

As an aside, it's been a little bittersweet seeing all of the attention and momentum going into Elixir. Yes it's great for BEAM and Erlang certainly benefits from those contributions as well. But there's just something so beautiful about Erlang, it deserves more attention than it gets. Thanks for keeping the dream alive!

rsrsrs86• 7 years ago

What is the use case for erlang given the current container technology and cloud providers? I mean many problems that erlang solved (and beautifully did it) can be dealt with docker and kubernetes

mononcqc11• 7 years ago

The Docker chapter does mention similar things, and José Valim recently wrote the following blog post on this: http://blog.plataformatec.com.br/2019/10/kubernetes-and-the-...

In a nutshell, docker and k8s are to OTP what region failover is to k8s tech. They operate on different layers of abstraction and impact distinct components.

OTP allows handling partial failures WITHIN an instance, something K8s can’t help with.

samvher2• 7 years ago

I agree that some of the problems that OTP solves can now be taken over by Docker/K8s if you modify your architecture a bit. I.e. if you only have a few processes running on BEAM it's likely you could replace them by a few containers with a supervisor that restarts containers with errors, though you would have to start thinking about how they communicate (HTTP?) and some other things. It wouldn't scale to the same extent either (you don't want 1000s of containers on a single node) and Erlang provides some other benefits as well (e.g. single interface to manage all your processes, consistency if all your services are in Erlang, efficient threading). I would say that the main use case is when the model of using 1000s+ of processes helps you build an easy to understand architecture, which can be in messaging, streaming, gaming, or other applications with many simultaneous users/clients.

sb8244• 7 years ago

Great stuff, as always. I liked this quote:

If you expect failure to happen on an external service, do not make its presence a guarantee of your system. We’re dealing with the real world here, and failure of external dependencies is always an option.

shijie• 7 years ago

As an Elixir dev for coming up on four (four! Jeez where has the time gone?) years, I’ve yet to really delve into Erlang. I can read the syntax well enough to figure out why my Hackney request is doing odd things with my HTTP headers etc..., but that’s about it. To all Erlangers out there, what advantages/niceties/powers do you have in Erlang that make you reach for the language over Elixir?

mononcqc11• 7 years ago

Mostly I just like it better. But otherwise in my day-to-day off the top of my head:

Less “magic” (fewer macros that may change semantics, straightforward config handling), focus on declarativeness, no variable re-binding, more powerful test framework (common_test), seamless Dialyzer support, better logging framework for structured logging, simpler/minimal syntax, personal preferences for tooling (until recently, direct support for releases was lacking in Elixir), and force of habit.

Adopting Erlang

Related Stories