OTP 23

http://www.erlang.org/news/140

by nifoc • 6 years ago

395 119 6 years ago

erlang.org

conradfr• 6 years ago

https://twitter.com/elixirlang/status/1260556559165292544

Elixir v1.10.3 is fully compatible with Erlang/OTP 23.0 - so up to date Elixir users are ready to give it a try!

hinkley3• 6 years ago

I'm curious what the short and long-term performance implications are for Elixir with OTP 23. Has anyone offered up opinions or benchmarks?

sitkack• 6 years ago

Support for SSL 3.0 is completely removed.

Yay! We need to actively and in a strong forceful manner remove broken security products.

ponyous• 6 years ago

Couple of times when somebody posted an OTP erlang update I had no clue what I was reading. Then I came back to HN and somebody broke down why some things are really useful/changed and it was mind blowing.

Can some erlanger explain why you are excited about certain things from the update?

brudgers3• 6 years ago

What I appreciate about this update is how little code it breaks and how little future code is likely to be backwards incompatible. Or to put it another way, what I appreciate is the sameness of the programming environment. To a first approximation nothing needs to be rewritten and newly written code will look just like existing code. What I appreciate is OTP is better and there's nothing programmers have to do (to a first approximation) to get that betterness. Instead, rolling out the betterness is a system administration task.

Basically, what is exciting is how well engineered it is.

yellowapple2• 6 years ago

What I appreciate about this update is how little code it breaks

Other than a bunch of code reliant on the stuff removed from erl_interface, of course. No idea how big of an impact that has on real-world projects, though.

The deprecation of erlang:get_stacktrace/0 also gave me a brief heart attack for one of my projects¹, but luckily I had the foresight to gate that to older OTP versions (and with newer versions the relevant code already removes dependence on that function), so I can rest at least a little bit easy.

----

¹: https://github.com/otpcl/otpcl/blob/73167c58771f700963cbd9f7...

dnautics9• 6 years ago

much of it is a lot of hard work getting to support TLS 1.3, which is nice. I have some TLS-dependent things in prod and let me say, it is really easy to do TLS in OTP. It's also easy to do TLS wrong in OTP, so I recommend watching all of Bram Verburg and Ingela Anderton Antin's videos first.

the socket library (if I am correct) is the beginning of redoing the entire network stack in a way that is more commensurate with the way that typical programmers expect (but with the right erlang-ey).

Allow underscores in numeric literals to improve readability

This is great whenever a language does it.

The embedded documentation is created as docchunks (EEP 48)

It looks like the erlang community is converging on the elixir strategy for documentation. This is fantastic, since elixir documentation is quite good, and it shows that the two languages are starting to put their friendly animosity aside and really working together.

distributed named process groups is introduced

This is kind of huge. Sometimes you want a locking, fully consistent process management (so CRDTs are inappropriate), and you don't want a full raft/paxos under the hood.

tyurok• 6 years ago

I personally like a lot the help in the shell feature that is one of the awesome things in elixir's shell that is now on erlang.

hinkley3• 6 years ago

Erlang may be the sturgeon of the software world. It has survived since the Triassic and yet is somehow prized for specific use cases and discerning tastes.

It is nowhere near as ancient as sharks, nor as vicious, but it employs some new physiology, and can survive just as long.

But that was a very different world and it sticks out like an outsider, when really it's been there the whole time.

(sorry, this analogy got away from me a bit).

tandr• 6 years ago

This is beautiful, and quite precise (imho). Thank you!

liveoneggs2• 6 years ago

the story I like to use is -- "imagine a world where java never happened, where servers were abstracted into HA clusters automatically by your programming language, and where horizontal scaling was automatic and free"

moviedo2• 6 years ago

I'm currently learning erlang/elixir and I'm really enjoying the language constructs.

I had originally taken a Programming Language Paradigms class in college with racket, and I really didn't appreciate functional ideas(i.e. syntax is my excuse). I'm now a really big fan of functional language idioms.

Anyone interested should try the futurelearn class, https://www.futurelearn.com/courses/functional-programming-e....

Grimm13• 6 years ago

I was in the same boat until discovering elixir/erlang it has by far been the most approachable functional language for me and it has been a gateway drug of sorts into experimenting with ocaml and various lisps.

I personally think elixir is really special and an amazing fit for any project that needs high IO concurrency.

unvs• 6 years ago

Agreed! I found learning Elixir to be a great gateway to learn Erlang as well. When I first tried learning Erlang (before Elixir was released), it became a bit too much to wrap my head around. It was a lot easier when I had gotten the fundamentals of FP down through Elixir, though!

dnautics9• 6 years ago

IMO erlang (and arguably even more, elixir) is "functional for the working programmer". It doesn't drown itself in academic abstractions, uses functional programming as a tool to guardrail you from mistakes (in the same way that C guards the programmer from making asm mistakes), with escape hatches that are battle-tested and justified based on decades of experience.

I would say the only other functional language that has the same bent is Julia, which is "functional for the working scientist". It makes different choices about where to expose state, understandable, since scientific computing has different tradeoffs from systems programming.

mcintyre1994• 6 years ago

On the Elixir/OTP side I've really enjoyed this course too: https://pragmaticstudio.com/elixir They have a current discount code LIVEVIEW (not affiliate), which is advertised on their currently free early access Phoenix live view course.

skrebbel• 6 years ago

I never understood why people rave so much about functional programming wrt Erlang/Elixir, when its functional programming is clearly only a means to an end (fast and safe message passing requires immutable data, which requires FP) and not a driving design goal in its own right.

I mean, unlike in typical hard FP languages like Haskell or Elm, mutable state is rampant in your average Elixir app, it's just spread out across many (global singleton) little processes. Only inside a process you're doing "true" FP but given how small the average process is in scope, in practice the only real big difference is that you can't to an `i++` style for loop. Oh no!

But once you leave the process boundary, and often even before it, all bets are off. The amount of Elixir forum messages I've read that go "you can't do X at that point, because Y hasn't completed yet" is nuts.

Eg you can't broadcast in a phoenix channel `join` because the channel hasn't been fully initialized yet. So you send yourself an :after_join message and do the broadcast in there. I don't know about you but to me this feels a lot more like C++ than like Haskell.

Or consider the library module Agent which is exactly identical in semantics to a global singleton variable in an OO/imperative language. It's just a blob of data that you can get or set.

Now, I don't think any of these are disadvantages. I did mostly C# and JavaScript before Elixir, so I'm used to the occasional mutable state flying around.

But I'll never understand that people like Elixir for being FP. You just get such a small subset of the usual advantages of FP that it feels like an implementation detail. There's lots of advantages, but freedom from thinking about state isn't one of them.

brokencode2• 6 years ago

If you look closely enough at Haskell, you’ll realize that it also can have a lot of mutable state. Haskell just puts state into various monads and STM to make its functions pure. You can even spawn numerous isolated threads, all with their own state, and have them communicate with one another like you do in Elixir. It does provide a lot of structure and guarantees compared to your standard imperative language, but I assure you mutable state is still there.

To be clear, the Elm model of putting everything into a big tree and and transforming that at every user input is very unusual among even FP languages, and is not the model typically used in Haskell. This might be what you are thinking of.

Edit: I also want to point out that Elixir processes can be registered globally to act as singletons, but by default you can spawn any number of agents or other processes at runtime, meaning they are not singletons.

hinkley3• 6 years ago

Practical tools often achieve that practicality by avoiding dogma.

nurettin3• 6 years ago

I have a prototype system where I am passing over 100k msg/sec between a dozen backend services written in python (asyncio+redis) and I keep on wondering when my bottleneck will become functional programming and safe message passing by making copies. When will the madness end?

pdimitar15• 6 years ago

But I'll never understand that people like Elixir for being FP.

To me the answer is: using mutable state is opt-in. I disagree that "mutable state is rampant".

By opting in to the mutable state constructs you are basically saying "I know what I am doing, let me do my work" which is IMO quite fine because "pure" FP languages like Haskell can be a huge hassle when you actually need to deal with the real world.

To me Elixir is a very nice compromise.

pera• 6 years ago

I mean, unlike in typical hard FP languages like Haskell or Elm

What is a "typical" FP language? Are Scheme, Common Lisp, OCaml, SML, atypical?

mutable state is rampant in your average Elixir app

I don't think this is generally true, or at least not on my experience, but I guess it may depend on what would one consider rampant.

dnautics9• 6 years ago

You just get such a small subset of the usual advantages of FP

Yeah but you're using that small subset in 90% of your code.

Look, this is not a thing to worry about in elixir:

    def p(my_array):
      do_something_with_this_array(my_array)
      return my_array

what is my array? I don't know. It could be anything. The company I work for just hired a sloppy python programmer that I don't want anywhere near my code, and you know what, if we change a section of our code to Elixir I am way more willing to have him work on our team.

AlchemistCamp• 6 years ago

Interesting. Is FutureLearn something analogous to Coursera and edX?

moviedo2• 6 years ago

Yes, very much the same as Coursera. Almost 1 to 1.

morty_s• 6 years ago

Same. Or similar, it’s taken longer than it should for me to appreciate the functional paradigm.

Rust and Scheme have been gateway drugs ha. I found I really like OCaml-y languages.

Now, I’m very interested in Erlang (and to some degree elixir). I’m learning as much as I can about Erlang and the ecosystem; I’m trying to answer the question, “why isn’t Erlang more popular?”

Any guesses? Reasons? (syntax aside)

pdimitar15• 6 years ago

I might sound bitter but after about 3.5 years with Elixir my answer is very simple and boils down to:

Habit, confirmation bias, sunk cost fallacy.

Namely: people have gotten a lot of battle scars by working with what pays their bills -- PHP, Ruby, Python, C#, Java -- and they refuse to look at an alternative because that would render their huge time and energy investment moot (in their eyes at least; I don't see why this has to be the case but plenty of people have been adamant about this without giving an explanation).

I've only become a better programmer since I adopted Elixir but I never stopped using other languages.

All of that plus what PG calls "the middlebrow dismissal" are the main reasons IMO. People are just too set in their ways.

empyrean• 6 years ago

Did you go to Northeastern? Glad to see a fellow Husky in the wild!

RcouF1uZ4gsC• 6 years ago

One of the really nice things about shared-nothing concurrency is that it scales to multiple machines. The same code that works locally will work across a data center. Along the same lines, shared-nothing tends to scale better as you add cores. With lock based programming, as you add cores, many times your contention increases minimizing the benefits of the additional cores.

lelf• 6 years ago

localcdn• 6 years ago

Nice to see default TLS 1.3 support. Though it looks like the use_srtp extension for DTLS is still being worked on.

mikece• 6 years ago

Does cloud-based serverless computing compete with or complement Erland/Elixir? As a dotNetCore developer who writes a lot of Lambda and Azure function code I am trying to understand how OTP relates to these paradigms or if they are apples and oranges.

bicx• 6 years ago

It's apples and oranges.

OTP paradigms are definitely intended for a larger system that needs its own distribution schemes, handles its own internal process uptime and restarts, and for some use cases, hot code updates (running servers that update without restarting). OTP nodes can also easily and natively communicate between each other across the network, allowing for some pretty sophisticated distributed systems.

There are many solutions that could be more easily implemented on some sort of serverless architecture, but in cases where you want to maintain a service with a high degree of reliability and distributed capability, OTP is a good choice. After all, it was developed for the telecom industry to handle phone system communications loads.

Worth noting: there are other solutions (some of which are already in serverless environments) that solve the same problems of service uptime during code updates/deployment by keeping an older instance running until the new instance is ready to accept traffic. Services like Kubernetes also implement a lot of cluster management capabilities that solve the same problems OTP was attempting to solve, and you can use whatever language and engine/framework you want. Personally, I think if you wrote a service purely in Erlang/Elixir and directly applied all OTP solutions for uptime, process communication, and distribution, it would be faster and more efficient than a typical serverless or container-based system. But, it would also lock you into that system and make any non-OTP tech more difficult to integrate.

macintux• 6 years ago

I’ve been out of the Erlang world for a few years now, but it doesn’t seem like a great fit for serverless. The VM has a non-trivial overhead, and the language (and VM) design is optimized for long-running, high availability services.

micmus• 6 years ago

I would say it's mostly that nobody optimized it (perhaps yet) for that environment. There's generally nothing inherently slow in the VM boot sequence - it's just that this was not a priority, so it's slow. There are some attributes suggesting it could be a good fit for that environment, for example, the VM is generally very small - you can get a full system in ~20MB - and that again wasn't something that was heavily optimized.

te_chris• 6 years ago

The VM has an overhead, but looking at our kube cluster right now which runs elixir and ruby pods (treating elixir as shared nothing, old school stateless, rather than distributed), mean RAM for our ruby pods is 1GB, Ex ones are 200MB. Response times: Ex: ~30ms, Ruby: ~150ms.

lawik3• 6 years ago

Sounds about right. With wasm being relevant for edge computing and serverless I'm keeping a close eye on the Lumen project compiler as I expect that to be more feasible for this sort of work. The advantages for OTP remain mostly in long-running services. But for Lumen that might be a more open question.

brudgers3• 6 years ago

Erlang is a message passing language for writing concurrent applications. Architecturally, Erlang programs share logical similarity with microservices. Architecturally, OTP shares some conceptual similarity with serverless frameworks.

Together, Erlang and OTP provide a batteries included environment for building and running scalable concurrent systems. Scalable down to any device that will run Linux, not just up into the cloud.

h91wka• 6 years ago

Erlang VM startup time is not great (in order of hundred milliseconds), but it has pretty unique features like hot bytecode patching in a running VM. So, naturally, Erlang shines when you need to build a system that runs for a long time, as opposed to short-lived lambdas.

voganmother42• 6 years ago

reminds me of: https://github.com/cloudozer/ling

ex_amazon_sde• 6 years ago

It competes, mostly. Amazon's internal SOA implements pretty much the same principle around resilience and load balancing.

Unlike OTP it must be language-agnostic and also aware of latency and bandwidth.

xutopia• 6 years ago

Though I am not working with Erlang or Elixir right now I am so pleased that this is seeing improvements. I had as much fun learning Elixir as I did learning Ruby ages ago. It's just a fun language with great construct!

organicfigs• 6 years ago

I've wanted to work with Elixir for 5 years now. I attempted to go through The Pragmatic Programmer's Elixir textbook and couldn't make it through. This comment applies solely to me, I feel I didn't have the aptitude to pick up functional programming (and that's coming from someone who graduated in CS from a T3, had years of experience, and starting med school next month).

dnautics9• 6 years ago

There's a reputation that functional programming is harder than object-oriented. I think this comes from the very difficult functional programming languages that exist out there. I think that can be an obstacle. People hear "fp" and think "hard".

I personally find Elixir's functional programming is easier, because there are fewer things you have to deal with than object oriented. There are basically no constructs in Elixir that you don't have out of the box in an OO language like Javascript or Python. You just have to deal with that you can't do things like assign variables in if statements (you just have to explicitly export them) and you don't have for loops.

Having said that, Enum.map and Enum.reduce are really hard to get used to and can be an obstacle in terms of "why can't I do this thing that is so easy in X".

In the long run, once you get used to reading them (which honestly took me about 4-5 months, I have like 20+ years of programming experience) I think they are easier because they are declarative -- you can see exactly what is happening to each piece of your list and in the case of reduce, what state is being passed through each iteration.

For loops are far more unstructured, literally anything in any of your parent scopes could be what you're keeping track of through iterations, which means, your mistake surface area is much much higher. The same goes for getting used to "if statements that export values" but, maybe, less dramatically.

I know this is difficult to hear since you gave up, but I promise you if you push through it, eventually it will feel like writing elixir is "programming with training wheels on" you can do crazy hard things without worrying about large classes of logical and structural programming errors. Hope you come back to it!

brudgers3• 6 years ago

Since it's already been five years, it might be worth considering Erlang instead of Elixr. I'd recommend Programming Erlang: Software for a Concurrent World. It was written by Joe Armstrong who designed Erlang and "sold" it internally across Ericsson to people who were not programmers. They were people who were using Erlang to solve some other problem.

The paradigm of Erlang is message passing. The primary idiom is logic programming (like Prolog). Functional programming is related to logic programming. Between message passing, logic programming, and simple primitives for concurrency; Erlang programs tend to look a little strange. But it all hangs together in a way that has been field tested for thirty years. Erlang is a language that was engineered for use by engineers for solving engineering problems.

Elixr is mostly a way of avoiding learning Erlang and making the Beam VM popular. It is a procedural programming abstraction layer. For me, Elixr's abstractions generate an impedance mismatch and Erlang's syntax better expresses the engineering mechanisms of concurrent systems and logic programming. YMMV.

schadara• 6 years ago

I did Ruby since 2011, and tried Elixir in December 2019. I really hated the verbose syntax compared to Ruby, which has a comparatively simple syntax. The next thing I hated in Elixir was the boiler plate for GenServers and having to deal with pids and named processes. However, this weekend, I rewrote about 2k lines of sloppy elixir that I hastily copy pasted, and condensed it down to around 700-800 lines in a few hours with barely any effort. It was easy because it's just functions calling functions. I guess I'm kind of a believer now. Besides that, Phoenix channels are very performant, which makes me think I might switch from Ruby to Phoenix permanently.

Try sticking through it.

mcmatterson2• 6 years ago

The use of prior matches in guards is super useful for parsing TLV style payloads where you first match on a size/length field and then match on the value itself as that number of octets. Does anyone have any insight into how / when similar support will make its way into Elixir?

dnautics9• 6 years ago

I think it's already there?

    Erlang/OTP 22
    Interactive Elixir (1.10.3)

    defmodule Test do
      def matchme(value) do
       x = :foo
       %{^x => bar} = value
       IO.puts(bar)
      end
    end

    > Test.matchme(%{foo: "bar"})
    bar
    :ok

mcmatterson2• 6 years ago

My goodness, you're right (well, your example isn't, but the feature I'm talking about has been present since at least 1.8 (the earliest install I have handy):

  > <<len::8, val::binary-size(len)>> = <<3>> <> "abc"
    <<3, 97, 98, 99>> 
  > val
    "abc"

I don't know how I'd missed that.

(Your example demonstrates 'pinning' which has been a thing since always I think).

waltfy• 6 years ago

Not mentioned in the link above but something super cool with this release is that

CPU quotas are now taken into account when deciding the default number of online schedulers. Thus, automatically making Erlang a good citizen in container environments where quotas are applied, such as docker with the --cpus flag.

Super happy about this!

bsaul4• 6 years ago

I would love it so much that someone took the time to correctly evaluate the pros and cons of OTP vs the current stacks we see those days (stateless services deployed and configured with kubernetes, with an event-bus backbone for service 2 service communication).

I've always wanted to try OTP for a real world project (simply because it looks more elegant and has less parts), and now may be the time, but i'm so scared of discovering huge pitfalls down the road that are better solved by more recent techs..

lawik3• 6 years ago

I think I understand your concern as the foundation is arguably old. But as someone actively following Erlang and Elixir the sentiment feels absurd. Elixir is very recent, very modern and Erlang/OTP has a very strong track record.

There are drawbacks of course but having worked with Python, PHP and JS. I haven't seen any real drawbacks. I wouldn't choose it for minimal overhead or the fastest compute maybe..

Edit: typo

bsaul4• 6 years ago

Are you "following" or are you actually using it in production for a real service ?

I'm asking, because to my knowledge, the only "huge number of users" recent success story is whatsapp, and after having viewed all the talks about their stack and experience, it seems that the team was so extremely talented that they probably would have made something good with any tech.

toast02• 6 years ago

I've used Erlang/OTP only on stateful servers, and observed its use in semi-stateless containerized deployment (but in that case, we were generating the containers to have a fictional state; we always had nodeX@hostnameY even if the container being that node moved around). However, OTP ships with a diskless mode, where all of the code and config, other than BEAM and modules critical to starting dist come from dist.

bsaul4• 6 years ago

Stateful servers seems like an atrocious point of failure nowadays (except if it's a DB server, or a persistent queue). And i barely see it anymore, now that everything seems to be running on containers or VM that are supposed to be destroyable at any time.

That's why the value of OTP seems to be less obvious at the moment (from the point of view of someone who doesn't use it). But there's surely something i'm missing.

rwmj2• 6 years ago

Breaks all of our Erlang bindings (which use erl_interface) ...

liveoneggs2• 6 years ago

bummer. What specifically?

rwmj2• 6 years ago

libguestfs is the main project we have with erlang bindings.

elcritch3• 6 years ago

Looking at the changelog I don't see what the current situation with erl_interface is. I've been working on some `erl_interface` code using C [2] and Nim [1], and the headers warn that older `erl_interface` headers would be deprecated in OTP 23. All the release note highlights mention is a new `erl_call` program.

Anyone know about the status / rational for the `erl_interface.h` API?

1: 1: https://github.com/elcritch/einode/ 2: https://github.com/elcritch/einode/tree/master/tests/c-nodes...

strmpnk• 6 years ago

Much of the erl_interface functionality prefixed with erl_ names has been deprecated since OTP 22 and has been removed. The API has shifted to newer ei_ prefixed functions. I believe there are some new deprecations in place as well which you can find the longer form release notes.

erl_interface itself is not going away but it is evolving with the BEAM VM. Deprecation warnings should be checked when compiling code on each major release to avoid surprises as features are usually deprecated for one release and then removed in the next release each year. The erl_interface documentation should be up to date with regards to new APIs and might be worth browsing again to get an idea of the what changes look like.

elcritch3• 6 years ago

Thanks! That's good info. The hardest part was that in OTP 22 the examples given used many of the deprecated erl_* apis. I was able to update the C examples (see the second link) to use non-deprecated ei_* api calls. Mostly small changes and a bit better buffer management. Though, I don't like the lack of buffer length check in even the newer ei_encode_* functions. :/ I added a 24 byte padding guessing most single item encodes are less than that, and then check variable length items for size. Still hard to use safely without a buffer overrun. I'll take a look and see what else may have changed. It's exciting seeing all the continual beam improvements!

alberth• 6 years ago

What’s the latest on BEAMJIT progress?

pbadenski• 6 years ago

A few random questions.. hopefully a person more knowledgable in the topic can help me out:

- from what I understand Akka (Scala) heavily borrows from Erlang/OTP - is this correct? What would be the biggest similarities/differences?

- to what extent OTP is unique to Erlang and to what extent it can be implemented in another language?

- any articles/materials out there showing the most valuable bits of OTP applied in JavaScript/NodeJS?

Multicomp• 6 years ago

I must admit, I have been watching erlang for a while trying to decide when to hop in, and I have increasingly come to the conclusion that I will not be learning erlang as such, although I will be buying the books and reading them, but I will be learning gleam, which is erlang with types.

I'm too used to the safety that rust or F# gives at compile time to give that up.

OTP 23

Related Stories