Typing lists and tuples in Elixir

https://elixir-lang.org/blog/2024/08/28/typing-lists-and-tuples/

by idmitrievsky • 2 years ago

231 52 2 years ago

munchler2• 2 years ago

F# has both a `head` and `tryHead` function to handle lists that may or may not be empty. In general, `tryFoo` is a good pattern for naming functions that might fail.

Having a separate NonEmptyList type might seem like a good idea in theory, but in my experience, it leads to code that is significantly more complicated.

truculent• 2 years ago

How does it make it more complicated?

In my view, you’re moving the potential for failure to a different place (the constructor), rather than changing some fundamental property or introducing new complexity.

Is it handling the construction of these types you find complicated? And is it simply not worth the guarantees?

greener_grass5• 2 years ago

The intuitive definition of NonEmptyList is:

    type NonEmptyList<'t> = 't * 't list

But this cannot be passed to any function that expects a List<'t>. This is odd though, since intuitively, all non-empty lists are lists.

See Rich Hickey's "Maybe Not" talk

OOP solution is to use inheritance. Typical ML solution is to use type-classes.

F# sits in an awkward middle-ground where neither is a perfect fit.

I believe that dependently-typed languages solve this more elegantly.

There's also the syntactic inconvenience of wrapping at construction, where in theory the compiler could figure it out for you.

For example:

    let xs = 1 :: 2 :: 3 :: []

Here xs is non-empty, but we must tell the compiler:

    let xs = 1, 2 :: 3 :: []

TypeScript does a better job here (although a great cost!)

What you end up with is massive code duplication or lots of extra function calls:

   xs
   |> NonEmptyList.toList
   |> List.map (fun x -> x + 1)
   |> NonEmptyList.unsafeFromList

(I say this all as someone who really likes F#)

aloisdg• 2 years ago

As a fellow F# dev currently learning Elixir, this kind of thing is a burden

bmitc2• 2 years ago

Just to clarify for the crowd though. In F#, `List.head` throws an exception when it fails whereas `List.tryHead` returns an `option`, which returns `None` when it fails instead of an exception.

A general confusion of mine in Elixir is generally how libraries and functions treat errors. There's the common idiot of returning either `{:ok, ____}` or `{:error, ____}`, but what can be inside the error tuple is not always clear. The other thing is that sometimes a function can both throw an exception and also return a success tuple. Such cases are confusing to handle, and there's a large gap between handling cases like that and the philosophy of "let it crash", which I think is preached a little looser than it should actually be practiced.

I do like F#'s way of disambiguating the two situations. The only issue I have in F#, which actually exists in every language that I know of that has exceptions, is that there is no way to know, up front and clearly, what exceptions can be thrown by a given function. This is particularly frustrating in F#, which has fantastic pattern matching for exceptions. I wish there was exhaustive pattern matching in F# for exception handling, such that it would warn you that you have an unhandled exception in a try/with expression (https://learn.microsoft.com/en-us/dotnet/fsharp/language-ref...) but of course would allow for wildcard patterns.

IshKebab• 2 years ago

Java has checked exceptions and they are used, e.g. in Android. But it seems to be the exception (heh) rather than the rule.

IMO the problem is proper exception handling with checked exceptions and wrapping each function (or at least small blocks) in try catch is just so insanely verbose that even though it is possible to get error handling as good as something like Rust, nobody actually does it in practice.

sodapopcan2• 2 years ago

There's the common idiot of returning either `{:ok, ____}` or `{:error, ____}`, but what can be inside the error tuple is not always clear.

That's the result of Elixir being dynamic and not having built-in monads. You always have to check the docs/code.

The other thing is that sometimes a function can both throw an exception and also return a success tuple.

I would say this is just poor design (though maybe someone could point me to an exemplary use of this?). As I see it (which is from how I've read it described and how most good lib do it):

  - If function can error and the caller can do something about it, return :ok/:error.
  - If the caller can't do anything about the error, raise.
  - If the function can't fail, return a raw value.

ku1ik• 2 years ago

I really respect Elixir core team’s approach to adding gradual typing to the language. They don’t rush it. They didn’t put too much focus on syntax so far (I’d argue the syntax in many cases is less important than foundations) and instead they focused on soundness of the system. With each new Elixir version the compiler is getting smarter, catching more bugs. Not hugely smarter, but smarter enough that I feel safer. Looking forward to Elixir 1.18!

jonnycat• 2 years ago

I'm not sure... I'm a huge Elixir fan and I trust José to build a great solution, but I've found the rollout to be a bit confusing. There was the announcement that "Elixir is now a gradually typed language" prior to 1.17 - but it seems that most of the changes were behind the scenes, and 1.17 largely didn't expose user-facing type errors or warnings.

Again, I definitely trust them to get it right in the long term, but in the meantime, the progress has been a bit confusing to me.

sodapopcan2• 2 years ago

The wording was a little odd, but there are certainly user-facing errors in 1.17, namely:

- Map keys (called with '.') are checked at compile time.

- Using comparison operators with different types causes a warning.

I may be forgetting something.

josevalim2• 2 years ago

Thanks for vote of confidence!

We need to type every data type and every function, so the type system will be rolled out over a long period of time.

The 1.17 release meant that we now have a gradual type system, which runs in every code being compiled, but it only supports a handful of types (including the dynamic one). The full list of supported types and examples of typing violations it can now detect is on the announcement: https://elixir-lang.org/blog/2024/06/12/elixir-v1-17-0-relea...

There is no support for type annotations, that comes in a later stage. The overall stages have been described in an earlier article (and I believe also in the paper): https://elixir-lang.org/blog/2023/06/22/type-system-updates-...

klibertp2• 2 years ago

but it seems that most of the changes were behind the scenes, and 1.17 largely didn't expose user-facing type errors or warnings.

That's how it normally goes with gradual type systems for existing languages, I think. The first step seems to be almost always adding a type checker that doesn't do anything in particular other than handling untyped code. Since being able to handle untyped code makes a type system gradual, announcing Elixir as "gradually typed" when this milestone is reached seems justified. After that, you're free to improve the type system and type checker(s), improve type inference, add specialized syntax, improve typed/untyped interactions, cover more language patterns, and so on. MyPy for Python also started without support for many things that were added later (and it's still being actively developed ten years later).

btbuildem• 2 years ago

I've always viewed Erlang (and by extension, Elixir) as safe from these types of improvements. Maybe I don't have enough experience in these languages, but having guards, and the different approach to conditionals seemed to do away with most of the pitfalls usually "safeguarded from" by the caution tape and excessive road signage of type systems.

I'm curious to learn more, but I can't shake a feeling of vague trepidation here.

rkangel• 2 years ago

Erlang and Elixir have long had "dialyzer" as a type checker. The problem is that it had some severe limitations in approach (and implementation) and so the cost/benefit ratio wasn't great.

This article really speaks to how they're thinking about the costs of the type system, so that you mostly get benefit, and that's great.

__jonas2• 2 years ago

having guards, and the different approach to conditionals seemed to do away with most of the pitfalls usually "safeguarded from" by the caution tape and excessive road signage of type systems

It doesn’t seem that way to me at all. The main pitfall static typing guards me from is runtime errors that can be easily avoided, guards don’t really help there.

If I do

  def double(num) when          is_number(num)  do
    num * 2
  end

I could have some call double(“4”) in my code somewhere, and if that’s reached it would throw a FunctionClauseError and crash the process. I don’t want to be able to do that, I want the compiler to scream at me when I write double(“4”) and not let me do it, the guard is doing nothing of the sort.

yellowapple3• 2 years ago

I want the compiler to scream at me when I write double(“4”) and not let me do it, the guard is doing nothing of the sort.

The optimal solution, then, is for the compiler to be smart enough to recognize that there's a guard and to preemptively enforce it at each callsite. You shouldn't need static typing for this, because the compiler should be able to figure out "oh, there's an is_number guard here, lemme whip up a type constraint for that".

Hell, the even-more-optimal solution would be to not need the guard in the first place, since the * operator already implies an is_number(num) constraint.

29athrowaway• 2 years ago

Erlang is an influential language, it has its uses.

But Erlang and by extension Elixir are a hard sell unless you are writing a system like Whatsapp.

SoftTalker• 2 years ago

Erlang/OTP is the nicest environment for building applications that I've ever experienced. I agree though that it's not popular and I've never actually been paid to use it, with the exception of some small projects that were completely under my control.

scop• 2 years ago

I build web apps. Basic CRUD stuff with a little bit of business logic sprinkled in. Elixir/Phoenix has been an absolute pleasure to use.

doakes• 2 years ago

Shouldn't be a hard sell to developers. According to this year's StackOverflow survey, Phoenix is by far the most admired web framework (10% above the second most popular). Elixir is the second most admired language, behind Rust.

https://survey.stackoverflow.co/2024/technology#2-web-framew...

insane_dreamer3• 2 years ago

The questions in that survey are poorly worded so as to make the results ambiguous, since they confound 1) languages in which X person has worked with in the past year, with 2) languages in which X person would like to work on next year. These express two very different concepts: 1) popularity in the workplace, 2) developers' personal preferences. (Neither of them have to do with the most "admired".

doawoo• 2 years ago

I disagree entirely. Having written everything form little tools for my own use to entire linux based firmware for small embedded machines, Erlang and Elixir are powerful tools. I genuinely find Elixir fun to write in a way no other language manages to match.

lawik• 2 years ago

Heck, I script with Elixir like it was Python.

insane_dreamer3• 2 years ago

That's because its inspiration was to create a language with the power of Erlang but that was as enjoyable to code in as Ruby (many people's "most fun to code in" language if they've actually worked with it; unfortunately it can't compete with Python's ecosystem).

sbuttgereit• 2 years ago

I agree. However, the reasoning behind that hard sell has less to do with actual technical considerations and more to do, I believe, with a combination of long held assumptions (not always right assumptions) about what Erlang is and the fact other other stacks have become entrenched and are good enough in the ways that are easiest to assess that Erlang/Elixir would be a tough sell. Add to that recruiting talent to work with Erlang/Elixir requires some outside the box thinking in HR hiring practices (something I rarely accuse HR departments of) and you have a tough sell.

This is all unfortunate. I do think the BEAM based stacks are underrated for applications outside of communications. Many of the same traits of resilience and resource utilization designed to facilitate communications systems actually apply to web apps and APIs, too. Elixir is very expressive and a good fit for writing business applications like accounting related software (what I'm currently working on). But... you have to think you can arbitrage those operational advantages into an overall competitive advantage... and that's a tough sell especially because it involves a lot of speculation which doesn't play out until you're getting to the end of a project.

hmmokidk• 2 years ago

L take

mikhmha• 2 years ago

I don't understand the reasoning? I've been working on an MMORPG game since quitting my job - and I settled on using Elixir for this project despite never using it before. I have a good understanding of distributed systems, and the features/tooling Erlang & Elixir provided were like a dream for me. Initially I thought I'd just try making some proof of concept thing. Fast forward several months later, and I seriously credit Elixir for how far I've gotten with this game. Most of my time is spent writing server-side gameplay code, not tracking down obscure networking & memory bugs. Its even caught cascading bugs caused by gameplay system interactions - when players and a.i were being (wrongly) resurrected from the dead the system just crashed! When 500 a.i agents doing pathfinding every single tick was starting to make the a.i system lag and delay inputs - it was trivial to understand the bottleneck causing the system to degrade. I can go on and on.

pdimitar• 2 years ago

Please go and on! If you write a blog post I promise you it will have an ardent following and a very warm reception, starting with yours truly.

Snacklive• 2 years ago

Please make a blog post it sounds awesome

mike1o1• 2 years ago

I disagree, and I think this sentiment is due to a failure in messaging from the Elixir core team and community which is a shame. There is a huge focus on LiveView (which is fantastic) in the Elixir community, but Elixir is so much more than LiveView! Even just writing a simple rest API (or GraphQL through Absinthe) is a great experience, due to libraries like Phoenix and Ecto. It's a joy to work with, even if you aren't writing a distrubuted/realtime system.

I wish there was more of an embrace in the broader ecosystem rather than the focus on LiveView.

yellowapple3• 2 years ago

Looks nice; I dig the $-prefixed type annotations. I also like that the types boil down to familiar conventions like atoms / atom-tagged tuples under the hood; that's exactly how I'd expect an Erlang/Elixir type system to work, and is exactly how I already tend to use tagged tuples.

If we get rid of this limitations, we could define head as follows:

    $ non_empty_list(a) -> a
    def head([head | _]), do: head

In this case, I ain't sure what the typespec is really contributing here:

- We can already infer the input type, thanks to Erlang/Elixir baking pattern-matching into function signatures

- We can already infer the output type, because there's only one possible output and it's coming directly from the input

This is exactly the problem I hit the last time I tried to chase down the "typespec ALL the things!" path with Dialyzer: the typespecs are just restating what's already obvious from looking at the function, making them redundant at best. Yeah, having a summary of the input and output types is valuable for more complex functions, but it's already a common Erlang (and Elixir, by descent from Erlang and Ruby) best practice to break down complex functions into smaller, simpler units, at which point the typespecs lose value again.

I'm probably missing something here, though - and maybe a more complicated example would better illustrate the value these typespecs add.

ccanassa• 2 years ago

I love that the creators are considering adding gradual types.

Given TypeScript’s popularity, it’s clear that developers really appreciate types. Honestly, the lack of types is what eventually drove me away from Clojure.

Typing lists and tuples in Elixir

Related Stories