Protecting sensitive data in Elixir GenServers (2023)

https://gabrielpereira.dev/en/posts/protecting-sensitive-data-in-elixir-gen-servers/

63 12 2 years ago

pawelduda• 2 years ago

It's good to know about format_status/2, but isn't deriving Inspect protocol for structs better by default? Assuming they're passed as state to a GenServer, it should achieve the same, no? I think this technique should be the first one listed because it covers more ground

latch• 2 years ago

No mention of setting the :sensitive process flag to true. Been a while since I've done Elixir, but I think sensitive: true disables additional things, possibly to the point of being a little too much in some cases.

rdtsc2• 2 years ago

I was going to mention that, too, but the author does provide a link below to https://erlef.github.io/security-wg/secure_coding_and_deploy... which does talk about the sensitive flag.

It also talks about the using `private` ETS tables. However, in general terms, I would say, stuff that prevents data being dumped into the logs is very useful, but trying to protect it against inspection assuming an attacker has gained remote shell access is a bit of a fool's errand, and it has to be balanced against the inconvenience of trying to debug the system when things go wrong -- tracing that sensitive process, checking the contents of that private ets table, etc.

behnamoh2• 2 years ago

On a side note: Those who went from a typed language to Elixir, how was your experience? Every time I read Elixir code such as the one in this link, I get worried and stressed due to the lack of type annotations. I guess my mindset is "write code that never crashes", but then BEAM apparently instills the mindset of "let it crash", which gives me anxiety tbh.

Lately I've been thinking that a dynamic language is fine as long as it gives me tools to recover my program in case things go down (and they will!). Common Lisp's restarts are interesting for this purpose. On the contrary, if my Python program goes wrong, it will crash, end of story.

I guess the benefit of Elixir/Erlang/Gleam is that they run on BEAM, so even though they're dynamic (except Gleam), the cost of crash is not that much as your main BEAM VM never crashes and you can literally login to your running VM like an OS and fix things (like changing the code while the rest of the program is still running!)

I think maybe this will alleviate my fear of no-types in Elixir, but I'm interested in hearing your experience.

jay-barronville• 2 years ago

Every time I read Elixir code such as the one in this link, I get worried and stressed due to the lack of type annotations. I guess my mindset is "write code that never crashes", but then BEAM apparently instills the mindset of "let it crash", which gives me anxiety tbh.

I don’t think the “let it crash” philosophy is about types though. It’s more about not needing to defensively handle every edge case due to fears of your program crashing. Crashing due to ending up in an exceptional state is a feature in Erlang/Elixir, but if a function expects a number and is passed a tuple, that’s still a bug.

saurik• 2 years ago

You don't have to add types to things, but you can via typespecs. The type analyzer then attempts to discover the types of things and can type check the program as well as it can based on what you give it. Some people use more explicit type signatures, others less.

behnamoh2• 2 years ago

I can't believe it's THE saurik I'm responding to! Thanks for all your work on iOS jailbreaking and Cydia (had a lot of memories with it).

qohen• 2 years ago

First of all, as of January, Elixir has gotten its own type-system[0][1]. And, even before, one could annotate Erlang and Elixir with typespecs and then run a tool called dialyzer over the code for type-checking[2]. (And let's not forget that, even without using actual annotations, BEAM languages have pattern-matching, which can give you a kind-of poor-man's typing, if you will).

Second, the point of "let it crash" is not that you can then login to the running VM, but rather that your GenServers, etc. would be part of a "supervision tree", i.e. your processes would be monitored by a Supervisor which would have a "restart strategy" that you wrote to deal with crashed processes. So, you can write code using the "happy path" in cases where you expect things to usually work and, if there's an error, you can let the process crash because it'll get restarted, in a matter of microseconds, by its Supervisor.

It's almost a cliche to hear stories about people running Erlang or Elixir who didn't realize they had a bug in their system for months until they happened to look at the logs and see notifications of a supervised-process getting restarted often.

(And, btw, in cases where you might expect to have lots of potential errors, e.g. from getting data from an external API, you can use exception).

[0] https://elixirforum.com/t/jose-valim-elixir-is-officially-a-...

[1] https://www.irif.fr/_media/users/gduboc/elixir-types.pdf

[2] https://fly.io/phoenix-files/adding-dialyzer-without-the-pai...

elbasti• 2 years ago

"Let it crash" is about code such that, if it crashes, there is a supervisor architecture that can keep the crash isolated and has sane ways of restarting.

This means you don't have to code defensively: if your code crashes, it's ok.

This is important because in erlang/elixir you have pattern matching, which allows you to code declaratively. Example:

Imagine you have a function that expects a two-element list, `[a, b]` and then adds them.

In elixir, you could write the function this way:

``` def add([a,b]) do a+b end ```

Note that the program will crash if you call: `add([1,2,3])` or `add([1])`.

In other words, you can write code such that the shape of the code matches the shape of the data.

If the data looks any other way than the expected one, it will crash. But this is a good thing! It's better than operating on data that has the wrong shape, and thhe OTP architecture should restart the process intelligently.

bryantwolf• 2 years ago

Elixir has had typespecs for a long time which can handle static type checking which is a lot like using typescript with vite. The compiler doesn’t type check, it relies on your IDE to do the dev time analysis.

https://hexdocs.pm/elixir/1.12/typespecs.html

They’ve recently started gradual typing which will feel a lot like typescript in the early days where you can type some things but you’ll frequently need the escape hatch ‘dynamic’ which is their ‘any’ to make it compatible with existing code

https://elixir-lang.org/blog/2023/09/20/strong-arrows-gradua...

sodapopcan• 2 years ago

Pattern matching doubles as native type-hinting (ie, not bolted on--it's always been there). For example:

    def foo(%{} = map) do
      map
    end

This will crash if `map` is anything other than a map.

There are also guards (which is another form of pattern matching):

    def foo(map) when is_map(map) do
      map
    end

Again, this will crash if given anything other than a map.

It isn't always necessary to do this since Elixir doesn't overload operators or many functions that work on types (although it does have the `Enum` module which works across collection types).

For example, `+` only works on numbers. So if you have:

    def add(a, b) do
      a + b
    end

You aren't going to run into those "subtle bugs" people talk about, ie, if you give it anything other than two numbers (like two strings or two lists) it will crash right there. These are only runtime checks but this is where a good set of sane tests help as there is no need to write any type-checking tests.

As mentioned by others, as of a couple of weeks ago the latest 1.7 release candidate has the beginnings of the type system. A big wart has been comparison operators as they will silently compare any two types. This has been half-fixed in 1.17 insofar as a compiler warning will be thrown if you compare two different types. Of course you could still run into the "subtle bug" issue, but this is where pattern matching and guards can help.

Until the type system is finished there is always still Dialyzer if you're into that! It will likely be a while before the type system is fully ready as they are really taking their time with it to do it right.

rdtsc2• 2 years ago

In Erlang (so probably valid in Elixir too?) if you're using OTP 25+, try using `format_status/1` (one argument) instead of the older `format_status/2`

https://www.erlang.org/doc/apps/stdlib/gen_server#c:format_s...

One new thing it does, compared to the old `format_status/2` is it allows removing the last message received by the process from, to prevent it from being dumped in the logs.

Protecting sensitive data in Elixir GenServers (2023)

Related Stories