The LMAX Architecture (2011)

https://martinfowler.com/articles/lmax.html

by mooreds • 2 years ago

113 35 2 years ago

chc4• 2 years ago

See also: the discussion on the LMAX disruptor github documentation https://news.ycombinator.com/item?id=38313457

senderista• 2 years ago

So much hype for a lock-free MPMC ring buffer. If you're familiar with the concurrency literature you'll see that nothing here was novel even at the time.

xxs2• 2 years ago

java had a proper memory model and the busy wait was a novelty. iirc, it was used mostly in a single thread processing (i.e. consumer) for the main business logic like matching forex order.

Realistically aside the disruptor LMAX had an interesting way for fail over by replaying all the input to reach the state.

usefulcat• 2 years ago

I thought disruptor was SPMC? Apart from that I don’t really disagree, I think it’s just an area that a lot of people didn’t/don’t know much about.

yxhuvud• 2 years ago

It comes in many variants with different properties.

anonymousDan• 2 years ago

Can you point to some relevant literature?

gpderetta• 2 years ago

I think that -MC is misleading. It would imply anycast (like what you would use for a work distribution queue), but the disruptor is multicast, I.e. all receivers potentially receive all the data. It is also lossy, like UDP.

bostik2• 2 years ago

It is quite fitting that another HN darling, Jane Street, uses similar architecture in their exchange(s).

Event input queue, fully deterministic matching engine, and an output queue. Plus importantly, a sequencer to guarantee that the events going into the input queue have a known order.

Aspects of their architecture and design decisions have been made public through their podcast series, Signals and Threads.[0]

0: https://signalsandthreads.com/

jnordwick5• 2 years ago

This isn't even close to unique. I['ve worked at a few HFT/MM places that all had a similar architecture in place before LMAX wrote this. This, multicast rings, and other patters are copied a lot in the industry because they work well for the problem space.

bostik2• 2 years ago

I'm not surprised. And I think you nailed the reason too:

copied a lot in the industry because they work well for the problem space

Finance as an industry could be described as ... incestuous. The people move around within the small circle of firms, so any architectural knowledge gets cross-pollinated quite rapidly. From what I understand, the disruptor pattern itself was reasonably well known in the academic circles well before LMAX wrote about it - but I'm not sure anyone had put publicly available numbers on a heavily optimised version before them. (FWIW, I use a simplified disruptor in one of my own projects because it works well and simplifies the overall design. )

We should also be honest about the technical challenges: there are very few as interesting problem spaces as an exchange or a low-latency MM.

hangonhn4• 2 years ago

Can you explain what a multicast ring is? The article itself just grossed over it.

Thanks!

pkaye• 2 years ago

Is the LMAX disruptor just a ring buffer? Anything else special about it?

xxs2• 2 years ago

It is a ring buffer indeed, with publish serialization (on producer side) and normally a busy wait. It sort of doesn't work on some VMs due to scheduling. (I mean the busy wait w/o a backoff)

jnordwick5• 2 years ago

The big win is how it allows bulk processing while taking serialization dependencies into account.

So you algos reading market data messages and yank all that is there and apply them in bunk instead of trying to do each one individually.

And your logger that is storing all the messages to a file and/or db knows what messages the algo has processed and can grab all the messages up to that point so it is always writing as much as it can and current.

qwe2645692• 2 years ago

Sounds so much like an io_uring model.

GenericCanadian• 2 years ago

I wrote this in Crystal lang to learn more about it and the language: https://github.com/nolantait/disruptor.cr

There is a Ruby version that is also very readable: https://github.com/ileitch/disruptor

nicholassm832• 2 years ago

And here's a version in Rust: https://crates.io/crates/disruptor

(Disclaimer: I wrote it.)

joezydeco• 2 years ago

Thanks for that. I was always curious why the finance bro down the block had an LMAX license plate on his car.

BrokrnAlgorithm• 2 years ago

Is there any didactic implementation of the Disruptor / multicast ring available somewhere? I've been curious in working through some practical example to understand the algorithm better.

itsthecourier2• 2 years ago

LMAX have an open source version of the disruptor in GitHub https://github.com/LMAX-Exchange/disruptor

nicholassm832• 2 years ago

Here's a high level description in TLA+: https://github.com/nicholassm/disruptor-rs/blob/main/verific...

(Disclaimer: I wrote it.)

There's also a spec for a Multi Producer Multi Consumer (MPMC) Disrupter.

whoevercares• 2 years ago

Fun fact, log4j use disrupter

bob1029• 2 years ago

One of my favorite patterns.

The implementations of this for C# support a value type as the buffer object. You can get upward of a half billion events per second in a SPSC arrangement.

My use cases for it are primarily in gaming or where I have a latency critical element. C#/.NET isn't necessarily ideal for these use cases, but you can do clever things like align GC to the disruptor batch processing boundaries.

If you are finding trouble with this pattern on small machines (1-2 cores), consider adding a heuristic that checks the core count of the machine and swaps from busy to a yielding wait strategy if there aren't at least 4 cores. Yielding wait isn't always good enough, but it definitely works for testing and prototyping.

neonsunset• 2 years ago

Is the company that uses this perchance located in Europe?

The LMAX Architecture (2011)

Related Stories