Data-Oriented Design Principles

by ingve • 3 years ago

102 51 3 years ago

jebarker4• 3 years ago

This might be an unpopular opinion, but I feel like these DOD posts should always mention that the ideas are borne out of experience in game engine development and don't necessarily easily apply in other domains.

In some general sense the principles are applicable everywhere. But trying to actually apply them in some domains is really difficult. For example, I work on developing a large scale ML training framework and I haven't seen an example where avoiding the use of objects and inheritance leads to better code. Having said that, the principles certainly apply lower down the stack when talking about implementing the GPU kernels etc on the hot path.

js6i• 3 years ago

That would be fair if OO/FP posts had always mentioned that the ideas were borne out of academia and don't necessarily easily apply in other domains, but that ship has sailed..

jebarker4• 3 years ago

Why can't we do both? i.e. give an honest account of the pros and cons of different approaches rather than treating them like dogma

rewmie• 3 years ago

That would be fair if OO/FP posts had always mentioned that the ideas were borne out of academia and don't necessarily easily apply in other domains, but that ship has sailed..

OO has been widely adopted because it maps almost perfectly to most domains.

FP is starting to be adopted because first class support from programming languages and frameworks lowered the barrier to entry by eliminating the need to implement all primitives and infrastructure. FP is being adopted because it maps almost perfectly to some domains.

What's your point, exactly?

loup-vaillant3• 3 years ago

Not sure if you’re being serious here: the whole point of academia is to eventually be useful to the outside world. Something that applies to academia and nowhere else is kind of useless. The very reason we take science so seriously comes from how it helped us completely transform the world.

So the idea that something would only (easily) apply to academia because it was born out of it feels kind of ridiculous to be honest.

nightski• 3 years ago

I feel the biggest weakness of PyTorch is that everything is an object with inheritance. It makes no sense that ReLu is a class for example. Seems like ML could take a few hints.

vore2• 3 years ago

Well, PyTorch uses classes more as a declarative way to construct your neural networks. It doesn’t get more data-oriented than pushing thousands of tensors through your GPU!

jebarker4• 3 years ago

I agree that often people choose to use the object versions unnecessary, but many operations (including relu) have functional versions: https://pytorch.org/docs/stable/generated/torch.nn.functiona...

vkazanov4• 3 years ago

Wouldn't it be nice if people would always follow the serious whitepaper structure: abstract, summary, conclusion, measurement...

But this wouldn't work for programming language ideologies. Not for the OOP proponents, anyway.

The DOD crowd can at least demostrate how CPU love blasting through data layed out in arrays.

The FP crowd can least explain how the languages can be formalised.

PS ML is the most natural application of dod principles, no?

jebarker4• 3 years ago

ML is the most natural application of dod principles, no?

Yes, the principles are helpful and natural in implementing the code that will execute the ML model pass with no unnecessary overhead. But in terms of writing the code that configures and defines a model and the pre- + post-processing, and to some extent dataloaders, it's natural to use objects + inheritance. Now I don't actually think using objects and inheritance are antithetical to DOD but it seems that DOD often gets presented that way.

discreteevent• 3 years ago

Not everything that matters can be measured.

tsss• 3 years ago

OOP can also be formalized if you really want to.

ptx• 3 years ago

But this wouldn't work for programming language ideologies. Not for the OOP proponents, anyway.

Why not? Are you saying that OOP isn't solving any problem?

As I understand it, OOP is mainly about allowing behavior to be reconfigured at runtime. As the introduction to "Design Patterns" puts it: "Polymorphism simplifies the definition of clients, decouples objects from each other and lets them vary their relationships to each other at run-time."

Using this indirection absolutely everywhere probably isn't great for performance, but it seems helpful in moderation.

xiasongh• 3 years ago

I feel that DOD is about performance, not understandability and usability, so it makes sense that the frontend of a framework, where usability is priority, doesn't benefit, and the backend and hot path does.

johnnyworker2• 3 years ago

Usability can only ever be as good as performance allows it. I don't care where the buttons are when it takes 5 seconds before they show up, or how pretty font is if the scrolling is stuttering, and so on.

From the user perspective, it really makes no difference if the same UI was hand coded in assembly or in something much higher level that is still fast because it doesn't work against the CPU, RAM, storage and other things.

ehaliewicz2• 3 years ago

I find simple loops over simple data more understandable than inheritance chains and pointer chasing.

flohofwoe• 3 years ago

...where avoiding the use of objects and inheritance leads to better code...

DOD isn't about "better code" but strictly about "faster code" by letting real-world hardware restrictions dictate how code accesses data (the way I wrote that it may sound like it's a bad thing, but it's really not) - writing software that doesn't fight the hardware it runs on can lead to simpler and easier to maintain code, but that's not guaranteed.

xgkickt• 3 years ago

AKA “Cache is King”.

(edit) Terje Mathisen's "almost all programming can be viewed as an exercise in caching" is often quoted amongst game developers, as it rings true with how we use hardware and pre-calculation of data.

dxhdr• 3 years ago

Mike Acton presented a great visualization of this in his 2014 DOD talk -- here's a link to just that clip: https://www.youtube.com/watch?v=rX0ItVEVjHc&t=1831s

bob10292• 3 years ago

I 2000% agree with everything in here except this line:

If you don’t understand the hardware, you can’t reason about the cost of solving the problem.

For me, "Data-oriented Design" mostly implies "try to use SQL-shaped things". In this context, understanding of the hardware is like seeing another side of the query planner blackbox. You should have a general sense of how many raw bytes you can move in a serialized manner based upon your machines & networks, but I feel trying to directly understand how exactly every H/W resource will be utilized goes against these principles at a certain point.

Most offerings of SQL provide exceptionally-powerful tools that attempt to answer questions like "how long might this query take to run based upon history and/or projected growth?". Is this not "reasoning about the cost of solving the problem"?

For non-trivial domains, focusing only on getting the data model logically clean might be a 100% full-time job. You should always worry about performance after correctness unless you are developing a piece of software where performance is the headline aspect of correctness (game engines, DAWs, etc).

munificent2• 3 years ago

The context isn't clear from the page, but data-oriented design mostly comes from the game industry where this level of performance often (but not always) does matter.

And in that context, they don't mean "data-oriented" in the sense of "declarative like SQL". They mean it in the sense of "how the bytes are arranged in memory". One of the primary motivations is being able to use the CPU cache well.

bob10292• 3 years ago

The context is absolutely clear. Data is data. You are simply biased towards a particular domain.

psfried• 3 years ago

In contrast to jebarker's comment, I actually think it's really interesting that a concept coming from game engine development actually seems quite applicable in some very different domains.

We (https://estuary.dev/) ended up arriving at a very similar design for transformations in streaming analytics pipelines: https://docs.estuary.dev/concepts/derivations/

To paraphrase, each derivation produces a collection of data by reading from one or more source collections (DOD calls these "streams"), optionally updating some internal state (sqlite), and emitting 0 or more documents to add to the collection. We've been experimenting with this paradigm for a few years now in various forms, and I've found it surprisingly capable and expressive. One nice property of this system is that every transform becomes testable by just providing an ordered list of inputs and expectations of outputs. Another nice property is that it's relatively easy to apply generic and broadly applicable scale-out strategies. For example, we support horizontal scaling using consistent hashing of some value(s) that's extracted from each input.

Putting it all together, it's not hard to imagine building real-world web applications using this. Our system is more focused on analytics pipelines, so you probably don't want to build a whole application out of Flow derivations. But it would be really interesting to see a more generic DOD-based web application platform, as I'd bet it could be quite a nice way to build web apps.

dkarl• 3 years ago

Different problems require different solutions.
If you have different data, you have a different problem.

What is unstated but perhaps implicit here is that you might have a different problem even if you have the same data, and this is why it is better not to bind functions and data together as in OOP.

Separating functions from data lets you organize them according to the functionality they provide, instead of grouping them all together with the data structure they operate on. When you bundle data and functions together into a class, it often grows into an oversized assortment of all the various operations you might want to do on that particular bundle of data. The OOP solution for splitting up a bloated kitchen sink class is decorators, which results in using multiple object instances, each with a different interface, to manipulate the same data. Instead of simply importing functions from a different module when you want extra functionality, you have to construct a new object instance to wrap your "basic" object and "enhance" it with extra methods.

StackOverlord• 3 years ago

As a Clojure programmer, this is not what I have in mind when I think of data-oriented-programming, but this:

Principle #1: Separating code (behavior) from data.

Principle #2: Representing data with generic data structures.

Principle #3: Treating data as immutable.

Principle #4: Separating data schema from data representation.

Source: https://blog.klipse.tech/dop/2022/06/22/principles-of-dop.ht...

I think using C++ gives a different twist to the meaning of data-oriented, mainly because with lisps code is data. As I read this "manifesto", it seems more focused on the data the program handles than handling the program with data: In Clojure I often use data-oriented programming for programs that barely deal with any data at all. I tend to lay what I call a "plan" that describes the computation that needs to be carried out. In some way this is similar to a DSL except that this "plan" won't run without also writing a "compiler" or "interpreter". If suddenly requirements change and you need to run your "plan" in a distributed way (or any other execution flavor you may think of), you just write another compiler.

Code being data, this is an approach you can take on code itself with macros, not just as a way to add behavior but to split different aspects of code: I once wrote a macro specifically for a block of complex code that I wanted to read without the clutter introduced by debug lines, so I moved this code in a macro that would add it back using a highly specific code-walker.

What is gained by introducing interfaces using data rather than an object system, must be repaid when writing and maintaining those compilers.

codethief• 3 years ago

As a Clojure programmer, this is not what I have in mind when I think of data-oriented-programming, but this:

Yeah, that's more or less how I would have defined DOP, too.

Could it be that there is a slight difference in meaning when people refer to "data-oriented design" (OP) vs. "data-oriented programming"? At least that's been my (anecdotal) impression so far.

lackbeard• 3 years ago

I'm not sure what is meant by "data-oriented programming" (I know what "data-driven" means...) but, yes, "Data-Oriented Design" has a distinct (if somewhat nebulous) meaning which comes from game programmer culture. (And my guess is the difference between it and data-oriented programming is not slight.) Data-Oriented Design is, basically, the name given to the bag of techniques listed on the submitted webpage. I don't know if the term was coined by Mike Acton, but it was popularized by the talk he gave that is linked to on the submitted webpage. (It's an inspiring talk! You should watch it, if you have not, yet.)

As far as I can tell, Data-Oriented Design is a reaction to trauma experienced by game programmers trying to undo damage inflicted by the... inapt... application of OO techniques in game codebases by their (probably well-meaning, but ignorant) peers. (Hence the contrast in the names: OBJECT-Oriented Design -> DATA-Oriented Design.)

The keystone idea seems to be, instead of organizing your program's data as objects in an inheritance hierarchy, figure out which data will be accessed together in tight loops, and pack that data together in arrays. (I.e., prefer structs of arrays to arrays of objects.)

P.S., There's an interaction during the Q&A section of that presentation by Mike Acton which I love: one questioner asks, somewhat incredulously, (and I'm paraphrasing here) "If I were to follow the principles you have laid out in this talk, then, if I ever needed to alter the layout of my data--after having invested (perhaps significant) time already writing my program--I would subsequently be required to rewrite all the code which accesses that data." and Mike Action answers, basically, with a stone-cold "Yes."

oersted• 3 years ago

This is also consistent with how it is thought of in GameDev. The trendy ECS (Entity Component System) architecture is usually implemented with a data-oriented mindset to maximize cache utilization, make allocations/deallocations of many small entities easy and fast, and facilitate concurrency.

vjust• 3 years ago

While Data is always associated with SQL, there is a world of data separated from SQL. SQL is a standard way in many cases to persist and work on data, but doesn't span the lifetime/journey of data. Software systems are built on classes, functions and structs, caches, interfaces, buffers. Its common to have hierarchical relationships between data objects. SQL doesn't naturally handle hierarchy, despite the fact it has syntax to do so.

As a software engineer, data modeler, and data engineer, DoD is a weird label applied to anything. I've decades of experience with SQL but don't gravitate towards it. Reality is messy.

ticklemyelmo• 3 years ago

Even with SQL we recognize that while aggregate roots and ORMS are great for storing and editing single items, they're terrible for use cases where the data is _used_, and you're better off using different query mechanisms to slice it in a better way. Indexing services and caches and transformation pipelines continue that.

I think CQRS is really just a glimmer of DoD in the enterprise world, the recognition that the system of record is generally a terrible resource for actually using the data, and that you need to rethink everything again if you want a performant system.

Data-Oriented Design Principles

Related Stories