What Happens to OO When Processors Are Free? (2015)

http://blog.metaobject.com/2015/08/what-happens-to-oo-when-processors-are.html

by mpweiher • 10 years ago

196 141 10 years ago

blog.metaobject.com

I guess to take this a bit further, the thought that came to mind for me is if we start processing every user on their own vm with own core and memory. Is that something anyone is doing today with web sites? It'd seemingly be horribly inefficient, but it's an interesting thought experiment.

masklinn6• 10 years ago

Is that something anyone is doing today with web sites?

It's a pretty common unikernel demo, see erlang on xen's "zerg" for instance: http://zerg.erlangonxen.org

The Mirage project's Jitsu also does something similar (though not identical): https://github.com/mirage/jitsu

abrookewood3• 10 years ago

It's not that inefficient if you are not working with real 'VMs', but with containers (or something similar). The unikernel projects mentioned by the peer below (Ling, MirageOS) can spin up a container per request, respond and then tear it down in milliseconds. They are pretty interesting from a security perspective, especially when you couple them with a read-only image - I imagine it would be pretty hard to attack something that only persists for the time it takes to handle a single request. EDIT: Peer is below.

icebraining• 10 years ago

I imagine it would be pretty hard to attack something that only persists for the time it takes to handle a single request.

I don't see why. If the request is the attack (as it usually is), then it'll persist for just long enough to accomplish it. What kind of attacks do you see it avoiding?

catern• 10 years ago

Yes, that's a fascinating new technique, known as "CGI scripting".

notamy• 10 years ago

Minor nitpick: (2015)

That aside, I wonder how hard proper software development would be for such a machine. It seems like multi-threaded software is something that we struggle to get right in our current low-core-count systems, so...

rntz• 10 years ago

Message-oriented no-shared-state approaches (whether you call it Actors, objects, CSP, session types, or whatever) to concurrency have proved much easier to write correct software in than the traditional threads, locks, and shared memory approach. And it's the message-oriented approach that the article proposes basing the hardware design on.

platz2• 10 years ago

Yes but Erlang provides a run-time that manages the messages between the actors/objects. Where does the run-time exist in such a machine? Would even be possible to have a run-time given the processor units exist at the lowest level?

masklinn6• 10 years ago

Assuming process residency/pinning and an erlang-like model it could be pretty easy. Multi-threading I think would be rather bad as you'd have a lot of churn trying to maintain cache coherency, at least at the higher end (hundreds+ cores). At 18 cores it's nothing out of the ordinary, Xeon E5 v4 (broadwell) top out at 22 cores + HT (so 44 threads), and Sun's Ultrasparc T2 topped out at 8C8T (64 concurrent threads).

Though you'd probably want to read the old Connection Machine papers too, the CM1 had up to 65kCPU (although individual CM CPUs were much, much simpler than ARM cores).

cjensen3• 10 years ago

The notion that multi-threading is hard to get right is a common one, but it does not match my experience. I've been running 100+ threads in C++ routinely for 20 years now.

So long as you are sensible (using mutexes for shared data structures, using queues or whatever for inter-thread messaging, using semaphores for inter-thread signaling, etc) it just isn't hard.

The only thing that briefly got me in trouble was when I tried my hand at lock-free data structures. Once they are working, they are fine, but it takes a few surprises before you really understand the A-B-A problem.

klodolph• 10 years ago

Yes, I suppose you can make multi-threading sound easy if you list all of the really easy problems. Mutexes, queues, and semaphores (or condition variables, if you have a modern code base) are all the easiest parts of multithreading.

But the "so long as you are sensible" advice is terribly, terribly wrong. You have to rethink how parts of your code are isolated from one another, and you have to avoid entirely new classes of errors—such as deadlocks, which can appear when you combine two well-tested and correct multithreaded components. And the number of threads has nothing to do with it.

7952• 10 years ago

It could allow us to represent data on the machine in a way that better matches the shape of the actual problem. So that data can be clustered together physically in two or three dimensions and can be processed locally. The current approach makes all data linear which is not a good fit for some problems.

ianhowson2• 10 years ago

It's hard.

The author has reinvented the architecture used by Cell (PS3) and Intel IXP. Both of these architectures are dead specifically because they are too damn hard to program for compared with a multicore ARM/x86 chip.

GPUs would be the most successful modern implementation of this idea. There are opportunities with FPGAs, but GPU silicon is so far ahead (and still advancing fast) that you're usually better off designing for GPUs.

You could also consider Cavium parts (16-64 way ARM chips) which ship today in high-end network hardware.

The common lessons across all of these are that:

* Memory is slow compared with computation

* Put caches everywhere and have the machine decide what to cache

* Synchronisation is hard and puts tremendous load on your scarce memory resources

* It's much easier to do the same job on different data a million times than to do a million different jobs on the same data. In other words, high throughput is easier to achieve than low latency.

matt4077• 10 years ago

I'm not sure if those architectures are comparable to the one discussed in the article, except that both are highly parallel. GPUs and Cell are, as you mention, data-parallel.

The article talks about a much more "anarchistic" parallelism where thousands of different (in code and data) objects are each doing their thing, sending messages to each other when necessary. I guess that Erlang/Elixir's threads are closest currently, as mentioned in the article.

gens3• 10 years ago

note: i didn't go to school for this

Communicating sequential processes[0][1] more or less models that. Golang has some history with CSP (they are based on the concept of pipes and CSP was described with pipes before) while haskel/erlang and similar functional languages model the "sequential" part of processing data.

[0]https://en.wikipedia.org/wiki/Communicating_sequential_proce...

[1]http://usingcsp.com/

readams• 10 years ago

This sounds like a good idea until you realize that your computations will be dominated by synchronization and/or communication overhead.

mpweiher3• 10 years ago

The point of the article was that there is so much extra parallel horsepower available that you would come out ahead even if overhead is something staggeringly ridiculous like 100x. That's what makes the idea interesting (though obviously not necessarily correct, it's just a thought-experiment).

njloof• 10 years ago

Ther Connection Machine was able to handle arbitrary n-processors-to-n-processors "sends" in (log2 n) time, but only by interconnecting the processors in a (log2 n)-dimensional hypercube. That's a lot of interconnects...

elihu• 10 years ago

Setting aside for a moment the wisdom of using thousands of cores to enable object oriented programming (which I'm dubious of), it seems to me that the way modern processors handle communication and synchronization is mostly a historical accident, and we could probably do a lot better than just letting processes share memory willy-nilly and having the processor cache sort it out. Like maybe we could give each core a set of registers dedicated explicitly for communicating with its neighbors, bypassing the memory hierarchy entirely for small messages.

gpderetta• 10 years ago

That didn't work very well for Cell.

mmstick• 10 years ago

Nothing happens to object-oriented programming because it is entirely unrelated to multicore processors, except for the fact that object-oriented programming makes for terribly inefficient data structures that cannot handle threading very well.

If you've written software before, you would know that most of your codepaths require prior data to be processed, and thus having multiple cores won't make software any faster than before. Additionally, there is a cost to starting threads, so the benefit of making your software multi-threaded has to outweigh the cost of launching a thread, sending data to that thread, and getting results from that thread.

Pooling and channels are often used to mitigate the initial cost of creating threads, but it still remains that it takes time to send data to a thread and receive data from that thread, and there will always be that one main thread that needs to handle the majority of the logic and keep results in sync.

Basically, it would be easier to see powerful cores sitting adjacent to miniature parallel cores that are able to respond quickly to requests.

masklinn6• 10 years ago

Nothing happens to object-oriented programming because it is entirely unrelated to multicore processors, except for the fact that object-oriented programming makes for terribly inefficient data structures that cannot handle threading very well.

OO doesn't specify the data structures though, if you model your objects as shared-nothing message-oriented entities it does concurrency pretty well (by not doing threading at all).

mpweiher3• 10 years ago

Exactly. In fact, Alan Kay very specifically wrote his model was that of little computers talking to each other. Having heap-allocated chunks of memory multiplexed onto a single processor, whether sequentially or by time-sharing, is/was an implementation technique that was necessary because massively concurrent hardware wasn't possible at the time.

The question is whether this is still true, and if it is not, what the consequences would be.

rubber_duck• 10 years ago

if you model your objects as shared-nothing message-oriented entities it does concurrency pretty

That's actor model, but it still doesn't feel as natural as with functional languages - dealing with immutable data in OO languages is tedious and error prone

cjensen3• 10 years ago

I think the author was specifically meaning types of OO, like Smalltalk, where each object sends and receives messages. That model works as-if each object is a separate thread.

vidarh• 10 years ago

But most implementation implements message sends/receives under the hood as method/function calls because it's far more efficient. The conceptual difference is small, and easy to support as method/function calls. The cost in overhead of putting them even in a different thread, much less on a different core or different CPU would be a huge problem for processes that aren't inherently possible to parallelise efficiently.

platz2• 10 years ago

But usually there is a run-time that manages the messages between the actors/objects (e.g. Erlang). Where does the run-time exist in such a machine? Would even be possible to have a run-time given the processor units exist at the lowest level?

For example, erlang queue messages into a 'mailbox' that is some out 'outside' of the actors.

I don't see where such message queues would reside in these microprocessors.

jinjin23• 10 years ago

object-oriented programming makes for terribly inefficient data structures that cannot handle threading very well.

The best approach I have seen to address this is the idea of transactional object as it is done in Realm (yes, the mobile database).

The idea that objects can freely be accessed on any thread without locks, with all changes protected by transactions, is so freeing, and the internal representation that Realm uses for the objects is apparently far more efficient than how they would natively be represented.

I really wonder why we are not seeing more of these kinds of transactional data structures. Seems like a natural next step for object oriented programming now that everything is turning multi-core.

louthy2• 10 years ago

You don't need transactions for this, just use immutable data structures and you get _transactional like_ behaviour for free.

comex• 10 years ago

That's sort of like software transactional memory, which has a reputation for being slow. However, limiting transactions to one object at a time may avoid most of the overhead; in fact I suspect it would often perform better than locking. On the other hand, that sounds tough to reason about… do you just always use persistent (immutable) data structures? Because those have quite a bit of overhead (if you don't need persistence anyway for some reason). If not, how do you prevent one thread from seeing another thread's transaction in progress?

cjensen3• 10 years ago

Having a dedicated processor for each object is a terrible idea in terms of energy.

Consider current high-core count Intel processors on Windows. Windows shuts down entire cores to save energy. Windows only powers enough cores to cover your current workload plus some margin. I assume other OSs work the same.

If each object has its own processor, then either (1) you must keep the core powered for the lifetime of the object or (2) the core must be energized each time the object receives a message, and re-energizing is not free.

I guess if the core could auto-detect its load (as current Intel processors do) that would help. But I think dynamically shuttling threads between cores and shutting down as much as possible (which is exactly what Windows does) is likely to be much more efficient.

So go ahead and have a thread-per-object for this hypothetical architecture, but let the OS decide when and where to assign cores.

BuuQu9hu2• 10 years ago

You don't want this at the object level anyway. You want to have objects clumped into vats, which are related object graphs that are isolated from other vats but freely intermingle otherwise. Then, messages between vats invoke turns of delivery inside vats independent of other vats, and we can schedule one vat per processor.

neolefty• 10 years ago

What if we separate this into two questions:

(1) What if you have a fabric of "free" processors, and the real cost is energy?

(2) How would <foo> programming paradigm work on this fabric?

dorianm• 10 years ago

Discussion at the time it was posted: https://news.ycombinator.com/item?id=10121974 (linked by the author in 2015).

dnautics• 10 years ago

Communications overhead will not scale and your parallelism will be amdahl's law-bound. You're better off being gustafson's law-bound, and to use those processors for running more copies of the single-threaded program with different values.

crb002• 10 years ago

Qt added signals/slots to C++.

FPGAs will get us most of the way there. We will want to express programs as concrete circuits and message passing. Something like http://www.clash-lang.org . Stream level operations are preferred over "Object" messages.

There is a tradeoff between large bandwidth large area circuits, and smaller circuits with less bandwidth. Inherently serial stuff goes on small circuits.

gricardo99• 10 years ago

Related, there have in fact been fabricated arrays of processors, at least in Academia, for some time: http://vcl.ece.ucdavis.edu/pubs/2016.06.vlsi.symp.kiloCore/2...

chris_st• 10 years ago

Slightly related is the video game "TIS-100"[1] which has you programming (incredibly) constrained processors that are arranged in a grid, with communication between them. The programming puzzles are not very difficult (so far! I'm about halfway through) and pretty entertaining to puzzle out. There's a "meta game" of comments by "your uncle" who was trying to get it working again.

I think it's closer to the GreenArrays, Inc chips, really.

[1] http://www.zachtronics.com/tis-100/

agumonkey• 10 years ago

There's a talk by moore on infoq about the ga-144. It's as exciting as a CPU talk can be.

abrookewood3• 10 years ago

It's a lot harder when you realise that everyone is trying to optimise for Cycle count, rather than the number of Instructions/Nodes.

stcredzero• 10 years ago

It's not the cost of processing that holds us back so much as it's the cost of coordinating. Doing things in parallel and concurrently are where things get tricky. So an opportunity to trade off software complexity for transistors is interesting, of course!

Animats4• 10 years ago

The real question is "what do we do with a lot of CPUs without shared memory?" Such hardware has been built many times - Thinking Machines, Ncube, the PS2's Cell - and has not been too useful for general purpose computing.

One of the simplest cases for broad parallelism is search where most of the searches fail. The required intercommunication bandwidth is low - you start up all the machines and wait for somebody to report success. Bitcoin miners and crypto key searchers are extreme examples. Problems which can be hammered into that form parallelize well.

Some problems that distribute spatially parallelize well on such hardware. Weather prediction, wind tunnel simulation, nuclear explosion simulation - hydrodynamics problems like that can be handled in sections, with communications only for the edge elements. That's what supercomputers are used for.

Neural nets and deep learning, though - finally, there's a broad use case for such architectures. Google is using lots of GPU-type boards that way now. This is a key concept. Massive parallelism on a single problem is really hard to apply to the things people used to do on computers. Instead, it makes possible new things to do.

An important question is how much memory each CPU should have. Thinking Machines, NCube, and the Cell (256K each) clearly had too little. You need enough memory per CPU that the intercommunication doesn't dominate the problem.

The other big problem is how to organize software for this. We have two main models of parallel software - shared memory, and clusters on a network. Those are the two extremes for parallelism. Systems where the CPUs are more tightly coupled than in an cluster but less tightly coupled than in a shared memory multiprocessor fall outside common programming paradigms. This is a very hard problem, with no really good general-purpose solutions.

highd• 10 years ago

One interesting thing going forward (over the next decade) regarding deep learning and high-performance computing will be if we can circumvent the big serial bottleneck - that is, depth. The best benchmarks have generally all been set by networks with significantly increased levels of depth compared to the previous records. Gradient descent, however, requires propagating residuals back through every layer of the network (to tune the lowest layers, at any rate). This is inherently a serial task. Recurrent networks can be considered to be infinitely deep - in practice they're truncated to some finite level during training.

There are a couple of solutions to these problems that are being worked on - including various "learning-to-learn" approaches. One approach (DeepMind published, I believe) is to locally train an auxiliary network to predict the back-propagated residuals coming down from the top. It will be interesting to see if these approaches pan out, or if eventually we start to run into real serial bottlenecks again as we build ever-deeper networks.

eternalban• 10 years ago

The real question is "what do we do with a lot of CPUs without shared memory?"

At the extreme we would go biological, that's what. But then we're talking 10^n in the billions ranges (say n = 15) and unlikely that we would directly "program" these tiny computing engines.

I think hierarchical models (in terms of system organization, with specialized sub-systems or 'organs' if you will) are inevitable unless these little machines have recourse to some sort of breakthrough in communication that would permit arbitrary [efficient and non-local] meshing.

mikekchar3• 10 years ago

If you are using immutable data structures/pure fp, then you can get a lot parallelism without the need for shared memory because the state is on the stack. As the article posits, you can just pass messages to the functions/objects and receive results back. Again, as the article suggests, the major problem is memory bandwidth. The trick is building a bus (busses?) that allows you to aggregate partial solutions.

This is why I'm very bullish on languages like Haskell in the future (say 10-15 years from now). It's interesting that the article suggest OO because I think the key to allowing massive parallelisation is immutability. Immutable OO and pure FP is essentially equivalent AFAICT, but the latter currently has nicer language support.

jandrewrogers2• 10 years ago

Having spent quite a bit of time designing massively parallel algorithms (concurrency starting at several thousand cores on up), computer scientists are often baffled when I tell them that FP and immutability don't help. In practice, it solves the wrong problem and creates a new one because massively parallel systems are almost always bandwidth bound (either memory or network) and making copies of everything just aggravates that. You can't build hardware busses to compensate or companies like Cray would have long ago.

If you look at effective highly scalable parallel codes of this nature, you see two big themes: pervasive latency hiding and designing the topology of the software to match the topology of the hardware. The requisite code to implement this is boring single-threaded C/C++ which is completely mutable but no one cares because "single-threaded". Immutability burns much needed bandwidth for no benefit. This among other reasons is why C and C++ are ubiquitous in HPC.

The challenge is that CS programs don't teach pervasive software latency hiding nor is much ink spilled on how you design algorithms to match the topology of the hardware, both of which are fairly deep and obscure theory.

We don't need new languages, we need more software engineers who understand the nature of massive parallelism on real hardware, which up until now is largely tribal knowledge among specialists that design such codes. (One of the single most important experiences I had as a software engineer was working on several different supercomputing architectures with a different cast of characters; there are important ideas in that guild about scaling code that are completely missing from mainstream computer science.)

fulafel• 10 years ago

If you have high bandwidth, you can do many of the same things as with shared memory. HPC users have historically used message passing on moderately communication heavy codes.

True, it has not been used for "general purpouse computing" but historically shared memory multiprocessors hadn't been either. We are just now starting to see things like parallelism in web browsers, 10-20 years after shared memory MP became common on desktop. Software is just slow to change.

Sean17082• 10 years ago

HPC is usually a balancing act between the performance you gain by adding another node and the time you lose passing messages around. You can't just keep adding nodes without introducing some form of shared memory because you will very quickly become limited by your bandwidth.

SwellJoe2• 10 years ago

Everything at web scale is embarrassingly parallel, at least on some fronts. There's still often the bottleneck of maintaining state (in a database, probably), but a heavily loaded server will have hundreds or thousands of processes or threads or some kind of async (probably epoll or something reasonably efficient on modern systems) to serve many clients. Which is why Erlang (and Elixir) has some very enthusiastic proponents.

Serving the web is, I think, a very big part of the "general computing" space, today.

dagss2• 10 years ago

Serving the web seems to split between a) IO-bound problems and b) databases. It is certainly not embarassingly parallel in the "would benefit from GPUs" sense.

Those thousands of threads just sit there waiting for the database to respond a lot of the time. The database is busy trying to keep things synchronized across nodes. It is just that the hard stuff is pushed down to the database layer.

loxs2• 10 years ago

Yeah, the Erlang ecosystem is so much better (for web) than everything else available today, it's not even funny.

Though it seems like there is some very major obstacle in people understanding it. It's so frustrating, I've already stopped advocating for Erlang/Elixir. Currently I'm concentrating in just reaping the benefits and doing my stuff.

ssully• 10 years ago

Just want to clarify the Cell[1] CPU was used in the PS3, not the PS2. The PS2 used the Emotion Engine CPU made my Sony and Toshiba [2].

[1] https://en.wikipedia.org/wiki/Cell_(microprocessor)?wprov=sf...

[2] https://en.wikipedia.org/wiki/Emotion_Engine?wprov=sfsi1

ChuckMcM• 10 years ago

This question, what to do with a 'shared nothing' machine, is the real heart of the question.

The original article talked about putting ARM-9 cores on a chip which is also overkill, your typical object neither cares nor needs any of the complex memory architecture an ARM-9, had they chosen a Cortex-M they could have put twice the number of cores on their chip with no loss of computability.

But as John points out, the challenge is how you actually program these things. One interesting proposal I heard was to compile code with every loop unrolled and every branch hard coded either one way or the other. So for every "branch" in a piece of code you would generate two binaries, one where it was taken and one where it wasn't. That means four if statements, 16 different binaries. (pruned for cases that couldn't happen though). And then you ran your input on every copy of the program simultaneously. [1] The issue though, and it became crystal clear to me as I was working my way through how Google processed logs and log data, was that feeding the output of one computation into the input of another became the limiter and the interconnect of the fabric was the real bottleneck. That analysis got a tremendous validation when the underlying network upgrades started getting processed. It made it possible to run the same sort of test on a group of machines connected in the 'old' way, and the same machines connected in the 'new' way. That demonstrated just how important inter-machine communication could be in making them productive.

So a model for optimizing both in parallelization (interdependencies in state) and scheduling (interdependencies in time) it essential for making a shared nothing cluster perform better than a very large shared memory machine.

[1] Branch prediction is good enough these days to make this not much of a win.

Animats4• 10 years ago

Today, loop unrolling is usually a lose if it makes the loop much bigger. Branch prediction is better today, but cache misses are worse.

sklivvz1971• 10 years ago

This article relies on the assumption that OO has been designed to work with single core or "few" core processor.

It's a false assumption. OO has been designed to model the world in the hope that this would make expressing business logic easier.

It's not hard to imagine what would happen with these many cores, because we already have specialized systems with many cores: GPUs and CUDA.

And as every single time people talk about parallelism as a holy grail of performance, the truth is obvious to whomever actually uses this hardware: not many problems are highly parallelizable and parallelism is always harder than single threaded code -- it's harder to think about, and this has nothing to do with the programming paradigm.

mikekchar3• 10 years ago

It think you are incorrect in your last statement. All functors are inherently parallelisable. This means that any map operation is parallelisable. Also as long as the operations are associative (which is a surprisingly large number of cases), your reduce operations will also be parallelisable. That means that any computation that can be represented by a map/reduce is usually fully parallelisable and always paralellisable in the map operation. The only operations that aren't paralellisable are those that depend on the previous values (and even then you can make judicious use of partial function application to cut down on the dependencies). Basically this is where you have effects where order matters and recursive calls.

I think the main reason we have difficulty imagining massively parallel computing systems is that we hold onto mutable state and imperative syntax. This pretty much forces us into a situation where order is significant. It seems to me that abandoning that idea is the key to freeing ourselves up, programming-wise. Personally, I don't find pure fp, or immutable OO that much harder than imperative. Different, for sure, but both have advantages and disadvantages.

dagss2• 10 years ago

No. As someone who has spent five years programming computing clusters, I have seen many problems that are fundamentally hard to parallelize. If it falls within mapreduce you are basically "done" no matter what language you use (no need to argue fp vs imperative) but very many interesting problems to solve doesn't.

I am a big believer in FP too, but for other reasons.

gens3• 10 years ago

OO has been designed to model the world in the hope that this would make expressing business logic easier.

OO was designed for multi processing by sending msgs between multiple processes[0]. Like nature does. However I do not find it suitable for most parallel computing work because race conditions.

[0] https://www.youtube.com/watch?v=fhOHn9TClXY

digi_owl• 10 years ago

The whole thing brings to mind the hermetic expression, "as above, so below".

It feels like every layers of computation logic ends up mimicking both higher and lower layers.

markhahn• 10 years ago

confuses throughput with latency. has nothing to do with OO.

What Happens to OO When Processors Are Free? (2015)

Related Stories