Futures for C++11 at Facebook

https://code.facebook.com/posts/1661982097368498

155 81 11 years ago

code.facebook.com

obstinate7• 11 years ago

I'm totally on board with futures, IF your application needs to scale to the point that blocking, threaded computation is infeasible. However, I honestly don't think that's the case for most folk. Blocking, direct-call computation is way simpler than futures.

pron3• 11 years ago

Or use fibers (lightweight threads) and enjoy the best of both worlds: simple, blocking code with the scalable performance of async.

obstinate7• 11 years ago

Fibers are fine as an idea. But all the C++ implementations I'm aware of require the programmer to mark up their code for cooperative scheduling. That's a non-starter for me.

dbattaglia• 11 years ago

I think the language and framework are also important in deciding to use them or now. In modern C# with MVC/WebAPI it feels wrong to me to NOT use async/await/Task (the .Net version of Futures and Promises). For fairly little change in coding style you can get some big performance wins on IO code.

obstinate7• 11 years ago

Unless you're writing pretty perf intensive code, part of me doubts you're going to see huge wins on something like async/await.

wcummings2• 11 years ago

Futures model async tasks, there's no reason you couldn't write an executor which uses OS threads using this framework (as they say in the post...)

we provide a flexible mechanism for explicitly controlling the execution context of callbacks, using a simple Executor interface

salibhai• 11 years ago

Futures sounds a lot like Promises to me. Or am I wrong?

matt_d• 11 years ago

Futures can be created with promises:

- http://en.cppreference.com/w/cpp/thread/future

- http://en.cppreference.com/w/cpp/thread/promise

- https://en.wikipedia.org/wiki/Futures_and_promises

- http://docs.scala-lang.org/overviews/core/futures.html

misframer• 11 years ago

Senders hold promises, and receivers hold futures. A promise represents a "promise" to deliver a value at some point to the holder of the future.

fugalh5• 11 years ago

Yes, the two terms are unfortunately both synonymous (comparing different implementations of the pattern in different languages), and usually have specific meanings in the context of any given implementation. Often, as here, Promise is the write handle and Future is the read handle.

ddoolin• 11 years ago

Promises, C++ style?

ConAntonakos• 11 years ago

That's what I surmised as well.

Aleman3602• 11 years ago

On a related note, MS has PPL on Windows: https://msdn.microsoft.com/en-us/library/dd492427.aspx

Very similar to C#'s TPL.

And VS2015 has experimental support for async/await in C++: http://blogs.msdn.com/b/vcblog/archive/2014/11/12/resumable-...

Matthias247• 11 years ago

There's also pplx, which is a cross-plattfor implementation of ppl: http://microsoft.github.io/cpprestsdk/namespacepplx.html

hatred3• 11 years ago

Can someone explain to me what does it add over http://www.boost.org/doc/libs/1_58_0/doc/html/thread/synchro... and pros/cons ? Thanks.

fugalh5• 11 years ago

It is along the same lines as boost's futures implementation. We have a different mechanism for expressing thread management, born out of trial and error and Facebook engineer feedback. At the time we set out to write this boost futures were slow and buggy (1.53), and C++ standard monadic futures proposals were in very early stages (it now appears that there will be monadic futures in C++17). I do not know if boost futures are now more robust and/or more performant in 1.58.

hatred3• 11 years ago

thanks for the explanation.

Serow2252• 11 years ago

Since it seems like you're at FB, do you know if anyone's working on an equivalent of FB's Swift but for C++ instead of Java? If so, I'd love an email to express interest in such a thing. Thanks in advance!

vsl• 11 years ago

They seem to be more robust, but boost::future::then() has weird design that makes it all but useless for real-life use. They return the same kind of future that std::async() does, i.e. its destructor blocks until the future is ready. In other words, you can't write "fire and forget" code:

    void run_and_report()
    {
       something_long_running()
       .then([=]{ report_results() });
    } // blocks here until the then() block executes

tangled• 11 years ago

I'd be interested in seeing why instagram didn't implement the suggested user service as a batch process that populated some KV store. Online evaluation is great for guaranteeing up-to-date results, but the tradeoff is that you now have to bring up and maintain a fairly critical production service.

The SU service fetches candidate accounts from various sources, such as your friends, accounts that you may be interested in, and popular accounts in your area. Then, a machine learning model blends them to produce a list of personalized account suggestions.
This enabled us to reduce the number of instances of the Suggested Users service from 720 to 38.

mikeyk• 11 years ago

We've seen huge gains from making the system as realtime as possible. Consider a new user who hasn't followed anyone yet--each new follow is super helpful in informing the recommendation system.

chadaustin• 11 years ago

Rats, the terminology is backwards from the E/JavaScript promise/future distinction: https://en.wikipedia.org/wiki/Futures_and_promises#Read-only...

There's been a push to standardize "Promise" to be the read-only side and "Future" to mean the resolver/promise pair, but Folly Futures use the opposite convention.

Agreeing on terminology is hard. :)

Edit: I think it's fine that Folly adopted the C++11 convention. I'm just whining in the open that that promise vs. future may never be agreed upon.

forrestthewoods• 11 years ago

I fucking hate the terms future and promise so god damn much. The names are beyond stupid. They're just confusing as hell. No one can read the names and make an educated guess as to which is which it what they do.

The following is my favorite interview response of all time.

Q: what's a quaternion? A: I don't know... But I bet it has four parts.

Genius! An honest answer followed by a wise educated guess. Now, what's a future and what's a promise? How can you remember the difference? Good luck!

Those terms should be retired forever.

gohrt• 11 years ago

A promise what you get when you can't get a result immediately: a promise to deliver a result later. "Future" is nonsense terminology.

Aleman3602• 11 years ago

Microsoft uses "Task". I like that way better.

obstinate7• 11 years ago

Make a header file for your project that exports the Facebook headers and typedefs the names to something that makes sense. I like source and sink. Producer and consumer are fine too. Emitter and receiver are fine as well. And writer/reader. If you do this, then you only have to make sense of which side is which once, then forever after you'll have sensible names for the things (at least in your project).

The important thing is to always treat the name the same. Generally, I treat types as nouns. So an emitter is something that emits something. It's passed to the routine that consumes the data. On the other hand, a consumer consumes, so it's passed to a routine that emits something for consumption.

rattray• 11 years ago

I've always found "promise" intuitive personally; it returns a guarantee about a future action. "Future" gives me a vague idea that it has to do with something later, but isn't great.

Learning these terms isn't that tough, either. I don't find "promise" any less intuitive than "statement", "expression", "parameter", or "argument". I'm just used to the latter ones.

fugalh5• 11 years ago

Scala also takes the same way C++11 does, and to my mind it makes more sense this way. So from my perspective Javascript is the backward duck here. I agree it would be nice if we could all just settle on something

No12• 11 years ago

E came out in 1997. From my perspective, Scala and C++11 are the backward ducks (unless you can point to something earlier using the Scala/C++11 nomenclature).

I agree the Scala/C++11 way might make a little more sense intuitively, but it really doesn't help flipping the definitions around.

vvnraman• 11 years ago

Excellent library and writeup !!

This is very similar to HPX, the general purpose C++ runtime system for parallel and distributed applications by the Stellar Group - https://github.com/STEllAR-GROUP/hpx.

I recently saw the excellent video presentation by Hartmut Kaiser on this https://www.youtube.com/watch?v=5xyztU__yys and a lot of concepts in folly futures are quite similar. However the most striking thing in HPX was that all the building blocks are serializable, and the presenter mentioned that it is so because you could serialize and move a thread to a different machine and run it there.

santaclaus• 11 years ago

Kaiser also gave a pretty good keynote at Meeting C++, "Plain Threads are the GOTO of todays computing:" https://www.youtube.com/watch?v=4OCUEgSNIAY

Same talk maybe?

amelius• 11 years ago

But how does one elegantly stop a chain of futures from executing when one is no longer interested in the final outcome of this chain of futures?

batou• 11 years ago

In microsoft-land where I reside, they are called tasks and we cancel tasks.

Technically all tasks can be collected in a list and then you just cancel everything in the list but I have yet to find a scenario where I've needed this.

jsedgwick• 11 years ago

Folly Futures support interrupts and cancellations. See https://github.com/facebook/folly/blob/master/folly/futures/...

stuhood• 11 years ago

com.twitter.util.Future supports raising interrupts that propagate 'backwards' through the chain (and across network boundaries.) This is used to implement cancellation, among other things.

https://github.com/twitter/util/blob/master/util-core/src/ma...

dang• 11 years ago

There is another fine post about this project at http://instagram-engineering.tumblr.com/post/121930298932/c-....

It was posted as https://news.ycombinator.com/item?id=9746522, but we merged that thread with this one so as not to have two on the front page.

millstone4• 11 years ago

I've never programmed seriously with futures, and I'm a little apprehensive.

For example, let's say I type the string "123". If there's a future / promise / async whatever anywhere, we risk processing the characters out of order. We may not catch that in testing, because it's usually fast enough that it comes out in the right order.

It is more efficient to make service A asynchronous, meaning that while B is busy computing its answer, A has moved on to service other requests. When the answer from B becomes available, A will use it to finish the request.

What if the first request is "set sharing permissions to private" and the second request is "upload this compromising photo?" Obviously it would be bad to process these out of order. How is this handled?

wcummings2• 11 years ago

Are you asking "how do you do concurrent programming?" These problems aren't specific to promises.

millstone4• 11 years ago

I don't think so. Here's a concrete example of what I mean.

Say we are making a Todo list app. We have a list of Pending and a list of Completed todos, and when the user completes one, we move it from Pending to Completed. But how do we ensure other threads can't see the transient in-between state? Traditional concurrent programming might solve this with a lock:

    lock()
    Pending.remove(todo)
    Completed.add(todo)
    unlock()

This problem is very well known, and any discussion of threads will spend a lot of time on locks, queues, serialization techniques, etc. for avoiding races. Now, with futures:

    Pending.remove(todo).then({Completed.add(todo)})

We've got the analogous race condition, even if we're single threaded. But articles on Futures never seem to discuss techniques for mitigating this. Why not? Is there a Futures equivalent for a lock?

lfowles• 11 years ago

What if the first request is "set sharing permissions to private" and the second request is "upload this compromising photo?" Obviously it would be bad to process these out of order. How is this handled?

Upgrade the user wetware. Have them wait until confirmation that sharing permissions are now private before beginning upload.

reagency• 11 years ago

I would never use your system. I expect a system to behave as advertised , not debug its internal state transitions

wtf_is_up• 11 years ago

Futures in Scala are composable, so this becomes:

    val upload = for {
      p <- Future(setPerms(private))
      u <- Future(uploadPhoto(photo))
    } yield u

    upload onComplete {
        case Success(uploadStatus) => // do stuff
        case Failure(why) => // do stuff
    }

These will not happen concurrently unless the futures are declared outside of the for comprehension.

nimrody• 11 years ago

I guess this should be

     u <- Future(uploadPhoto(p))

right?

bklimt• 11 years ago

How does this project relate to Facebook's other futures/promises library, Bolts?

https://code.facebook.com/posts/225525624316574/building-and...

hatred3• 11 years ago

They might look related in terms of various "idioms/building blocks" used but they cater to two completely different programming languages i.e. Objective C/C++.

damart• 11 years ago

Futures/promises seem like they're basically just a limited subset of FRP (Rx, RAC) observables/streams - such that they either "next" once and "complete", or "error" without "nexting".

I suppose they also have caching built in, but that is easy to build on top of FRP.

learc832• 11 years ago

Yeah, I think you're right on the money. Matt Podwysocki has a chart explaining how promises fit in to all of this but I can't seem to find it. Here's what he says in the RxJs documentation.

"One question you may ask yourself, is why RxJS? What about Promises? Promises are good for solving asynchronous operations such as querying a service with an XMLHttpRequest, where the expected behavior is one value and then completion. The Reactive Extensions for JavaScript unifies both the world of Promises, callbacks as well as evented data such as DOM Input, Web Workers, Web Sockets. Once we have unified these concepts, this enables rich composition."

fugalh5• 11 years ago

Indeed, any good intro to Rx points this out: Observable is the plural of future/promise.

When people are trying to do fancy streaming stuff with futures and come to me for help, I generally recommend they do the streaming with Rx instead. (eg rxcpp from Microsoft)

blakesmith• 11 years ago

I'm a huge fan of Finagle futures, which this project seems to draw a lot of inspiration from. My biggest challenge is actually the fact that Folly as a dependency is so heavyweight memory wise: I have trouble building anything on my laptop with 4 gigabytes of memory. Most of the Folly code is template code - I guess lots of template code leads to large memory compile time footprints?

Anyway - great project. I'd love to use it sometime maybe when I get a bigger dev machine.

fugalh5• 11 years ago

Yes, templates are expensive at compile time (memory and time both). folly/futures only depends on a few pieces of folly/, you may find that you can use the futures headers just fine. If building libfolly is the showstopper, you might be able to comment out all the non-dependent files.

learc832• 11 years ago

Has anyone used the Reactive Extensions for C++? I've used RxJs fairly extensively, but I haven't tried it any other languages. Rx solves a similar problem, but observables are more general than promises.

https://github.com/Reactive-Extensions/RxCpp

iambvk• 11 years ago

Am I the only one who thinks ".then(...)" code is also unreadable? Composability with ".then(...)" looks like a workaround for C++ limitations.

Why can't we have this instead:

  fooAsync(input) {
    //...
    USING_B_THREAD_POOL
    //...
    USING_A_THREAD_POOL
    //...
    USING_B_THREAD_POOL
    //...
  }

What are the disadvantages?

jasonzemos• 11 years ago

.then() is a hack as much as the entire async paradigm is a hack to make up for the weight of OS-supported threads used in their intended way. We have to step back and ask: what is the ideal mode of development? What does one want to do but can't? The ideal answer is to continue developing in the traditional, stackful, RAII-made-easy way where the operating system can provide a suitable natural API and runtime environment (i.e threads and kernel scheduling), and the programmer can state their intent as clearly and naturally as possible.

The std::future interface is good, but it has no truly suitable implementation. I'm not familiar with FB's Folly but at a glance I don't see it solving the fundamental problems with both std::future and callback-hell; .then() seems to just move callback-hell into one place rather than having it spread out -- it's still hell. std::future on the other hand is tied directly to the OS thread interface. Are you able to .wait() on say, 1,000,000 items at once? You'd either need 1,000,000 threads or you're bound by the slowest waiting entity in the current thread. It's definitely flawed.

The solution is to once again break down the execution context. Like threads did with processes, a new division is necessary in userspace based on stackful context switching (think boost::context crossed with boost::asio using the yield_context feature). Userspace contexts can "go to sleep" and "wake up" while being agnostic to the bounds of the thread-pool, and working fine on even a single thread. This model allows synchronous-looking programming while really acting in an asynchronous way -- which is what the obsoleted OS process/thread system offered.

"Async" and "callbacks" are really just a model of no-model -- it hasn't inverted the stack, it's basically eliminated it -- and I treat it as nothing more than temporary.

xyzzy123• 11 years ago

Fibers which could do async IO coded in a blocking style (co-operatively tasked) would rock so hard that one day it has to work without sucking...

TwoBit• 11 years ago

In game development we don't use futures or similar mechanisms. We use job systems. They are much more powerful for serious muktithreading requirements. I'm not saying futures are useless at all, but we wouldn't use them where performance matters.

jnordwick• 11 years ago

Total strawman on the code example. They don't really need the mutex and all the complexity, but they add it so they can overstate their point -- make the most complex code you can and compare it to a simple example you code. Blah.

georgerobinson• 11 years ago

Threads are heavyweight — switching threads is inefficient, they have considerable memory overhead, and the OS will bog down if you make too many of them.

I didn't realize Posix threads were that inefficient?

First, I can't see why the memory overhead would be anything more than having a separate stack and entry in the TCB?

Second, can you not just save the stack pointer, program counter and registers into the TCB, then do the reverse when restoring a thread?

Finally, I wouldn't have thought you'd have too many TLB misses either, thus leaving the only expense being trapping to the kernel to switch between threads?

Can anyone explain?

albinofrenchy• 11 years ago

I can't explain completely, but check out http://blog.tsunanet.net/2010/11/how-long-does-it-take-to-ma....

Relatively speaking, I don't think context switching is _that_ expensive compared to other areas you can focus on like doing smarter memory management.

I don't know exactly what they mean by threads having heavy memory overhead, but possibly they mean cache interference mentioned in that link? I'd be curious if there is an actual large memory chunk other than stack/scheduling details in play here too.

On modern cores too, there are a decent chunk of registers, including floating point registers. I've looked into timings before for embedded applications and the performance hit here isn't trivial when looking at interrupts and the like; but I'd be surprised if it was that overwhelming on servers.

jasdlfkja• 11 years ago

Names go against C++ convention, should be. Future<T> => future<T> makeFuture => make_future

TheMagicHorsey• 11 years ago

This is quite nice. I still wish C++11 would add some CSP style channels into the core language, with appropriate elegant syntax (maybe they could steal Go's syntax for channels).

I'm sure there is a good performance reason they haven't done this yet. I don't know enough about the design of programming languages to know all the complexities involved. I'm just idly wishing.

uuilly• 11 years ago

Another option that has been around for a number of years is: http://doc.qt.io/qt-5/qtconcurrent-index.html

Futures for C++11 at Facebook

Related Stories