Thread Safety in C++ and Rust

https://blog.reverberate.org/2021/12/18/thread-safety-cpp-rust.html

by todsacerdoti • 5 years ago

147 72 5 years ago

blog.reverberate.org

Nice overview. One other thing to add about thread comparability is that different objects must be able to be used concurrently safely. This is important with objects that may have hidden shared state - for example a copy on write string.

haberman7• 5 years ago

Yes that is a good point. I will see if I can fit this into the article without introducing too much digression.

pornel3• 5 years ago

Another difference is that in C/C++ thread-safety documentation usually talks about functions being thread-safe or not. OTOH in Rust, the thread-safety information is attached to data types, not functions. Even closure types like `Fn() + Sync + Send` are not about the function itself, but the data it is allowed to capture. So in a sense, you're allowed to call any function anywhere you want, but Rust will stop you from giving it data that it couldn't safely touch.

haberman7• 5 years ago

Which documentation do you mean? The Abseil docs I linked to (https://abseil.io/blog/20180531-regular-types#data-races-and...) are talking about thead-safety of types, not functions.

gumby• 5 years ago

OTOH in Rust, the thread-safety information is attached to data types, not functions.

Which as the article points out means you pay the overhead for the thread safety every time you use the object wether you need to or not.

So it's a design tradeoff, and I can make the argument for either case. However in my opinion Rust made the wrong choice. They chose the "you don't have to think of it" case, which I think is wrong for a systems programming language.

If I want to make a bunch of changes to an object atomically (or a bunch of bookkeeping to multiple objects) I still need a manual lock (mutex), and then the internal bookkeeping is deadweight. I also might use an object on a single thread for a while and then hand it off to another thread which then has ownership (so still requires no locking). Or even hand it off to a group of threads which then need to use it in a safe way, presumably by calling different functions.

I don't mean to imply there is One True Way or that Rust Got It Wrong.* This is merely one choice of many when exploring the design/functionality space.

* OK on mixed case identifiers they did :-)

geofft2• 5 years ago

Most of this internal bookkeeping is at compile time, right?

Your average object that contains a bunch of mutable fields will, even in Rust, be expressed using a mutex. For instance, here's a very simple race-free program that transfers $20 to Alice from Bob and $30 from Bob to Alice: https://play.rust-lang.org/?version=stable&mode=debug&editio... It uses

    struct Bank {
        alice_balance: i32,
        bob_balance: i32,
    }

and keeps the bank inside a Mutex (or more specifically inside an Arc<Mutex<Bank>> to manage ownership across threads). This is a pretty straightforward pattern based on the Mutex docs: https://doc.rust-lang.org/std/sync/struct.Mutex.html

In this case, the individual threads that perform the transfer are not applying any additional overhead to alice_balance or bob_balance. Those are regular old int32_t variables in a regular old struct, same as C/C++. The only thing that's being prevented is that the compiler won't let me write a version of this program that doesn't have the Mutex. There's a lot of analysis going on inside the compiler to confirm this, but once the compiler is happy, there's no runtime overhead compared to the equivalent C/C++ program.

(So I don't think "you don't have to think of it" is really what's going on. You have to think of it. It's just that if you don't, your program won't compile.)

Now, you can write a version of the program that uses thread-safe data types for the balances directly, maybe something like this: https://play.rust-lang.org/?version=stable&mode=debug&editio...

    struct Bank {
        alice_balance: AtomicI32,
        bob_balance: AtomicI32,
    }

And it is true that there is now extra overhead in this version of the program, and the "advantage" is you could get rid of the mutex and it would still compile. But why would you write it this way? You know that you want the accesses to be transactional. You know you're going to want a mutex (or something) around the Bank. Why add the atomic types?

If Rust somehow used atomic integers by default (or something like a global interpreter lock) so you didn't have to think about it and code compiled whether or not you locked things properly, then yes, I'd agree. But it doesn't - it uses normal integers and things fail to compile.

kouteiheika• 5 years ago

However in my opinion Rust made the wrong choice. They chose the "you don't have to think of it" case, which I think is wrong for a systems programming language.

As someone who has programmed in C++ for over 10 years and in Rust for over 5 years I have to hard disagree on this one. (:

Rust made precisely the right choice because by default in gets rid of all of the mental overhead of "is this actually safe to call/move this across multiple threads?" while giving you the tools to bypass it if you actually want to do it the C++ way, and then you have to think about it. (But only if you explicitly choose to!)

If I want to make a bunch of changes to an object atomically (or a bunch of bookkeeping to multiple objects) I still need a manual lock (mutex), and then the internal bookkeeping is deadweight.

Then just... don't add any internal bookkeeping to your type? (:

Or, depending on the kind of bookkeeping you use, you might not have to pay for it anyway. For example, Rust has atomic integer types which require you to access them in a safe way, however, they also have this method on them:

pub fn get_mut(&mut self) -> &mut u32

...which means that if you wrap that in an outer Mutex you will have an &mut to it, which means that even though the type is thread-safe by default you do not have to pay anything extra to use it if you decide to put it in an outer Mutex and do the synchronization yourself. The overhead of its "built-in" synchronization is then completely gone!

Isn't this the best of both worlds? (:

I also might use an object on a single thread for a while and then hand it off to another thread which then has ownership (so still requires no locking).

...so you'll have a &mut reference on that single thread, in which case no locking is required. Am I missing something?

Or even hand it off to a group of threads which then need to use it in a safe way, presumably by calling different functions.

Fair enough. But in that case you can just use an UnsafeCell and do it the C++ way if you really want to.

matklad2• 5 years ago

you pay the overhead for the thread safety every time you use the object wether you need to or not.

That’s not correct. See, eg, `Mutex::get_mut` (https://doc.rust-lang.org/stable/std/sync/struct.Mutex.html#...). It gives access to the underlying data without any locks or synchronization, if you can prove that no other threads ca access the mutex concurrently at the point of call.

apatheticonion• 5 years ago

As someone new to Rust I appreciate the way you have articulated this.

ridiculous_fish• 5 years ago

This seems rather like a hole in the scheme. There really are thread-unsafe functions like `setlocale` or `setupterm`, and the corresponding Rust crates allow you to use these functions from arbitrary threads. It's easy to produce a SIGSEGV by just invoking `ncurses::initscr()` a bunch in separate threads, without writing any unsafe code.

firethief• 5 years ago

That's only a problem if you use code that is unsound. Well-written crates do not incorrectly mark functions safe.

pornel3• 5 years ago

Thread-unsafe C functions are a problem for Rust indeed. For example, C `setenv()` is unfixably broken for multi-threaded programs, and C code invoking it can crash Rust programs. Currently it's an open question what Rust should do about that.

However, for Rust functions there's no hole, because Rust doesn't allow thread-unsafe global state. All globals always have to be immutable or use some synchronization primitive. Therefore, a safe Rust version of the `setlocale()` function would be forced to use atomics or mutexes to set the locale without data races. It's still a bad API, and Rust can't fix that.

masklinn2• 5 years ago

OTOH in Rust, the thread-safety information is attached to data types, not functions.

That’s not entirely true, in the sense that you can have non-threadsafe functions operating on thread-safe types e.g. Mutex::get_mut.

pornel3• 5 years ago

But `Mutex::get_mut` takes `&mut Mutex`. In this case the exclusivity and safety is ensured by the exclusive reference `&mut`. So get_mut can be called from any thread where you can get hold of a `&mut Mutex`.

couchand4• 5 years ago

The author seems to have missed that operations on atomics in Rust don't need &mut references.

haberman7• 5 years ago

Not exactly. I was just trying to show the most direct port possible of the C++. In C++, Increment() is a non-const method, so in Rust I made increment(&mut self).

The point is to show that thread-safe types are expressed differently in C++ and Rust. If you try to do things the C++ way in Rust, it doesn't work.

couchand4• 5 years ago

Strange choice of demonstration, then. The C++ example is thread-safe, so you don't demand any further synchronozation from the caller. The Rust example, by contrast, DOES demand additional (superfluous) sychronization by the caller.

The article makes it seem like Rust type system is just throwing up blockers for no reason.

If your API is satisfactory with an &mut reference, then you don't need an atomic inside.

fulafel2• 5 years ago

This is very kid-gloves on C++. In C++ these are basically just aspirations without any actual safety assurances.

kwenfjkwenf2• 5 years ago

Yes, of course. C++ is not safe. That's the whole point of the language.

stathibus• 5 years ago

I wouldn't say that's the _point_

jstimpfle4• 5 years ago

A few thoughts and questions by a layman interested in distributed and concurrent programming. I would be very interested to hear back from someone with deep experience in this area.

In my limited perception we are moving further and further away from fine-granular synchronization, and many use cases can be solved efficiently with off-the-shelf synchronization methods like queues. Given that, is thread safety that much of an issue anyway? What are folks' use cases where thread correctness is hard to get right? Where are more complex concurrent data structures needed - like, say, a concurrent hashmap?

I gather that in kernels like Linux for example, there is a lot of this because it is attempted to serve as many syscalls as possible in the calling thread. My idea is that complexities like this come partly due to the fact that a running OS is a loose collection of software that wasn't developed by a single entity, so necessarily there was a lack of opportunity to measure requirements and allocating appropriate storage and computing resources ahead of time.

In systems where the requirements are clearer cut, maybe appropriate resources can be more easily allocated upfront to achieve simplicity and possibly higher throughput at the cost of a little added latency (communication).

It also seems like the fine granular model could become more obsolete over time as shared state becomes costlier with growing core count and the higher synchronization cost that I assume must come with it.

mrfox321• 5 years ago

Depends on what you are working on?

Cross-request cache's naturally require synchronization. Although you could simply get away with thread-local storage.

In huge systems, there's always some other thread doing stuff that you may be using in a way that hides the fact that it is thread-safe.

I'm no expert, mind you.

I avoid concurrency problems if possible, but they are sometimes unavoidable. I touch shared resources that would OOM if made thread-local.

Animats2• 5 years ago

I'm working on a client for a virtual world. This has a whole 3D world, or at least the part you can currently see, in memory. Anything can change, but most of it is static. There are quite a few threads working in coordination to keep the world updated. Some of this is painful in Rust, because Rust really wants an ownership tree. It can be made to work with reference counts (rust "Rc" and "Arc") and their weak forms as backlinks. It's kind of clunky. Single owner with backlinks, along with the static analysis needed to guarantee safety, is needed. That's a hard problem.

Much of the "use a queue" thinking comes from the web back end crowd, where you're managing a large number of connections which have very little interaction with each other. Most of the state is in the database, anyway.

Now that is a very real problem. The next frontier in language design is support for non-shared-memory systems all working on the same problem. I knew someone who spent too much time on that with the Playstation 3's Cell processors. Sony went back to more traditional architectures after it became clear how hard it was to do anything that way. The supercomputer people have been dealing with this for years, but mostly they want to handle big matrices, where the inner loop is tiny. Same with GPUs - shaders are highly parallel but small programs.

Hercuros• 5 years ago

Another option instead of using Rc/Arc is to use integer indices/handles into (e.g.) a vector holding your objects. Then you can have safety (in the sense of there being no language-level UB) by bounds checking your indices (you can do even better if you use a generational arena and attach a generation counter to your indices), and you can still have arbitrary object graphs that way.

I think that this pattern is common in C++ game programming as well (e.g. when using entity component systems), because using pointers or references everywhere and building complicated object graphs is a recipe for object lifetime issues.

I would be surprised if anyone could handle the mental complexity of managing ownership in arbitrary object graphs without there being some simplifying idea (like that of an ownership tree) involved, even if it's not enforced at the compiler level (as it is in Rust, but it isn't in C++).

bluGill• 5 years ago

CPUs stopped getting faster about 20 years ago. There have been minor improvements, but up until 20 years ago you could count on the a doubling of MHz every couple years, today clocks are not faster.

Unstead we are given SIMD and more core cores. Both can greatly increase the speed of your code, but only if you can take advantage of them. SIMD requires specific data structures. Cores need threads. As such how to take advantage of threads is a critical question for any program that isn't fast enough.

If your code is fast enough then don't worry about it. If your code isn't then you need to to deal with hard areas of programming.

geofft2• 5 years ago

I think this is an interesting question and I'm not sure why it was downvoted.

One argument here is that the whole mental model of Rust is that yes, most things can be solved with off-the-shelf synchronization methods. What the Rust type system gives you is a way for library authors to write those methods and say "Yes, this hashmap is actually a concurrent hashmap, safely usable from multiple threads" - and in turn for these types to compose, and to see if the data stored in the hashmap is itself thread-safe. Then the application author doesn't need to worry about any of this. As long as the code compiles (and the library authors weren't going out of their way to lie about types and make the Rust compiler ignore it), you know there are no thread-safety issues and you can just treat synchronization as a problem for someone else to solve. That's the ideal - just like the average application shouldn't have its own crypto implementation, it shouldn't have its own concurrency implementation.

A simple case where you need a concurrent hashmap (at least conceptually) is writing a multi-threaded web server or similar. When a thread wakes up, it gets told that there's data on some particular connection. It then needs to look up that connection in a table and manipulate the request object associated with that connection. Any thread could be woken up for any incoming traffic, so this table needs to be cross-thread. You could do one thread per connection, and arguably more people should, but people very often find that they hit scaling limit with that approach.

But yes, most of the time you should avoid shared mutable state. The Rust type system is designed around making it hard to accidentally have shared mutable state. Anecdotally, that feels like the most common cause of thread-safety issues - not people thinking up front about how they need to design their program around a concurrent hashmap.

jstimpfle4• 5 years ago

Thanks for commenting :-). I probably would have classified what you describe as a coarse-grained synchronization problem though. There needs to be a little runtime-y support for the connection and request object handling, but that can live in a central place and be exposed as a simple (blocking?) function call. It's almost an "off-the-shelf" situation. Individual request handlers can be programmed without any concern for synchronization (at least with regards to the request object, and apart from that most request handlers are rather isolated from each other). Any particular request is never handled by more than 1 thread at once. It's only 1 logical thread, as is evidenced by the existence of thread-per-connection implementations.

I am starting to think that this is a good example of what I wanted to express in my parent post - maybe synchronization issues can more often than not constrained to a few central places, maybe we don't have to litter code with locks and unlocks so much? Maybe web request processing architecture is not that primitive and other domains can learn from it? Maybe a lot of nitty-gritty synchronization could be centralized if the "it needs to happen right here, right now" constraint is lifted?

alexchamberlain2• 5 years ago

I think that's fair to a certain extent - a queue is the simplest form of message passing or work distribution. There are many other data structures that have been developed that work well with multiple threads. I think what _is_ out of vogue is just protecting an arbitrary data structure with a mutex and calling it thread safe.

tptacek3• 5 years ago

This is a pretty great explanation that links to a rabbit hole of further intuition tuneups around &mut. I come to Rust's way of looking at the world via C/C++, and after following the links here it turned out I had a bunch of fucky intuitions about what Rust is trying to express. They really should have renamed `mut`!

Having said that: I'm proof that you can get a bunch of stuff done in Rust without having a perfect mental model of this stuff. :)

saghm2• 5 years ago

It's common to hear people refer to &T and &mut T not as "mutable" or "immutable" but "shared" versus "unique". What probably muddles this is that the `mut` keyword _also_ is used for declaring things mutable or immutable, just at the binding level rather than the type level. These are related due to the fact that you can't take a `&mut` reference to a binding that isn't declared as `mut`, which probably also muddles the water a bit.

tptacek3• 5 years ago

&mut is kind of gross syntactically anyways; another thing that would have clarified this would have been to rename & to &shared, which has symmetry.

nyanpasu642• 5 years ago

There's one other oddity, where `let x: &mut T` allows mutating the target, while `let x: Box<T>` or `T` doesn't.

  fn box_immut(x: Box<i32>) {
      // illegal
      *x += 1;
  }
  
  fn ref_immut(x: &mut i32) {
      // legal
      *x += 1;
  }

IIRC under the current current Rust-to-LLVM compilation rules, &mut T is noalias (because it's exclusive), and &T is noalias or readonly or something (because all other &T are immutable) unless T contains an UnsafeCell. It actually took many years to get &mut T to be noalias without exposing LLVM bugs resulting in miscompiled Rust (which went undiscovered because Rust is the first LLVM language to use noalias so heavily).

And noalias and pointer aliasing is a huge mess, with crates like owning_ref and Tokio's intrusive linked lists and Pin<Box<T>> being unsound under the proposed Stacked Borrows rules. And I just found a LLVM codegen soundness issue: https://github.com/rust-lang/rust/issues/63787.

Matumio• 5 years ago

This was my biggest misconception when learning Rust. I was thinking "&mut" can modify data, so "&" cannot. Wrong.

The "mut" keyword is not the opposite of "const" from C++. It's declaring that you have exclusive (unique) ownership at this moment.

smoldesu• 5 years ago

Having said that: I'm proof that you can get a bunch of stuff done in Rust without having a perfect mental model of this stuff. :)

That's one of the best features, IMO. If you tried approaching C or even C++ with that mentality, you're going to hit brick walls, hidden errors and antipatterns before you know it. If you try it in Rust, you're going to get a couple linting errors and and a non-functional build. Not a perfect catch-all for every error you can cause, but like you suggested, it's plenty to get stuff done with.

jjtheblunt• 5 years ago

fucky intuitions that's got to be a typo

tptacek3• 5 years ago

I'm not so much a typo kind of commenter.

natded• 5 years ago

One of the best post-beginner / intermediate learning experiences wrt. Rust is to try to model the Borrow Checker behavior with Rust. Just write small (>10 LoC) programs within safe subset of Rust:

1. Rejected unsafe programs (and the why)

2. Rejected safe programs (and the why)

.. and try to convince yourself that you understand the logic behind the two.

https://whileydave.com/2021/12/06/modelling-borrow-checking-...

https://github.com/rust-lang/unsafe-code-guidelines/blob/mas...

efokschaner3• 5 years ago

This is the first time I've heard of "thread compatible" objects/state. The author states: "Most types in C++ are thread-compatible, as this guarantee comes mostly comes for free", and that "Any concurrent call to a non-const method must be synchronized by the caller."

My understanding is that due to things like memory tearing, or the possibility of invariants between object members being in intermediate states, reads are always unsafe when there is a possibility of writes. Therefore, you can't selectively skip having some kind of synchronisation for the const calls (the reads), as something needs to be synchronizing all the reads against any potential writes. So to me this just reads as, "the caller has to manage all the thread safety for all the accesses", which I'd agree is what typically "comes for free" in c++, but I'd hardly call it a "guarantee".

I'm wondering if I'm not seeing some meaningful level of "thread compatibility" in between "thread-safe" and the "accessor beware" behaviour of all c++ that wasn't designed to be thread-safe. Or is there an example of some c++ that is not even "thread-compatible" once the caller has accepted the burden of synchronization?

ReactiveJelly2• 5 years ago

My understanding is that due to things like memory tearing, or the possibility of invariants between object members being in intermediate states, reads are always unsafe when there is a possibility of writes.

Correct. Rust calls this "aliasing XOR mutability". You can either have any number of readers reading the data as constant, or you can have one writer mutating the data. You can never have reads and writes at the same time. (unless you use, e.g., a mutex, which allows writes that _look like_ reads from the type system's PoV, which is safe)

nyanpasu642• 5 years ago

I interpreted the author saying "thread-compatible" to mean "accessor beware". Surprisingly in some cases, in C++ it's unsafe for multiple threads to even call const member functions on the same object, for example when the const method modifies a `mutable` field. One way this could happen is when a "const" method lazily computes a value and stores it in a field.

haberman7• 5 years ago

reads are always unsafe when there is a possibility of writes.

That is correct for any reads that could race against a write. Writes must be synchronized with all other operations. But reads may proceed in parallel with other reads without synchronization.

For example, if you have an object that is written in the thread where it is constructed, and only released to other threads once the mutation is complete, then you do not need to synchronize readers. Because the nature of the data flow ensures that none of the writes can race with the reads.

So to me this just reads as, "the caller has to manage all the thread safety for all the accesses"

Not all accesses. Just the ones where a writer may race with another write or a read.

Or is there an example of some c++ that is not even "thread-compatible" once the caller has accepted the burden of synchronization?

Thread-unsafe types do not even allow two reads to be unsynchronized.

efokschaner3• 5 years ago

Thanks for the response. I think my disagreement is in describing that as "without synchronization", somewhere the caller is somehow ensuring there are no writes during their safe parallel reads.

Orou• 5 years ago

I was just reading an article earlier today on the same subject. I thought the "pedagogy" section was interesting when considering if `mut` is the right term to be using for exclusive references.

https://docs.rs/dtolnay/0.0.9/dtolnay/macro._02__reference_t...

exdsq2• 5 years ago

I read all these posts on Rust and it makes me wonder if I'm being stupid spending time on C++? I want a language that I can focus on for the foreseeable part of my career. I'm looking for stability and am interested in domains like finance and robotics. My key wants are that I can deep dive a language for as long as possible, as I'm burning out from constantly changing languages after ~8 years. I need some sort of external validation before I jump on five different things and fail lol.

rapsey• 5 years ago

Rust job market is 1% of the C++ job market. Most programmers never heard of HN. Most jobs don't advertise here.

tomjen3• 5 years ago

Even if there is no new C++ project ever, there is tons and tons of legacy products that needs to be maintained (they are not all going to be rewritten, just as the old cobol projects aren't) and C++ is a deep language that really could benefit from extreme knowledge (a const rvalue reference? wtf).

Rust may fail, but it is probably worth learning some of the ideas - maybe you could use those ideas in your C++ projects?

exdsq2• 5 years ago

Oh I've spent years working with Haskell and work with Rust right now! And I absolutely love diving into the pros of awesome type systems. I've actually worked with Golang, F#, Scala, Typescript, Java, C#, C++, Python, and probably a few others (all professionally, as in I've shipped something to prod) over the last few years.

I used to love this but now I'm getting burned out with it all and want to learn a language in more depth.

otabdeveloper4• 5 years ago

The actual stuff is being written in C++ still.

Rust is for "programming is hard, let's go (language) shopping" programmers. Unsurprisingly, they don't tend to finish their projects.

adrianN• 5 years ago

You can still make a living as a COBOL programmer even though it has (hopefully!) been a very long time since any new project was started with COBOL. There is enough C++ around that you'll have no trouble finding a job as C++ programmer for next couple of decades at least.

matklad2• 5 years ago

If you pick any language which is non-esoteric today (C, C++, Python, JavaScript, Java), you should do fine for the next 20 years.

If you want to make a bet on a language which isn’t supper big yet, but might be in the future, than Rust is definitely the way to go. It’s already big enough that it’s clear that it isn’t going away, but it is still growing pretty fast. It might or might not end up as big as C++.

sakex• 5 years ago

It's missing channels which are used more on more for thread safety

sudarshnachakra• 5 years ago

Agree with the conclusion of the article that both C++ (mutable/const_cast) & Rust (interior mutability) to hand-wave certain properties they vouch for to enable performance/flexibility.

I guess Rust is better behaved in this regard to ensure that they do not violate invariants in case of types like Cell/RefCell/Mutex, but indeed the design of the language itself caused all these problems for Rust. Because the raison d'etre for Mutex is to share something between threads and calling a lock on it must mutate it, but the language does not allow mutable instances to be shared.

Essentially Mutex is a paradox in Rust, which cannot be implemented but for unsafe.

Thread Safety in C++ and Rust

Related Stories