Why did we choose Rust to develop TiKV?

https://pingcap.github.io/blog/2017/09/26/whyrust/

by cyber1 • 9 years ago

159 191 9 years ago

pingcap.github.io

jcelerier8• 9 years ago

After years of usage of GC, it is very hard to go back time for manually managing the memory.

... are you guys sure of your "experienced C++ developers" ? There's as much memory management in modern C++ than in GC'ed language: none. Create your objects with `make_unique` or `make_shared` according to what makes sense (or just enforce `make_shared` if you're really dubious of the coding abilities of your team but at this point you'll have problems whatever you do).

jhasse7• 9 years ago

You'll have to manually break cycles using weak_ptr though.

tentaTherapist3• 9 years ago

But you'd have to do that in Rust too, or at least use a library like petgraph, because Rust hates cycles even more than C++.

blub9• 9 years ago

Yeah, that read like some Java advertising from the 90s.

I wish more of these posts were honest and said "we picked X cause we think it's cool and we're gonna get paid to learn it". But they have to make up some convoluted explanation that sounds rational and acceptable instead.

c4pt0r• 9 years ago

Hahahah, you're right. We picked Rust cuz our CEO and I think it's cool... XD

deadjoe• 9 years ago

from the post , looks like they really do think picking rust is dope and already built a cool db system upon it. Maybe they already got bunch of bucks in the pocket. Huhhh..

latch2• 9 years ago

First sentence is why I dread reading comment. Why you gotta be like that?

jcelerier8• 9 years ago

sorry, could you explain a bit more ? does '... are you guys sure of your "experienced C++ developers" ?' sound harsh to you ?

fsloth2• 9 years ago

"...make_shared"

Sorry, I'm going to be a bit harsh now.

There is a host of distinctive differences between garbage collection and reference counting. Yes, both are memory handling strategies. That's where the similarities end.

"Just slap it in a shared pointer" is never a good advice without knowledge what 'it' is or in what kind of system it exists in.

jcelerier8• 9 years ago

There is a host of distinctive differences between garbage collection and reference counting.

No, reference counting is commonly seen as a specific implementation of garbage collection. https://en.wikipedia.org/wiki/Garbage_collection_(computer_s...

"Just slap it in a shared pointer" is never a good advice without knowledge what 'it' is or in what kind of system it exists in.

I agree, but it seems from their blogpost that they are not sure that their developers are able to handle the "mental overhead" of managing ownership, hence the simple solution of going for shared ownership everytime.

(btw, I re-read your post three times and could not find any hint of harshness !)

pjc504• 9 years ago

And do all the C++ libraries take only smart pointers as arguments and return only smart pointers as return values?

jcelerier8• 9 years ago

And do all the C++ libraries take only smart pointers as arguments and return only smart pointers as return values?

C++ libraries that would be as recent as Rust libraries would certainly take things by value or reference so there would be no problems. I honestly don't know libraries with raw pointers in their APIs that aren't from the 90's or before; and I don't think you want to use those in a current product anyways.

Koshkin2• 9 years ago

They don't, and it is (or should be) considered bad practice for functions, in general, to take/return smart pointers. Normally these should only be used as variables or class members; functions, on the other hand, should take/return raw pointers (except some special cases) or, better yet, references (except when there is a need to check for null value).

tentaTherapist3• 9 years ago

But then your program uses slow, cache-unfriendly and much-reviled reference counting. I'd even prefer Ocaml and its fully-featured GC if C++ is only fast in artificial benchmarks and not in idiomatic code, which apparently must use RC.

Note: I haven't used C++ at all on any project larger than a single file.

stepik777• 9 years ago

And how would smart pointers help you if you need to return pointer to a member from a function? Does C++ protects you from moved from unique_ptr? Or from iterator invalidation? Or maybe you can safely use non-atomic shared pointer if you don't need to send it across threads?

duneroadrunner• 9 years ago

While "just enforcing `make_shared`" wouldn't solve all the memory safety issues, it actually can be somewhat practical to "just avoid using any (or most) C++ elements that can access invalid (or uninitialized) memory", instead using the safe, compatible substitutes in the SaferCPlusPlus[1] library.

[1] shameless plug: https://github.com/duneroadrunner/SaferCPlusPlus

p0nce3• 9 years ago

I think you are being needlessly inflammatory. The list of 3 items against C++ makes it pretty clear they do have experienced C++ developers.

eikenberry• 9 years ago

TLDR; the author likes rust and wanted to use it. The article reads like some dev's rationalizing what they want to do to the management. These types of things are fine, but as a dev to a dev it is obvious that they just want to use this cool tech. Good for them.

dullgiulio3• 9 years ago

Even more, Go is excluded as (author's opinion) goleveldb is not as mature as RocksDB. Thus they should have used CGo, which is way suboptimal, slower etc.

The title should have been: "C++11 or Rust? We chose Rust". In a greenfield project like this one seems to be, it's I choice I would approve.

A personal note regarding future comments to this thread: I have had enough of negative advertising against Go in every language related thread. People who use Go are not stupid: they know the language limits and tradeoffs and are okay with them. Deal with it.

edem• 9 years ago

I haven't used Go or Rust but when I compare them I have a gut feeling that I'd be better off with Rust primarily because Go's build tools are not adequate. What do you think about this?

peoplewindow3• 9 years ago

Re your last part - is that fair? It's very common that programmers have to end up maintaining code written in languages they don't like, due to the original authors moving on, lack of better jobs in their area, whatever.

People advocate against Go partly because they'd rather not have to work on Go codebases in future.

jackmott• 9 years ago

I will probably use Go as soon as it has generics.

alexnewman5• 9 years ago

Nonsense. I love go and rust but I would never develop a database in a garbage collected language again. Btw I have done it a few times. Go is better than java but it still

Can’t call into c as fast Has micro gc pauses which effect performance Doesn’t give me explicit control of the hardware

Go is great for middleware but he authors go it right

lobster_johnson2• 9 years ago

That's not a fair summary at all. For example, they explain why C++ was off the table. They also use Go for TiDB (which is the high-level distributed query engine layered on top of TiKV), but for TiKV they needed fast and easy access to C, which Go, in its current state, can never provide.

baq• 9 years ago

it'd be enough for me to say 'rust is kinda like c++ in terms of performance and complexity but without the 0 pointer'

jhasse7• 9 years ago

Rust also has a "0 pointer": https://doc.rust-lang.org/std/option/ The equivalent of a null pointer exception in Rust is an unwrap panic.

(IIRC if you use Option<Box<...>> None will even be represented by a null pointer internally)

Ygg24• 9 years ago

Umm unwrap panic is definitely not the same.

For one it's heavily discouraged. Any example using it is not best practice.

tentaTherapist3• 9 years ago

Yeah, but unwrap is very, very explicitly marked, and its use is discouraged, and unlike null it's nothing close to an undesirable edge case on every single pointer.

Edit: In my perfect language there wouldn't be an unwrap, only matching on an Option, but I guess Rust is too pragmatic for that.

gypsyharlot• 9 years ago

I'd say: "It's like C++, only you get the runtime errors at compile-time".

Shorel• 9 years ago

And lack of experience with D.

Just kidding, great job!

pornel• 9 years ago

I can't wait until Rust is no longer be perceived as hipster trendy choice.

It's a very solid language in the no-GC niche and shouldn't need a blog post from every project that uses it.

Does it have to be 30 years old before it's not "new" and weird?

mac01021• 9 years ago

“new and weird" is always relative to the industry/domain on application. In each domain , whether it be avionics or web development, managers are not going to want to use it until it has a proven record of success in that domain.

zerr3• 9 years ago

On the other hand I can't wait when Swift becomes general, non-Apple language, available on most platforms (including the most popular one) "with batteries" - that will be the end of Go and Rust I think :)

rf15• 9 years ago

while I think that that would be a good thing (I like what happened with C#), I see the languages specified and controlled by Microsoft, Google and Apple as second-class languages, since they are often lacking in community input and are usually designed with certain platform-specific goals in mind instead of being cross-platform. (or company-strategic goals when it comes to Google)

cyber14• 9 years ago

Swift can't replace Rust in system programming domain because he uses ARC for memory management but Rust gives you choices.

readittwice• 9 years ago

that will be the end of Go and Rust

Will it? I don't think so, although I think Swift is great. IMHO Rust targets another market compared to Swift. Rust feels more low-level, e.g. Swift uses Reference Counting for everything. Swift also can't give you some of the nice guarantees you get in Rust.

Go also has some nice stuff compared to Swift: simplicity, goroutines or tracing GC. Go is also already widely used.

Swift is certainly great for Mac/iOS-Development, but I am somewhat skeptical for other platforms. I am afraid that non-apple-platforms will always feel a bit like second-class. It's just not Apple's main priority and most Swift-devs will always be paid by Apple. But who knows, I may be wrong.

hashmal2• 9 years ago

All three have different goals and solve different problems. While you can compare things like "they both have MATCH statements" or "unions are safe", deeper they have nothing to do with each other.

You would be crazy to use swift (or objc) in real-time stuff. You would be crazy to write a UI-heavy app in rust. (note: I'm talking about high level DOM-like manipulation, not about rendering engines)

geodel• 9 years ago

Yes next version of Swift will be so awesome it will kill all programing languages.

greydius• 9 years ago

Haskell is almost 30 years old and still considered 'new' and 'weird'.

TremendousJudge• 9 years ago

I don't know about new but it is pretty weird

luckydude3• 9 years ago

I'm (very) old school, C is still my language of choice (though it could use some help). I like the syntax, it's pretty simple.

I tried playing with Rust and found the syntax to be off putting. I really wonder why each new language feels it is important to come up with a different syntax to say the same stuff. Go did a lot better than Rust in this respect, at least in my opinion.

It may be that appealing to C programmers isn't a thing any more, but if it is, then Rust could have done better. And, yes, I get that the syntax isn't the selling point of Rust, trust me, I get it. I just don't get why make people wade through some weird syntax when you don't need to.

ufo2• 9 years ago

Your comment about syntax sounds like you just want everything to look like C.

wyldfire2• 9 years ago

When I look at Rust it seems to me to be very much C-inspired syntax. The code-blocks-return-a-value style makes it much easier to understand some conditional paths.

"Why would Rust change the syntax to say the same stuff?" Well, Rust is able to meet all of C's requirements while addressing one of C's big old warts: lexical analysis of C code is Very Hard. (e.g. if this particular build added "-DFOO" or "-Ibar/" to the command line then maybe there's a syntactical error or not).

This means that writing robust tools to analyze or modify C code is Very Hard. I don't know if this was an inspiration for Rust's syntax but regardless I'm thankful that they didn't try to reuse C syntax.

Most other successors to C (save perhaps C++) have taken on features that make them unable to write code that we have been able to write in C (bootloaders, OS kernels, interrupt routines, device drivers, etc). Teams had a really legitimate and mostly sane reason most of the time for saying "we couldn't possibly consider Java/Go/Python/etc because we won't be able to meet our product's latency requirements."

Ar-Curunir• 9 years ago

What is the biggest difference between C syntax and Rust syntax? I can only really think of how types are stated: in C they precede the variable, whereas in Rust you put a colon and then the type (but often you don't need to do this cuz of type inference).

This change is much better IMO because there's a clear delineation between types and variables.

The same idea applies to function signatures, which are much clearer to me in Rust than in C/C++.

AYBABTME4• 9 years ago

Syntax is a method to express ideas. Ideas at the core of a language should be easy to express with that language's syntax.

Some ideas are not possible in C as they are possible in Go/Rust/other languages. With the years, some ideas become more mainstream and then get integrated into the new languages that arise. These new languages need to express ideas that were not expressible in C, and thus may be better served by adopting different syntaxes.

In short: the ideas at your disposition and the ergonomic with which you can express them is a function of the syntax used to represent them. Different languages focus on different ideas, and so it follows that different syntax might be warranted.

wyldfire2• 9 years ago

Here at HN Rust seems to come up frequently and it's indeed why I decided to take a look.

For my many colleagues who do not read HN, Rust is something that they might have heard of but haven't given any real attention to. For that matter, most haven't looked at golang either and they probably couldn't tell you what the difference between the two are.

IMO it will take a while before it grabs the attention of the mainstream software community.

klakier• 9 years ago

When I see posts like "Why did we choose something over something other", I'm like "Nobody cares"

sgift• 9 years ago

And with that attitude we get the same problems over and over again.

meche123• 9 years ago

This post is just an advertise for their product.

oelmekki• 9 years ago

Looks like you care enough for opening the comment page and dropping yours :)

amelius2• 9 years ago

If you had lots of circularly referencing data structures, would it make more sense to choose a garbage-collected language like Go?

int_19h2• 9 years ago

Most circular data structures still have some node that is semantically the owner of the whole thing. Having true circular ownership is much rarer.

amelius2• 9 years ago

Closures naturally generate cycles in the data dependency graph. A way out would be to copy the environment of a closure, but that would mean a performance penalty.

cyber14• 9 years ago

I think TiKV is a good example where team chose Rust over Modern C++. Rust gives the same performance and is close to metal like C when it is necessary. All possible memory management mistakes it catches at compile time if it's not "unsafe" and this is a really great!

With Rust I can hack without fear! I shouldn't remember tons of C++ rules which are described in http://isocpp.github.io/CppCoreGuidelines/CppCoreGuidelines, "C++ programming language" book, also here https://herbsutter.com/gotw/, etc and I can focus on algorithms and implementation.

C++ combines a lot of different paradigms m.b. more correct I would like to say "C++ paradigms hell"! Which of C++ subsets is the right way, no one understands. Even Bjarne Stroustrup said, "Within C++, there is a much smaller and cleaner language struggling to get out." - and where is this "smaller and cleaner language"?. What is the idiomatic style in C++? Is it Google guidelines, CoreCpp guidelines or other enormous guides?

I look inside a lot of C++ projects and each of them has different styles, use different paradigms, sometimes look like different languages!

Rust, Go, C, Java code bases look the same, they have their own idiomatic style, their own way.

I think Rust is the next step in the evolution of system programming language.

blub9• 9 years ago

C and Java code bases certainly don't look the same. In fact, claiming that about C is simply insulting to the intellect of anyone reading your overenthusiastic message.

I also sincerely doubt that Rust code bases will look the same in ten years. It supports functional and OO paradigms and it's attracting very different classes of programmers. Recently someone wrote a post about difficulties with some OO concepts in Rust, and a top reply said that they never encountered such issues because they program in a functional way.

Go is an exception here, but as soon as they extend the language in a significant way (e.g. templates) differences will start to appear.

C++ doesn't have an idiomatic style because it's used in very different ways by different people. It's impossible to have a fixed style and address the mass market.

Ygg24• 9 years ago

   > It supports functional and OO paradigms.

[Citation Needed]

I don't think anything in Rust supports OO paradigm. There are instances where people ask for OO paradigm, but so far, there wasn't any movements towards it.

There is no inheritance and no polymorphism based on it. There is generics polymorphism, but nothing like that.

Hell, even things like

    object.method()

is a sugar for:

    method(&object)

And I mean that in literal sense, you can use the second form always. https://play.rust-lang.org/?gist=6a8111b51c3862caa52c498c79e...

usrusr• 9 years ago

I think you misread a terribly misreadable statement: what he meant wans't that C and Java look the same, he meant that any two C codebases will share a lot of similarity, any two Java codebases will share a lot of similarity and so on. C++ on the other hand can be anything from "C with objects" to deep template metaprogramming to "has there even been a C++ before 2011?" styles, which are as dissimilar from each other as you can get without crossing language borders.

noncoml• 9 years ago

The one reason I would give is Algebraic Data Types.

sriram_malhar3• 9 years ago

Considering that their team likes Go, it seems strange to me that they would consider Rust over Go for the storage layer. A storage layer should be IO-bound, and should hardly trouble the CPU; the choice of language really should not be a determining factor. The big wins in that space are architectural, not language specific.

sanxiyn3• 9 years ago

Rust would use less memory than Go. (Dropbox also likes Go and used Rust over Go for the storage layer, and when asked, memory usage was their primary reason.)

jerf3• 9 years ago

"A storage layer should be IO-bound, and should hardly trouble the CPU; the choice of language really should not be a determining factor."

This used to be true, but it's out-of-date now. You can now get a network pipe in to a system that a rather beefy multi-core CPU using a user-space TCP stack can barely keep up with, let alone do any real work, and if you can scrape up the PCI express lanes, putting a few of the latest SSDs into a system can start getting you theoretical maximum bandwidth numbers that just a few years ago looked more like what you'd expect for a RAM bandwidth number.

I'm of the opinion that it was already not as true as commonly supposed 5 years ago (in my experience using slow languages on putatively IO-bound tasks was still noticeably slower than using fast languages), but the latest in network pipes and SSDs have really ended it. It's true that on most desktop systems you've still got more CPU than you know what to do with, but as you step into the serious database space that's not true anymore. For a serious database I wouldn't be perturbed if someone looked at Go's performance and just plain discarded it on the spot, even before considering GC issues. It's very fast for a scripting language; it's fairly slow for a compiled language. "The compiler spends hardly any time on optimization" is not what you want to read about your database implementation language.

(I've got one of the nvme SSDs in my laptop, and it is interesting to see just how many CPU bottlenecks there still are in systems nowadays. In some sense, I really shouldn't ever see a "loading" screen because you "ought" to be able to read things off of my SSD fast enough to completely fill my RAM in 5-10 seconds; "merely" loading Firefox ought to be somewhere in the 50ms range. In practice I still see loading screens and load waits, because the CPUs are still doing things. Lots of things that used to be dominated by and hidden in the load time, but aren't anymore.)

Santosh83• 9 years ago

And cache. Compiled languages tend to be more cache friendly, which contributes a lot to speed of execution.

And too many apps still do cause CPU spikes, often for quite a bit of time, and no doubt much potential for optimisation lies there. The popular perception that CPUs are fast enough to deal with nearly every workload is inaccurate. Even an innocuous bit of Javascript on a webpage, probably doing some trivial stuff, causes 100% usage for several seconds.

sriram_malhar3• 9 years ago

You can now get a network pipe in to a system that a rather beefy multi-core CPU using a user-space TCP stack can barely keep up with.

I'd love to learn more about this. Can you please point me to some links that elaborate on this point with benchmarks? Thanks much.

AYBABTME4• 9 years ago

Developing a storage layer as elaborate as RocksDB from scratch is quite an endeavor, and wanting to just use RocksDB instead of making your own is a smart decision. From there, Go is sort of easy to throw out of the picture: using cgo kills performance and safety. I say this as a person who uses Go as my workhorse, have used it for many years and has a favorable opinion of it.

sriram_malhar3• 9 years ago

Ah, my mistake. I misunderstood. I thought that they were replacing the C++-based RocksDB storage with one written in Rust. Yes, it makes perfect sense to use something if it is already proven itself.

baldfat• 9 years ago

its innovation in the type system and syntax gives it a unique edge in developing Domain-Specific Libraries (DSL).

I think Racket still has the edge for producing DSL?

StreamBright• 9 years ago

It is kind of funny how software engineers can engage in lengthy discussion about tooling. Imagine the same for architects. Instead of looking at the building they would talk about the type of hammer they used while building it.

arjie• 9 years ago

With software the material you construct your creations influences the means. Architects most certainly do argue about whether they should use cross-laminated timber, reinforced concrete, glulam, or steel. They talk about these things and write long pieces on them. The materials influence the design of the building.

They don't talk about it on blogs on the Internet because that's not where the audience is. But they do talk about this.

new299• 9 years ago

They don't talk about it on blogs on the Internet because that's not where the audience is. But they do talk about this.

It's a real shame. I love reading about other disciplines and knowing about the practical trade offs.

mrkgnao• 9 years ago

Is this a hammer question, or one of e.g., concrete vs. wood?

steveklabnik7• 9 years ago

Imagine the same for architects

Do you know the history of the term "design pattern" in software? It comes from this book: https://en.wikipedia.org/wiki/A_Pattern_Language

AYBABTME4• 9 years ago

I think "hammer" is pretty diminutive as a parallel for a choice of language. I feel a better parallel would be "architects talking about steel alloys versus composite materials" which is not crazy.

bpicolo2• 9 years ago

Anybody have experience with TiDB? How does it stack up against CockroachDB? Seems hard to find comparison. Probably hear less about it mostly because it's developed in China? Looks like it's an impressive piece of tech, though.

sanxiyn3• 9 years ago

http://weekly.pingcap.com/2016/10/17/how-we-build-tidb/#atom... explains their main difference from CockroachDB.

bpicolo2• 9 years ago

Mostly curious how it works out in practice. Performance, etc.

lobster_johnson2• 9 years ago

A big difference is that TiDB is not ready for production yet.

Having followed to the project for a while, another distinction is that TiDB is operationally more complex. You need to build and deploy TiDB (high-level query engine), TiKV (key/value store) and PD ("placement driver", which coordinates sharding and data migration) separately. TiDB is stateless and can be scaled freely, but TiKV and PD are both stateful and implement their own distributed consensus systems. PD actually embeds Etcd, whereas TiKV has its own Raft implementation in Rust. Compare this to Cockroach, which has a single monolithic daemon that you deploy everywhere, which contains both the distributed query engine, the key/value store, consensus/cluster coordinator, etc. (There may be benefits or drawbacks to the difference in design; I don't know the internals of either project well enough to debate that.)

For an internal project I'm working on, running TiKV standalone actually looks very interesting, but it's not very well documented yet.

Why did we choose Rust to develop TiKV?

Related Stories