The Road to Rust 1.0

http://blog.rust-lang.org/2014/09/15/Rust-1.0.html

by steveklabnik • 12 years ago

570 238 12 years ago

Adopting the channels system is interesting. Are there any other languages that have a scheduled release pattern like this?

steveklabnik19• 12 years ago

I'm not aware of any, but if there are, I'd like to talk to them about their experiences doing so. :)

howeman• 12 years ago

Go sort of does this, though on longer timescales. Developers are at "tip". Actual releases are every 6 months, and before the release there is always at least one release candidate, and sometimes an actual beta.

Ixiaus• 12 years ago

The key to all these changes has been a focus on the core concepts of ownership and borrowing. Initially, we introduced ownership as a means of transferring data safely and efficiently between tasks, but over time we have realized that the same mechanism allows us to move all sorts of things out of the language and into libraries. The resulting design is not only simpler to learn, but it is also much “closer to the metal” than we ever thought possible before. All Rust language constructs have a very direct mapping to machine operations, and Rust has no required runtime or external dependencies.

Almost sounds like they borrowed this thinking from Exokernel design... I think Rust is shaping up to be a very exciting language.

AlyssaRowan2• 12 years ago

It'd be wonderful if they kept the ability to define that a certain destructor does zero memory.

Sometimes, you need that.

gpm• 12 years ago

Zeroing out memory isn't sufficient, as discussed here: http://www.daemonology.net/blog/2014-09-06-zeroing-buffers-i...

That said, this was discussed on reddit[0], it sounds like there is a way to guarantee that you did zero out memory (but not necessarily copies of that memory, as discussed in the link above), and because Rust is intended to be memory safe, it's not as much of an issue if you don't/can't.

http://www.reddit.com/r/rust/comments/2fnb82/zeroing_buffers...

AlyssaRowan2• 12 years ago

Not in C. Rust is, as you say, intended to be memory safe.

That means it has the hope of getting it right.

pacala• 12 years ago

The ownership idioms are very similar to idiomatic C++11 and std::unique_ptr. Which is to say that Rush has got an industrial strength safe memory management system.

But Rust stands out because the rest of the language is such a joy to use, compared to pretty much any other 'systems' language out there.

Congratulations to the team!

wycats9• 12 years ago

One of the cool things about the Rust ownership system is the lease/borrow system. Moving is cool, but much of the time you want to synchronously call another piece of code and give it a temporary (stack-frame-long) lease for that pointer.

Rust starts with ownership, but makes it easy to ergonomically and safely lend out that ownership (including one-at-a-time mutable leases) of both stack and heap allocated pointers.

I've been programming with Rust since last December, and I have had essentially zero segfaults coming from Rust code during that time frame, roughly equivalent to what I would have expected writing code in a language whose safety guarantees come with a runtime cost (GC or ARC).

JoshTriplett3• 12 years ago

I used to describe my preferred family of languages as:

- C when I absolutely had to (kernel/modules/plumbing).

- Python for scripting and broad accessibility.

- Haskell when I had the choice and I knew everybody who would work on the project.

I was skeptical of Rust when it first came out, due in large part to the many different kinds of pointers it originally had, many of which involved significant manual memory management. But now, with a strong static type system, garbage collection, pattern matching, associated types, and many other features, Rust is looking like a serious contender to replace all three of those languages for me.

Still waiting to see if it develops a strong following, community, and batteries-included library ecosystem, but I need to start doing more experiments with Rust.

Disappointing to see yet another language-specific package management system (Cargo), though.

steveklabnik19• 12 years ago

Rust does not have any garbage collection, to be clear. All your other features are correct though :)

(We have previously said "opt-in GC" but that was a lie. See https://news.ycombinator.com/item?id=8312327 for more.)

JoshTriplett3• 12 years ago

Yeah, I should have said that more clearly. Automatic memory management (never having to call free), not garbage collection. Which is arguably more awesome: whenever the compiler can figure out at compile time when you'll stop using memory, it can statically decide to reclaim it there.

haberman3• 12 years ago

I am a huge fan of no required GC, and of precise refcounting. But does Rust have anything on the roadmap to handle truly cyclic structures that don't lend themselves to a weak pointer approach?

pestaa2• 12 years ago

Curious to hear more about language-specific (though OS-agnostic!) package management systems. IMO composer is the best thing ever happened to PHP, Ruby gems are huge, Python eggs also make a very useful ecosystem.

OpenSUSE's Open Build System would be great to ship independent packages, but those are again heavily tied to Unices, hence leaving other platforms behind.

JoshTriplett3• 12 years ago

Curious to hear more about language-specific (though OS-agnostic!) package management systems.

As far as I can tell, one of the main justifications for most language package management systems is "we also run on Windows/OSX, which has no package management, so we'll invent our own". As a result, users of systems that do have sane package management get stuck with multiple package management systems, one for the distro and one for every language. Even then, I find it disappointing that nobody has built a cross-platform package management system for arbitrary languages to unify those efforts.

yla92• 12 years ago

Lately, IMHO, Gradle in Android Development(applicable for Java development as well) is a huge improvement over managing dependencies with pom.xml(ant) and linking jar files manually. Besides, you can totally customize build.gralde too.

pcwalton6• 12 years ago

Disappointing to see yet another language-specific package management system (Cargo), though.

It's been a huge step up from makefiles in developing Servo. I can barely ever bring myself to go back to using make and git submodules now.

The workflow of breaking up your project into small, self-contained packages that people hack on independently and are all built with a package manager that natively understands the language and can build without a single line of shell is a huge improvement over vendoring dependencies and dealing with a maze of configure scripts or arcane make/(insert your preferred replacement here) files. It improved compilation time a lot too!

bjz_7• 12 years ago

It's been a huge step up from makefiles in developing Servo.

As a maintainer of a number of Rust libraries I highly agree! It has definitely won over a few skeptical collaborators of mine (who were fans of Make).

doe883• 12 years ago

Disappointing to see yet another language-specific package management system (Cargo), though.

Coming from Python I find Cargo very very smart and very well thought so far, it is not feature heavy, but everything has a very clear and useful purpose. For instance today I found that if I created a file .cargo/config I could override my dependancies to make Cargo search projects on my fs instead of grabbing them on Github, while doing developments it's a big thing I think.

Ygg2• 12 years ago

Disappointing to see yet another language-specific package management system (Cargo), though.

I don't think it is. You need support for Rust modules on various platforms Linux/Mac/Windows(possibly Android). No single tool works on all those platforms. Cargo does and it has minimal dependencies.

Not having to juggle three different configuration (CMake, Makefile, etc.) on different platform is actually pretty great.

pjmlp4• 12 years ago

Disappointing to see yet another language-specific package management system (Cargo), though

So what is the solution to have portable packages for:

- RPM systems

- Debian systems

- tarball systems

- Pkg systems

- MSI systems

- Mainframe OS

- Embedded OS

- Aix/HP-UX/Solaris package systems

- ...

adrusi• 12 years ago

The goal of the [nix](http://nixos.org/) project is to solve this, and every time anyone brings up a package manager on HN, someone has to mention nix. The reality is that nix is really nice, but isn't any better than making a new package manager until it has wide adoption, so no one is using it.

dbaupp19• 12 years ago

As others have said, having a package management system that deeply understands the language and tooling is awesome. Examples:

- Rust has a distributed-by-default documentation generator (rustdoc), cargo knows this and provides `cargo doc` to render a library's docs with it.

- rustdoc can run code examples in the documentation as tests, to check that everything is up-to-date, `cargo test` does this (along with running the in-source unit tests and any external tests).

- the Rust compiler allows for plugins, which are dynamic libraries loaded into the compiler and can be used for things like custom macros (aka procedural macros aka syntax extensions) and custom compiler warnings. cargo understands these, allows them to be 'imported' via the normal dependency mechanism, and specifying `plugin = true` in a package makes cargo do the right thing, e.g. building as a dynamic library (static libraries are the default) and compiling for the correct target when cross-compiling.

I'm sure all of this is possible with other systems, but it seems unlikely to be so nice to use.

seabee• 12 years ago

It's very similar to Racket, and yes, it is nice to use!

Other systems can get you much of the way there (node, Python are the only ones I'm really familiar with) but I suspect you need a little language help to achieve the same kind of convenience.

sigzero• 12 years ago

Disappointing to see yet another language-specific package management system (Cargo), though.

No, I don't want my language to be bound to someone else's package manager at all.

dscrd• 12 years ago

Disappointing to see yet another language-specific package management system (Cargo), though.

As a packager in a Linux distro, I'm disappointed every time somebody tries to cram in PL-specific packages inside distro packages.

kazagistar5• 12 years ago

Interesting, how do you get around it, when those packages are depended upon by user facing software?

InclinedPlane2• 12 years ago

Disappointing to see yet another language-specific package management system (Cargo), though.

That's a much bigger problem to tackle, and one that nobody has really thought much on or done much work with. Package management is a tough problem in the abstract, and it's a big challenge to create a meta-package management infrastructure, more so when there doesn't seem to be any money in it (at least not easily).

rubiquity3• 12 years ago

Green threading: We are removing support from green threading from the standard library and moving it out into an external package.

I only ever looked at Rust from a 500 foot view while toying with it at a Hackathon, but I had no clue it had so many different types of threading models. This seems like a step in the right direction, indeed. If Task is going to your unit of concurrent execution, as much transparency around that as possible is a good thing.

masklinn2• 12 years ago

It had just two: libgreen backed tasks with "green threads", libnative backed them with OS threads.

rubiquity3• 12 years ago

Does that mean every task will be an OS thread? Or are tasks handled by a runtime specific scheduler and will use a pool of threads for IO?

hornetblack• 12 years ago

Green threading used to be default. And used libuv for io. Then native threading was avaliable. Then it became default. Now green threading is being moved out of builtin libs.

Starting with green threads is really easy too:

    extern crate green;
    extern crate rustuv;

    #[start]
    fn start(argc: int, argv: *const *const u8) -> int {
        green::start(argc, argv, rustuv::event_loop, main)
    }

    fn main() {
        // this code is running in a pool of schedulers all     powered by libuv
    }

Edit: You can also start up a new libgreen scheduler pool at any time and add some native threads to the pool. So you can have some threads running with libgreen and some with libnative. (So you could theoretically embed a go program inside a rust program)

MichaelGG6• 12 years ago

Rust looks fantastic, and has a lot of things I wish I could do while in a higher level language like F#.

I just wish Rust was a bit less verbose. Requiring, for instance, type annotations on function arguments because it's sometimes helpful is such a weird decision. Let the programmer decide when an annotation is needed. This gets annoying when you get into functions with complex arguments. Especially for local functions where the signature could be messy, but the limited scope means annotations just clutter things. I'm not sure why Rust forces us to compromise here.

steveklabnik19• 12 years ago

Even in languages with whole-program inference, it's generally regarded as best practice to write out the types of your functions, hence Rust's choice here. You're right that there's a tradeoff. In general, Rust follows 'explicit over implicit.'

This gets annoying when you get into functions with complex arguments.

Have you seen the where clauses yet? This should significantly help complex function declarations.

MichaelGG6• 12 years ago

I just don't understand why there has to be a tradeoff. I just don't get why the compiler should decide on such a large thing, instead of letting the programmer do it. One can always be more explicit if one feels they're getting value from it. If someone wants to write a bunch of terse code, why stop them? Does the compiler gain a large benefit from not having to include this feature? Who loses by allowing users to do what they want?

Comparing C# and F#, the extra annotations change the frequency in which I'll introduce an inner (local) function. For instance, here's a little helper in a piece of code I'm writing at the moment. It's part of a larger function and isn't exposed beyond 5 lines of code.

  let runWithLog f name =
    try run f name with ex -> logExn ex

Used in: runWithLog "sessions" collectSessionStats runWithLog "visitors" collectVisitorStats

Having to add "(f: string -> RunParams -> Whatever -> unit) (name: string)" almost doubles runWithLog helper yet provides no benefit. And this an extremely simple case! Once the arguments are generic, higher-order functions themselves, it gets quite noisy.

Sure, if it's a top-level export, then maybe annotating is a good idea. But if it's limited in scope then what's the harm?

Not that it'll change when I use Rust - there's nothing competing in this category. It'd just be nice if the language let the user decide on the style.

pestaa2• 12 years ago

Fully typed function signatures form a kind of contract you can program against.

Haskell and other extremely strongly typed languages can infer the types of function parameters, yet the community still agrees it is good practice to annotate your work.

MichaelGG6• 12 years ago

Would Haskell be better off if the compiler enforced this community agreement, instead of letting users decide?

Also, the type annotations can be added on later. While you work and play with ideas, leave everything unannotated. After it's cemented and perhaps refactored a bit, add the "contract". In Rust, even while working things out, the user has to figure out and jot down the types.

pcwalton6• 12 years ago

Requiring, for instance, type annotations on function arguments because it's sometimes helpful is such a weird decision.

In separate compilation you have to annotate functions anyway, and in most large ML projects I've worked on people tend to annotate just because the error messages get much better. This is a common design in functional languages these days.

doe883• 12 years ago

What I think is an important feature of this language is the ease with which it can interact with other languages. Especially the possibility for Rust code to be called from foreign languages such as C very easily.

I'm looking forward for even better support of iOS with the support of arm64, I think it is really important to offer an alternative.

BTW is there an RFC on dynamically sized types? I can't find any, I'm looking to learn of it works.

steveklabnik19• 12 years ago

Especially the possibility for Rust code to be called from foreign languages such as C very easily.

The second production deployment of Rust is a Ruby gem, written in C, that calls out to Rust. It's used in skylight.io, if you're curious.

BTW is there an RFC on dynamically sized types?

IIRC, DST was before the RFC process even existed, it's just taken forever to implement. The Duke Nukem Forever of Rust. :) http://smallcultfollowing.com/babysteps/blog/2014/01/05/dst-... is what you want to read, IIRC.

doe883• 12 years ago

IIRC, DST was before the RFC process even existed, it's just taken forever to implement. The Duke Nukem Forever of Rust. :) http://smallcultfollowing.com/babysteps/blog/2014/01/05/dst-.... is what you want to read, IIRC.

Thanks for the info and the link!

wycats9• 12 years ago

The second production deployment of Rust is a Ruby gem, written in C, that calls out to Rust. It's used in skylight.io, if you're curious.

Yep! I'm one of the authors of that project. The fact that Rust provides automatic memory cleanup and the attendant safety without runtime overhead (even ARC has non-trivial runtime overhead) was a huge win for us, as was the transparent FFI.

We were looking for a way to write fast code that was embeddable in a Ruby C extension with minimal runtime overhead and without a GC (two GCs in a single process is madness). We also wanted some guarantees that we wouldn't accidentally SEGV the Rails apps we were embedded in. Even last December, Rust was a clear winner for us.

We've been shipping Rust code to thousands of customers for many months, and given the language instability, it's worked really well for us.

xngzng• 12 years ago

Agreed. Two areas I can think of: 1. Rust library that is callable from Python. Many are using C/C++ for optimised Python program. 2. Able to create iOS framework library in Rust to be callable from Swift/Objective-C; similarly Rust library callable from Android NDK.

3289• 12 years ago

It seems that 1.0 is going to be a solid release. But the post-1.0 Rust is going to be even more exciting once they have added inheritance and subtyping which enable true polymorphic reuse!

Rusky• 12 years ago

Why are trait objects not "true polymorphic reuse" already? Inheritance is pretty much just a performance and expression optimization to be avoided in most situations (and I believe is going to be in 1.0)

strcat2• 12 years ago

Object inheritance is only useful in rare edge cases so your statement doesn't make much sense. Traits have default methods, inheritance and can be used as bounds on generics (no dynamic dispatch, type not lost) or as objects (dynamic dispatch / type erasure).

What makes you think object inheritance is such a sure thing anyway? I don't expect either object inheritance or optional garbage collection to materialize, ever. In fact, I'd be pretty sad if the language was degraded with that extra complexity - I think it would be a worse situation than the mess that is C++. There would no longer be a shared idiomatic Rust.

wtetzner2• 12 years ago

I think optional garbage collection might materialize, but I would imagine it would end up being a third-party library.

kvark• 12 years ago

I'm excited for the release too. Know many people who hesitate to touch Rust, even if interested, due to the fact language is still in active development.

On minor concern though, I don't see how "where clauses" are simplifying the language. Looks like something that could be added after the release.

steveklabnik19• 12 years ago

On minor concern though, I don't see how "where clauses" are simplifying the language. Looks like something that could be added after the release.

Where clasues simplify Rust code, they don't simplify the language itself. They're also important for associated items. For more: https://github.com/aturon/rfcs/blob/associated-items/active/...

eCa2• 12 years ago

I haven't looked at Rust, but it seems from the outside that releasing a stable version of a language every six weeks is very aggresive?

steveklabnik19• 12 years ago

Continuous deployment is common in the web world. It's true that it's aggressive, but we think it's going to have significant benefits.

pbsd5• 12 years ago

People tend to expect more stability from a systems language than a web one. Can I trust that my Rust 1.0 code will work, unchanged, 20 years from now? If not, the language is likely to remain in the enthusiast realm.

eCa2• 12 years ago

Yes, I'm not saying it's a bad thing. It does imho put a lot more pressure on the language developers than a cd'd web app, with regards to backwards compatibility and such. (With ~10 releases a year there's bound to be accidental breakage that passes the beta period.)

Ambitious might have been a better word than aggresive..

wycats9• 12 years ago

I'm sure that seemed true about web browsers too. The trick is feature flagging all new API surface and only including flagged code when it's deemed stable.

hadoukenio2• 12 years ago

Still sitting on the fence as to which language I should pick up on next - the only contenders are C++11 and Rust.

How does Rust compare with C++11 as a language? C++11 seems to (in some ways) have caught up with what Rust has to offer (compared to older C++ versions) e.g. smart pointers, concurrency and regexes part of the standard library

pcwalton6• 12 years ago

Rust is memory-safe; C++11 is not.

Q6T46nT668w6i3m• 12 years ago

C++ is useful when learning Rust since a lot of documentation assumes the reader is familiar with C++ and there’s plenty of C++ documentation that has no pre-requisites.

jmgrosen2• 12 years ago

Hmm. I learned Rust without any C++ knowledge (though I did have a fair amount of experience with C), and it was pretty easy. Recently I've had to start writing some C++ (to my dismay), and it has not been so easy... the language is really complex.

ranran876• 12 years ago

I'd definitely pick C++11 unless you need to use Rust.

Rust is inherently memory safe - however in practical terms this isn't important for most applications. If you are writing security critical applications Rust will provide you with some very important guarantees (ie. there are certain mistakes which are inherently not possible in the language). C++ doesn't really guarantee anything and if you're an idiot you can shoot yourself in the face. However in practical terms memory management in C++11 is very straightforwards and C++11 compliant code (ie. using the STL and not writing it like C) is very safe and clean. You're not mucking with raw pointers anymore

The main issue I see is that Rust is still in early development. It may or may not get "big" in the coming years. And library support is ... lacking

In contrast C++ has the STL and boost and every library under the sun. I haven't working with a lot of other languages extensively, but I've never seen anything as clean, robust and thorough as the STL and boost. C++ will remain relevant for a long long time. If Rust takes off in a big way, you'll be well positioned to jump ship.

hadoukenio2• 12 years ago

Wow, that was a great comment and exactly the type of info I was after.

I think (coming from a dynamic language world) the memory safeness is what pulls me towards Rust. But from what you say and what I've read elsewhere, that was old-style C++ and not C++1[17].

Thanks!

jroesch5• 12 years ago

I'm not sure in what world writing C++11 is "very straight forward ... safe and clean". You can easily write unsafe code without thinking about it. For example one can write a function that lends out a reference or pointer to memory that may or may not exist after the call exits, and this is impossible in Rust.

Jweb_Guru• 12 years ago

I definitely understand where you are coming from, and I agree that C++11's ecosystem maturity is a great reason to choose it at the moment.

However, I cannot agree with you that Rust's safety guarantees are not useful for most C++ programs, or that you have to be an "idiot" to do memory-unsafe things in C++11. Someone at Yandex recently did a presentation about Rust [1] in which they pointed to a bit of (completely idiomatic!) C++11 code that caused undefined behavior. The audience, full of seasoned C++ and Java developers, was asked to identify the problem. Not one of them could point to what was causing it (the compiler certainly didn't). The next slide demonstrated how Rust's compiler statically prevented this issue. The issue could have taken weeks to surface and days to track down, and the C++ compiler simply didn't have enough information to determine that it was a problem. This is something that happens over and over to anyone using C++, in any application, not just security-critical ones.

I'm not saying C++11 doesn't improve the situation, because it does--it would be disingenuous to say otherwise. But it's equally disingenuous to imply that C++11 makes memory management straightforward or safe. It does not.

[1] http://habrahabr.ru/company/yandex/blog/235789/ (note: the presentation and site are in Russian).

dbaupp19• 12 years ago

There's more to memory-unsafety than raw pointers; all of these are problems even in the most modern C++ versions:

  - iterator invalidation
  - dangling references
  - buffer overruns
  - use after move (and somewhat, use after free)
  - general undefined behaviour (e.g. overlong shifts, signed integer overflow)

And there's more to memory safety than security critical applications. Rust means you spend a little more time fighting the compiler, but a lot less time fighting the debugger and a lot less time trying to reproduce heisenbugs caused by data races/undefined behaviour.

Of course, the library/tool support is indisputably in C++'s favour.

if you're an idiot you can shoot yourself in the face

If you're a human you will shoot yourself in the face. It just takes far too much brain power to write correct C++ always (a single mistake leads to brokenness), especially in a team where there can be implicit knowledge about how an API works/should work that may not be correctly transferred between people.

pnathan• 12 years ago

I've had a vastly happier Rust experience than C++ experience (I've written in both, well beyond the "zomgz 50line starter program").

The Rust compiler is vastly smarter and gets you type checking you have to pay out the nose for in C & C++. I'm a fanboy of Rust, but I would suggest looking hard at Rust for any C or C++ production code going forward. (My default going forward for this space will be Rust unless overriding considerations say otherwise).

BruceIV• 12 years ago

C++11 is still C++ though. I've been writing pretty much exclusively C++11 for the last year, and, while they've added a lot of very nice features and I find it painful to go back to C++03, it's still a huge language with lots of hoary edge cases and everything and the kitchen sink built in (now with rvalue reference and move semantics for extra verbosity!).

I haven't touched Rust since 0.7 (where I abandoned it after filing two bugs on the project before I even got argument parsing working), but it's the most promising of the new batch of languages in my mind, and I have some side projects in mind I'd like to try when 1.0 comes out.

lmm• 12 years ago

Why are those your only choices? What makes your problem unsuited to e.g. Nimrod?

egonschiele• 12 years ago

Could someone lay out the advantages of Rust over C/C++/Dart/Go/other languages that cater to a similar space?

whyever2• 12 years ago

AFAIK Rust is the only language that offers memory safety without garbage collection.

st3fan• 12 years ago

Swift

fanf22• 12 years ago

What about Ada?

_pmf_• 12 years ago

C++11's std::unique_ptr and std::shared_ptr are also nice features. Does Rust provide more guarantees?

sriku• 12 years ago

This is all good. No higher kinded types for v1.0?

steveklabnik19• 12 years ago

Nope. We're pretty sure they'll be backwards compatible, and there are too many other outstanding issues (eg, syntax) and too little time. Lots of us want them though!

spott• 12 years ago

There isn't an RFC for them, at least that I could find. Have they just not gotten that far in terms of thought?

tomdale2• 12 years ago

We've been using Rust in production for Skylight (https://www.skylight.io) for many months now, and we've been very happy with it.

Being one of the first to deploy a new programming language into production is scary, and keeping up with the rapid changes was painful at times, but I'm extremely impressed with the Rust team's dedication to simplifying the language. It's much easier to pick up today than it was 6 months ago.

The biggest win for us is how low-resource the compiled binaries are.

Skylight relies on running an agent that collects performance information from our customers' Rails apps (à la New Relic, if you're more familiar with that). Previously, we wrote the agent in Ruby, because it was interacting with Ruby code and we were familiar with the language.

However, Ruby's memory and CPU performance are not great, especially in long-running processes like ours.

What's awesome about Rust is that it combines low-level performance with high-level memory safety. We end up being able to do many more stack allocations, with less memory fragmentation and more predictable performance, while never having to worry about segfaults.

Put succinctly, we get the memory safety of a GCed language with the performance of a low-level language like C. Given that we need to run inside other people's processes, the combination of these guarantees is extremely powerful.

Because Rust is so low-level, and makes guarantees about how memory is laid out (unlike e.g. Go), we can build Rust code that interacts with Ruby using the Ruby C API.

I'm excited to see Rust continue to improve. Its combination of high-level expressiveness with low-level control is unique, and for many use cases where you'd previously use C or C++, I think Rust is a compelling alternative.

alelefant• 12 years ago

I've never built a Rails app, so perhaps there's something simple that I'm missing (I'm mostly a C# guy), but how would I use Skylight to monitor my on-premise application? The pricing makes it seems like a hosted service. I would have expected some sort of profiler to be needed on the server and then perhaps a centralized location for the data to be pushed to for display.

The design of the site is great and I'm mostly curious because it sounds like something I wish was available in the .NET world!

tomdale2• 12 years ago

Thanks for asking. If you're interested in the nitty-gritty, Yehuda and I gave a talk on the architecture behind Skylight at RailsConf: http://www.confreaks.com/videos/3394-railsconf-how-to-build-...

The short version is that the agent runs on your servers and collects information from your Rails app using the ActiveSupport::Notifications instrumentation built in to the framework. We serialize that into a protobuf that's transmitted via IPC to a background daemon (written in Rust).

That daemon batches multiple reports into a single payload that is sent to the Skylight servers, where we use Storm and Cassandra to process the requests, and periodically do aggregate roll-ups.

Unlike New Relic, Skylight gives you access to the entire distribution of response times, not just the average response time. (According to DHH, averages are "useless.")[1] This ends up being a lot of data, and a lot of CPU-intensive processing, which is why we sell it as a hosted service.

[1] https://signalvnoise.com/posts/1836-the-problem-with-average...

hit8run• 12 years ago

Hey Tom, did you also consider googles golang? Currently I'm building my backends with golang and I am pretty satisfied with the performance and the whole flow. Maybe it is sometimes a bit cumbersome to check on errors like a paranoid but in the long term it helps to predict what happens in error cases. How about rust? Is the workflow comparable to golang? Is there an equivalent to gofmt and some ide support for code completion? Thx and regards Bijan

shadowmint• 12 years ago

There are some work-in-progress code completion tools https://github.com/phildawes/racer).

No format tool yet, but its often talked about and Id be surprised if it doesnt happen.

The biggest infrastructure difference to go at the moment is that its tricky to cross compile binaries in rust.

dbaupp19• 12 years ago

I don't speak for tomdale/wycats, but embedding a language with a GC (Go) inside another GC'd language (Ruby) only leads to worlds of pain, especially when trying to efficiently transfer data between them.

In languages without compulsory GCs (like Rust, which doesn't have one at all) this works because it's easy to have complete control about when memory is freed, but a GC'd language may free memory that was passed back into Ruby since the GC can no longer find any references to it.

I'm sure there's ways around this (e.g. using unsafe/raw pointers), but this control is the default in Rust, and there are a pile of language mechanisms (ownership, in particular) that make making this safe much easier.

(Also, AIUI, Go's FFI story is relatively inefficient, and, has some rather bad bugs; apparently https://code.google.com/p/go/issues/detail?id=7978 is the Go GC trying to free FFI memory.)

Xorlev2• 12 years ago

Love that you guys are using Rust in production. Very elegant design (AS::Notification IPC -> Rust).

I'd dump the part of the marketing site talking about the fast UI on Ember. It's cool that you guys use Ember and you're proud of it, but I want to see more about your product. Show me how easy it is to drop into an app, or reports you generate, alerts you flag, etc..

Ultimately, while I nerd out on what a product is built with, I only care about what it can offer me. Fast UI isn't a feature, it's an expectation.

aaronbrethorst• 12 years ago

Somewhat OT, but...

I'm looking at your pricing, and I don't have any idea how you define the notion of a "request". Right now, I can't even begin to guess what pricing bucket my apps might fit into, and that reduces my motivation to try out your service. I speculate I'm not the only person who's felt this way.

(also, I know why you include the Ember mention at the bottom, but I really don't care. I'd rather see your awesome UI than hear what technology it's based on.)

Edit: Seriously, I'm getting downvoted for offering someone feedback on their product experience?

annnnd• 12 years ago

Don't let it get to you - upvote and downvote are very close for mobile users. Here you have a +1 to compensate. :)

chrisseaton• 12 years ago

"We end up being able to do many more stack allocations, with less memory fragmentation and more predictable performance, while never having to worry about segfaults."

But isn't this stuff the job of the VM? There's no reason why a program written in Ruby can't do the same stack allocations, reduced memory fragmentation and predictable performance automatically - if the Ruby VM was better designed.

If Ruby had a better VM would you chose to use it over Rust? In Rust are you doing things that the VM could be doing for you?

dbaupp19• 12 years ago

It is very hard to have all this done reliably, especially in a rather dynamic language like Ruby. Maybe the Ruby VM could be better designed to perform these sort of optimisations, but it will never give the same control as a language like Rust, C or C++; i.e. a small code change could cause the optimiser to fail, leading to slow-down and bloat.

Furthermore, the hypothetical "sufficiently smart VM" isn't much value for code written now.

pcwalton6• 12 years ago

There's no reason why a program written in Ruby can't do the same stack allocations, reduced memory fragmentation and predictable performance automatically - if the Ruby VM was better designed.

Escape analysis falls down pretty regularly. No escape analysis I know of can dynamically infer the kinds of properties that Rust lets you state to the compiler (for example, returning structures on the heap that point to structures on the stack in an interprocedural fashion).

This is not in any way meant to imply that garbage collection or escape analysis are bad, only that they have limitations.

shin_lao4• 12 years ago

The JVM, who is several order of magnitudes better than the Ruby VM, constantly gets smashed in terms of performance and memory usage by a well designed C++ program.

Why? Because in theory, a VM should be able to do as well, if not better, than a programer.

The limit of a VM is that you are extremely limited in the amount of resources you can allocate to the VM to determine how much resources should be freed. See what I mean?

In the end the VM is nothing less than a program. With RAII, for example, there is zero overhead associated with the decision of releasing a memory block, because it's compiled into the program.

On top of that if you start to add some nasty memory optimization tricks allow you, you start to get why manually managing memory is still very interesting when performance and/or memory usage is important.

InclinedPlane2• 12 years ago

Meta note: I saw that this post was being pretty heavily downvoted and upvoted it to keep it from falling out of the discussion. I disagree with the assumptions in the post, but they are handily refuted by replies. I'd rather have a robust discussion with every reasonable side well represented than a boring echo chamber.

I urge other folks here to not merely blindly downvote posts they disagree with, reserve downvoting for posts that don't deserve to be seen at all because they don't contribute to the discussion.

austinz2• 12 years ago

Congratulations to the Rust team! Can't wait to start learning the language and building stuff using it.

I'm looking to learn about how Rust's refcounting memory management works (and how it differs from how, e.g. Objective-C or Swift's runtime-based reference counting works), mostly for personal edification. Can anyone point me to any good resources?

steveklabnik19• 12 years ago

Is worth noting that you don't reach for reference counting by default in Rust: your reach for references, then boxes, THEN Arc.

You can find documentation on these types here: http://doc.rust-lang.org/guide-pointers.html

austinz2• 12 years ago

Awesome! Thanks.

eddyb• 12 years ago

Well, you wouldn't go for Arc (atomic RC) unless you need to share data across tasks (threads), within the same task you can use Rc.

This is different from shared_ptr in C++ which is always atomic and LLVM has to try hard (in clang, no idea what GCC does) to eliminate some redundant ref-counts.

Oh and since Rc is affine, you only ref-count when you call .clone() on it or you let it fall out of scope.

Most of the time you can pass a reference to the contents, or, if you need to sometimes clone it, a reference to the Rc handle, avoiding ref-counting costs.

burntsushi7• 12 years ago

Steve already told you that refcounting isn't reached for by default, and I wanted to share an anecdote to emphasize that.

I've written a regex library, CSV parser, command line arg parser, elastic tabs and quickcheck in Rust. Behold:

    $ find ./ -type f -name '*.rs' -print0 | xargs -0 grep Rc
    $

I've definitely reached for it a few times as an escape hatch, but I've always ended up finding a cleaner approach to persist without it.

Of course, there are plenty of legitimate uses of refcounting. I just haven't hit them yet. :-)

steveklabnik19• 12 years ago

You're logged in as root? O_O :p :)

MoOmer• 12 years ago

We are removing support for green threading from the standard library and moving it out into an external package. This allows for a closer match between the Rust model and the underlying operating system, which makes for more efficient programs.

That's an interesting move in comparison to Go, which multiplexes coroutines onto threads.

pcwalton6• 12 years ago

It's more systems-like. We don't pay any overhead for calling into C code, and M:N scheduling doesn't really provide advantages over 1:1 scheduling when you don't have a GC (and even when you do, the differences are fairly negligible on modern Linux).

ngrilly2• 12 years ago

With 1:1 scheduling, how do you limit stack size to something reasonable (a few kB per thread), which is necessary when you need to launch tens of thousands of threads?

forrestthewoods• 12 years ago

When will it be pleasant to use on windows?

cdnsteve• 12 years ago

Can rust be used to power http endpoints like a REST API? Or is it more designed to be system type daemon stuff? I guess I don't fully understand the marketing of it, however I haven't ever written anything in C or C++ either.

tmzt• 12 years ago

There are already a number of libraries and frameworks that implement an HTTP setver in Rust, though things seem to be coalescing around Teepee.

http://chrismorgan.info/blog/introducing-teepee.html#main

There are few REST api libraries as well.

toolslive• 12 years ago

How difficult is it to start from a huge C++ codebase and start adding new features in Rust? How bad is such an idea? I know there is interoperability, but those are toy examples, does anybody have real life experiences?

illumen• 12 years ago

Does it use the GPU, memory compression, code rewriting, automatic vectorisation, multiple cores, or any other performance techniques from the last 10 years?

dbaupp19• 12 years ago

Rust is a low-level language so everything user-space can be implemented, and the main (and only) compiler rustc uses an industrial strength optimiser (LLVM) which has support for automatic vectorisation.

Furthermore, the type system is designed to be very good for high-performance concurrency.

See http://blog.theincredibleholk.org/blog/2012/12/05/compiling-... for an example of using Rust on a GPU, and I can only imagine that it has become easier since then.

alkonaut4• 12 years ago

Hopefully the tooling will take off once the language stabilizes. Using a "newish" language is a total exercise in frustration when you are a spoiled kid who expects an IDE to come with your language configured, and a nice big play button for running your first program.

For a language to take off, it badly needs a very good (ideally "official") development experience, such as a custom eclipse impl, or a very good IntelliJ plugin. When a dev experience comes with batteries included it lowers the treshold substantially from just "use whatever text editor you like and compile on command line, here is a readme".

Chirono• 12 years ago

Wait, they've got rid of unique pointers? When did that happen? They were there a couple of months ago. That was one of my favourite language features...

andrewaylett• 12 years ago

My understanding is that they're still there, just as part of the standard library rather than as a language feature.

steveklabnik19• 12 years ago

That's correct. What used to be ~T is now Box<T>. We say 'boxes' instead of 'unique pointers' now.

theavocado2• 12 years ago

When I first looked at Rust, I recall being very confused about the distinctions between crate, package, module, and library. It seemed like an area which could use some simplification.

steveklabnik19• 12 years ago

"Package" and "library" are two words that mean the same thing. They're more generic terms for "crate," which is Rust specific. "module"s are ways of splitting up your code inside a crate: one crate has many modules, and each module belongs to one crate.

theavocado2• 12 years ago

Thanks, Steve! Have not yet had a chance to go through the new guide, but looking forward to it!

The Road to Rust 1.0

Related Stories