Announcing Rust 1.19

https://blog.rust-lang.org/2017/07/20/Rust-1.19.html

by steveklabnik • 9 years ago

318 80 9 years ago

Incredibly excited to see unions available in stable Rust now!

The release notes mention my RFC, but a huge thanks also to Vadim Petrochenkov for the implementation, and all the myriad RFC contributors.

GolDDranks3• 9 years ago

Now that unions have stabilised, is there any significant un-expressable things in C APIs left? Bitfields?

DDSDev2• 9 years ago

I have run into issues with bitfields, and flexible array members.

JoshTriplett4• 9 years ago

Yes, to the best of my knowledge, bitfields are the last thing in C APIs you can't express conveniently in Rust; you can handle them with careful macros, but that's annoying, just as it was for unions.

In terms of things you can express now but that could still use some work: anonymous structs and unions. You currently still have to name all the intermediate aggregates, even when the C API doesn't. See https://internals.rust-lang.org/t/pre-rfc-anonymous-struct-a... .

jmcomets• 9 years ago

I was pretty surprised, how the heck are unions different from enums?

I read that they're essentially the same as C unions (~untagged enums), but I have no clue how `match` works in that case.

Rusky• 9 years ago

A pattern on a union just interprets it as the specified field. It doesn't try to determine which one was actually written to last.

Cieplak2• 9 years ago

Great running into you at Shizen @steveklabnik!

Just started using Rust in a serious capacity this month to secure some C++ functions that are called by our Erlang apps, with great assistance from Rustler [1]. Several people have complained to me about the decision to remove async IO from Rust, but I'm really grateful that it happened, because it lets Rust focus on being the best at what it is. Erlang's concurrency primitives and Rust's performance & security are a match made in heaven.

[1] https://github.com/hansihe/rustler

steveklabnik6• 9 years ago

Hey hey! You too. Sorry I had to skip town before we could grab a drink, it was a hectic trip!

the decision to remove async IO from Rust

I'd be interested in hearing more about this, did they maybe mean green threads? https://tokio.rs/ is the big async-io project, it's certainly not removed in any sense!

Cieplak2• 9 years ago

I should clarify the "remove async IO" as the decision to not include an event-loop primitive in the core language or stdlib, which is tantamount to complaining about GC not being part of Rust core :)

Metal IO is an awesome library as well:

https://github.com/carllerche/mio

jmull• 9 years ago

Hm. I'm not sure about the addition of unions. Why add something that is unsafe to read or write? You need an additional mechanism to let you know which type it is OK to access.

They mention the case where the type can be distinguished by the lest significant bit, but wouldn't it be better to handle that case as an enum? That is, the least significant bits define the enum tag, while the remaining bits define the associated value.

(By the way, I really mean this as a straight question, not a criticism in the form of a rhetorical question. I really don't know enough about it to be criticizing it.)

vvanders2• 9 years ago

They're largely meant as an easier interop with the C ABI and C APIs that use unions. You wouldn't be using them in your higher level Rust library.

steveklabnik6• 9 years ago

https://github.com/rust-lang/rfcs/blob/master/text/1444-unio...

TL;DR: FFI with C is much harder without unions. There are smaller reasons as well.

kbsletten• 9 years ago

I believe that they've framed it in this article as mostly for compatibility with C/C++. A simple example is JS objects are often represented as a single f64 with the NaN space punned for 32-bit pointers and 53-bit integers. If you want to put some code written in Rust in that environment, if you could define a union that could expose a safer API by using bit twiddling you could. Presumably if you have the memory to spare or are unconstrained in design you won't use this feature which is part of why it took so long to get into the language.

GolDDranks3• 9 years ago

Unions are used in C APIs so having them is nice for FFI.

Besides, enums aren't always optimal for some things. Let's say that you have an array of 32-bit values whose prime-numbered indexes contain either a signed or unsigned 32-bit integer, depending on a single global boolean tag. You can't encode that invariant into a (non-dependent) type system. If you want to do it safely, using enums, you are going to have a tag for every value, and that's going to cost you. (A bit crazy example, but bear with me.)

Unions are a way to circumvent the type system a bit. They allow you to keep track of the stuff inside memory slots using the way you see the best. But there's a reason they're unsafe - the responsibility is on you!

fanf22• 9 years ago

You don't need to get as tricky as that.

Rust doesn't let the programmer specify the layout of enums in detail right now, so you can't specify where the compiler should place the discriminator.

The example I care about is a word which is a pointer if the bottom bit(s) are 0, but otherwise contains a bunch of packed fields, where the least significant bits are a nonzero value I can do something useful with.

kibwen8• 9 years ago

As other commenters have mentioned, the demand here is almost all due to the desire for smoother interoperation with C code. What's gone unsaid so far is that, despite still requiring the `unsafe` keyword for many operations, this feature helps make Rust code more safe when calling into C, because it simplifies the interface and eliminates the need for hacked-up handwritten workarounds. IOW, a little bit of standardized unsafety to replace a larger amount of bespoke unsafety.

Manishearth6• 9 years ago

FWIW the hacked up workaround that bindgen uses is _beautiful_

https://github.com/servo/rust-bindgen/blob/master/tests/expe...

Basically, if you have a union of A, B, C, you create a struct with three zero-sized fields using BindgenUnionField<A>, and then add a field after that containing enough bits to actually fill out the size. Because the BindgenUnionField is zero sized, a pointer to it is a pointer to the beginning of the struct, and it has an accessor that treats the pointer as the contained type.

This makes the API for field access `union.field.as_ref()` instead of `union.field`, but that's still pretty clean.

It's still a hack, and I'll be happy to see it go, but it's a really fun hack.

Manishearth6• 9 years ago

Other commenters have mentioned the use case for FFI, and there's a very small niche use case for type punning, but there's one thing that this enables you to do that was absolutely not possible with the language previously.

This is zero cost stack allocation of data in a way which avoids destructors.

Basically, currently, in Rust, if you want to allocate a type and avoid destructors being run, you have to write a wrapper around `Option<YourType>` that nulls the option in its destructors. Or you heap allocate it and turn the box into a raw pointer after allocation. Both have overhead. The zero-overhead way of doing it is to stack allocate an array and cast pointers, but then you need to know the size beforehand.

With unions, you can stack allocate a `union Foo {x: YourType}` and just use that. Unions don't have destructors, so this stack allocates enough space for your type, and lets you unsafely but conveniently access it as your type (no ugly pointer casting hacks), but you can guarantee that destructors won't be run.

The obvious question here is -- why is this even necessary? Surely you can just call mem::forget to avoid destructors right before the function returns.

However, destructors also get run whilst panicking, so if your function accepts a callback, and that callback panics, you can't avoid destructors without the overhead mentioned before.

For a concrete example of this use case, check out ArcBorrow::with_arc() (https://doc.servo.org/servo_arc/struct.ArcBorrow.html). ArcBorrow<'a, T> is basically a borrowed reference to a T that is known to be backed by an Arc (atomic reference counted type). You can obtain borrowed references to an Arc normally, which is great -- lets you share the data without bumping atomic reference counts all the time and paying the atomic overhead. But if you have an &T -- a borrowed reference to a T -- there's no way to bump the reference count on that if you need to escape the borrow; since there's no guarantee that the &T borrows from an Arc allocation and not something else. So you must pass down an &Arc<T>, and that has double indirection. There are other reasons (pertaining to the existence of RawOffsetArc, which has to do with FFI constraints) as to why &Arc<T> won't work for us there, but I won't get into those. ArcBorrow<'a, T> lets us freely pass around borrows of &T which can be cloned as an Arc if necessary.

Anyway, ArcBorrow has a with_arc method (https://doc.servo.org/servo_arc/struct.ArcBorrow.html#method...). This takes a closure and passes an &Arc<T> to it.

But we don't have an Arc<T>, we have an ArcBorrow<T>, which has a different representation (in particular, ArcBorrow contains a pointer to the data, whereas Arc contains a pointer to the allocation, which starts earlier because of the refcount).

So we construct a fake Arc<T> on the stack (https://doc.servo.org/src/servo_arc/lib.rs.html#884-907), and share it with the closure. Because it's a fake Arc<T> we can't actually let its destructors be run, so we put it inside NoDrop, which on nightly uses unions (but on stable uses the non-zero-cost methods I mentioned above).

I use this same trick in array-init (https://github.com/Manishearth/array-init/blob/a0cb08928b42d...), where I stack allocate an uninitialized array and let you fill in the elements with a closure. If the closure panics the destructor of the _partially_ initialized array should not run, so again, it's in a NoDrop.

In general when writing unsafe abstractions you often need escape hatches like these.

Mouse47• 9 years ago

Stupid question, since I'm not sober...why doesn't ArcBorrow just store a reference to the Arc and also a reference to the underlying T? That would solve the double indirection and also give the ability to clone the Arc?

Great effin post btw, i spent about 40 minutes readin that shit

ofek• 9 years ago

Wow, a break yielding a value from within a loop is awesome! Do any other langs have that?

kibwen8• 9 years ago

I feel like this has to have precedent from somewhere, most likely from another expression-oriented language, but I can't seem to find one right now. I've checked the RFC expecting to see a discussion of precedence from other languages, but no luck so far.

joshmarlow• 9 years ago

If you're using the loop macro, you can do it from Common Lisp (see [0]).

  (block outer
    (loop for i from 0 return 100) ; 100 returned from LOOP
    (print "This will print")
    200) ==> 200

  (block outer
    (loop for i from 0 do (return-from outer 100)) ; 100 returned from BLOCK
    (print "This won't print")
    200) ==> 100

[0] - http://www.gigamonkeys.com/book/loop-for-black-belts.html

EDIT: I always do formatting wrong on here.

pdw• 9 years ago

Not just with LOOP, RETURN works with every standard iteration construct.

lispm• 9 years ago

One can also break out of the LOOP and return the so far accumulated value:

  CL-USER> (loop for i from 1 below 10
                 sum i
                 when (= i 6)
                   do (loop-finish))
  21

msbarnett• 9 years ago

Sure, Ruby:

     x = (1..100).each do |i|
       if i > 9
         break i
       end
     end

Evaluating that gives

    > x
    => 10

You can also supply a value to next within a loop, which is occasionally useful

matthewrudy• 9 years ago

ruby does this

have code around my app doing things like:

  loop do
    code = random_string(6)
    break code unless code_already_used?(code)
  end

static_noise4• 9 years ago

One thing that recently surprised me is that Rust lacks default values and named arguments for functions.

Some consider it a counterpattern or bad memory to be able to not fill out all parameters or reference them by name. However some interfaces easily require like 50 different function parameters that cannot be removed in a simple way and they all make sense in different configurations. Without default values and named parameters you're lost there. I don't get this design decision on Rust side at all.

vvanders2• 9 years ago

Can't you just do that with passing a struct + Default[1]?

[1] https://doc.rust-lang.org/std/default/trait.Default.html

static_noise4• 9 years ago

It seems like you can... but then this could be done implicitly at no loss and lead to less clutter. It's not hard to understand what default parameters or named parameters do.

krisdol• 9 years ago

Maybe they'll get there with time, but with 50 different parameters (... for one function? really?) I'd say put most of them in some kind of an options object, which certainly can have defaults and also checks your values at compile-time.

steveklabnik6• 9 years ago

Yes, the currently-considered-idiomatic solution here is the builder pattern.

static_noise4• 9 years ago

Here

https://matplotlib.org/api/pyplot_api.html#matplotlib.pyplot...

I count about 40 plus more which are not listed.

You surely could first make a dictionary or structure and fill it with these arguments. But then "kwargs" are just that, a dictionary.

I find the interface of the plot function pretty straightforward. There are a lot of options but it's pretty clear what they do and when to use them. Most often you use only some of them - but often different combinations. Splitting this up into multiple helper objects that need to be constructed and filled beforehand would turn the default one-liner into a ten-liner which is not better.

shepmaster• 9 years ago

If you do have the struct implementing `Default`, as suggested from sibling comments, and it's fully public, you can also use the "functional update syntax" (where the `..` is):

    #[derive(Default)]
    struct Config {
        interesting_thing: bool,
        a: i32,
        b: i32,
        c: i32,
    }   
    
    fn main() {
        Config {
            interesting_thing: true,
            ..Config::default()
        };
    }

In addition, the builder pattern is especially useful for larger configuration objects like this.

fnord123• 9 years ago

I think the derive_builder crate is worth a mention here:

https://docs.rs/derive_builder/0.5.0/derive_builder/

xzel• 9 years ago

Builder pattern/defaults. I personally find it much more straightforward/organized than using default values and/or named parameters. Python loves named parameters and sometimes their over usage drives me crazy.

piaste• 9 years ago

I agree about default values, but named parameters have other benefits beyond acting as makeshift configuration options.

1) More readable code. The classic is foo.Bar(true). It breaks the flow of reading to have to hover and see what that 'true' means. Much nicer to see foo.bar(launchMissiles: true).

2) Protects from a particular class of dumb mistakes. You have a function foo(x, y) where x and y have the same type. You refactor it so that one of the parameters isn't needed anymore. It's surprisingly easy (read: I've seen it, and I've done it), when you clean up the function calls, to accidentally delete x instead of y or vice-versa. Named parameters prevent that.

kibwen8• 9 years ago

Coming from Python I agree that the lack of keyword/default arguments is initially conspicuous, though in practice it seems that library authors seem content to implement builder patterns to work around their absence. While ideally I do think that the builder pattern is less elegant than supporting keyword arguments, in practice it does avoid the unfortunate pattern I've seen in some Python APIs to just cry #yolo and cram a hundred parameters into a single function call. I've never seen a good interface that requires "like 50 different function parameters"; call me zealous, but if I were to design my own programming language, I'd make it a hard compiler error to have more than four parameters to a function. :P

static_noise4• 9 years ago

While you can misuse a feature, you don't have to. My example is extreme and you probably don't run into something like that often depending on what you do. There could be other solutions. There surely are other solutions that are neither more clear nor better in any way. Sometimes you just got to do what you got to do. What surprised me about Rust is that it has all those cool features but lacks a very basic one that most programming languages have.

tikue_• 9 years ago

Saying library authors are content with their absence is probably a bit too strong. There's not really another option, and I doubt many folks would choose not to author libraries for this reason alone. I personally have found their absence annoying when writing Rust libraries.

I'd probably be in favor of a hard cap on # of function arguments. :)

iopq• 9 years ago

I'd make it a hard compiler error to have more than four parameters to a function

There's already such a language, it's called Haskell and it only allows one parameter per function.

cowsandmilk• 9 years ago

However some interfaces easily require like 50 different function parameters

Please tell me you're not responsible for this monstrosity https://salilab.org/modeller/9.18/manual/node315.html

dragonwriter• 9 years ago

However some interfaces easily require like 50 different function parameters

These interfaces are poorly designed; as in natural language, you almost never need more than a subject, direct object, and one or two indirect objects, any of which may themselves be compound entities.

A call with more than about four parameters has usually decomposed at least one thing that should be composite in the argument list.

Announcing Rust 1.19

Related Stories