Implementing Go's defer keyword in C++

https://oded.blog/2017/10/05/go-defer-in-cpp/

by ingve • 9 years ago

86 85 9 years ago

junke• 9 years ago

I am not sure capturing variables by reference implements the expected behavior; what would the equivalent C++ code return for this Go example?

    func a() {
        i := 0
        defer fmt.Println(i)
        i++
        return
    }

JBReefer• 9 years ago

Wouldn't this work by reference? The referenced value would be the same in both contexts.

Honestly, am I missing something?

Vendan• 9 years ago

Go prints 0:

https://play.golang.org/p/kdi60DOln-

parenthephobia• 9 years ago

The syntax of defer is a function call, but it isn't called like a normal function: the arguments are evaluated at the time the defer is encountered, and then the function is called later. That's one reason why defer is very often called with an anonymous function, so that variables can be captured.

So, the version that prints 1 is

    func a() {
        i := 0
        defer func(){
          fmt.Println(i)
        }()
        i++
        return
    }

albinofrenchy6• 9 years ago

This seems unnecessary and possibly even harmful.

In C++ the destructor handles this as part of the lifespan of the object. Making users of the object encapsulating the resource handle resource cleanup is an antipattern, which is why the standard library does this for you.

Occivink• 9 years ago

There's plenty of one-time cases where you don't want to declare an entire class but still enjoy scope-based functions. However, the approach in the post is not very C++-esque, I like this one much more: https://github.com/mawww/kakoune/blob/master/src/utils.hh#L5...

albinofrenchy6• 9 years ago

The only time I can think of where this is useful is when you are trying to be super defensive with possible exceptions being thrown.

Otherwise it seems a lot like you can just make the call at the end of scope yourself for the same line count and _much_ more readable code.

I guess if you have multiple return paths it's more tenable to use scoped dtors for this but it all really seems questionable to me -- I can't think of a single use-case where it'd be preferable to use something like this rather than a dedicated class.

Too2• 9 years ago

std::unique_ptr has a widely unknown second template argument called deleter. It is designed exactly for this purpose. You just do roughly like this(written from a phone):

    typedef fdwrapper = unique_ptr<int, close>;
    fdwrapper fdhandle(open(filename));

alexgartrell• 9 years ago

ScopeGuards as imagined here (Facebook's folly library has one) are near-essential for writing reasonable systems code in C++.

   int fd = socket(...);
   if (fd < 0) return false;
   SCOPE_EXIT { ::close(fd); };

   if (ioctl(fd, ...) != 0) return false;
   if (ioctl(fd, ...) != 0) return false;
   ...

   return true;

You could totally just write the FdWrapper class, but then you're still a user handling resource cleanup, but with more lines.

ot• 9 years ago

You could totally just write the FdWrapper class

Which, BTW, is exactly what folly::File does :)

bodyfour• 9 years ago

You could totally just write the FdWrapper class

Yes, and that RAII class would be reusable all of the places you need to close a file descriptor. It's even a handy place to hang more methods that work on the descriptor... for instance if you wanted to abstract those ioctls.

That's why I personally never use ScopeGuards in C++. You are correct that writing a single-use RAII class requires a few lines of extra code. However I have found that after I write such a class I often find myself using it over and over.

butterisgood• 9 years ago

Sometimes you need to wrap C APIs. Sure you could put their guts in a class or struct, but you can also use a unique_ptr for some of the same lifetime stuff.

Also, defer in Go always uses the function exit, not the scope. Not sure you can do that in C++. Everything seems scoped.

Maybe don't do this :-)

albinofrenchy6• 9 years ago

You usually shouldn't use unique_ptr for C api stuff unless you really only need to free it.

Typically though there are custom delete/free API calls you need to make, and to use them with unique_ptr you have to pass that function pointer in as the second argument to the unique_ptr constructor.

At the very least you should have a C++ function that gives you the unique_ptr back that does the right thing; but at that point you almost might as well just make a class and give it member functions to map the C calls.

evincarofautumn2• 9 years ago

If you wanted to do things on function exit, you could have an “ENABLE_DEFER” macro that declares a scope for guards; the “DEFER” macro would register the deferred lambda with that scope, not define a new object, and the destructor of the scope would run all the registered guards.

Of course, in my C++ code I would just write a class/struct to wrap any RAII-like logic, not use a “defer”. A marginal amount of extra work that pays off in consistency and readability.

tcbawo• 9 years ago

It seems that this might be for only limited cases. I'm not that familiar with Go, but I suspect that due to coroutines, the deferral may not execute for quite some time, while in C++ this could be limited to the single-threaded flow of the calling method (unless you manually move the scoped object out of the call). To match Go's behavior, you would need to attach the object lifetime to whatever future/asynch mechanism you were using. One very nice implementation (if heavyweight) is Seastar's then chaining: http://docs.seastar-project.org/master/md_doc_tutorial.html

lucio• 9 years ago

go's "defer" is like java/net "finally" for a try/catch enclosing the entire function.

pro: you can push more than one "defer" in the same function.

con: "finally" puts the cleanup code at the end of the function. With "defer" you have it scattered at the beginning.

nothrabannosir2• 9 years ago

con: "finally" puts the cleanup code at the end of the function. With "defer" you have it scattered at the beginning.

Quite commonly an advantage, e.g.:

    f, err := openFile('...')
    if err != nil {
        ...
    }
    defer f.Close()

is how it's commonly used. This makes sense because the cleanup lives next to the initialization of the actual thing it's cleaning up. In a way it becomes part of the initialization, itself.

Same with locks and release. Now you don't have to scan across the entire function body to make sure your inits and cleanups match up.

karmakaze• 9 years ago

con: "finally" puts the cleanup code at the end of the function. With "defer" you have it scattered at the beginning.

I would consider this a 'pro' as each action and defer are paired, executes in reverse order and a conditional action/defer doesn't need a second matching check.

markbnj• 9 years ago

It's a nit I guess but I feel like RAII is a pattern, not a feature of the language. Haven't done C++ in years but as I recall it doesn't have anything like python's with or finally blocks, which would both serve the purpose illustrated here. Using destructors to perform things when exiting a block is fine, I guess, but it does introduce hidden behaviors that will need to be well documented.

stabbles• 9 years ago

It's a known trick also used in the standard library of C++. For instance in multithreading

    {
    std::lock_guard<std::mutex> lock(some_mutex); 
    ...
    }

owns a mutex for duration of the scope. The constructor claims it, and the destructor releases it.

But yeah, it has downsides. If you would not name the variable, and do `std::lock_guard<std::mutex>(some_mutex);`, then its scope is limited to a single line, and the lock is immediately released.

masklinn5• 9 years ago

And to toot its horn, Rust combines that with its ownership semantics to create a very neat pattern: many locks have to do with resources yet almost all languages split the resource and the lock and you've got to know that you need to acquire one lock before you use a resource without any intrinsic relation between the two.

In Rust however sync::Mutex::new takes a value, which the lock owns, and acquiring the lock yields a MutexGuard which acts as a reference to the wrapped value. You must acquire the lock to get access to the value (since it's owned by the lock itself), and once the lock is released you have no way to access the locked value (unless you created a copy).

If you would not name the variable, and do `std::lock_guard<std::mutex>(some_mutex);`, then its scope is limited to a single line, and the lock is immediately released.

OTOH you can pretty trivially create a wrapper/extension which takes a lambda, acquires the lock, calls the lambda then drops the lock. It could even return the lambda's result while at it.

Too2• 9 years ago

Every single variable in C++ is implicitly surrounded by a with-block. And __exit__ is called destructor.

Animats2• 9 years ago

The best solution to this problem seems to be Python's "with".

    with open(oldfile, 'r', encoding='utf-8') as infile:
        with open(newfile, 'w') as outfile:
            ....

The implied close will be executed on exiting the "with". "with" calls .__enter__ on the "open" object at entrance, and .__exit__ at exit. This works even if exit is via return or an exception.

If the implied close generates an exception, everything still works reasonably. __exit__ has a "traceback" parameter, which indicates an exception is in progress. An exit operation should do its closeout, then re-raise the exception sent as the traceback, if any. This works even when a destructor raises an exception. (A typical case is closing a network connection. If the other end dies, close will generate an error, which should not be lost.) The default use of "with" in Python thus comes with correct single and nested error handling.

That's not true of RAII or Go's "defer". In C++'s destructors, you can't return an error code, and an exception in a destructor is usually a big problem.

It's hard, but not impossible, for a deferred call in Go to report an error. The deferred function can muck with the parameters of the function which called it. This only works if the deferred function was written specifically for the context in which it was called; just deferring a "close" won't do this. So you don't get proper error handling by default.

(Languages seem to need exceptions. Both Go and Rust started with an always-aborts panic mechanism. That was too drastic in practice, and in both languages, panicking was made recoverable and provided with unwinding machinery. So both languages now have the heavy machinery for exceptions, without the language support for them. This results in kludges to simulate exceptions.)

mappu2• 9 years ago

One minor downside to `with` is how it brings normal control flow out to the right.

eesmith5• 9 years ago

If that's an issue, one option might be https://docs.python.org/3/library/contextlib.html?#contextli... .

tedunangst2• 9 years ago

This is still tied to a variable of some type, requiring a dummy class for arbitrary functions.

eesmith5• 9 years ago

The variable isn't necessary, and contextlib.contextmanager is the standard wrapper for arbitrary functions. Here's the example code from the documentation:

  from contextlib import contextmanager

  @contextmanager
  def tag(name):
      print "<%s>" % name
      yield
      print "</%s>" % name

  >>> with tag("h1"):
  ...    print("foo")
  ...
  <h1>
  foo
  </h1>

RX143• 9 years ago

Surely ruby's blocks are much better here. For one they're a much more general construct than with, and second exception handling is done with normal language constructs like rescue and ensure. Implementing one is simple and obvious and is nicely contained in a single method.

For example

    File.open(args) do |file|
      # use file
    end

And the implementation is simply

    def self.open(*args)
      file = File.new(*args)
      begin
        yield file
      ensure
        file.close
      end
    end

lstyls• 9 years ago

I know Python far better than Ruby, but it looks like the semantics are equivalent, it's just that Ruby's standard `open` implementation includes the cleanup rather than leaving it up to the caller as in Python.

int_19h2• 9 years ago

Ruby does this at the cost of having non-local returns from those blocks, which complicates semantics quite a bit (since now you need to distinguish closures that capture "return" and other control transfer from those that don't, and you need to handle the scenario where a control transfer happens when the target is already gone).

asdfaoeu• 9 years ago

You can do basically the same thing in python

    @contextlib.contextmanager
    def open(*args)
        # file open
        try:
            yield file 
        finally:
            file.close()

    with open(args) as file:
        # use file

masklinn5• 9 years ago

On the other hand, it means any helper/builder has to handle this pattern on its own rather than be able to return a file object which can be "context managed" by a caller, which Python allows.

ohnoesjmr• 9 years ago

The problem with contexts is that can't just innocently enter one in some if branch where the rest of the code would be scoped.

I ended up an OnExit() context that does:

    if conn is not None:
        ctx.call(conn.close)

weberc2• 9 years ago

I agree that Python's `with` is nicer than Go's `defer` (and I use both languages almost daily). I'm not sure I agree that languages seem to need exceptions--stack traces in the presence of errors are certainly handy, and there's certainly a case for panicking, and it's arguably more ergonomic to have syntax or type-system support for eliminating error-handling boilerplate, but I still prefer the simplicity of returned errors. That said, I'm really interested to hear different view points.

kibwen2• 9 years ago

Languages seem to need exceptions. Both Go and Rust started with an always-aborts panic mechanism. That was too drastic in practice, and in both languages, panicking was made recoverable and provided with unwinding machinery. So both languages now have the heavy machinery for exceptions, without the language support for them. This results in kludges to simulate exceptions.

This is completely incorrect with respect to Rust, both historically and practically. Rust started with unwinding, and then grew opt-in abort machinery for those use cases where it matters. And, for what seems like the millionth time, unwinding does not equal exceptions; this is like saying that C has exceptions because of setjmp/longjmp. Unwinding exists in Rust solely to give one the ability to fail gracefully, running destructors that might have valuable side effects and allowing Rust-written subsystems to avoid taking down the entire system. For an example of the latter, Skylight, who ships a Ruby gem written in Rust, "catches" any unwinding so that an errant failure in their gem does not bring down a client's entire Ruby process. I have been writing Rust for years and, without exaggeration or (heh) exception, have never once seen the ability to "catch" unwinding used as a basis for constructing ad-hoc recoverable exception-handling, especially since abort-on-panic is an entirely valid mode for Rust applications (this is how Dropbox's backend Rust is compiled, for example) and completely obviates any sane attempt to use panics for recoverable error handling.

epage• 9 years ago

To add to this, people's perception of Rust:

- Exception -> Rust's panic

Reality:

- Runtime assert -> Rust's panic

- Exception -> Rust's Result (ie error monad)

lloeki• 9 years ago

And the same goes for Go. The only legitimate use of panic is still always-abort; recover merely helps one define what part of the program has aborted, and is a noop except within a deferred call — you'd be hard pressed to use recover as casually as exceptions in Java, Python or Ruby. Typically, this is the case when a master/workers scenario is implemented[0], where the master has to survive a worker's death, because they're different "processes", but the latter definitely dies from panicking (possibly even caused by a call to a third party lib such as described by kibwen)

[0] https://github.com/urfave/negroni/blob/master/recovery.go#L1...

kentonv5• 9 years ago

The best solution to this problem seems to be Python's "with".

Ehhh... `with` is a crude substitute for destructors. C++ is a whole language designed around RAII. Python really is not.

You do make a good point that the handling of exceptions in destructors in C++ is problematic. But you can actually throw from destructors, as long as the destructor wasn't called during unwind for another throw. I'm of the opinion that the inability to throw again during unwind is a glaring bug in the standard, since it presumes people can know if their code is going to throw, which is like presuming people know where their bugs are. But the reasonable way to work around it is to squelch exceptions from destructors when another exception is already in-flight. This is usually fine, as the "secondary exceptions" thrown during an unwind are usually side-effects of the "primary exception" anyway.

mappu2• 9 years ago

The ScopeGuard RAII destructor will be called at the end of the scope, but Go's defer will be called at the end of the function.

e.g. in Go can you can defer inside a for loop.

mhh__• 9 years ago

I feel that the way this is implemented in the D programming language is more natural (Subjective, of course): https://tour.dlang.org/tour/en/gems/scope-guards

Although this approach would probably not work like this in Go given Go's lack of exceptions.

saurik• 9 years ago

This is not the same as Go's defer, as this honors scope; this is similar to D's scope guards (and hence the name of the C++ class...). In Go, when you use defer, the code in question is deferred until the end of the function, no matter in which scope you used the feature: this means that if you have a loop in which you use defer, everything is deferred until well after the entire loop finishes, not during each execution of the loop. This also means that Go has extremely... interesting?... behavior with relation to argument capture for the expressions used in a defer statement to make this even remotely sane. There are reasons the people who work on Go think this makes sense, but whether or not you agree with those semantics, the semantics implemented in this blog post are very different.

maxpert• 9 years ago

Great implement a coroutine, channels, and async io using those coroutines... WOLLAH we have a golang replacement.

missblit• 9 years ago

Boost asio has async IO that uses coroutines. I only ever used it for toy code, but it seemed to work pretty well.

gmfawcett• 9 years ago

(It's "voilà", not "wollah". It's a borrowed French word meaning "there it is!" or "behold!". In English, the accent is optional: "voila" means the same thing.)

beached_whale3• 9 years ago

The macro will break with templates though. Just name it an be explicit

randyrand• 9 years ago

why?

beached_whale3• 9 years ago

the preprocessor does not understand the commas as not being part of the macro

slavapestov• 9 years ago

The Swift compiler uses this trick: https://github.com/apple/swift/blob/master/include/swift/Bas...

oppositelock• 9 years ago

This is a neat C++ exercise, but when you're working daily with a language and intend to write maintainable code that's going to be around for a while, it's best to stay within common, boring conventions of that language. In C++'s case, just stick to explicit RAII-based guard wrappers and your life will be a lot easier. Macro-based emulations of other language features can lead to lots of unexpected behavior.

Sorry for pooh-poohing someone sharing some ideas, but being someone who has to maintain long lived code in quite a few languages, code like this immediately sets off my alarm bells.

Don't get me started on the dude who use #define magic to make C look like Pascal.

xyzzy_plugh• 9 years ago

I've seen a lot of C++, and there's no consensus on what "boring conventions" are.

Some places ban templates, some ban RAII. Some ban operator overloading. Some ban lambdas or C++11 features. Some ban macros.

There's a lot of variety, and just because you use something like this in your library or project doesn't mean you can't come out and play. C++ is the most "be yourself" language I've encountered. There is no idiomatic code.

Now, I get where you are coming from, and I'd love C++ a lot more if there was less feature abuse, but I've accepted that the ship has sailed.

kentonv5• 9 years ago

I disagree.

If you tell people that the only way they can do something "on scope exit" is to write a whole class with a destructor and manually capturing variables, the result is they won't bother. They'll just put the code at the end of the function, and say "I'm too lazy to make this exception-safe."

This defer() macro is almost exactly what I've been using for the last 5 years or so:

https://github.com/capnproto/capnproto/blob/master/c++/src/k...

And I can say pretty confidently that it's a really good idea to have. I'm sort of surprised, actually, that not everyone is already doing this...

pjmlp2• 9 years ago

Don't get me started on the dude who use #define magic to make C look like Pascal.

I did it for memory allocation, because C designers weren't able to create something as simple as new, rather requiring us to keep using sizeof on malloc calls.

That was only for an year, until I got hold of Turbo C++ and only dropped pure C when my teachers required me to deliver assignments in C.

Thankfully most of them accepted C++ as well.

foo101• 9 years ago

My intention is not to nitpick but I am genuinely trying to understand this from a pedantic perspective.

C++ has a neat feature called Resource acquisition is initialization, a.k.a RAII.

Is this really a feature of C++, or is this a design pattern/best practice that developers have discovered and use to write safer C++ code?

Merad• 9 years ago

A bit of both, IMO. RAII is a design pattern, not a language feature. However, it’s a pattern made possible by the semantics of destructors and object lifetimes in C++. The creator of C++ (Stroustrup) developed RAII within the first few years of the language’s existence.

ovao• 9 years ago

It's not a discovery per se, since its creation was deliberate, but it's a practice exercised throughout the STL (the C++ standard template library), and is a term coined by C++ creator Bjarne Stroustrup himself. It could be considered a "standard pattern" in lieu of a better term.

pjmlp2• 9 years ago

It is a bit of both, as replied on siblling comments.

While RAII was coined by the C++ community, it is not unique to C++, there are other languages that support it.

millstone• 9 years ago

What are the advantaages/disadvantages of a function-scoped destructor like Go's defer vs a lexically-scoped destructor like C++ offers?

kentonv5• 9 years ago

I see a lot of people in this thread dumping on this idea, but in my opinion as someone who has built several large systems in C++, this pattern is not just good, it's essential. KJ (the C++ framework library built with Cap'n Proto that I use in all my projects now) has had KJ_DEFER() since just about the beginning, working basically exactly as described here, and I use it constantly.

If you write C++ with exceptions, you need this. If you simply place tear-down code at the end of the function instead, you are creating exception-unsafe code. Of course, ideally state is encapsulated in classes with destructors and tear-down happens there, but writing a whole class can often be overkill for a one-off use case. If you don't have a defer macro, you're going to get lazy and introduce bugs.

If you don't use exceptions, then you still need this, because your code almost certainly has lots of "if (error) return nullptr;" or whatever. Without `defer`, you need to stick the teardown code before every one of those returns, or you need to resort to "goto fail" error handling.

quietbritishjim3• 9 years ago

If you simply place tear-down code at the end of the function

No one is suggesting this. They are saying that you use an RAII class such that it doesn't need tear down. In the example in the article, if std::ofstream calls close() in its destructor (which it does) then you don't need to DEFER a call to it. If you lock a mutex, use a RAII locker class so that you don't need to DEFER a call to unlock(). As someone once said of C++: "my favorite feature is the close brace", because it calls destructors for all objects on the stack, even during exception unwinding.

Of course, the article is just using RAII. But I disagree with the style; rather than re-writing a DEFER(myfile.close()) in every single function that uses a file object, you should make the file class have an appropriate destructor, or at least write a wrapper that does. That way you re-use that code rather than having to re-write it over and over.

If you write C++ with exceptions, you need this.

You absolutely do not. You need to use the core language feature that it is using: deterministically-called destructors.

kentonv5• 9 years ago

No one is suggesting this.

At least one person in this thread is, in fact, suggesting this: https://news.ycombinator.com/item?id=15524405

They are saying that you use an RAII class such that it doesn't need tear down.

Yes, as I said in my original comment: "Of course, ideally state is encapsulated in classes with destructors and tear-down happens there, but writing a whole class can often be overkill for a one-off use case. If you don't have a defer macro, you're going to get lazy and introduce bugs."

I do know how destructors work, thanks.

evincarofautumn2• 9 years ago

Side note, that DEFER macro should be variadic:

    #define DEFER(...) ScopeGuard CONCAT(_defer, __LINE__) = [&] () { __VA_ARGS__ ; }

Otherwise commas will be problematic—it’ll be treated as too many arguments to the macro if they’re not wrapped in parentheses. The reason being that the preprocessor understands parentheses as grouping delimiters, but not other bracket characters. Also note that I changed the double underscores to a single underscore, since the former is reserved for use by the implementation & standard library.

It’s this type of constant minor language-lawyering and clerical work that makes me both very good with C++ and very averse to using it anymore. (Though to be fair this particular case is also a problem with C, which I like nevertheless.)

Implementing Go's defer keyword in C++

Related Stories