The Defer Technical Specification: It Is Time

https://thephd.dev/c2y-the-defer-technical-specification-its-time-go-go-go

by mattjhall • 1 year ago

131 64 1 year ago

Mond_• 1 year ago

Seems like a perfect fit for C, and glad to see we're trying to avoid stepping into that funny pitfall Go has with its function-scoped defer keyword.

Glad to see C is evolving and standardizing.

codr73• 1 year ago

I've been doing properly scoped defers in C since forever, as long as you have access to cleanup attributes and nested functions it's no big deal.

https://github.com/codr7/hacktical-c/tree/main/macro

wahern• 1 year ago

Yes, the proposal is tailored so that other than simple syntax support no new semantics need to be implemented within GCC to support defer, though clang will need to finally add support for nested functions--in spirit if not the literal GCC extension.[1] The proposal also gives consideration to MSVC's try/finally to minimize the amount of effort required there to support defer.

[1] Because defer takes a block, not a simple statement. And deferred blocks can be defined recursively--i.e. defer within a defer block.

fuhsnn2• 1 year ago

the proposal is tailored so that other than simple syntax support no new semantics need to be implemented within GCC

Not just GCC, but you're right it's tailored, to the same "unwinding" queue that C++ destructor, stack-VLA de-allocation and __attribute__((cleanup)) shared, won't fit into the current state of language otherwise.

Clang share more frontend between C and C++ so I imagine they can implement it as hidden C++ lambda scope-guards, the nested scenario is just full-capturing lambdas inside another.

saagarjha2• 1 year ago

Please don’t use nested functions; they’re a security nightmare.

codr73• 1 year ago

Curious: why?

Animats3• 1 year ago

Ugh.

Go's "defer" is reasonably clean because Go is garbage-collected. So you don't have to worry about something being deleted before a queued "defer" runs. That's well-behaved. This is going to be full of ugly, non-obvious problems.

Interestingly, it's not really "defer" in the Go sense. It's "finally", in the try/finally sense of C++, using Go-type "defer" syntax". This mostly matters for scope and ownership issues. If you want to close a file in a defer, and the file is a local variable, you have to be sure that the close precedes the end of block de-allocation. Most of the discussion in the article revolves around how to make such problems behave halfway decently.

"defer" happens invisibly, in the background. That's contrary to the basic simplicity of C, where almost nothing happens invisibly.

codr73• 1 year ago

The point of defer is to put the cleanup logic in one place for local variables though, so the risk of someone else deleting it isn't a thing.

jayd16• 1 year ago

It's "finally", in the try/finally sense of C++

What sense is that? C++ doesn't have finally and the article explicitly calls out how its not like destructors.

Animats3• 1 year ago

Right, C++ doesn't have "finally". But "defer" defers to when a "finally" section would run in a language that has it. I think.

Jtsummers3• 1 year ago

The central idea behind defer is that, unlike its Go counterpart, defer in C is lexically bound, or “translation-time” only, or “statically scoped”. What that means is that defer runs unconditionally at the end of the block or the scope it is bound to based on its lexical position in the order of the program.

The only reasonable way for defer to behave. Function scoped never made sense to me given the wasted potential. The demonstration with loop and mutex being a good one.

chrsig2• 1 year ago

yeah, I've always just extracted the loop body into a new function as a result

infogulch2• 1 year ago

I've created a lambda and called it inline to force lexically scoped defer semantics. Works fine and reads fine imo.

gblargg2• 1 year ago

Block-based defer is also important when using macros that inserts blocks. They can use defer without care for how nested they are invoked.

topspin2• 1 year ago

Regarding the statements on golang's defer:

  "the defer call is hoisted to the outside of the for loop in func work"

Astonishing. Add that to the list of golang head scratchers. That is one of the biggest "principle of least astonishment" violations I've ever seen.

Disclaimer: Not a golang hater. Great language. Used it myself on occasion, although I remain a golang neophyte. Put away the sharp objects.

dgunay• 1 year ago

It's incredibly ugly but you could sort of hack in a smaller-scoped defer using anonymous functions: https://go.dev/play/p/VgnprcObPHz

topspin2• 1 year ago

Yeah, I get it. There's an idiom. Still, that glitch is guaranteed to catch everyone off guard, experienced or otherwise, when taking up golang. As I said, it's an entry on the list, and such a list exists for most (all?) mainstream languages. At least it's minor compared to nil, a flaw somehow promulgated in a brand new language many years after anyone purporting to be a language designer would or should have known to avoid. That's a mystery for the ages right there.

hinkley2• 1 year ago

I love the Principle of Least Astonishment, but I first encountered it in the Ruby book and I gave up reading it halfway through because I kept thinking, "He and I have very different definitions of astonishing..."

Etheryte• 1 year ago

In a way that makes sense though, once you're at a level where you can not only write Ruby, but write books about Ruby, surely very few things would astonish you.

majormajor2• 1 year ago

It's definitely a footgun but I think it's also pretty clear in go docs that defer is a function-return-time thing, versus a loop iteration thing. "A defer statement defers the execution of a function until the surrounding function returns." from https://go.dev/tour/flowcontrol/12

I think per scope-level is probably better, but honestly still - as a I mention elsewhere - still something that seems fairly limited compared to writing code inside blocks that clean themselves up in the Ruby world. The more we're messing with scope, the more it seems like it would be possible to go all the way to that? The go-style defer appears likely to be simpler from an implementation POV; if we're gonna make it harder let's go all the way!

I know a lot of people hate the nesting of indentation from that, but it makes so many other things harder to screw up.

wruza• 1 year ago

My biggest astonishment is how people continue to shoot themselves in the foot by not making scope vs function declarations explicit. For the reasons that “someone will misunderstand complicated ideas, so let’s make it implicit” or something. While there could be just:

  defer x // scope scoped
  defer fn x // function scoped

Also:

  var a = 0
  fn var a = 0
  for fn i := …

But we have this allergy to full control and invent these increasingly stupid ways to stay alert and get unpleasantly surprised anyway.

Edit: Same for iifes. Everyone uses them, so turn these into proper embedded blocks!

zombot2• 1 year ago

I wouldn't want my life turned into an embedded block, whatever that is.

rollulus2• 1 year ago

Say that the defer would execute inside for loops, what would make you more astonished: loops and functions are the exceptions, or defers execute at the end of any block? I would prefer the latter of these two. But then the consequence is that a defer in an if-block executes instantly, so you cannot conditionally defer anymore. So it seems that the rules for when deferees execute need to be arbitrary, and "only functions" seems fewer exceptions than "only functions and loops", isn't it? And what about loops implemented through gotos? Oh boy.

lmm2• 1 year ago

a defer in an if-block executes instantly, so you cannot conditionally defer anymore.

Of course you can, using ?: (or && and || if you prefer), just like any other case where you want an expression rather than a statement. Or simply using the non-block form of if. (Some stupid autoformatters or tech leads insert extraneous braces, but you should be avoiding those already).

Someone• 1 year ago

so you cannot conditionally defer anymore.

I think you can, for example this way:

- declare a function pointer variable cleanup - initialize it with no_op - call defer ‘call cleanup’ - if, inside a block, you realize that you want to do something at cleanup, set cleanup to another function

That’s more code, but how frequent is it that one wants to do that?

One thing this doesn’t support is having a call into third party code defer cleanup to function exit time. Does golang supports that?

throw-qqqqq• 1 year ago

Another cool difference between this and Go’s ‘defer’, is that it doesn’t allocate memory on the heap. Go’s ‘defer’ does and it has a small performance cost compared to just calling the .release() or whatever yourself… shrugs

At least this was the case last I did benchmarks of my Go code. Dno if they changed that.

klodolph• 1 year ago

Does go’s defer allocate on the heap? I thought it would only do that if necessary.

pkaye• 1 year ago

I know they implemented an optimization back in go 1.13. Not sure if that will help.

https://github.com/golang/proposal/blob/master/design/34481-...

zyedidia• 1 year ago

What is the recommended way to use defer to free values only on an error path (rather than all paths)? Currently I use goto for this:

    void* p1 = malloc(...);
    if (!p1) goto err1;
    void* p2 = malloc(...);
    if (!p2) goto err2;
    void* p3 = malloc(...);
    if (!p3) goto err3;

    return {p1, p2, p3};

    err3: free(p2);
    err2: free(p1);
    err1: return NULL;

With defer I think I would have to use a "success" boolean like this:

    bool success = false;

    void* p1 = malloc(...);
    if (!p1) return NULL;
    defer { if (!success) free(p1) }

    void* p2 = malloc(...);
    if (!p2) return NULL;
    defer { if (!success) free(p2) }

    void* p3 = malloc(...);
    if (!p3) return NULL;
    defer { if (!success) free(p3) }

    success = true;
    return {p1, p2, p3};

I'm not sure if this has really improved things. I do see the use-case for locks and functions that allocate/free together though.

lelanthran• 1 year ago

I don't even bother with `error1`, `error2`, ... `errorN`.

I initialise all pointers to NULL at the top of the function and use `goto cleanup`, which cleans up everything that is not being returned ... because `free(some_ptr)` where `some_ptr` is NULL is perfectly legal.

ayende• 1 year ago

That is a well structure system, yes Both cleanup for error and allocation happens in the same place

That means you won't forget to call it, and the success flag is an obvious way to ha dle it

loeg2• 1 year ago

Can also use a different variable name for the success case and null out any successfully consumed temporaries.

    void* p1 = malloc();
    if (!p1) return failure;
    defer { free(p1); }
    ...
    someOther->pointer = p1;
    p1 = NULL;
    return success;

bobmcnamara• 1 year ago

I'm not sure I'd do either for this trivial case, but it might make sense where the cleanup logic is more complex?

    void* p1 = malloc(...);
    void* p2 = malloc(...);
    void* p3 = malloc(...);

    if(p1 && p2 && p3)
      return {p1, p2, p3};

    free(p3);
    free(p2);
    free(p1);
    return NULL;

lukaslalinsky• 1 year ago

Ever since I started working with Zig, I came to realization that its errdefer is even more useful than defer itself. But you can't implement errdefer in C, since there is no standard/disambiguous way of returning errors.

infogulch2• 1 year ago

Can you expand on how errdefer works in zig? I'm not familiar.

Jtsummers3• 1 year ago

https://ziglang.org/documentation/master/#errdefer

defer always executes on scope exit, errdefer executes on an error exit. In principle, this is similar to the logic of a try/catch/finally:

  try {
    // whatever
  } catch {
    // errdefer would belong here
  } finally {
    // defer would happen here
  }

loeg2• 1 year ago

Zig has a special / compiler-known ADT for "value OR error". This is similar to Result<T,E> in Rust. Or in C++, e.g., folly::Expected<T,E>.

The Zig one is so special and compiler-blessed that there is special syntax for defer blocks that only run when the function return is an error variant of that result ADT -- errdefer.

fwlr• 1 year ago

The author takes great care to rebut a common theme among objections to the proposal - “this isn’t necessary if you just write code better”. I am reminded of this fantastic essay:

If we flew planes like we write code, we’d have daily crashes, of course, but beyond that, the response to every plane crash would be: “only a bad pilot blames their plane!”
This doesn’t happen in aviation, because in aviation we have decided, correctly, that human error is an intrinsic and inseparable part of human activity. And so we have built concentric layers of mechanical checks and balances around pilots, to take on part of the load of flying. Because humans are tired, they are burned out, they have limited focus, limited working memory, they are traumatized by writing executable YAML, etc.
Mechanical processes are independent of the skill of the programmer. Mechanical processes scale, unlike berating people to simply write fewer bugs.

(https://borretti.me/article/introducing-austral#goals)

smadge• 1 year ago

I agree with that, but:

- the language still allows you write the unsafe version even with defer. By your logic fallible humans will continue to write these class of bugs because they can. - adding a whole new flow control construct will introduce a whole new class of bugs. The dog barking example is cool for demonstrating how defer works, but is completely unreadable for what it does, programmers will write code like that because they are allowed to, and unreadable code becomes buggy code. - to make a language safer you should remove the things that make unsafe behavior possible, not add constructs which make safe behavior easier.

globnomulous• 1 year ago

is completely unreadable for what it does

Maybe I'm putting too much emphasis on "completely unreadable" rather than the rest of the quotation, but I find the example crystal clear, and I'd never expect code intended to illustrate, clearly and loudly, language features to read naturally.

to make a language safer you should remove the things that make unsafe behavior possible, not add constructs which make safe behavior easier.

Some of this guy's other (equally superb) blog posts explain why this isn't an option: it breaks decades' worth of C code, and the C standards group is strongly committed to ensuring that C that compiled 20, 30, 40 years ago continues to compile.

Regardless, I find it incredibly weird to read the statement "you should [...] not add constructs which make safe behavior easier," no matter the contents of "[...]." If your goal is to improve the security of a programming language or the maintainability of code in that language, and you don't want breaking changes, this isn't just your best option. It's your only option, I think.

majormajor2• 1 year ago

Defer is often quite nice when you have many return paths but it also seems fairly limited if our goal is to assume human error is unavoidable. Ruby block-based stuff or "with" in Python seem like the clear winner there.

zombot2• 1 year ago

they are traumatized by writing executable YAML

It gives me solace to know that I am not alone.

ChrisMarshallNY• 1 year ago

Pretty much the only time I use it, is if the act of doing some cleanup might cause a change (like a mutable function in a communication API, or letting go of a reference may interfere with a last operation).

Generally, I find it isn’t necessary. I can usually figure out a way to make it work with standard flow control.

In my case, it’s the Swift language.

Wumpnot• 1 year ago

Can't really blame MS for saying ..just use C++.. they aren't exactly wrong.

neilv• 1 year ago

Here’s a basic example showing off some of its core properties

Why not make the string literals in the code identify their positions in the output, to expose the behavior, rather than obfuscate it?

Then the reader only has to work through the code, to see why it would have that order.

It currently looks like a puzzle intended to be harder for the reader to understand than it needs to be.

hyperhello2• 1 year ago

I was thinking it might be clearer with defer printf("2"); printf("1"); for example.

gblargg2• 1 year ago

Agreed, the example immediately made me see it as an example for the Obfuscated C contest.

fuhsnn2• 1 year ago

You can play with defer in Linux/VM with slimcc[1] today! It only diverges from the TS in keyword being _Defer, as well as several goto constraint violations not detected, bright side is you can witness why they are constraint violations...

[1] https://github.com/fuhsnn/slimcc

hyperhello2• 1 year ago

What does this return?

  int x = 1;
  defer x = 2;
  return x;

gizmo6862• 1 year ago

That will return 1. The defered code is executed after the return value is computed. This lets you do things like:

  char *str = foo();
  defer { free(str); }
  return strlen(str);

Jtsummers3• 1 year ago

https://thephd.dev/_vendor/future_cxx/technical%20specificat...

Right, there's a demonstration of GP's question (or a variation) on page 10 of the draft technical specification.

sbrudenell• 1 year ago

I always thought golang's defer was a readability nightmare because it obfuscates execution order. OP's "basic example" is ... a great example of obfuscation. try/finally doesn't have this problem. It can add indents, but I'd so much rather read a function with 4 indents than 4 defers

aeijdenberg• 1 year ago

The TS doesn't seem to provide for a way to modify return values for the function. For example the following is a common pattern in Go using defer to ensure that errors closing a writeable file are returned:

    func foo() (retErr error) {
        f, err := os.Create("out.txt")
        if err != nil {
            return fmt.Errorf("error opening file: %w", err)
        }
        defer func() {
            err := f.Close()
            if err != nil && retErr == nil {
                retErr = fmt.Errorf("error closing file: %w", err)
            }
        }()
        _, err = f.Write([]byte("hello world!"))
        return err
    }

The Defer Technical Specification: It Is Time

Related Stories