Perils of Constructors

https://matklad.github.io/2019/07/16/perils-of-constructors.html

by jasonpeacock • 7 years ago

197 53 7 years ago

matklad.github.io

meddlepal• 7 years ago

I don't really feel like this is an argument against constructors but perhaps an argument that some language designs are better than others...

For example Go does not have constructors and it's very easy to get a nil deref error if you aren't careful.

stinos2• 7 years ago

I don't think the author's intent was to argue completely against cnstructors. At least to me this reads more like: here are some pros, here are some cons. But you're right: in some languages some aspects are better than in others. Take the vector constructor example for instance: that's not really just a case against constructors. The exact same, i.e. not knowing which argument does what, can be a problem in any language with no named arguments, and not just for constructors but for anything which takes arguments. But is that a problem? Do I really need to know the order of arguments of everything? Nope, it's 2019, my editor and internet know the answer.

Sharlin• 7 years ago

The point is that with (unnamed) constructors the problem is exacerbated. There’s a reason functions are named descriptively.

skybrian• 7 years ago

Although it doesn't have constructors, Go has zero values. I can't find a reference, but I believe this is so that you can create arrays with any element type and they can be automatically initialized?

Naturally, if pointers must have a zero value (like any other type), nil is a logical choice.

The pervasive support for zero values (or not) seems like a pretty fundamental design choice with far-reaching consequences. It works okay when there is a logical use for a zero value (as encouraged in Go) and is awkward otherwise.

If you don't have zero values, you end up with something like Rust's array initialization syntax:

https://www.joshmcguigan.com/blog/array-initialization-rust/

khold_stare• 7 years ago

I completely agree with most of the post. In C++ I use the static factory method "trick" mentioned in the post, when constructing subobjects may throw or result in some kind of error. Throwing from the middle of a constructor in C++ is a quagmire, so it's best not to do it. Bonus, you can mark the constructor and factory functions noexcept.

The part I disagree with relates to "relying on the optimizer for placement". Even in C++ using the above factory pattern, you are returning the constructed object from a function - and there is no problem if it is ultimately part of some larger object. The C++ standard specifies copy-elision very precisely so you don't have to hope the optimizer does it - it is required to. To demonstrate you can do stuff like this even if you object contains non-moveable members, like std::mutex

  class Foo
  {
  private:
    std::mutex mutex_;
    SomeComplexSubObject sub_;

    Foo(SomeComplexSubObject sub) noexcept
      : sub_{std::move(sub)}
    { }

  public:
    static std::optional<Foo> make(SomeParams params) noexcept
    {
      try {
        return Foo{SomeComplexSubObject{params}};
      }
      catch (std::exception const& e) {
        return std::nullopt;
      }
    }
  };

I think Rust can also specify something like this (ie. "copy-elision") as part of its unwritten spec. Anyway, great article! :)

stinos2• 7 years ago

Throwing from the middle of a constructor in C++ is a quagmire, so it's best not to do it

https://isocpp.org/wiki/faq/exceptions#ctors-can-throw for one disagrees, but it might not cover what you mean exactly. So: also in modern C++ (i.e. using RAII types)? Or do you mean other problems than leaking, do you have an example?

UncleMeat• 7 years ago

Even though cpp11 has been around a while, there is a ton of code that isn't using smart pointers everywhere. This makes exception handling messy, especially with partially constructed objects.

kibwen2• 7 years ago

Rust's informal spec actually does guarantee return-value optimization (at least, it did last time I checked; the perils of being informal) so that data returned by-value is constructed in-place in the stack of its caller, but the wrinkle is that it has to be able to determine what data is going to be the return value; simple cases like returning a record literal directly (without first creating any intermediate values that might want to live in the return value) are easy, but if you do more complex operations in your factory function then you do have to start putting faith in the optimizer. Suggestions about adding "placement" semantics to Rust are concerned with allowing the user to write code guaranteeing such behavior.

Thorrez2• 7 years ago

Is the spec specific about which cases it's required and which cases it's optional? Because simply saying "it's required if the compiler can determine it" is quite open ended.

matklad3• 7 years ago

There are cases where copy elision doesn't work (don't know how important is that in practice).

One representative example would be something like this (I think this can't use copy-elision, but I am not an expert in C++, please correct me if I am wrong!)

    void f(std::vector<Foo>& xs) {
        xs.emplace_back(arg1, arg2);
    }

Here, the semantics is that xs is resized first and then the constructor for Foo(arg1, arg2) is called. So, if resizing xs throws, no Foo is constructed at all. If we replace emplacement with push_back and a factory function, then the semantics becomes 1) construct Foo 2) extend the vector 3) copy/move Foo into the vector.

Thorrez2• 7 years ago

I don't understand the emplace_back example. I don't see any copies (or moves) happening there at all, so yes, no copy elision can happen. But it doesn't make sense to want copy elision when you already have no copies.

But yes, I don't think push_back can do full copy elision, but I don't think there's a real need, because it can use move instead of copying.

kbwt• 7 years ago

In your example, how do I embed a Foo in my FooAndBar object?

mothsonasloth• 7 years ago

Nowadays I use builders with validation and defaults. It makes life so easier when creating instances or testing them.

This is for c and Java. Not sure how the builder pattern would work for Rust.

kibwen2• 7 years ago

Many Rust libraries actually employ the builder pattern extensively (sometimes as a reaction to the lack of default/keyword arguments); it's so widely-accepted that the necessity of error-handling while constructing items via method-chaining (as featured by the builder pattern) was a key argument for the postfix `?` error-propagation operator.

hobofan2• 7 years ago

The builder pattern works just as well in Rust, and is in fact used pretty commonly in the ecosystem whenever you want to create a struct that has quite a few configuration options, but you don't want to specify all of them (Rust doesn't have default argument values).

munchbunny• 7 years ago

In my experience with C/C++/C#/Java I pretty much always use factory methods and hidden constructors for complex object construction in order to hide the constructor semantics and failure modes behind a standard function abstraction. It has the added bonus of being more testable.

ape4• 7 years ago

Constructors can definitely be misused. But they can be handy too. What if the initial values of the members are just constants. Perhaps it takes a bit of computing (not too much) to set them up.

exDM69• 7 years ago

What if the initial values of the members are just constants

In idiomatic rust

    impl Thing {
        fn new() -> Self {
            Thing { ival: 13, fval: 42.0 }
        }

Another option is to use Default::default if appropriate.

davesmith1983• 7 years ago

C# has `readonly` variables. The values of these cannot be changed after the constructor has finished running.

mattnewport• 7 years ago

I don't find this that useful in C# since the majority of types in C# tend to be reference types and you can still change the values unless they are immutable types which though considered good practice aren't that common in most C# code in my experience.

spankalee• 7 years ago

Dart fixes this with initializer lists, which must run for every class in the hierarchy before any constructor bodies. Initializer lists have restricted access to `this` so you can't call methods or pass `this` to functions.

agumonkey3• 7 years ago

I often think that Dart is a failure.. but every time I read about it, they have interesting ways to do things. Maybe it's yet another language that only exists to die alone and spread its gene for the next decade to pick up ?

aembleton• 7 years ago

I think Flutter is helping to keep Dart alive - https://flutter.dev/

thosakwe2• 7 years ago

Dart isn't dead, though. Its use has been increasing because of Flutter. Beyond that, there are still new features coming out with every release, so I don't really think dead is the best word to describe it at this point.

markdog12• 7 years ago

It also has named constructors, factories, const constructors for compile-time constants.

https://dart.dev/guides/language/language-tour#constructors

klodolph• 7 years ago

This is kind of a pattern with “design flaws in C++” if you’ll excuse the phrasing. C++ conflates a lot of different concerns with each other, so you end up with some reasonable objective (enforce invariants) and some reasonable way of achieving that objective (constructors & destructors) and it works well enough 90% of the time but the other 10% of the time it's a problem. Then there's a ton of design patters that crop up to manage the 10% of use cases where the language doesn’t quite give you the right tools for the job.

For example, access control. In C++, access control boundaries are classes, which is a reasonable choice except there are a ton of situations where this is the wrong choice, so you have “friend”. Some C++ designers will tell you “friend” is a code smell, which is true, but the fact is you can’t always avoid it. In languages where access control is defined relative to modules this is rarely a problem. So I say that C++ conflates access control boundaries with class boundaries.

Another example is syntactical blocks and variable lifetime. Object lifetime is an important concept in C++, but object lifetimes often don’t line up with the extent of syntactical blocks, even if it doesn’t make sense for the objects in question to have dynamic storage duration.

My hot take here is that C++ is a very opinionated language, in the same sense that Go and Python are opinionated languages, it’s just that C++, Go, and Python have strong opinions about different aspects of the language. Another difference is that people in school falsely equate traditional object-oriented design with good design. This makes sense, because it’s much easier to teach object-oriented design than it is to teach good design.

adrianmonk2• 7 years ago

The footgun argument is not applied very consistently here.

In other languages, it's an argument against constructors. You might do things that are bad like have members with default values or call methods on an object that isn't ready. You could avoid doing those things by establishing conventions.

For example, the default values thing can be avoided in C++ by using member initialization lists. Or in Java, use final fields and definite assignment (https://docs.oracle.com/javase/specs/jls/se10/html/jls-16.ht...) will protect you from default values.

But the requirement to maintain those conventions is an unreasonable burden when it comes to constructors.

However, when it comes to maintaining conventions to avoid other issues, Rust then gets a free pass:

A perceived downside of this approach is that any code can create a struct, so there’s no the single place, like the constructor, to enforce invariants. In practice, this is easily solved by privacy: if struct’s fields are private it can only be created inside its declaring module. Within a single module, it’s not at all hard to maintain a convention like "all construction must go via the new method".

And it even goes on to say "One can even imagine a language extension" for Rust. Fair enough, but in languages that use constructors, one could imagine language extensions too. Default values for fields could be an explicit opt-in thing. Or calling methods on a not-fully-built-yet object could be banned or could require an explicit syntax.

fluffything• 7 years ago

FWIW I disagree with the author in that, within the module that declares a struct you must follow a convention in Rust. You don't. You have full access to the struct private implementation details and can do whatever you want.

If you care about privacy, you'd just keep your modules small.

matklad3• 7 years ago

Sorry if the post sounds like I am trying to argue that one approach is inherently better than the other. It is not my goal. Rather, I want to show the amount of language level machinery (extensive static checks, or runtime nullability) that is required if the language has constructors.

adrianmonk2• 7 years ago

OK, that's very reasonable. I think you make some good points that constructors can be problematic. Maybe they need to be reinvented or something.

Just ditching them entirely as Rust does seems like an honest attempt at moving things forward, though I don't really see it as the long term solution because neither approach is all that great when you consider all the weaknesses that have been pointed out with both.

couchand• 7 years ago

I doubt there will ever be a language that doesn't have its own conventions and patterns. The practical question here is the risk when the convention is missed. In Rust, the object cannot be half-initialized when the pattern is missed. And not using public fields is rightly the default mode in all three...

rzwitserloot• 7 years ago

I haven't written it at all, but the Pony language showed up here on hackernews a while ago, and it has the kotlin solution except the notion that the variable isn't yet initialized is type-carried there. You couldn't 'fake out' the process and observe a null on a non-null variable there.

In other words, with some type system antics you can have your cake and eat it too, and that makes 'rust did this right', without also tacking on a deep dive on why pony's approach also has downsides, disingenuous or ill informed. (which I'm sure it does; my point is, this article doesn't cover it).

matklad3• 7 years ago

Sorry if the post reads like “rust did this right”: my intention was to show different trade offs around static/dynamic checking of constructors. I didn’t mean to argue that certain approaches are right, and others are wrong.

In particular, the section about Swift shows that you can fully statically checked constructors if you dedicate enough language machinery to it.

I don’t know a lot about Pony, but it seems like it uses a strict subset of Swift’s rules? There’s no inheritance so two phased initialization is condensed to “don’t call methods until all fields set”. It’s also not possible to call one constructor from another, so designated/convenience constructor split is also absent.

scubbo3• 7 years ago

I don't have a CS background, so apologies in advance if these are basic questions, but:

For this layout to work though, constructor needs to allocate memory for the whole object at once. It can’t allocate just enough space for base, and than append derived fields afterwards. But such piece-wise allocation is required if we want a record syntax were we can just specify a value for a base class.

1. Why can the constructor not allocate memory "progressively" for the object as the construction chain descends the class hierarchy? Is it because, in a multi-threaded program, something else might allocate some of the "extended" memory, causing a clash when the constructor attempts to append fields? I assume that an approach of "if the memory that a constructor is trying to append into is already allocated, first move the occupying object to a free memory location, and then continue with allocation" is inefficient because it relies on an "overseer" that can coordinate and resolve these clashes?

1a. Sub-question - why does the memory for an object need to be contiguous? Is this purely an efficiency concern ("read a whole object sequentially" being more efficient than "read an object by reading a bunch of pointers and then reading the locations they point to"), or are there other considerations? I was under the impression that RAM (unlike hard drives) has no (or, negligible) performance penalty to random access - but maybe those intermediate "jumps" add up to a measurable impact?

2. "But such piece-wise allocation is required if we want a record syntax were we can just specify a value for a base class." I don't understand this claim at all. Why is this required? What does it mean to "specify a value for a base class" - is this shorthand for "specify a value for a field of a base class"?

Recommendations for further reading are received just as gratefully as direct explanations - I'm sure these concepts are already covered in the literature or a course syllabus, but I have no idea where to start!

firethief2• 7 years ago

If you want to understand this article you'll need to learn a programming language like C. Once you know what the stack, the heap, and records/fields are you'll be able to ask meaningful questions.

morelisp2• 7 years ago

It's weird to see a recommendation to learn C when 'scubbo's questions already indicate some knowledge of memory management - far more than you will learn "learning C". The suggestion to "allocate memory progresively" and not-necessarily-contiguously resembles Lisp cons cells, and the questions about how to handle conflicts are the first steps to inventing (and then finding the downsides of) https://en.wikipedia.org/wiki/CDR_coding.

The second question is also reasonable, "a value for a base class" is somewhat ambiguous - if I read the post right it means specifying a value for the derived class's inherited fields by using an instance of the base class without copying, but in languages with more dynamic approaches to dispatch it would also be reasonable to talk about reassigning the base class itself (e.g. MRO tricks in Python). And "such piece-wise allocation" is indeed possible in various Lisp-inspired object systems.

brianberns• 7 years ago

An object has to occupy a contiguous region in memory in order for dereferencing to work efficiently. We need both "this.baseField" and "this.derivedField" to directly access the desired bytes. If either of those has to chase through a chain of memory regions, performance will be much worse.

morelisp2• 7 years ago

Statically-known field offsets are at least as important as contiguous memory; you can have either without the other, though a major "achievement" of classical OO is that a single-inheritance class gives you both without really thinking about it.

aw1621107• 7 years ago

I'm no expert, so take my answers with a grain of salt.

Why can the constructor not allocate memory "progressively" for the object as the construction chain descends the class hierarchy?

I think it's a matter of allocating everything up-front being easier than the alternative. Even with a single-task OS/kernel and a single-threaded program, memory fragmentation could mean that a chunk of memory that is large enough to start construction wouldn't be large enough to finish it. As you mentioned, the OS/kernel could move the object under construction to a new chunk of memory, but there's no guarantee that the new chunk would be large enough, and performing said move is extra work that is relatively easy to avoid in the first place.

Sub-question - why does the memory for an object need to be contiguous?

I think it has to do with a combination of predictability and efficiency more than it being technically infeasible.

The predictability side is probably most important for systems programmers, who are also the ones who need to worry the most about allocation behavior. For them, control over object layout in memory can be important, and the choice between using "normal" members instead of pointers can be quite deliberate. Splitting objects across multiple allocations would necessitate inserting/dereferencing pointers regardless of what was written in the source code. This makes object size, memory access patterns, and potentially performance quite unpredictable.

There are several aspects to the efficiency component:

- The compiler cannot use fixed offsets to access fields of an object if the object can be split across multiple allocations at runtime.

- There is additional bookkeeping required on the runtime (OS/kernel) to ensure that a split objects gets allocated/deallocated correctly.

- Splitting an object across multiple allocations would require pointers to be inserted/dereferenced at some point or another. While you are correct in stating that RAM allows for approximately the same speed in accessing arbitrary memory locations, the same cannot be said for the CPU cache. Memory is quite slow compared to the CPU cache, and the cache is usually very limited in space compared to RAM. Thus, for optimal performance, you typically want predictable access patterns that are located in "close" memory areas. Pointers usually make doing so difficult, if not impossible.

- Object size is no longer predictable, which is very problematic for more memory-constrained devices (or if you want to pack as much data as possible into cache).

- Probably more I'm not thinking of at the moment

"But such piece-wise allocation is required if we want a record syntax were we can just specify a value for a base class." I don't understand this claim at all. Why is this required? What does it mean to "specify a value for a base class" - is this shorthand for "specify a value for a field of a base class"?

I'm guessing that they mean that they want the ability to specify the values for the base class (or the fields of the base class) without having to allocate memory for the unused field(s) of the subclass(es) while still retaining the ability to specify subclass fields later.

scubbo3• 7 years ago

Fascinating - thanks for your input! I wasn't aware of the CPU cache's extra limitation - that makes a lot of sense. And, yes - as you say, it makes perfect sense that having all the data of an object colocated (and at predictable offsets from one another) would make for a more efficient operating experience.

panic• 7 years ago

I think the article is wrong here. This is the code it says won't work:

    struct Base { ... }

    struct Derived: Base { foo: i32 }

    impl Derived {
        fn new() -> Derived {
            Derived {
                Base::new()..,
                foo: 92,
            }
        }
    }

You don't need to allocate memory progressively to support initializers like this. You could allocate the entire Derived struct, copy the return value of Base::new() into the beginning of the allocated struct, then initialize the rest of the fields. If you wanted to avoid the copy (and deal with complications like internal pointers), you could even make Base::new() write its return value directly into the Derived struct's memory (like C++'s "return value optimization" does).

TazeTSchnitzel• 7 years ago

PHP 7.4 adds optional type declarations for properties on an object, and so there was the problem of how to avoid an inconsistent state (type not matching before initialisation). The route that was eventually taken was to have the properties not be null, but be unset. If you try to read from them at runtime without having written to them first, an error is thrown. Notably, there's no requirement everything be defined when the constructor exits.

broth• 7 years ago

Any programming language or scripting language can be misused — especially when it comes to misusing constructors or some other mechanic. I think it boils down to knowing these language pitfalls and establishing standards to follow and help guard against them.

kazinator• 7 years ago

The easiest answer is to set all fields to default values: booleans to false, numbers to 0, and reference types to null. But this requires that every type has a default value, and forces the infamous null into the language. This is exactly the path that Java took: at the start of construction, all fields are zero or null.

I don't think this forces us to have a null.

Firstly, if an object has another one as a member, then we just recursively default-construct that member. If the member is optional (e.g. next node of a linked list that may not be there) that can be addressed with a sum type, like a "maybe" type, whose default value for construction can be the "not there" variant of the type. (If that is considered morally equivalent of a null, I don't know what to say; but it's certainly not there because of default construction, but because we wanted linked lists that somehow terminate, and it makes sense for construction to produce a node that has no next node.)

The concept of a default value isn't problematic at all. Every type has a set of values in its domain. If that domain is empty (like the nil type in Common Lisp at the bottom of the type spindle), then the default value of an object of that type is that only possibility: to have no value. If the domain has exactly one value, we have no choice but to establish that value as the default. Otherwise, we can designate one of the two or more values as the default.

In Rust, there’s only one way to create a struct: providing values for all the fields.

If that's the design, we could require the programmer to specify that default literal when the type is defined. Then that literal is used by default construction. Problem solved.

If we can have literals, we can have always have default construction, if we inconvenience the programmer to supply us the literal that is to be used for it.

hobofan2• 7 years ago

I agree that it doesn't really force you to have a null. I found that line on reasoning especially surprising, as Rust also gets around this with sum types and does have a `Default` trait in the standard library.

I do however think that beyond the empty/one element domains you mentioned, default values can be a bit problematic. An acceptable default value for a type is not really determined by the type itself but in what context it is used.

Imagine a config struct with multiple boolean flags:

    struct SomeConfig {
      feature_a: bool,
      feature_b: bool,
      feature_c: bool,
    }

Would it be a good idea to use the default value `false` for all of them? I think in a lot of cases like these it is very much preferable to force the programmer to provide values for all the fields.

If that's the design, we could require the programmer to specify that default literal when the type is defined. Then that literal is used by default construction. Problem solved.

Rust does kind of have a way to do it, though it's a bit more explicit. If you have e.g. `std::default::Default` (which would be the conventional trait for that) implemented, you can easily create a struct without needing to specify the default fields:

    let instance = SomeStruct {
      foo: 1,
      ..SomeStruct::default(),
    }

ncmncm• 7 years ago

Not buying this argument, not at all.

In the constructor, there is no object yet. You have a bunch of subobjects with no relationship besides proximity. The constructor is already a confined space analogous to the "module" recommended in the article. Its job is to tie the subobjects up into an object, and it is a great good that there is a specific language construct for this purpose.

It is true that you need to be careful, in the constructor, not to call members that assume class invariants have already been established while you are still establishing them. But nobody forgets they are coding construction, when doing it. The article invents a non-problem, and then a solution that solves nothing -- apparently just because there is no other choice in Rust.

Weak thesis, weak argument. Rust has strengths, but lacking constructors is not one. To present failing arguments risks suggesting that no better arguments for your favored language are available.

Perils of Constructors

Related Stories