NULL: The worst mistake of computer science? (2015)

https://www.lucidchart.com/techblog/2015/08/31/the-worst-mistake-of-computer-science/

by BerislavLopac • 8 years ago

209 368 8 years ago

kekzzz• 8 years ago

Why can't people just get over it and stop blaming the language for their own sloppy code?

Because everybody is working on the limit of complexity they can comprehend, so there is no headspace left for dealing with bad ergonomics. We need all the help we can get.

jstimpfle10• 8 years ago

everybody is working on the limit of complexity they can comprehend

That's the problem right there.

timvisee• 8 years ago

Well, apparently it doesn't work that way.

goto114• 8 years ago

Because languages are tools, and tools should help you do your job, not hinder you.

masklinn13• 8 years ago

Why can't people just get over it and stop blaming users for the tools failing them?

thwy123212• 8 years ago

"The key point here is our programmers are Googlers, they’re not researchers. They’re typically, fairly young, fresh out of school, probably learned Java, maybe learned C or C++, probably learned Python. They’re not capable of understanding a brilliant language but we want to use them to build good software. So, the language that we give them has to be easy for them to understand and easy to adopt. – Rob Pike"

When did computer science become about hand holding? has it always been this way? Look at react. It was designed to force functional programming concepts in an OOP manner. Is the future of programming the implementation of tightly controlled interfaces with extreme type safety? I would argue thats where we are going. Things are becoming less expressive, not more.

skgoa2• 8 years ago

It's the Java approach. Don't give people footguns and force them to write software in a readable, testable, maintainable style. It works extremely well in software engineering, because you want systems that work reliably and that can be maintained/extended by any other engineer at your org. In the professional software engineering world, "clever" programmers are almost always a horrible drag on their team.

masklinn13• 8 years ago

It's not the Java approach at all. The first languages to remove ubiquitous nulls (e.g. MLs) were looking to increase expressivity and modeling abilities, not to force bondage and discipline upon developers.

coldtea2• 8 years ago

When did computer science become about hand holding?

When people with pragmatic goals want to get large teams of new programmers productive fast, and can't expect everyone to be able to fend on their own or can afford the cost of accumulated mistakes.

Look at react. It was designed to force functional programming concepts in an OOP manner.

Whatever that means, as React has little to do with "OOP manner".

thwy123212• 8 years ago

OOP meaning React.Component, functional meaning immutable html state, property inheritance, render(), etc

pjc508• 8 years ago

Prevention of user error is a good design principle: https://justuxdesign.com/blog/poka-yoke-design-the-art-of-er...

edoo4• 8 years ago

Nulls in strongly typed languages can get rather weird but from a C/C++ perspective it is the same as 0. nullptr is just a correctly casted 0.

saagarjha5• 8 years ago

nullptr is actually "a prvalue of type std::nullptr_t" or something like that. Since C++11 NULL is the same as nullptr.

edoo4• 8 years ago

And it is directly convertible to a bool. The primary effectiveness seems to be with operator overloading where a NULL could trigger an integer method instead of a pointer method.

saurik2• 8 years ago

To the small extent to which "since C++11" is even relevant to "from a C/C++ perspective", even in C++11 NULL is still allowed to simply be defined as 0.

jcelerier2• 8 years ago

Actually nullptr is a struct.

trhway• 8 years ago

reminds that https://en.wikipedia.org/wiki/Blind_men_and_an_elephant :)

grive2• 8 years ago

This is only true in C, which is part of why C is less insane than C++.

The two languages should not be conflated anymore.

mbel• 8 years ago

This is not even true in C. 0 is a "null pointer literal" when used in pointer context, this does not imply that the actual null pointer has a value of zero.

masklinn13• 8 years ago

That's not correct.

In C, the literal "0" is a null pointer constant handled at the compilation stage, but casting a runtime zero is not specified to yield a null pointer (and the address zero can be perfectly valid and usable), nor are null pointers specified to be zero-valued (quite the opposite).

edoo4• 8 years ago

I disagree. nullptr is directly convertable to a boolean 0:

#include <stdio.h>

int main(int argc, char *argv[]) {

  printf("%d\n", nullptr == 0);

  return 0;
}

jeromebaek• 8 years ago

At least C# has the syntactic sugar to easily check for null references which lets you avoid the horrors of code like `(if s != null && s.length)`. Instead you can type `s?.length`. Never have I appreciated syntactic sugar as much.

rusk15• 8 years ago

A lot of higher languages use the concept of truthiness such that null, or 0-length both evaluate to false:

    if (s) then do stuff with s;

ohazi• 8 years ago

Other candidates:

- null terminated strings

- machine dependent integer widths

saagarjha5• 8 years ago

null terminated strings

This is mentioned.

machine dependent integer widths

What exactly do you dislike about this?

rusk15• 8 years ago

What exactly do you dislike about this?

Or what do you even do about it?

A handful of solutions already exist: - Use a higher-level language - Java

pjc508• 8 years ago

It's an area of the language which can produce "works on my machine" bugs?

It becomes less important as we all converge on 64-bit machines with 32-bit "int" types, but it's still a monumental pain point for portability.

DaiPlusPlus• 8 years ago

Null-terminated strings are useful for buffering values of unknown length (e.g. copying from a network stream). Length-prefixed strings have issues of their own (e.g. what size length prefix to use, efficient encoding of variable-length integer prefixes, what happens if a length-prefixed string's calculated end-position is outside the process' memory space?)

thaumasiotes3• 8 years ago

Null-terminated strings are useful for buffering values of unknown length (e.g. copying from a network stream).

That doesn't seem like a good example. The data arriving over the network arrives length-prefixed, because 0 is a legal byte value for arbitrary data. What do you then gain by throwing away your existing knowledge of the length?

altmind• 8 years ago

I came here to write that ASCIIZ is the more costly design decision than null, that not only led to crashing programs, but also to security vulnerabilities, sloppy APIs that cannot handle "binary data" and subpar performance.

skgoa2• 8 years ago

I wonder whether the author also hates the 0 and 1 elements of natural numbers. Since they have the same flaw of having weird, special semantics that all other other numbers don't share. In fact 0 is not even a number, but a placeholder for the concept of the absence of a number. Just like NULL.

BlackFly2• 8 years ago

I can assure you that the number 0 in 0℃ is a number and not the absence of temperature.

But other than that, excellent point about the wonkiness of special values.

rusk15• 8 years ago

the number 0 in 0℃ is a number

If we're going to get pedantic ...

0 in this case is an offset. The number is what's behind it: The point at which water freezes (aka 273 °K).

In this case 0 indicates the absence of any offset.

TheOtherHobbes• 8 years ago

That's exactly the point the author isn't making - which is that there should be an explicit distinction between Exists But Holds No Value and Does Not Exist.

0 is neither. It's clearly a number which can be used in arithmetic and defines a specific integer quantity. (Unless you think 0 is identical to NaN...)

The problem is that in some languages it's used for either or both of the above logical definitions, when it shouldn't be used for either.

pjc508• 8 years ago

They have all the same semantics as regular numbers? They all derive naturally from the Peano axioms.

You may be thinking of NaN.

boomlinde• 8 years ago

No, 0 is definitely a number.

giornogiovanna7• 8 years ago

Zero's behavior is totally consistent with the other numbers, though - it doesn't break associativity, commutativity, or any of the other stuff you'd expect. On the other hand, NULL takes every type I've ever written and adds an instance whose behavior with every function is, at best, to crash my program, and at worst, completely undefined. Its behavior is not at all consistent with the other instances.

AnimalMuppet2• 8 years ago

0 breaks division, though...

saagarjha5• 8 years ago

it means that C-strings cannot be used for ASCII or extended ASCII. Instead, they can only be used for the unusual ASCIIZ.

This is a very pedantic quibble, and I'm not even sure it's correct. ASCII has NUL as well, and ASCIIZ isn't a character set AFAIK.

C++ NULL boost::optional, from Boost.Optional

First of all, nullptr, second, std::optional.

Objective C nil, Nil, NULL, NSNull Maybe, from SVMaybe

Nil is not a thing in Objective-C, to my knowledge.

Swift Optional

You're looking for nil. So it should be four stars?

Swift’s UnsafePointer must be used with unsafeUnwrap or !

You're confusing Optional and UnsafePointer.

saurik2• 8 years ago

What the comment about ASCII means is that NUL is a valid character in an ASCII string, but it can't be represented in C's null-terminated string encoding as the format (sometimes called ASCIIZ, but yeah: not an encoding... but I mean, come on... the article is clear here) terminates at the first NUL.

Also, Nil is absolutely a thing in Objective-C: it is a null pointer of type Class (whereas nil is a null pointer of type id; you should avoid mixing them up, though I will admit nothing much bad will happen as Class and id are generally co-polymorphic due to the type system being kind of lame. I am not sure they always have to be, though).

(And as someone who has been programming in C++ since before it was standardized at all, I frankly think listing NULL and boost::optional is totally acceptable and complaining about it as if C++11 is more canonical is just being annoying.)

Doing a quick search for how nil works in Swift, it apparently isn't a null pointer, so you are wrong there as well :(.

nil means "no value" but is completely distinct in every other sense from Objective-C's nil.
It is assignable only to optional variables. It works with both literals and structs (i.e. it works with stack-based items, not just heap-based items).
Non-optional variables cannot be assigned nil even if they're classes (i.e. they live on the heap).
So it's explicitly not a NULL pointer and not similar to one. It shares the name because it is intended to be used for the same semantic reason.

Given that I don't think any of the rest of your comment was legitimate criticism, I am frankly betting that your comment about UnsafePointer is also not useful, but I am kind of tired of having to analyze this comment at this point (I stepped in due to the note about character sets and the floor kept sinking).

saagarjha5• 8 years ago

What the comment about ASCII means is that NUL is a valid character in an ASCII string, but it can't be represented in C's null-terminated string encoding as the format (sometimes called ASCIIZ, but yeah: not an encoding... but I mean, come on... the article is clear here) terminates at the first NUL.

I think ASCIIZ is the more common format, so I replied with all that in response to ASCIIZ being called "unusual". Most popular languages that actually allow NUL bytes in strings usually tend to support some encoding of Unicode anyways…

Also, Nil is absolutely a thing in Objective-C: it is a null pointer of type Class

You got me…in my defense, I didn't know about this to my knowledge. It was still stupid of me to assume that DuckDuckGo would be case-sensitive when I searched that. I guess I should use this for Classes now instead of nil.

I frankly think listing NULL and boost::optional is totally acceptable and complaining about it as if C++11 is more canonical is just being annoying.

One ships with C++, one doesn't; that's like saying Joda-Time is the canonical date library for Java instead of java.time. Although, I should probably ask you if you consider Joda-Time to be "more canonical" before listing it as an example…

Doing a quick search for how nil works in Swift, it apparently isn't a null pointer, so you are wrong there as well :(. I am frankly betting that your comment about UnsafePointer is also not useful

I should have been more explicit, since these kind of run together when you bring pointers into the mix, which I guess I should have realized once I read the footnote. I'm not really satisfied with the explanation given in the article, nor your rebuttal of my argument. Swift's nil is overloaded in a sense: for native Swift structures, it's the whole "Optional-as-an-enum" abstraction that we know about. For class types, and pointers, it's a bit more complicated: you just cannot assign nil to a UnsafePointer or a SomeClass unless it's "Optional", but the "Optional-ness" is completely in the type system and under the hood, in order to facilitate interoperatability with C, Objective-C et al. you need to actually have the type of the size be sizeof(void *), have zeroes in it, etc. You cannot actually set either of these types to nil if they are non-Optional unless you do illegal things. So when you set a "pointer" (being an Optional<SomeClass>, UnsafePointer?) here to nil, you are literally shoving a nil it in, which also happens to work well with Swift's type system and Optional.none abstraction because there is no way to subvert it legally. All of this was basically a long-winded way of saying that yes, Swift's nil is actually NULL, but the type system makes sure that you don't get a "bare NULL" which lets you pretend like the Optional enumeration abstraction works but under the hood, and semantically, it's the same thing as NULL.

Zarel2• 8 years ago

ASCII has NUL as well

Yes, this is why C-strings can't be used for ASCII, because C-strings can't contain NUL as a character.

You're looking for nil. So it should be four stars?

nil isn't a null by the definition this article is using, because it's not a subtype of every reference type.

jillav• 8 years ago

Whether null/NULL is a good a idea or not (I like it, just yesterday it saved my ass) it saddens me that more and more articles I stumble upon are made to criticize technologies but not to talk about solutions or innovations

saagarjha5• 8 years ago

The article talks about a solution, suggesting that optionals are a much better way to handle this.

revskill• 8 years ago

The problem is in tooling. If all compilers/builders out there could detech null for us, those kinds of error could be taken care with much more ease.

rusk15• 8 years ago

A lot of static analysis tools will do this.

I can't speak for other IDEs, but Intellij will always warn me if I'm exposing myself to some NULL operations.

I do think better first class language support is the way to go though.

giornogiovanna7• 8 years ago

TypeScript can be easily configured to do this[0], and Kotlin always does this[1]. The future is now!

[0] https://www.typescriptlang.org/docs/handbook/release-notes/t... [1]: https://kotlinlang.org/docs/reference/null-safety.html

ozim• 8 years ago

In VS with resharper I get "Possible null exception" as warning, but null exception is runtime thing and not compile time, so I don't see how it is tooling problem. Compiler does not know your user did not fill in some field in form.

empath754• 8 years ago

A higher level programming language is a kind of tooling. Use a language that doesn't have NULL and problem solved.

jchw8• 8 years ago

This tidbit gets a ton of mileage but I think it's overrated. There are a lot of unsafe shortcuts we take to get better ergonomics and NULL is one of them.

I think it's a bit unlikely we'll fully get rid of null, but we can get rid of some of the pitfalls. TypeScript for example pretty much fixes the problem, by enforcing you check for null when needed, though TypeScript takes a handful of other soundness shortcuts. Go makes null less harmful by treating nil pointers like empty values by convention.

cyphar5• 8 years ago

I would recommend looking Haskell's Maybe and Rust's Option type to get a better idea of how this can be solved -- and how this article isn't really overrated (just commonly misunderstood).

They allow for explicit NULL-ness (which is a necessary concept) without falling into the trap of making everything implicitly possibly NULL. And when NULL-ness is explicit you are then forced to explicitly handle it in order for your program to compile (in the case of Haskell and Rust).

jchw8• 8 years ago

I dunno why everyone's assuming I don't know about the common solutions to the problem. TypeScript does explicit nullness without needing monads, and I actually mentioned that one. I still disagree that this is not overrated.

Go's idea of nil, for example, seems OK to me, and the language would need to be way more complex to fix it. For example, it would need a type system with explicit nullness, or maybe even actual generics. But it mostly doesn't matter because doing things with nil doesn't crash nearly as much in Go. Like a nil slice just acts like an empty slice. You can even append to nil and it returns a non nil slice. You can call methods on nil. Etc.

The trouble with getting rid of nil to me is that it requires you to either have values at all times, or deal with the possibility that you don't at all times. Go has the very very nice property that you can initialize any type to a zero value and it should work as an "empty" object. Pointers without nullability don't have a zero value. Fixing nulls at the cost of getting rid of Go's properties for zero values would not be worth it.

virtualized• 8 years ago

It's the worst mistake because it made you believe that its atrocious ergonomics are actually superior to more sensible solutions. Implicit nullability doesn't really save you any null checks. It just makes it possible to forget necessary checks.

It was fine to design a language with nullable pointers in the 70s. It's unacceptable nowadays. nil in Go is a major mistake.

jstimpfle10• 8 years ago

Implicit nullability doesn't really save you any null checks.

It sort of does, because explicit nullability forces you to do many redundant null checks when you actually knew that something could not possibly be null.

jchw8• 8 years ago

Okay. So let's say we get rid of nil in Go. Now, structs with pointers have no zero value. Slices and maps have no zero value. Funcs have no zero value. Reflect can no longer create objects because it can't possibly enforce that you initialize the pointers. Functions that return either an error or a value now need a new pattern, probably requiring generics or another special type. Map access needs to return this special type.

Did we win? Did that make Go better? Fuck no. Most people aren't frequently hitting nil pointer errors in Go because unlike C the behavior is a lot more reasonable and the conventions a lot simpler. And by the way, we didn't fix all the runtime errors. Nil pointers are just one possible runtime error. How about out of bounds array access, memory exhaustion, race conditions?

And yeah, I get that you can also fix all of those things, which is then called Rust. But we don't need another Rust, Rust is a fine Rust. Go has, imo, much better ergonomics and most of the time it's just fine for what I'm doing. Like, writing small to medium size servers and utilities in Go has rarely been a regretful experience. And, even if we had no runtime errors we would still need unit testing to ensure our components are functioning correctly. So, most of the time I'm aware of when my code has runtime errors anyways.

Getting rid of null is not magic. It does not get rid of all runtime errors. And yes, it does impact ergonomics. I will take Go zero values at the cost of nil pointers, every day.

giornogiovanna7• 8 years ago

I totally disagree - NULL gives much worse ergonomics all-around. While modifying code, I'm constantly afraid of whether the value I'm accessing could be NULL. Most SQL schemas are filled with "NOT NULL" to the point of ridiculousness, and most Java methods that I've seen tend to have @NotNull used everywhere too. Not having NULL gives you a lot of confidence when reading and writing code, by guaranteeing that your object does indeed exist.

bryanrasmussen2• 8 years ago

previously: https://news.ycombinator.com/item?id=11798518

merricksb• 8 years ago

Also...

https://news.ycombinator.com/item?id=10148972 (150 points | Aug 31, 2015 | 143 comments)

bryanrasmussen2• 8 years ago

just about nothing is as popular as null is.

rusk15• 8 years ago

NULL is a convenient way to map singularities in your model of the problem.

I have on a few occasions tried to write NULL-less code, and it adds a good bit of work.

- model all possible states for a value

- determine appropriate default actions for all types

- meaningful place-holder values

It's a good exercise, and I think more code should be written this way, but - as an Engineer I'm trying to model just enough of the problem to solve it. I'm not trying to simulate every possible outcome in that domain.

Certain corners of your problem simply don't need to be modeled, and what's more the effort needed to model them can just be too much.

NULL is a great way to just throw up your hands and go "I don't know and I don't care". Much as when modelling a physical system singularities typically represent phenomena that the model doesn't take account of, so it goes with NULL. It simply says "Don't Go There".

knocte2• 8 years ago

Have you ever dealt with the Maybe(Haskell)/Option(F#) types? If not, then you don't understand what's wrong with NULL and how to easily avoid it without much work.

jstimpfle10• 8 years ago

I find Maybe a bad idea. It forces me to write denormalized code when I know that something is not NULL. It's not possible to specify this knowledge as a data structure since data structures are static but context is dynamic. I much prefer the simple NULL sentinel that blows up like an assertion when I made a mistake. That said, there's not very often a need for NULL at all if you structure the code correctly.

rusk15• 8 years ago

I see these as a more sophisticated way of dealing with NULL. It allows me to define alternative default behaviour beyond just throwing an undeclared exception.

They are still NULLs however under the hood and I still need to do the work of defining what I want to happen when they occur. It's just neater.

jcelerier2• 8 years ago

I have on a few occasions tried to write NULL-less code, and it adds a good bit of work.

Really depends on the language. On Java / C#, it's hard to avoid null-checking a lot of stuff. In C++ you will never encounter a null std::string outside of code written by indian students learning on Turbo C++ 3.5. So you don't need to check for anything: if it's a value, it exists.

rusk15• 8 years ago

Strings are an easy case. Default initialisation to "" (empty string). You'll still encounter NULL when modelling other domains however.

AJRF• 8 years ago

Do people generally agree that Java Optional == Scala Maybe / Haskell's Maybe?

Java's Optional seems fundamentally flawed in that Java allows any reference to hold a null value and Optional can still throw a NPE when calling isPresent on it so it still gives people a footgun.

based2• 8 years ago

Maybe Not - Rich Hickey (clojure), 29 nov. 2018

https://www.youtube.com/watch?v=YR5WdGrpoug

https://dotty.epfl.ch/docs/reference/intersection-types.html

fmjrey2• 8 years ago

Yes I was going to mention that video in which Rich makes an important point in my opinion: database tables, or objects, or structs, still live within the Place Oriented Programming (PLOP) mindset. That mindset was born in a time where disk and RAM were the expensive resource therefore update-in-place was the default. I insist on the "in-place": you need to know where something is so you can update it. The downside of PLOP is that if you have no value for one of the slot in your generic form (be it a table, object or struct) then what can you put there?

The alternative is to use data shapes that do not require something to be in a certain place. Hence the use of maps as the most basic data shape: you either have an entry in it, or you don't, no need to have a null entry. Expanding that thinking to databases, and you realise tables is not the right aggregate, instead you need to go one level down to something that datomic calls datom, or RDF calls a fact.

To summarise, PLOP forms that package together a set of slots to be filled magnify the issue of NULL/null/nil. Instead make the slot your primary unit of composition and make sure you use aggregation of slots that does not force you to have slots filled with a null value when there is no value in the first place.

kybernetikos• 8 years ago

Yes, 'maybe not' is very relevant to this discussion, but few people seem to agree with my understanding of what he says about the right solution:

Optionality doesn't fit in the type system / schema, because it's context dependent. For some functions, one subset of the data is needed, for others a different subset. Trying to mash it into the type system / schema is just fundamentally misguided.

lmm21• 8 years ago

Java's Optional seems fundamentally flawed in that Java allows any reference to hold a null value and Optional can still throw a NPE when calling isPresent on it so it still gives people a footgun.

The existence of Optional makes it possible to migrate away from using "null". E.g. Map#get could be replaced with a method that returns Optional.

Calling isPresent is the wrong way to use an Optional. But yes, it's an ordinary Java value, it can be null as long as Java-the-language permits null. That's not a problem with Optional, it's a problem with Java, and introducing Optional as a plain old normal Java class is the first step to fixing it.

doubletgl3• 8 years ago

Isn't there an inherent need in programming to express an explicit "nothing" value? Coming from Python and JS, I never found None/null to be much of a problem. I in fact like the distinction of null and undefined in JS. Using null allows you to distinguish from the accidental undefined.

zoul• 8 years ago

The key word here is explicit. Explicit Maybe/Optional types from Haskell, Swift or Rust are wonderful. After my experience with Elm and Swift, the mere thought of going back to implicit nullability everywhere hurts.

sitharus• 8 years ago

No, there’s no inherent need for every variable to allow a nothing.

There is a need for a nothing value in many cases and languages without an implicit null, such as Haskell, F# and Rust, employ an ‘option’ type. It’s a dedicated type that either contains a value or nothing. It forces you to declare when you expect a potential null and to check for it.

doubletgl3• 8 years ago

I didn't use the word variable. I think we're talking about the same thing here.

enriquto• 8 years ago

Isn't there an inherent need in programming to express an explicit "nothing" value?

In numerical calculus: yes, most certainly. What do you expect the result of log(-1) to be? The alternative is to use specially tagged particular numbers as "no-data", and pray so that they do not appear naturally as the result of computations.

gpderetta• 8 years ago

Or exceptions.

da_chicken10• 8 years ago

Some systems already do that. C#'s Double type includes Double.NaN, Double.NegativeInfinity, and Double.PositiveInfinity. Math.Log(-1) does indeed return NaN.

kazagistar2• 8 years ago

Technically, you could force the value to be one constrained to a valid range, rather then augmenting the domain, but this is a lot of work and maybe not worth it for pracial use.

TheCoelacanth• 8 years ago

That doesn't mean numbers need to always be nullable. It means that `log` doesn't return a number, it returns a nullable number.

syrrim• 8 years ago

What do you expect the result of log(-1) to be?

i pi, or more generally i pi n for odd n.

goto114• 8 years ago

The major criticism towards NULL's does not apply to dynamic languages. The problem in languages like Java is that nullability is not represented in the type. This criticism is not relevant in a language without static type checking. The criticism is also not relevant in a language like TypeScript where nulls are specified in the type.

In short, the problem is not nulls per se, the problem is static type systems which does not allow you to specify if a certain value can be null or not.

But I hereby predict that soon people are going to declare that "nulls are bad" and eliminate them even in languages where they are totally fine.

chc• 8 years ago

The OP specifically calls out ways that nulls cause problems in dynamic languages. In fact, they're very similar to the problems nulls cause in statically typed languages, just ignoring some of them because danger is already priced in with a dynamic language.

Dynamically typed programs are usually informally "duck typed," since it's impossible to do something meaningful with truly arbitrary types most of the time. But just like in statically typed languages, there's always the very real possibility that your duck will instead be a null — and the bug is not the function returning a non-duck, it's you ever assuming you have something that quacks like a duck.

mehrdadn3• 8 years ago

In C++ I practically never feel like I need a null for anything but a pointer. A null string or integer makes no sense to me.

cmrdporcupine3• 8 years ago

std::optional has the potential to eliminate many of the null-pointer uses.

kazagistar2• 8 years ago

Even in dynamic languages, it's generally bad practice to have functions that return multiple possible types, and you are often better served by returning a more proper "null object" that accepts the same messages/methods as what would normally be returned. But for the most part, the problem people have with null is related to compile time checks and it not being opt in, which is unrelated to dynamic languages where every return type cold contain... whatever.

doubletgl3• 8 years ago

I agree that returning the same type, if possible, is better, since you can model it so that one specific value (zero, empty string, etc.) expresses the "nothing" accordingly. But I have found that several other developers don't share this view and prefer functions to return false, null, etc. instead of a value in the same space as the normal return type.

laichzeit0• 8 years ago

So how does NaN in Python and NA in R relate to NULL? I know Python has the None type, but it's not the same as NaN. One of the most annoying things in Numpy is that there is no way to indicate that an integer value is "missing", similar to NaN for floats. In R both integers and strings can be NA (if I remember correctly). So for numeric types at least, there is definitely the need to somehow indicate that a value is "missing".

hurrrrr• 8 years ago

Numpy has masked arrays [1]. Though I can't say how well they work.

[1] https://docs.scipy.org/doc/numpy/reference/maskedarray.gener...

Nullabillity• 8 years ago

NaN is not the same as None/null, and using it that way is asking for bugs.

None/null is for missing/unknown values (which may be a reasonable thing in your domain), while NaN is for the result of an illegal/undefined operation (which is definitely a bug, such as division by 0).

pdkl95• 8 years ago

NULL is a value that is not a value. And that’s a problem.

The problem isn't NULL, it's languages not enforcing the necessary checks for the "no data" condition. Option can still be NULL ("None" in rust), wrapping NULL in a struct doesn't provide any safety. The safety of Option wrapper types is from the other language features (like rust's "match") and a stricter compiler that forces the programmer to write the NULL check.

NULL would be fine if C required you to write this:

    foo_t *maybe_get_foo(/*...*/) {
        if (/*foo_is_available*/) {
            return foo;
        } else {
           return NULL;
        }
    }

    foo_t *f = maybe_get_foo();
    if (!f) { /*...*/ }   // REQUIRED or compile error
    do_something(f->bar); // only allowed after NULL check

Obviously implementing that requirement would be difficult in C. Languages like Rust were designed with enforcement features (match + None, much stronger type/borrow checking), but lets your have "a value that is not a value".

masklinn13• 8 years ago

The problem isn't NULL, it's languages not enforcing the necessary checks for the "no data" condition.

Talking about "NULL" pretty much implies that. When Tony Hoare talks about null references, it's about every reference being nullable in languages like Java or C#, not about the ability to conceptually wrap/opt non-nullable references in a nullability thingie.

danesparza2• 8 years ago

Psst: I think the thingie you're referring to is called a 'monad': https://en.wikipedia.org/wiki/Monad_(functional_programming)

r_c_a_d2• 8 years ago

Or add exceptions like C++ did,

if (foo_is_available) return foo; else throw FooNotAvailable();

r_c_a_d2• 8 years ago

Not sure why I am getting downvotes for this. The parent proposed a language feature to support an enforced code execution when a return value was not available:

    foo_t *maybe_get_foo(/*...*/) {
        if (/*foo_is_available*/) {
            return foo;
        } else {
           return NULL;
        }
    }

    foo_t *f = maybe_get_foo();
    if (!f) { /*...*/ }   // REQUIRED or compile error
    do_something(f->bar); // only allowed after NULL check

And I was pointing out that C++ exceptions do exactly that.

    foo_t *maybe_get_foo(/*...*/) {
        if (/*foo_is_available*/) {
            return foo;
        } else {
           throw FooNotAvailable();
        }
    }

    try {
        foo_t *f = maybe_get_foo();
        do_something(f->bar); // f cannot be NULL
    }
    catch (const FooNotAvailable& ex) {
        /*...*/
    }

Sorry if that wasn't the answer you wanted because C++ is not fashionable. But it is a valid option for how to avoid the use of NULL.

SamReidHughes• 8 years ago

You might instead argue that the problem is allowing checks for the null condition. If you make comparing a pointer with null crash, or just remove the keyword to make its use non-idiomatic (or only allow it as special initialization syntax), people won’t casually use null to mean something. It’ll serve its role as a pointer value that crashes cleanly if you dereference it.

ummonk2• 8 years ago

Yup, this is what happens when you use Typescript with strict null checking.

chongli2• 8 years ago

No, the problem with null is the inability to enforce, at the type level, that a particular value is not null. C simply does not have a type for "guaranteed valid pointer to x".

kzrdude3• 8 years ago

C allows defining new types though, so it can be done.

empath754• 8 years ago

Since every type can be NULL that’s a lot of verbosity.

jhomedall• 8 years ago

Option can still be NULL ("None" in rust)

'None' in Rust is part of the Option enum, not an equivalent to null.

  pub enum Option<T> {
      None,
      Some(T),
  }

estebank• 8 years ago

It can be at the machine level. If T is NonZero, then sizeof(Option<T>) == sizeof(T).

vmchale• 8 years ago

Option can still be NULL ("None" in rust), wrapping NULL in a struct doesn't provide any safety. The safety of Option wrapper types is from the other language features (like rust's "match")

The advantage here (especially true in Haskell) is that you can use monadic error handling to make this far more pleasant.

masklinn13• 8 years ago

That's a secondary nicety (alongside the ability to make everything "nullable" not just the references/pointers subset), the primary advantage is that:

1. it clearly separates "nullable" and "non-nullable" providing better modelling tools

2. the compiler assists/mandates null-checking, providing better type-safety

sidlls• 8 years ago

Even Rust doesn't have the strictness in your comment. It's perfectly fine by the compiler to make use of `x.unwrap()`: if x is None (or Err, in the case of Result), you'll just get a panic at runtime. The features you note are superior to C's offering, but purely optional.

dbaupp• 8 years ago

There's a fundamental difference between the Rust approach of the library providing a function for opting-in to potential crashes and the C/Java approach of not distinguishing that case at all. The programmer still is forced to write a null check, it's just a check that crashes the program.

slavik81• 8 years ago

I largely agree, but that would get very tedious because there's no way to create a pointer type that is guaranteed to be non-null in C. You'd end up having to do a lot of unnecessary checks because of that.

People say optionals are the solution, but the way I see it it's the other way around. Pointers types that allow NULL are basically optionals, and the problem is that we use them everywhere, even for things that are not optional. What we are missing are pointer types for things that are not optional.

And the pointer type with a guaranteed value needs to be as easy to use as the nullable pointer type, if not easier. Otherwise it won't always be used when it should be.

pierrebai• 8 years ago

Not very difficult. Just provide a not_null pointer attribute, just like const. Then require that all dereferenced pointers must have the not_null attribute. Problem solved.

(Other note: C++ has a not_null pointer-like type: it's references. Unfortunately, C++ references cannot be reseated, which makes wholesale replacement of pointers not feasible. Plus, the language doesn't actually forces you to check pointers before assigning to a reference.)

burfog• 8 years ago

Linux does well with some macros: IS_ERR, IS_ERR_VALUE, PTR_ERR, ERR_PTR, PTR_ERR_OR_ZERO, ERR_CAST, IS_ERR_OR_NULL

https://elixir.bootlin.com/linux/latest/source/include/linux...

No, it doesn't totally replace NULL, but it does solve some of the problems in a high-performance way.

jmfayard• 8 years ago

Progress, it's now a solved problem in modern languages like Rust, Swift or Kotlin See for example: https://kotlinlang.org/docs/reference/null-safety.html

masklinn13• 8 years ago

I mean it's been a solved problem since the 70s if not earlier, the problem has always been uptake. And that is not solved e.g. C++ recently introduced std::optional, which is not a type-safe version of a null pointer but is instead a null pointer wrapper for value types.

cmrdporcupine3• 8 years ago

NULL in 'relational' databases in particular is a disaster. Or at least according to the notorious Fabian Pascal.

http://www.dbdebunk.com/2017/04/null-value-is-contradiction-...

Codd never proposed it in his original relational model. For good reason.

pleasecalllater• 8 years ago

I'm afraid current relational database have little common with the Codd's ideas.

goto114• 8 years ago

Disagree that it is a disaster. Nullability is explicit in the column type, so it doesn't have the "billion dollar mistake".

Furthermore you need to represent missing values somehow if you perform a left join.

lucideer• 8 years ago

null is often used in relational databases to be semantically equivalent to "info missing". I get that this use of null isn't ideal.

However there are quite a lot of cases where it is semantically equivalent to "not applicable"; while some of these cases can be avoided with a restructure, some can't, and in either case I don't see a compelling argument to do so.

e.g. a parent_id column that is null for top-level objects in a hierarchy. Restructuring the database to avoid this seems like moving to an unintuitive paradigm in pursuit of rule-following for its own sake.

molf• 8 years ago

I think it would be nice if `NOT NULL` was set by default on columns. However there are a lot of legitimate use cases that can't be (practically) solved by restructuring.

Data can be incomplete. Maybe only because it is not (yet) known. If NULL values were impossible it would create the need for one additional table with a foreign key relationship for every attribute that can be independently NULL. Sometimes this pattern is a good idea, but I don't think I can be convinced it should be the only possible solution.

Dirlewanger• 8 years ago

I think it would be nice if `NOT NULL` was set by default on columns

Then make sure to have explicit defaults?

nerdponx• 8 years ago

How do you propose we represent the absence of data? At least in SQL "logically missing data" and "missing data in memory" are properly divorced, unlike many languages in which the latter is abused and conflated with the former.

da_chicken10• 8 years ago

Nonsense.

First of all, Codd's early work didn't propose it because Codd's propositions were idyllic and rooted in mathematics and not software engineering. In Codd's world, you would never create a FirstName/GivenName and a LastName/Surname field. You'd just create a Name field. The fact that you might want to sort by last name or the fact that someone named "Alice Taylor Smith" has a surname of "Smith" while "Robert Taylor Smith" has a surname of "Taylor Smith" isn't relevant. In Codd's world, you'd just define Name as a Name type, and the Name type itself would know everything about the entire domain of names. It would be about as complicated as working with a DATETIME. The same would be true for an Address type. Such a type would support both Western and Japanese addresses, which are wildly different. Codd doesn't need NULL because he has complex types which perfectly and comprehensively represent their domains. That's not realistic. Parsing Names and Addresses for even a single culture is notoriously difficult.

Second of all, Codd's later major work, The Relational Model for Database Management, includes his Twelve Rules (numbered 0 to 12) for database design in order for a DBMS to be considered relational[0]. Rule number 3 is:

Rule 3: Systematic treatment of null values:
Null values (distinct from the empty character string or a string of blank characters and distinct from zero or any other number) are supported in fully relational DBMS for representing missing information and inapplicable information in a systematic way, independent of data type.

So Codd clearly thought that nulls were essential.

[0] https://en.wikipedia.org/wiki/Codd%27s_12_rules

axilmar• 8 years ago

NULL is certainly a mistake, but even more of a mistake is not allowing distinct states in variables.

NULL is just another case of a state of a variable. Other states are 1, 15, 0xffffffff, etc.

That mainstream languages don't handle this is the worst mistake of the computer industry.

rusk15• 8 years ago

Most languages do. Java will always initialise a non-initialised value to NULL for instance (EDIT or 0 or false for primitives).

It's simply a reality of how computers operate that when you allocate a piece of memory (a variable) it will have something in it that you'll need to clear or initialise.

In this respect, NULL is doing you a favour.

zxczxc111• 8 years ago

Only for field variables. Local variables most definitely have to be initialised by the programmer, otherwise it's a compiler error

david04• 8 years ago

Scala alternative to null: https://www.scala-academy.com/tutorials/scala-options-tutori...

captainmuon• 8 years ago

I've made my peace with null. Null is basically just an implicit

    assert(valid(x))

before every time you call a method on x. Similary, I think of exceptions as explicit "crash-unless-caught" commands.

If you write your program with the "blow up early" mentality anway, or use static checking tools and a bit of discipline, I've found that null looses it's terror.

pjmlp• 8 years ago

That is a very big IF, considering how many teams are actually structured.

lawn• 8 years ago

Oh I'm fairly disciplined. But we use a 15 year old code base which should've been made with more discipline than it was.

Additionally the program shouldn't "blow up early" but it was in many ways coded that way.

gre• 8 years ago

“Looses its terror” here means the opposite of “loses its terror”.

piyh• 8 years ago

Cry havoc and let loose the null pointer exceptions

kzrdude3• 8 years ago

If it's C, it's far more terrifying than that.. there is no assertion! just a vauge threat that something will go wrong if you stick a null in there, with no guarantees and no checks.

lmm21• 8 years ago

Sure, null can be a non-problem for the low, low price of 3 or 4 extra lines on each function. But those extra lines distort your program architecture, pulling you away (perhaps without even noticing) from short, composable functions.

arcticbull3• 8 years ago

In market terms, sir, you've entered the capitulation phase haha. It's actually not correct to say that accessing null will always blow up. In embedded systems without memory protection address 0 may well contain valid data, usually a vector table. In WASM address 0 is totally valid also, if I'm not mistaken, as memory is represented as a big ol' array with an offset and checking for 0 would be too inefficient.

masklinn13• 8 years ago

I've made my peace with null. Null is basically just an implicit

That very much depends on the language:

1. it can be a compile-time error (Swift)

2. it can be a runtime error (java, C#)

3. it can depend on what you're actually using (Python, Ruby, and on runtime extensions you might have loaded in the latter case)

4. it can be a no-op (objective-c)

5. it can depend on the combination of platform, compiler and surrounding code going from a segfault to deleting your program's security and/or causality (C, C++)

6. it can depend on the implementation and exact codepaths (Go)

etc...

userbinator• 8 years ago

To someone who has been using Asm and C for decades, these arguments just make no sense. Reading this article reminds me of the arguments against pointers, another thing that's frequently criticised by those who don't actually understand how computers work and try to "solve" problems by merely slathering everything in thicker and thicker layers of leaky abstraction. It's not far from "goto considered harmful" either.

any reference can be null, and calling a method on null produces a NullPointerException.

...which immediately tells you to go fix the code.

There are many times when it doesn’t make sense to have a null. Unfortunately, if the language permits anything to be null, well, anything can be null.

That's not an argument. See above.

3. NULL is a special-case

...because it indicates the absence of a value, which is a special case.

though it throws a NullPointerException when run.

...and the cause is obvious. I'm not even a regular Java user (and don't much like the language myself, but for other reasons) and I know the difference between the Boxed types and the regular ones.

NULL is difficult to debug

Seriously? A "nullpo crash" is one of the more trivial things to debug, because it's very distinctive and makes it easy to trace the value back (0 stands out; other addresses, not so much.) What's actually hard to debug? Extraneous null checks that silently cause failures elsewhere.

The proposed "solution" is straightforward, but if you reserve the special null value to indicate absence then you can make do with just one value instead of a pair, of which half the time half of the value is completely useless. If you can check for absence/null, you will have no problems using Maybe/Optional. If you can't, Maybe/Optional won't help you anyway --- because it's ultimately the same thing, using a value without checking for its absence.

TickleSteve4• 8 years ago

Completely agree, this is CS theory gone off the deep end...

cultus10• 8 years ago

A simple, safe way of tracking nulls like option is "CS theory going off the deep end?"

Implicitly allowing all code to return nothing, and manually trying to remember what can return null and what can't, and checking that value, is incredibly error prone. It's really crazy that this has been the dominant way of handling the problem for many decades when their is a dead-simple way of ensuring it can't happen.

Null pointer errors, contrary to many claims, show up in production code all the time. Eliminating them is of huge value.

gnulinux3• 8 years ago

Calling this "CS Theory" is intellectual dishonesty.

kiriakasis3• 8 years ago

Cannot understand this position... a non-nullable pointer is pretty much just a normal pointer but the compiler checks if you have tested for null. In rust (and I believe Swift) optional references also have the same size as normal pointer. In some version of non-nullables you only need checks for dereferencing and not for other handling.

giornogiovanna7• 8 years ago

Just look at the example on Wikipedia[0]. Tagged unions are super-simple, and make NULL completely unnecessary. NULL causes lots of headaches, while tagged unions have never caused anyone headaches, so removing NULL is kind of the obvious thing to do. In a sufficiently advanced language, such as Rust, they get optimized to equivalent code anyway, so there isn't even any performance loss.

[0] https://en.wikipedia.org/wiki/Tagged_union#Examples

Ygg2• 8 years ago

   > ...which immediately tells you to go fix the code.

Assuming your code isn't deeply nested. I've seen cases where null was triggered years after code went into production. In that case you have to:

A) Assume value isn't null and have more readable code

B) Litter the code with null checks.

e.g.

    if (a.getStuff().getValue() == "TEST")

becomes

    if (a != null && a.getStuff() != null && a.getStuff().getValue() == "TEST)

Thing with Maybe/Optional you have to check for presence of None, otherwise your code won't compile. Another smart way is what C# did. Integer can't be null. Integer? can be null.

seanalltogether• 8 years ago

But expanded null checks could be automated by the compiler if so desired right? Without having to change the nature of null into an optional.

@MAYBE if(a.getStuff().getValue() == "TEST")

hesselink• 8 years ago

The thing is, you're focusing on when you've detected that there is an issue (a crash). A lot of the issues with NULL are the fact that you can't easily detect if beforehand. It's not indicated in the types, or the syntax. That means that it's incredibly easy for a NULL issue to sneak into an uncommon branch or scenario, only to be hit in production.

nsajko2• 8 years ago

But why was the scenario not tested before production? Should that not be the case anyway?

cyphar5• 8 years ago

another thing that's frequently criticised by those who don't actually understand how computers work and try to "solve" problems by merely slathering everything in thicker and thicker layers of leaky abstraction

I think that you're being quite unkind. Haskell's Maybe type and Rust's Option types are very far from "leaky abstractions" and were developed by people who definitely understand how computers work. In fact, your description of them appears to indicate that you aren't really sure how they work (None doesn't take up "half of the value") -- the point of typeclasses is that cases where NULL is a reasonable value are explicit and your code won't compile if you don't handle NULL cases. Allowing NULL implicitly for many (if not all) types is where problems lie.

It also appears you're arguing that languages which don't have models that are strictly identical to the von Neumann architecture are "merely slathering everything [with] leaky abstraction". Would you argue that LISPs are just leaky abstractions?

ummonk2• 8 years ago

Well, a better comparison would be Typescript with strict null type checking enabled. You just use algebraic data types to specify whether a value can be null.

giornogiovanna7• 8 years ago

Yep! It's worth restating the fact that wrapping a pointer in an Option in Rust actually takes up NO extra space, because Rust's smart enough to just optimize it back into a nullable pointer.

https://doc.rust-lang.org/std/ptr/struct.NonNull.html

flipgimble• 8 years ago

If your whole world is asm and C, then I take it you don't care much about type systems. Bless your heart, you lonely programmer of ephemeral software, may you be employed gainfully fixing your own bugs for decades. The saltiness if mostly for entertainment, please don't take too much offense. For everyone else working at a level of complexity where mistakes are inevitable and costly, types are an essential bicycle for the faulty minds of programmers.

The article is not arguing that we shouldn't express or model the absence of a value. It is arguing that all types having to support "no-value" case leads to error prone code that has historically cost immeasurable amount of money and much credibility and respect. If everything can be null then it takes too much effort to null check every reference, so developers routinely forget or think they know better. Instead it argues that we should model the idea a possible empty values as a separate composable type. Then you can write a large percentage of your code with a guarantee that objects/type/values are never going to be nil, while still handling that possibility in the smaller error checking parts of your code base.

One interesting anecdote is that our team, working in Swift, had to integrate a crash reporting tool and verify that it works. The challenge was that we haven't seen a runtime crash in several months in production.

A "nullpo crash" is one of the more trivial things to debug

If it happens in your debugging environment in front of your eyes then maybe. Some of us work on software that is used by millions over decades and would never get to see any reports from a majority of crashes.

tjoff• 8 years ago

Not sure what world you are from. But here on earth apparently garbage collection and javascript is all the rage ;)

Issues with null doesn't even register in comparison.

badestrand• 8 years ago

> There are many times when it doesn’t make sense to have a null. Unfortunately, if the language permits anything to be null, well, anything can be null.
That's not an argument. See above.

It is actually their best point, IMO. I really like how RDBMS/SQL solve this: fields hold values and you specify beforehand whether they can hold NULLs. The author is right, sometimes it does not make sense for variables to be null-able (think ID fields or usernames) but often it does (e.g. a user's avatar). Being able to indicate that would be a nice idea. C++ for example does that, as `Field x` is not nullable but `Field* x` is.

nsajko2• 8 years ago

Also, when you are aiming for full test coverage null dereferences will be caught during testing.

adrianN• 8 years ago

Nobody does full path coverage, not even NASA.

lmm21• 8 years ago

...which immediately tells you to go fix the code.

But which code? The point where you observe the error could be many compilation units away from the code that's broken; it might be in a separate project, or even 3rd-party code.

...because it indicates the absence of a value, which is a special case.

Why does it need to be a special case? Is your language incapable of modelling something as simple as "maybe the presence of this kind of value, or maybe absence" with plain old ordinary, userspace values?

Seriously? A "nullpo crash" is one of the more trivial things to debug, because it's very distinctive and makes it easy to trace the value back (0 stands out; other addresses, not so much.) What's actually hard to debug?

"Tracing the value back" is decidedly nontrivial. And totally unnecessary if you just don't allow yourself to create that kind of value in the first place.

if you reserve the special null value to indicate absence then you can make do with just one value instead of a pair, of which half the time half of the value is completely useless.

What do you mean? If you're talking semantically, you want absence to be a different kind of thing from a value: it should be treated differently. If you're talking about runtime representation, you can pack an Option into the same space as a known-nonzero type if you want to (Rust does this), but that's an implementation detail.

(Confusing sum types with some kind of pair seems to be a common problem for programmers who haven't used anything but C; sum types are a different kind of thing and it's well worth understanding them in their own right).

If you can check for absence/null, you will have no problems using Maybe/Optional. If you can't, Maybe/Optional won't help you anyway --- because it's ultimately the same thing, using a value without checking for its absence.

Nonsense on multiple levels. Maybes deliberately don't provide an (idiomatic) way to use them without checking. By having a Maybe type for values that can legitimately be absent, you don't have to permit values that can't be absent to be absent, and therefore you don't have to check most values - rather you handle absence of values that can be absent (the very notion of "checking" comes from a C-oriented way of thinking and isn't the idiomatic way to use maybes/options) and don't need to consider absence for things that can't be absent.

pjc508• 8 years ago

The whole point of using type systems is to prevent human errors; a "poka-yoke" for programming.

The great advantage of Maybe/Optional systems is that only some of your references have to use them. You can draw a clear boundary between the parts of the code that have to check everything, and those that can prove it's already been checked.

In assembler we have no real type annotations, but for a long time I've considered trying to design a type-checking structure-orientated assembler.

lalaithion2• 8 years ago

Option<&T>, in Rust, will compile down to a C pointer to T, with Nothing represented by the null pointer.

taco_emoji3• 8 years ago

...because it indicates the absence of a value, which is a special case.

But that's exactly the problem -- it's special, meaning it's only useful for certain situations. An ideal type system would provide compile-time guarantees, rather than having to wait for users to report issues. A type system which A) allows you to define variables as non-nullable and B) requires a null guard before every dereference of nullable variables eliminates this entire class of bug. What on earth is wrong with that?

EDIT:

Seriously? A "nullpo crash" is one of the more trivial things to debug

Even if this were true [0], wouldn't it be easier if such a crash was just never even possible?

[0] which it's not - all a nullpo stack trace tells you is that something important didn't happen, at some point before this

setr2• 8 years ago

In general, it's difficult to make the case that runtime errors are preferable over compile-time errors, unless the difficulty to enable compile-time errors is significant. In the case of Optional<> as a language/DB type, I can't imagine much effort involved, except for uptake.

Especially given that it can trivially be optimized out, since the eventual assembly should very well make use of the 0x0 property. But if you can encode the guarantee in your "high-level" language, why would you not want it?

In fact, I'm not sure how anyone could imagine assert(n != null) scattered throughout the codebase is a pleasant situation, unless of course, as most do, you're skipping the safety check for unsafe reasons.

tarkin2• 8 years ago

I've been using kotlin and swift. They've partly removed null with the 'maybe' feature.

So instead of calling methods on null objects the methods are just not called if the object is null.

This helps when there's a race condition, and you attempt to call a method on a null object, and then that solves itself by the same code being called again without the race condition.

But a lot of the time if the object is null and the method is not called you still have an error, but it's just not a null pointer error now.

This 'nullless' code is nice in some places, especially with UI lifecycles calling code repeatedly, but other times it just changes the type of error you debug.

jillesvangurp• 8 years ago

Actually in Kotlin, nullability is part of the type system and denoted with a ?. So String and String? are two different types. Dereferencing a nullable type is a compile error until you do the null check. Doing that triggers a smart cast to the non nulled type. So the inferred type becomes non nullable and you don't have to do any casts. You can force this cast by using the operator !!, which you should avoid for obvious reasons but is useful with some legacy code. If you get this wrong you still get an npe.

It also provides backward compatibility with java where java types are considered nullable by default unless other wise annotated with a @NonNull. Also you get nice warnings about redundant null checks.

This provides for a lot of extra compile time safety and it largely removes the need for Maybe, Optional, and other kludges that people have been coming up with to force programmers to replace null checks with empty checks.

sifoobar• 8 years ago

Nah. There will always be missing values, no matter how many layers of safety measures we wrap around the fact. Hitting a NULL in C is very unforgiving; but that's just the spirit of C, there are plenty of ways to provide a less bumpy ride in higher level languages.

My own baby, Snigl [0], uses the type system to trap runaway missing values without wrapping. Which means that you get an error as soon as you pass a NULL, rather than way down the call stack when it's used.

https://gitlab.com/sifoo/snigl#types

bunderbunder4• 8 years ago

You know who works on a platform with NULL but doesn't have quite so many problems with it? DBAs.

There's some need to draw a distinction between the basic idea of NULL, and the way that NULL has been implemented in most high-level programming languages.

In most RDBMSes, values can't be null unless you say they are. Sometimes explicitly, as in table definitions, sometimes implicitly, when you select a JOIN type. Either way, though, the fact that the developer is in control of when it can and cannot happen means that it always has a knowable meaning. (Or should, anyway.)

The problem with many programming languages is, you're given it whether you want it or not. In a low-level language like C, that's reasonable, because it takes a sensible approach to how it works: Only pointers can be null, and all pointers are nullable for obvious (especially in the 1970s) reasons.

More generally, I'm not going to fault languages from that era for trying it out, because this stuff was new, and things were still being felt out. So I don't really fault Tony Hoare for giving null references a try in ALGOL W.

What seems much more bothersome is high level languages like Java and C# cargo culting this behavior. They could have followed the lead from languages like SQL and let the programmer be in control. They should have. They already throw exceptions when a memory allocation fails, and they allow inline variable initialization, and declaring variables at the point of usage, and composite data types have constructors, so they lack all of (early) C's reasons why ubiquitous nulls were a good idea. They could have, I think quite easily, made nullability optional. At which point it'd have basically the same semantics as optional types from functional programming, so I doubt we'd be worrying about it anymore.

But they didn't.

lmm21• 8 years ago

NULL in SQL is a notorious source of errors and confusion (particularly when it comes to e.g. tri-state boolean logic). It certainly can come from nowhere and surprise you - if anything the behaviour is even worse than in Java or C#. So I don't think there's anything to learn from there. (Rather what modern languages should have done - and increasingly do - is follow ML practice and avoid null entirely, implementing option types as ordinary library types where the programmer explicitly wants to represent absence).

bunderbunder4• 8 years ago

I guess. My tendency is to think that it's more a problem for developers who are new to SQL, and are surprised to find out that, despite having the same name, nulls in SQL don't have the same semantics as nulls in other languages.

Once you get a handle on the semantics, though, they make a lot of sense. The trick is to understand how SQL's NULL is rooted in mathematical formalism, not the pragmatics of dealing with pointers. It has more in common with NaN in floating-point numbers. So, for SQL, "null <> null" behaves like "NaN <> NaN". For C and friends, "null == null" for the same reason that "0 == 0".

da_chicken10• 8 years ago

That's not really NULL's fault that it causes confusion in SQL. That's just ternary logic. People who don't handle NULLs in SQL aren't really mishandling the NULL. NULL is just a value. They're simply failing to understand the Boolean value of UNKNOWN and what that means. They're so used to thinking only in bivalent logic that the additional complexity throws them off.

However, "It's more complex for me to think about," or, "I don't understand the convention," or even, "It's easy to forget the convention," are not a very convincing arguments. It's similar to arguments about little endian vs big endian. Yes, big endian is how we write our positional numbers, but little endian makes casts a noop. Or arguments about zero-based array indexing. These concepts aren't difficult. They're just more complex. Negative numbers aren't difficult, but they're more complicated than just cardinal numbers. Fractions and decimals aren't difficult, but they're more complex than integers. Multiplication and division aren't difficult, but they're more complex than addition and subtraction.

jdblair• 8 years ago

Lua's version of NULL, called nil, once bit me badly due to behavior in a sqlite library. The sqlite library I was using represented SQL NULL as nil, a perfectly reasonable choice. However, in lua there is also a convention to use nil as the end of table sentinel.

This meant innocent looking code using ipairs() would stop iterating on a row of results once it hit a nil, which could occur in anywhere. It meant we were missing data (the lua code uploaded locally collected data to a server) until I figured out the cause and explicitly iterated over the expected size of the table.

nayuki3• 8 years ago

NULL in SQL really isn't great. For one, nullable table columns is a bad default, and you have to explicitly write out "NOT NULL" to avoid this behavior. I'd say that 90% of the time I want not-null table columns, and only 10% of the time do I want a nullable column.

Secondly, NULL has weird arithmetic. It turns out that NULL=NULL is false, and NULL<>NULL is also false. (This is unlike C/Java/Python/etc. by the way.)

Thirdly, even if you design all your tables to have NOT NULL on all columns, your queries can still synthesize NULL values in the results. For example, LEFT JOIN, RIGHT JOIN, FULL OUTER JOIN, (but not INNER JOIN). For example, computing max(column) on a table with zero rows.

cpburns2009• 8 years ago

I find NULL to be incredibly useful. I do agree that it has bad defaults in SQL, and equality is annoying similar to NaN in other languages.

dpcx• 8 years ago

I can get behind your first statement. Having NULLable as a default on columns is "probably" a bad idea.

I'm not so sure I can agree with the other two. NULL<>NULL (and NULL=NULL) both return false for a very simple reason: truly missing data _can't_ be equal to anything, including missing data... Because it's missing. You cannot with certainty say that value1 is or is not equal to each other.

For the third point... What should max(column) return when there's no data? You're telling the engine "give me the maximum value of something that doesn't exist". That is, in my experience, "missing data."

grumpydba• 8 years ago

NULL in SQL really isn't great. For one, nullable table columns is a bad default, and you have to explicitly write out "NOT NULL" to avoid this behavior.

This is not true on many dbms. It's an implementation choice.

empath754• 8 years ago

Are there any dbs based on modern type theory?

porpoisely• 8 years ago

NULL can mean and be different things in different domains of computer science. NULL in the database world isn't the same thing in the programming world. In the programming world, null is a result of the system architecture, systems programming, etc. In SQL, NULL is a result "lack of data". There have been debates on whether there should be different types of NULL. A NULL type for "data that is available but we don't have it yet" - like car make and model for a car owner. A NULL type for "data that does not apply" - like car make and model for an adult who doesn't own a car. A NULL type for "data that never applies" - like car make and model for a child. Then you get into the philosophical debate on whether a NULL can ever equal a NULL. Does it even make sense to even think of NULL in terms of equality. How can an unknown entity ever equal another equality? But what if you are just asking "are they both unknown"? Then you probably can think of two NULLs as "equal".

In higher level programming, the consensus seems to be the less nulls the better. Which is why languages like C++, C#, etc are introducing Option-like syntax ( mostly to accommodate the database world and their NULLs ).

NULL exists to solve particular problems in computer science. It can also cause a lot of problems. You can argue it's the best solution and worst mistake depending on the situation.

lalaithion2• 8 years ago

All of those different nulls can be solved by not having null as a special case of your database specification, but as a first class type construct.

    data MightBeData a = Yes a | Unavailable | NotApplicable | NeverApplicable

Twisell2• 8 years ago

NULL in databases have many properties that save a shitload of coding time and help write more secure code. To cite only one of theses useful properties NULL automatically propagate through all operations and aggregations.

karmakaze• 8 years ago

All this is true. I immediately took it to mean the Tony Hoare invention of Null. That of the zero memory pointer. Other types of null usually have other names with the notable exception being databases. Also Go uses nil for its null which puzzled me for a while then realized its behaviour is different, you can work with them to a point, e.g. len() or append().

davemp4• 8 years ago

This is hard to reconcile with type theory for me.

NULL, to me, implies and uninhabited type, i.e. there can never be a value with a NULL type. Using null for a "data isn't there, apply, available, etc" seems like an abuse of the type system. I see no reason that the former needs to be supported at the type level. These properties are just responses to queries, not some mystical, uninhabitable oblivion. Unnecessary type features just make verification and learning a language much more difficult.

SpicyLemonZest2• 8 years ago

It needs to be supported at the type level, whether by null or by options, simply because “data not available” is a common value people need to use. When there’s no good way to express it, they’re forced to invent special sentinel values, and you end up in the situation where array index -1 means “value not found in the array”.

giornogiovanna7• 8 years ago

NULL isn't the uninhabited type, that's the bottom type. NULL is a value that inhabits every type.

kazinator2• 8 years ago

This is clear in (Common) Lisp.

There is a null class/type whose one and only instance is the special object nil.

This nil also itself names a type which contains no instance: its domain of values is the empty set. This is the type at the bottom of the type spindle, corresponding to the NULL that you're talking about.

nil is a special self-evaluating symbol representing Boolean false, the empty list, and this bottom type.

So you see, if we just don't conflate the null object with the null set type, everything works out.

bytematic• 8 years ago

NULL should mean an intended lack of data. Undefined should be unintended.

sharpercoder• 8 years ago

Yet `Maybe<T> | Option<T> | ...` is not an option (pun intended), as Rich Hickey explains here: https://www.youtube.com/watch?v=YR5WdGrpoug.

In effect, his argument is: 1) You have `public X Do(Y y)` changed into `public X Do(Option<Y> y)`. This will break your API. 2) You have `public X Do(Y y)` changed into `Option<X> Do(Y y)`. This will break your API.

Thus, do not use Option<T> or equivalent in your API's. Only use a language-supported construct such as C#8's upcoming `string?` and `string`.

bunderbunder4• 8 years ago

This is a spot where I've got to respectfully disagree with Mr. Hickey.

Changing a public API call that used to guarantee that it returned a value so that it might now return nothing is a breaking change, and, as an API consumer, I want my APIs to broadcast that change loudly. Compiler errors are a good (but not the only) way to do that.

Changing a public API member so that its arguments are now `Maybe[T]` is just silly. There's no need to introduce a breaking change there. Just overload it so that you now have versions that do and do not take the argument and get on with life.

If there's an argument to be made here, it's that statically and dynamically typed languages require different ways of doing things. In a statically typed language, I expect the compiler to keep an eye on a lot of these things, and I'm used to leaning on the compiler to catch things like a function's return value changing. In a dynamic language, I'm not.

I'm also, when working in a dynamic language, used to having to deal with the possibility that, at all times, any variable could contain data of literally any type. Removing nullability there changes the set of possible "this reference does not refer to what I expected" situations from (excuse the hand waving) a set with infinite cardinality to a set whose cardinality is infinity minus 1. If you think of NULL as effectively being a special type with a single value (call it "void"), then eliminating it reduces the number of classes of errors I have to worry about in a dynamic language by 0. I'm hard pressed to see any value there.

tatut• 8 years ago

This is backwards. Rich did not advocate for changes that break promises.

The point in the talk is that "strengthening a promise" should not be a breaking change. Changing return type from "T or NULL" to always returning T. The case where you previously couldn't guarantee a result, but now you can.

The other case "relaxing a requirement" also should not be a problem. The case where you previously had to give me a value, but now I don't need it and can do my calculation without it.

MichaelMoser123• 8 years ago

Checking if the optional is present is very similar to checking for NULL values. Now if you have a nice match statement like rust and lambda functions for streams, that may make things a bit more readable.

You will still need analysis tools to check that all code paths check for none before accessing that value ..

cphoover• 8 years ago

Option monad fixes null reference problems. Does not fix the ambiguity between not found vs doesn't exist.

lmm21• 8 years ago

If you have multiple possible reasons for a value to be "absent", that sounds like a case for an "either" or "result" type.

vectorEQ• 8 years ago

in C NULL is just 0. a nullptr in c++ is just a pointer which points to 0. so it's not an undefined value... it's set to 0 on purpose so you can check it.

consider this: char ptr; ptr = (char )0xb8000;

before assigning ptr, ptr can be ANY value from 'random memory'. (compiler trickery aside.. because it might initialise it to 0 anyway...)

so you want to have: char ptr = NULL; ptr = (char )0xb8000;

So you can then do IF(ptr != NULL) { do_stuff(); } you could not check for validity of the ptr value or it being present otherwise. an if(ptr) or if(!ptr) would only work if it's initialised and reset to NULL each time before assignment, so you can validate the assignment.

This is not mistake but a tool.

for a hardcoded offset like this it might be fair to say you could do if(ptr == 0xb8000) {do_stuff()} but what if it was a ptr returned by a new allocation or so? Or by taking the address of another variable or object? In that case setting things to null and checking them is absolutely essential to assuring your code works like you intended it to.

this whole article seems just a bunch of nonsense. for some languages it might hold true, but i can't beleive it would do for any. perhaps this original algol null... who knows.

pjc508• 8 years ago

Proper typesafe systems wouldn't let you use C-style reinterpretation casts either.

It's quite instructive to see how the low-level Rust people handle this.

AnimalMuppet2• 8 years ago

So if you have memory-mapped IO, how would you write to a specific address? In C/C++, a reinterpret cast is exactly what you need there. What would you use in Rust?

nayuki3• 8 years ago

I agree with pretty much everything in the article. However, I would give Java a lower score because no one uses java.lang.Optional in practice, and there is too much legacy libraries and application code that cannot or will not be changed. Also, the @NotNull annotation isn't in Java SE; it is made available through various third-party libraries.

A language with a null value can dramatically simplify things for a language designer, though. In the case of Java, we know that every array of objects is initialized to null references. Thereafter, we can construct and assign objects to each slot of the array. Otherwise we run into issues that C++ faces - when we construct the array, the field of every object is uninitialized, so they are potentially dangerous if read or destructed, and need the special syntax of placement new to be initialized. The trick to avoiding null here is to avoid pre-allocating an array, and instead to grow a vector one element at a time. The C++ std::vector<E> is very accessible and performant, whereas Java java.util.List<E> is very clunky to use compared to native arrays.

Another case that gets simplified is object construction. When the memory for an object has been allocated but before the user's constructor code has run, what values should the fields have, assuming that they are observable? In a Java constructor, all fields are initially set to null/0, then you simply assign values to fields in the body code of the constructor. In C++ constructors however, you should initialize fields in the initializer list, and then you have still have the option to initialize fields in the body.

I still think pervasive null values are bad for the programmer (rather than the language designer). Now that I have preliminary experience in Rust, I see that its design is much safer and still practical, so I think this language shows the way forward.

arcticbull3• 8 years ago

Re: Array Initialization

One approach you can take is the Rusty "hang up a technical difficulties sign" (unsafe) while you mess around with potentially uninitialized memory, which is valid, but places the burden on you as the library writer. Another would be to initialize your array of pointers as an array of Option<Box<T>> pre-filled with None. Due to pointer alignment you can actually optimize Option<Box<T>> by turning it into a tagged pointer (which I believe is what Rust does) so that None == null at the machine level, while the language exposes a safe interface on top. [1]

Re: Object Construction

With object construction in Rust you can either (a) create all fields in advance and specify them at construction [best] (b) use mem::uninitialized() [bad] or (c) create a builder which has optional fields for everything and yields a constructed option via 'a' later [most work].

[1] https://doc.rust-lang.org/std/option/

bluejekyll• 8 years ago

It appears that Java will eventually get Value types, which will allow for Optional to be defined on the stack, at least making it actually do something useful.

pella• 8 years ago

Julia Missing Values

"Julia provides support for representing missing values in the statistical sense, that is for situations where no value is available for a variable in an observation, but a valid value theoretically exists. Missing values are represented via the missing object, which is the singleton instance of the type Missing. missing is equivalent to NULL in SQL and NA in R, and behaves like them in most situations."

https://docs.julialang.org/en/v1/manual/missing/index.html

"First-Class Statistical Missing Values Support in Julia 0.7"

https://julialang.org/blog/2018/06/missing

ChrisRackauckas• 8 years ago

The nice thing about Julia is that it separates `nothing` from `missing`. nothing<:Nothing is a null that does not propagate, i.e. `1+nothing` is an error. It's an engineering null, providing a type of null where you don't want to silently continue if doing something bad. On the other hand, missing is propagates, so `1+missing` outputs a missing. This is a Data Scientist's null, where you want to calculate as much as possible and see what results are directly known given the data you have. The two are different concepts and when conflated it makes computing more difficult. By separating the two, Julia handles both domains quite elegantly.

tabtab• 8 years ago

From my practical experience, Null has a use, but is over-used or misused. If you concatenate a Null string to other non-Null strings, the Null should usually be treated like a zero-length string instead of nullifying the ENTIRE expression result. I know this differs from how numbers typically are treated, but so be it. Strings are not numbers.

Without that behavior, one often has to write verbose statements such as: denull(stringA,"") || denull(stringB,"") || denull(stringC,"") || denull(stringD,"") etc. ("denull" name varies per vendor. "||" is concatenate here.)

zyxzevn• 8 years ago

No. Undefined behaviour is.

prmph• 8 years ago

It is not possible to have a NULL type that works for all situations and has stable semantics.

The issue is, NULL should be a concept, not a value. I see no problem with using sentinel values, so long as they are well designed, and such good design comes with skill and experience, just as with all other aspects of architecture. The quest to have a single value that can be used for all the various possible meanings of NULL, to me, is the root of the problem.

lisper• 8 years ago

The quest to have a single value that can be used for all the various possible meanings of NULL, to me, is the root of the problem.

Exactly right. In particular, the conflation of nulls to indicate both error and non-error conditions (e.g. out-of-memory vs end-of-linked list) makes it impossible to distinguish errors from non-errors in many situations, and that is obviously bad.

Ideally you want nulls/sentinels that carry information about where, when, and why they were generated. You want separate nulls for numerical overflow/underflow, end-of-linked-list, out-of-memory, timeout, suppressed error/exception, unpecified/unknown value (preferably a separate one for each type) yada yada yada.

trophycase• 8 years ago

I don't write business software that needs many 9s of uptime so I find null to be fine. Yes, returning null is kind of throwing your hands up but I find that to be kind of the point. It allows my software to fail fast if it does and makes it incredibly obvious where things are going wrong. Generally if a reference has a value of null where it shouldn't, I can pinpoint the location of the bug within a few minutes or even seconds.

IMO it makes the program much easier to reason about compared to returning some sort of empty value and then failing much much later in the program.

jonstaab• 8 years ago

Thanks for the article! I've often heard that null is bad, but haven't ever seen such a thorough, readable explanation.

Just so I can think it fully through for myself, it seems that the problems with null are:

1. Its semantics are different from whatever type it is substituted for, so can't be used as a value

2. Superficially, it looks identical to a missing record value. This difference might be something you want to ignore (isNullOrEmpty), or something you care about (cache miss or hit with null)

3. It is used both for missing data, and missing functionality, which confuses two separate systems.

I agree that null as a type generally works better than null as a value, but I don't know if you can always articulate it as a type, especially in dynamic languages. A pragmatic solution seems to be a combination of:

- A Maybe type or monad. This forces you to unpack the nullable semantics of the thing, either in the type system or by unwrapping the value. A Maybe monad is a well designed interface for dealing with the edge cases, but it doesn't make the edge cases go away. This eliminates problem #1, and manages problem #2.

- Nil punning. (concat nil nil) yields an empty list in clojure. Same for +, string/join, etc. This is really similar to Monads/Types, but switches the responsibility for handling null intelligently from the data structure to the standard library. Putting null in the type forces you to opt in to null; nil punning forces you to opt out. This makes for more terse code, which is nice, but probably has a slightly narrower scope of application than monads, since it tackles problem #1 by making it make sense in most cases rather than eliminating it entirely, and nil punning doesn't always make sense. Incidentally, this seems closest to PHP's and javascript's strategy; their real problem is that they extend nil punning to cases where nil isn't involved (1 + '1' anyone?).

- Key or attribute errors. This is sort of a fallback to compensate for failing to handle the null case, but often works well when something just "shouldn't be null". This is probably just a substitute for a lack of compiler checks, but works well enough in the python world; sometimes failing hard is the right thing.

- Distinction between code and data. I like higher-order functions, so I'll just say that "sometimes data includes functions". But in most cases, the function you're calling should be resolved at compile time. Interfaces should be fully implemented, and (as in python), there should be a distinction between missing functionality (AttributeError) and missing data (KeyError).

Ultimately, it seems to be a question of language/api/user interface design: there is a difference between present, present and empty, and absent. Regardless of what strategy you use to manage the difference, there has to be one.

btbuildem• 8 years ago

Author forgot about Erlang, and it's wonderful lack of NULL

yxhuvud5• 8 years ago

Honestly, I feel that the problem isn't null but that type systems (at least earlier on) tended to allow other types to be null, willy-nilly. Null is best considered a separate type to non-null values, and is basically not a problem if the type system handles that in some way. Be it option or union types - both solve it and it mostly stops being an issue.

bitL• 8 years ago

If you have self-discipline and aren't lazy, NULL is totally fine. Maybe language designers could have made it behaving more like a 0-dimensional sentinel for beginners.

gambler2• 8 years ago

Rich Hickey's "Maybe Not" should be watched by anyone who thinks nulls/nils/undefineds are okay. It should also be watched by anyone who think that Optional/Maybe/Nullable and good enough:

https://www.youtube.com/watch?v=YR5WdGrpoug

shittyadmin2• 8 years ago

Conflating NULL with NUL terminated strings seems a stretch... both have problems, but separate problems. I suppose they're both related in that they provide a "special" value rather than separating that information though.

mynameishere• 8 years ago

I don't care. java.lang.Optional is a PITA in a way null never is.

sfilargi• 8 years ago

The problem is not the NULL but the lack of types.

beardyw2• 8 years ago

That 'nothing' is inconvenient applies the same in mathematics with zero. Why do we have to have a number we can't divide by? What is 0 to the power of 0? It's a special case we always need to worry about. But its inclusion in the number system is not in question.

And I remember my distress using a financial package being told that my unused zero value still MUST have a currency! My pocket is empty, how can it have a currency? If a farmers field is empty - must I say what it is empty of - cows, sheep, aardvarks?

I think worrying about inconsistency here is worrying about the inconsistency of the world we live in. 'Nothing' is a mysterious thing we need to accept and respect.

billpg• 8 years ago

I recall having a friendly argument with a friend who insisted that 30°C was exactly twice as hot as 15°C.

beardyw2• 8 years ago

It looks as if even in temperature zero is a bit problematic. https://en.m.wikipedia.org/wiki/Zero-point_energy

NULL: The worst mistake of computer science? (2015)

Related Stories