Retrofitting null-safety onto Java at Meta

https://engineering.fb.com/2022/11/22/developer-tools/meta-java-nullsafe/

by mikece • 4 years ago

170 225 4 years ago

engineering.fb.com

BiteCode_dev• 4 years ago

For the same reason, this alone is why I use mypy: it will tell me when I risk of nulling myself with None in Python.

I still wish for ?, ?[] and ?. to offer failsfe for None, IndexError/KeyError and AttributeError one day.

djinnandtonic• 4 years ago

Or use Kotlin, it's near perfectly interoperable with Java and the acclamation period is measured in weeks if you're a Java dev

pjc502• 4 years ago

As they point out, they already have some Kotlin and use it interoperably, but "rewrite existing code in Kotlin" is not a realistic demand.

mdaniel• 4 years ago

My experience has been that "migrate on touch" is a reasonable strategy, so if you have to make a change to a file, use the "Code > Convert Java to Kotlin" (control-alt-shift-k)

lesuorac• 4 years ago

IIUC, there's already automated tooling that will do the conversion and not produce a giant mess like c/c++ to rust does so the cost is predominately CPU and not SWE.

stjkalsfd• 4 years ago

And simply delete however many millions of lines of java code already exists :)

iLoveOncall• 4 years ago

I can't think of any popular language that would take more than a few days to get acclimated to as an experienced developer, so that's not a very compelling argument.

rr888• 4 years ago

Ooof, I think most devs can modify an existing code base in a few days. To learn all the idiomatic styles, tradeoffs of the major libraries and different build systems take months, maybe years IMHO.

jlund-molfese• 4 years ago

Well, C and Scala are some counter examples that immediately come to mind.

Kotlin is probably more similar to Java than any other mainstream language. There’s almost no learning curve there, while going from Java to other “easy” languages like Python requires significantly more time to get used to.

yamtaddle• 4 years ago

It's always the (usually quite bad) tooling, learning about platform/SDK shittiness and pitfalls, and figuring out which parts of the open-source library ecosystem you want to engage with, that takes like 90+% of the time getting decent with a new language, in my experience. Getting comfortable with the language per se takes low tens of hours at most, as you wrote.

afandian• 4 years ago

I can't think of many other languages that will compile into a Java codebase, and be interoprable in both directions, as well as Kotlin. It's a lot quicker to pick up than e.g. Scala IMHO.

howinteresting2• 4 years ago

Rust can take a while if you don't know either C++ or Haskell.

TylerE• 4 years ago

C++? Java? Rust?

anyonecancode• 4 years ago

the acclamation period is measured in weeks

Ah, "acclimation" -- thought at first you were saying that Java devs love it for a few weeks and then hate it lol.

sedro2• 4 years ago

Kotlin interops nicely with Java but it's not null-safe (all references from Java are assumed non-null).

Larrikin3• 4 years ago

That is wrong all, non annotated Java code gets a special type denoted with a ! . You can use ?. or assign it to a new val with ?: return.

Or you throw all the null safety away and write unsafe java code but in Kotlin the same as the Java code you're working with.

pron21• 4 years ago

But then you have to use Kotlin, which isn't just Java with nullability types, but a language with quite a different design philosophy, and a language that is increasingly at odds with the evolution of the JDK (partly but not solely because it also targets other platforms, such as Android and JS). It appeals to some but certainly not to all (interestingly, it hasn't significantly affected the portion of Java platform developers using alternative languages, which has remained pretty much constant at about 10% for the past 15 years).

jgavris• 4 years ago

Migrating from Java to Kotlin looks nice and easy on the surface (optionals!), but the lack of checked exceptions will absolutely bite you sooner or later if you are consuming Java code. Better carefully read the docs and source of all your transitive Java dependencies.

dehrmann2• 4 years ago

I tried Kotlin, and while I liked it, iterop was still somewhat annoying, Java's lambdas are better, using the Java 8 stream API is ugly, and the code ends up being similar enough that I'd rather use Java and avoid tooling hassles.

jillesvangurp• 4 years ago

The article actually addresses Kotlin. They'd love to switch to it but they just can't do it overnight because they have so much mission critical Java code. So, this is a stop gap solution for legacy code. They published another article some time ago how they are switching to Kotlin: https://engineering.fb.com/2022/10/24/android/android-java-k...

Migrating millions of lines of code is a non trivial effort. They'll be stuck with bits of Java for quite some time. So, this helps make that less painful.

pjc502• 4 years ago

C# has made it possible to gradually roll out stricter nullability checking as well. The static analysis gets integrated into the regular language analyzers. Incremental migration is the only way to go.

mhoad• 4 years ago

In a similar fashion I was extremely impressed with how Dart approaches this as well and was able to move the entire ecosystem along in about 18 months. https://dart.dev/null-safety

endtime• 4 years ago

Yeah, I did a bit of Dart null safety migration work at Google and I thought they did a good job. (The auto migration tool sometimes made things nullable unnecessarily, and so you had to carefully review its output, but you could always just not use the tool.)

munificent• 4 years ago

I work on Dart.

It was a lot of work to get there and we're still not totally there yet. Dart still supports running legacy code that isn't null safe. But I'm really glad we did this and very grateful that Leaf and others on the team were able to design a null safety type system that:

* Defaults to non-nullable.

* Is fully sound.

The latter in particular is very nice because it means the compiler can take advantage of null safety to generate more efficient code, which isn't true in languages like Java, C#, and TypeScript where their null safety type rules deliberately have holes.

zasdffaa• 4 years ago

Some rare nice words from me about C# - the not-null facility is great. It's very basic propagation that should be better and I've fought the compiler too many times over it, but I really like the compile time guarantee it gives.

AtNightWeCode5• 4 years ago

C# have taking it further. New projects give errors if you don't declare vars as nullable when they may be null in some context. It will help for static analyzers to find these cases. I see a big risk in it. That people will just init nonsense objects to get around the warnings.

Deukhoofd2• 4 years ago

That people will just init nonsense objects to get around the warnings.

You technically can do "= null!", which means assign null by default, but assume it is not null. This is currently the recommended way to do deserialization, where you know the value is not null, but you have not explicitly filled it.

rndgermandude• 4 years ago

It sure can be a problem when people initialize stuff with nonsense to avoid running into nullable warnings. However, accidentally running into null pointer exceptions is still worse than people deliberately footgunning themselves with bad workarounds.

I feel C#'s nullable has helped me personally to avoid a lot of potential bugs and also changed the way I write code in a lot of places - like creating `bool Try...(..., out var)` style APIs instead of "old school" returns-null/throws style stuff, which I think make a lot of code cleaner and more easy to read.

Sometimes nullable can get a little messy and annoying, especially when retrofitting old code to make use of it without breaking existing APIs, and all in all the way C# does it is a clear net win in my opinion.

pjmlp6• 4 years ago

And we are still not using it, due to the pile of code out there.

Lets not forget the whole drama with prefix ! planned for .NET 7.

revskill• 4 years ago

OK, then i realize GoF design patterns lack the single most important pattern: Option data structure.

twawaaay4• 4 years ago

Nobody ever claimed that GoF is a complete set of patterns. It is just a subset of patterns from infinite variety of possible solutions to problems.

howinteresting2• 4 years ago

A very good argument is that the GoF stuff should be looked at with suspicion because they didn't bother nailing down the fundamentals.

jerf• 4 years ago

Nobody should have ever claimed the GoF is a complete set of patterns. The GoF themselves vigorously said it was not intended to be.

But it has definitely been raised up to The Official Set Of Design Patterns by a lot of people. I've lost count of the number of languages I've seen someone write about "design patterns" in, and what they mean is they show a complete ported implementation of all the GoF design patterns, including all OO quirks, even in languages where they are manifestly inappropriate, including dynamically typed languages (where many of them apply, but are optimally designed quite differently to account for the huge differences in the type system) and functional languages (when many of the GoF patterns just dissolve into the language, but a whole other set of patterns is necessary).

justin_oaks3• 4 years ago

The null problem has been around a very long time. It's been a huge source of errors in programming. I'm glad to see it actively being worked on, even if it is awkwardly being retrofitted on an old language.

I can only look on with disappointment when new (or newish) languages include null as part of the language instead of doing something more sensible like having an Option/Either type. Of course, I won't name names... cough Go cough

brundolf• 4 years ago

It's okay for null to be a part of the language as long as the type system can reason about it (via nullable-types, sum types, etc). Dart, TypeScript, Kotlin are examples. But yeah, Go is the problem-child that doesn't have any way of statically reasoning about nulls

justin_oaks3• 4 years ago

Yes, it's OK to have null in the language in limited cases. I'm fine with null so long as the language enforces checking for null when it could cause a NullPointerException (or that language's equivalent).

Java's biggest failure in null handling was allowing EVERY reference type to be null without a way of specifying that it could be non-null. This basically ensured that null could creep in almost anywhere.

Allowing a type to be null should always be opt-in. It still bothers me that I have to put NOT NULL in almost every SQL column definition because the SQL spec has columns default to being nullable. Ugh. At least SQL does allow me to choose nullable vs non-nullable, unlike Java.

Arbortheus• 4 years ago

I agree about Go. One quite frustrating scenario I have ran into multiple times is if you unmarshal (parse) some JSON into a struct, you can’t tell if an integer field with value 0 was specified in the JSON input as 0, or missing from the input. AKA there’s no way to differentiate the zero value from undefined for primitives.

There are roundabout hacky ways like parsing the JSON into a map[string]any to check if it’s there, but it’s so ugly and requires so many lines of code.

coin• 4 years ago

cough Go cough

Go is an old style language developed and released in modern times

pjmlp6• 4 years ago

Very old, given the time it has taken to catch up with CLU.

travisd• 4 years ago

I also hate this about Go, but its (partial) saving grace here is the `x, err := NewX()` pattern which (at least for me) tends to prevent a decent number of these issues in practice since usually either `x` is non-nil xor `err` is non-nil.

Makes the

    // Java
    name = personService.getPerson(123).getName()

problem less likely since you'd generally have to write:

    // Go
    person, err := personService.GetPerson(123)
    if err != nil { ... }
    name, err := person.Name()
    if err != nil { ... }

Definitely more verbose than maybe

    // TypeScript
    personService.getPerson(123)?.getName()

but I think that's part of Go's tradeoffs -- much more likely that errors will be annotated more correctly (i.e., in Go you'd be more likely return an error like "failed to load person: 123" if it was the GetPerson call that failed rather than a generic error that doesn't describe which step failed)

exabrial• 4 years ago

This is just covering up design problems. NPEs show you were you have design deficiencies.

If you have getAccount().getContact().getPhoneNumber() and contact is null, you'll get an NPE. The question shouldn't be: "How do I shove the NPE under the rug for the next 1337 coder to deal with?", the question should be: "How did I initialize an Account without a Contact?"

random314• 4 years ago

Yes and Null safety forces you to ask that question at compile time instead of runtime

chrisseaton• 4 years ago

the question should be: "How did I initialize an Account without a Contact?"

Well yeah that's exactly what static analysis can tell you - what code path led to that.

You're saying this project should be what it already is.

commandlinefan3• 4 years ago

How did I initialize an Account without a Contact

Actually I already know - he got it from a JSON feed from an external system and ran that JSON through an automated code generator.

mrkeen7• 4 years ago

This is just covering up design problems.

Static analysis (detecting) is the opposite of "covering up".

"How did I initialize an Account without a Contact?"

As long as your language lets you do it, your teammates will do it. And in many cases it won't be accidental either. Your teammate will want an Account with its Contact set to null.

commandlinefan3• 4 years ago

getAccount().getContact().getPhoneNumber()

Every time I see people "deal" with this problem, it looks like this:

    if (getAccount() != null &&
        getAccount().getContact() != null &&
        getAccount().getContact.getPhoneNumber() != null)  {
       // do something
    }
    // don't put an else condition in, just keep going and let the program
    // produce the wrong result in a confusing way when it happens in production

I actually blame rampant code generation and an adamant refusal to even consider object-oriented design for this problem - this sort of case ought to be handled by something like:

    getAccount().callContact()

(or whatever you were going to do with the phone number when you got it). Insistence on generating code from SQL schemas or XML DTDs and then writing procedural code makes that pretty much impossible, though.

colejohnson66• 4 years ago

With that awful `if` statement, you're calling the functions multiple times. It's negligible with the simple getters Java users use everywhere, but if that `getAccount()` call involved, say, the database, now you're making multiple calls when you don't have to.

C# at least has null propagation and pattern matching that can make that line:

    if (getAccount()?.getContact()?.getPhoneNumber() is string pn) {
        // do something with `pn`
    }

The "idiomatic" C# way would also include properties:

    if (GetAccount()?.Contact?.PhoneNumber is string pn) {
        // do something with `pn`
    }

return_to_monke• 4 years ago

Surprised that guard clauses weren't mentioned here yet, you could also write

``` if (account == null) { throw NoAccountException }

if (account.contact == null) { throw AccountWithoutContactException }

if (account.contact.phoneNumber == null) { throw ContactWithoutPhoneNumberException }

// do stuff here ```

kaba03• 4 years ago

You mention code generation as a problem, but it is actually a solution as well for this particular problem, I have really grown on mapstruct, where you create an interface declaratively specifying what field maps to what, and it will generate at compile time fast, efficient, correct code you might write by hand.

bcrosby95• 4 years ago

That sort of design easily leads to a god object.

Class based OOD is way harder than that. There's a reason codebases are full of these train wrecks (the unofficial official name of the pattern). And it's not because of auto generation and/or laziness. It's because, despite what's on the tin, passable - nevermind good - OOD doesn't look anything like how we actually think of objects as humans.

Thaxll• 4 years ago

There are other problems such a null vs empty string in the database for example.

Once you start dealing with API + conversion + db, if your language exposse null, it will be painful.

java_expert_123• 4 years ago

No it doesn't, it just forces these designs to be apparent at compile type and fail there instead of blowing up at runtime.

Do you not know why Kotlin exists or why dart added null safety or why C# added null safety? Or were they also wrong in their choices?

godshatter• 4 years ago

An aside here, but wouldn't it be simpler to eschew the syntactic sugar and get the account first and then if it's not null then get the contact and if it's not null then get the phone number and return it, returning null if any of those conditions fail? Then there's no exception to handle and flow is not broken.

A null phone number would indicate one wasn't found, which could mean the contact record didn't have one, the account didn't have a contact record, or there was no account. If this violates a business rule, then throw a MissingAccount or MissingContact or MissingPhoneNumber exception at the appropriate place that would be more understandable to the end user or could be handled more specifically than a generic null pointer exception.

I'm a C programmer, though, so I'm used to taking the long way around to do things, usually because there's no other choice.

BoorishBears7• 4 years ago

It's sad because while I don't think the second concern is so big, the first half of your comment is deceptively important, but this is a conversation that really difficult to have.

I've brought it up when using Swift and Kotlin and it takes so much energy for people to realize that safety operators, which feel really really good to use, accidentally sweep up major issues

People assume you're missing something when you explain that they need to get more comfortable with force unwrapping (which is intentionally introducing a NPE)

But for example, if you have getAccount().getContract().getPhoneNumber() so you can show it in a settings page, you should not throw an NPE. You can easily correct the issue in a non-invariant breaking way by say defaulting to "<missing>" in the UI.

But if you have getAccount().getContract().setSomeFieldWithRealWorldImportance(true), you should either be force unwrapping or returning a Result type, and ensuring someone is actually handling it with a hard stop on the user.

The problem is reality is often much more subtle than "doSomethingImportant()".

For example, I worked on an iOS app that was used for medical work, and one habit iOS devs had was wrapping access to weak references to objects with a null safety:

So they were super comfortable writing things equivalent to viewController?.showSomeDialog()

because in most situations crashing over lifecycle issues is a bad idea... but here we could be creating incorrect output in a highly sensitive situation.

"someDialog" could be "Drug interaction between X and Y detected" for example, but the mental patterns wouldn't detect that a safety operator was hiding that.

It was better for the app to crash than silently hide that information, since at least then it'd be known the app was in an invalid state.

Now the problem is here devs start to think "well I'm not making medical apps" or "well I wouldn't make that mistake"...

But go through any moderately large codebase and you will find important user actions that silently fail where a crash would actually serve the user better.

It's how to convince people that a hard crash can ever be good though.

jmartrican• 4 years ago

Thank you for stating this. I been pulling my hair out trying to comprehend how more focus is not put on that fact. Whether your language is null-safe or not, you have to deal with this. Like for example, if I have a type that is serialized from JSON to Kotlin type and one of the fields is not-nullable but the JSON does not have that field, you still going to get an error, right? With Java, at least you'll be able to do something about it and not just fail the request at the these null-safe touchpoints.

driggs• 4 years ago

It's so sad that Java 8 had the chance to really fix the null problem, but gave us only the half-assed `java.util.Optional<T>`. Rather than implementing optional values at the language level, it's just another class tossed into the JRE.

This is perfectly legal code, where the optional wrapper itself is null:

    Optional<String> getMiddleName() {
        return null;
    }

sanedigital• 4 years ago

Don't people use @NonNull everywhere now? It's been a few years since I've programmed in Java but even then I feel like that was common practice.

tyingq• 4 years ago

Seems common, though it only helps so much with 3rd party libraries.

mrkeen7• 4 years ago

Yes, it's always been possible to check for nulls at runtime.

Personally I use notNull(..) over @NonNull since it actually fires when you expect it to (as opposed to whether your framework dispatcher interceptor trigger decided to invoke it)

BoorishBears7• 4 years ago

In my experience mostly people forced to use Java who are then consuming said code in Kotlin do that

melling• 4 years ago

“Unannotated types are considered not-nullable”

Defaulting to not-nullable is a great idea. Much less boilerplate.

joshlemer• 4 years ago

Which @NonNull are you referring to, Lombok's? Or javax.annotation.NonNull, or something else?

mrkeen7• 4 years ago

It may have been hoped for, but it was an unrealistic hope.

They couldn't have fixed it while retaining backward compatibility.

masklinn4• 4 years ago

They couldn't have fixed it while retaining backward compatibility.

Opt-ins exist. That's what .net ended up doing: by default the language uses "ubiquitous" nullability, but you can enable reference nullability support at the project or file level.

If you do that, references become non-nullable, and have to be explicitly marked as nullable when applicable.

bagels• 4 years ago

I love this solution. Now you have two kinds of nulls.

modeless• 4 years ago

Just like JavaScript!

britch• 4 years ago

I disagree. The problem with Null in my opinion is that it is the default and can easily be created accidentally.

There's nothing inherently wrong with an "absence of value, value"

Optional.empty is not null, it's no-value

pron21• 4 years ago

What do you mean by had a chance? That chance isn't gone. This area is under investigation, and when we have a solution we like, we'll implement it. We can't address all issues at once.

Optional isn't half-assed because it was never envisioned as a general solution to the null problem. It's simply an interface that's useful in streams and other similar cases for method return values, and it does the job it was intended to do reasonably well. It's possible that a solution to nulls in the type system could have obviated the need for Optional, but I don't think that delaying lambdas and streams until such a solution was found would have been a better decision.

johannes1234321• 4 years ago

So then there is language-level Optional and library-level Optional?

jimbob45• 4 years ago

This area is under investigation, and when we have a solution we like, we'll implement it

What's to investigate? Just copy what C# did and be done with it.

BoorishBears7• 4 years ago

It's like when people say Java had the chance to do generics right (like C#) and then didn't

Yeah technically tomorrow morning Java could fix it, but there have been kingdoms built on the current situation. C# took its lumps years back on breaking things and so there were fewer kingdoms to demolish. And if you fix it, it'll be years before it trickles down to a large portion of devs who don't get to work at the bleeding edge, or even near the edge.

hn_throwaway_992• 4 years ago

I have definitely come to the conclusion, especially after doing a lot of work in Typescript, that class Optional is a big mistake, whether from the JDK or other libraries that preceded it.

First, because exactly the type of code that the parent commenter showed. I've actually seen this in production code (and shrieked). The fact is that without language-level support, you can end up getting the worst of all worlds.

Second, like all things in Java along the lines of "why use 1 character when 10 will do?", the Optional syntax is verbose and annoying to use.

But, most importantly, the fundamental issue is that all classes are optional by default in Java (indeed, that's the problem at hand). Adding an Optional class doesn't really mean you can make any non-nullability assumptions about other classes, especially when using different libraries. The Optional class just added more complexity while simplifying very little.

hedora• 4 years ago

Optional isn't half-assed because it was never envisioned as a general solution to the null problem. It's simply an interface that's useful in streams and other similar cases for method return values, and it does the job it was intended to do reasonably well.

Regardless of the intent, once it got into the hands of enterprisey java devs, it immediately got used in two ways:

Tri-value nulls: treating null, None and Some in different ways.

Annotation hell: spam nullabilty annotations like @Nonnull Optional haphazardly everywhere with no semblance of soundness in sight.

When actually used as intended (by the 90 percentile coders on the team), it just bludgeons the GC to death because the JIT does not always do the right thing and elide the extra heap indirections.

They’re almost as bad an idea as type erasure!

dehrmann2• 4 years ago

I wonder if the plan is to add syntactic sugar for Optional<T> in a future version.

layer8• 4 years ago

It wouldn’t be compatible with regular nullable references.

nigerianbrince• 4 years ago

    @NonNull
    Optional<@NonNull String> getMiddleName() {
        return null; // error
    }

kaba03• 4 years ago

Or even just setting the default to non-nullable with any static analysis checker.

joefkelley• 4 years ago

Do you feel that a language has to have something at the language level to prevent NPEs?

In my experience, Scala does pretty well without it.

I guess is your point that the language should make it impossible to write bad code, not just make it easy to write good code?

kaba03• 4 years ago

Just a heads up, Scala 3 actually has a compile flag that will make types exclude ‘null’ as a valid subtype, so every nullable variable will have to have type signatures like String | Null.

lazulicurio• 4 years ago

imo it seemed like there was this phase of "if we pretend null doesn't exist maybe it will go away" that resulted in a bunch of design issues. Beyond Optional, the other big one to me is Map.compute/Map.merge, which subtly breaks the previous interface contract for Map. As annoying as null is, I'd rather have null than have broken apis.

Aardwolf• 4 years ago

Isn't Object itself (and any class) already an optional at the language level?

Either it's null, so has no value, or it's not null so has a value, and you can check for null-ness. What more is needed?

Seems like the opposite, a guaranteed non-null value at core language level, would be novel instead...

usrusr3• 4 years ago

Right, if only there was a language on the JVM that did that /s

It's really an eye-opener when you compare kotlin and scala, with all their superficial similarities: Where kotlin simply takes the @Nonnull annotation, promotes it to a core language feature and drowns it in convenient function-scope syntactic sugar until it's actually nice to use, scala opts for Option and stacks layer upon layer of architecture trying to make Option somehow disappear. I lost half a decade holding on to plain java snobbishly dismissing kotlin as a second class scala before I finally got converted.

cheriot• 4 years ago

Scala had the same possibility, but the ecosystem of libraries used Option<T> appropriately so in practice I never thought about null. That might be harder in an older, larger ecosystem like Java...

dheera2• 4 years ago

I'm not a Java person but isn't this just very similar to std::optional in C++?

https://en.cppreference.com/w/cpp/utility/optional

masklinn4• 4 years ago

No, because in C++ bare types are not nullable. In java, reference types are nullable, Optional being a reference type, Optional<T> can be null, Empty, or have a value, whereas T can be null or have a value.

Also because std::optional does essentially the opposite. It’s a more efficient _ptr rather than a safer one. Optional<T> is strictly less efficient as it implies an additional allocation.

krzyk8• 4 years ago

One can abuse everything, but should one? (and that error should be catched by any IDE or some static analysis).

Optional are quite nice and we use it where appropriate, they play nice with streams which is a nice bonus.

pharmakom3• 4 years ago

Optional is not just another class. The control flow in the compiler will check that optionals are checked.

didip2• 4 years ago

Java has so many deficiencies in its design that so many frameworks are invented to cover its flaws.

Just give a real Optional type at a language level. It’s clearly possible in other JVM languages.

rad_gruchalski2• 4 years ago

Other jvm languages compile to the byte code and fix this by inlining the optional transparently.

What you mean by “at a language level” would break the byte code for jars compiled with older jdks.

didip2• 4 years ago

Breaking backward compatibility is not the end of the world.

mrkeen7• 4 years ago

Java has so many deficiencies in its design that so many frameworks are invented to cover its flaws.

Java the language has its flaws, but the frameworks make the situation worse.

Just give a real Optional type at a language level. It’s clearly possible in other JVM languages.

It's not the presence of Optional that we need, it's the absence of nulls. And nulls are in all the libraries. And you also have to convince your fellow Java developers that null is bad.

brabel• 4 years ago

We've been using `@Nullable` as Meta does in our code base for many years now (everything else is assumed to be non-null). Initially, we had CheckerFramework actually doing the checks at compile time... but eventually had to remove it because it's unstable, slow, has not kept up with Java evolution (we're on JDK 17, I think Checker still barely works on JDK 11) and also, because our IDE can be configured to flag errors on `probable nullability issue`! That was really a boon for us... it costs nothing at compile time and it's pretty easy to not mess up as the IDE keeps us in check whenever we make any change.

At the same time, we have adopted Kotlin as well. Interestingly, most of our developers prefer to stay on Java... not because they love Java, but because of familiarity and that they seem to think it's "good enough".

Anyway, we haven't had a large rate of NPEs for a very long time and when one actually happens in the wild we're extremely surprised!

That said: looking forward to the specification[1] that's being cooked up as Java absolutely should have had this a long time ago (as mentioned in the post).

[1] https://jspecify.dev/docs/start-here

clumsysmurf• 4 years ago

At the same time, we have adopted Kotlin as well. Interestingly, most of our developers prefer to stay on Java... not because they love Java, but because of familiarity and that they seem to think it's "good enough".

I have been using Kotlin on Android for a few years now, and really enjoy the extra expressiveness and conciseness that the language offers ... but have come to the conclusion its much easier to write unreadable kotlin code than java code.

One of the things I think makes Kotlin hard to read is scoping functions (apply/also/let/run/with). Based on their names alone, the newcomer can't tell what they do (I had to consult a table for weeks on whether they returned the original object, last expression, and whether they passed it, this, etc).

I do love these when used at the right place, right time, and "just enough" but I keep coming into contact with code that nests these 4-5 levels deep:

  aaa?.run {
    bbb?.run run2@ {
      ccc?.run {
        ddd?.run {
          this@run2.foo()
         // whats in scope here urgh
        }
      }
    }
  }

Other pathologies I see are making any method (foo), that operates on X, as an extension method X?.foo(). Then the namespace for X gets polluted with all these things.

One of the great things about Kotlin's nullability in the type system is the operators like ?. and ?: ... but often see code like this:

  aaa?.bbb?.ccc?.ddd()

Is it safe, yes, yes it is. It won't crash, but what happens if one of these is null, and should not be? There is nothing in Kotlin forcing people to write this way, but find Java's explicit checking of null made people think more of the "negative case" (throw an exception? etc)

Like anything else, I guess we could use linting rules, it comes to having good taste ... which is not in abundance.

What worries me the most though, is that in the beginning, Kotlin was mostly additive on Java. I read, in biological systems, adding is easier that changing or removing. It seemed like Effective Java + Typed Groovy. Now as Java has "last mover" advantage, and adopts the good parts of Kotlin, I am worried its not just additive, they need interop between different implementations (records vs data classes).

titzer• 4 years ago

Whoa, interesting. I didn't know Kotlin had all those constructs.

In Virgil, a method on an object (or ADT) can declare its return type as "this". Then the method implicitly returns the receiver object. That trick is very useful to allow a chain of calls such as object.foo().bar().baz(). I find it readable and easy to explain:

https://github.com/titzer/virgil/blob/master/doc/tutorial/Re...

Larrikin3• 4 years ago

In your first example, thats poorly written code that should never make it past code review. I'm also having a hard time thinking of a case where run would even be useful in that situation that wouldn't also fix your second example.

If you need all aaa through ddd to be non-null you return early or you check ahead of time

  aaa ?: return
  bbb ?: return
  ccc ?: return
  ddd ?: return

  if (aaa != null && [the rest] != null) foo(aaa, bbb, ccc, ddd)

Since if foo needs them all to be not null they should be passed in

  aaa?.bbb?.let{ bbb -> //definitely should not be null in this case
    bbb.ccc?.ddd()
  } ?: return // or throw an exception

BoorishBears7• 4 years ago

My workplace is trying to get onto the KMM train and I get sweaty palms trying to imagine people onboarding onto Kotlin from other languages.

I mean I love Kotlin, but as you mention there is just so much syntax. The simplest things can be done 100 different ways and it's hard to say what's idiomatic.

I'm personally going to be ruthless about "syntax simplicity" in code reviews. Otherwise I'm imagining a situation where the dream of shared code really just becomes "Android devs throwing a black-box library over the wall at iOS devs" because only they spend enough time with Kotlin to understand it

sixbrx• 4 years ago

Checker framework works fine on JDK 17 since not long after the release

_ZeD_• 4 years ago

I really hoped that projects like lastnpe[0] will have more visibility.

0. http://www.lastnpe.org

Traubenfuchs• 4 years ago

I have been writing Java for money for more than ten years and never ever have I had non-trivial problems with null. Less than one percent of the bugs I fixed were caused by nullpointers, less than one percent of write-deploy-test loops were caused by it.

I either have code that can't be null (e.g. getters of lists that create a list if the field is null, outright validation before usage), code where null has a desired meaning that must be handled (new JPA entity with no key yet) or code where the Nullpointer will lead to a client/user/implementor caused bad request style response.

What the hell are you guys doing for you to spend significant amounts of time on NullpointerExceptions?

On the same note, I didn't understand the inclusion of Optional in Java. Always felt like some annoying special flavor type of custom Java nuisace like vavr or jooq.

rightbyte• 4 years ago

Null pointer exceptions are like free unit tests. The programmer assumed something never signaled an invalid result and was wrong.

I don't understand the fuss either. Logic problems from improper error handling is way worse than NPE.

Edit: Pointers that can be null actually are a sum type but without the fancy pants hype.

justin_oaks3• 4 years ago

There are patterns that experienced developers will use to avoid issues with nulls. Here are a couple off the top of my head.

1) Reversing string comparisons:

    "literal".equals(variable)

instead of

    variable.equals("literal")

2) Always initializing lists instead of leaving them to default to null

   public class SomeClass{
      private List someList=new ArrayList(); //or Collections.emptyList()
      ...
   }

The problems with NullPointerExceptions can mostly be solved by teaching programmers to use these strategies and teaching them to look for boundaries where nulls can slip in.

The real problem happens when things get scaled up. If you have thousands of developers then it makes sense to take steps to automate the problem away. Also, if the code is used by billions of users then even rare NullPointerExceptions will happen frequently. So it makes perfect sense for Facebook to be the one to work on this.

ben77992• 4 years ago

The funny thing is these techniques were introduced in books like "Effective Java" before many of the developers who complain were even born.

Everyone should know these things by now. They are good things to ask about in interviews.

ahtihn• 4 years ago

On the same note, I didn't understand the inclusion of Optional in Java.

It interacts nicely with stream pipelines.

ben77992• 4 years ago

Way too many developers are lazy and haven't actually studied the language they're using to adapt their code. So instead you get discussions where the tools are blamed.

Obviously there are a lot of lazy/bad developers and the larger the company/team the more likely you have some of them. The irony is when this comes up in discussions about places like Google and Facebook that have built up a cult of believing they are better than everyone else while seemingly having lots of trouble dealing with these kinds of issues.

I don't hate Optional, but what I find is the same people who can't handle using null also fall into the Optional anti-patterns.

mrkeen7• 4 years ago

The slow-down doesn't come from NPEs occurring as bugs.

The slow-down occurs from all the extra thinking that happens once [your teammate] allows nulls into the code "as a feature".

If you assume no nulls, then you get to lean on the type system. A String is actually a String, not an instruction to GOTO the nearest enclosing RuntimeException handler.

And honestly, I'd be mostly happy to work on the no-null assumption, hit a few NPEs, and fix them as I go. But someone else will decide that nulls are OK as values, and now every value is suspect. Which means you can no longer just blindly assert against nulls.

And that's the value-add of Optional. You can use it to deliberately represent things that aren't there, meaning you can go back to treating every null as a bug to squish, rather than a design choice.

mrcrumb1• 4 years ago

If only there were a language feature that could tell you which case it was and ensure you handled it correctly...

thedougd• 4 years ago

At a former workplace, someone had a great idea to collect and aggregate the logs of our software from across our test labs. At the time, the idea was a bit novel for a bunch of programmers and testers who shipped and didn't run their own software. The plan was to drive elimination of commonly logged errors through bug fixes or reducing the severity of the messages.

I was a lowly programmer, invited to a meeting of distinguished engineers and senior staff to review some of the findings from the log collections. When NPEs came up, their comment was that 'there should be zero NPEs' and that they would handle them punitively through reports to managers. So, I asked, "What's your recommendation to the developers to prevent the NPEs?". Silence, nothing. Findbugs existed at the time, and I believe JSR303 was in draft. Happy to report they decided to treat it the same as any other error or bug.

pulse7• 4 years ago

@Oracle: Please add this solution to the Java language and make Optional deprecated!

pron21• 4 years ago

Why this solution? Maybe there's an even better solution? We're following the various experiments in the area, and when we have a solution we believe is the right one, we'll provide it, but not sooner than that.

holysantamaria• 4 years ago

Just use C#. :P

pjmlp6• 4 years ago

Not always an option, and even if that is possible, plenty of existing code doesn't work with non nullable references.

fedeb95• 4 years ago

Beware of expected nulls and expected NullPointerException in your legacy code when exterminating nulls.

mbfg2• 4 years ago

I always wondered why enums in java were allowed to be null. Seemed like something they could have enforced. Could have just had some sort of 'default' attribute on a value.

masklinn4• 4 years ago

I always wondered why enums in java were allowed to be null.

Because they’re just the “type-safe enum” pattern, built in, with some compiler support (e.g. switches). Besides that they’re normal reference types.

Could have just had some sort of 'default' attribute on a value.

Universal defaults are, if anything, even worse than universal nulls: they generate garbage states you can’t guard against at all.

mbfg2• 4 years ago

Just trying to learn here, not being combative, couldn't the same compiler support have just enforced the non nullness? At first you think, how do you enforce

someEnum = getAnEnumSomewhere(), but at some point there's got to be an original reference being hard-set to null that the compiler could see, right?

also i don't understand your point about universal defaults?

enum Foo { bar default, baz };

why can't you guard against the default? how is checking a foo enum against bar different then checking against null?

tinglymintyfrsh• 4 years ago

Null References: The Billion Dollar Mistake by Tony Hoare

https://www.infoq.com/presentations/Null-References-The-Bill...

cletus• 4 years ago

So after working at Google for 6 years (and doing Java and other things prior) I went to Facebook and programmed in Hack for 4 years. Hack is actually a really interesting and convenient language in many ways. One way I really appreciated is how nullability is built into the type system. Having to deal with nullability in Java is actually way more of a chore than any of the verbosity that people normally complain about (eg boilerplate around anonymous classes).

The story I like to tell about how people abuse Java is this:

    @nullable Optional<Boolean>

I don't know if this is still the case but you could literally find many instances of that in Google3 (and not all of them were the product of an auto-generated tool). I mean it's a super-large code base. There was a cleanup at one point for the (many) instances of 1204 in the code base.

The way I like to describe this is "for when 3 values for your boolean just aren't enough" (since you obviously now have 4 states: 1) null 2) Not set 3) Set, true 4) Set, false).

I have a few rules that have held up v ery well over the years including:

1. If you don't do arithmetic on it and it's not an ID of some kind, it's almost certainly not a number (eg people who store SSNs as INT fields); and

2. It's almost never a boolean. You're almost always better off replacing any boolean with an enum. Not only is this typesafe but if done right adding an enum value will cause compiler errors because you've been exhausitve with switches without a default case.

RedNifre• 4 years ago

Oh yeah, I wrote one of those, too, complete with safe navigation through type aligned sequences etc. https://github.com/cosee/null4j

If I were still writing Java I would throw in some exception throw/catching based Haskell style do notation for binding nullables, i.e.

  var nullableResult = nullable(() -> {
    var a = bind(getNullableA());
    var b = bind(getNullableB());

    doSomethingRequiringBothNotBeingNull(a,b);
  });

In my experience, Java developers don't want null safety though (the ones that do switched to Kotlin anyway).

Zak• 4 years ago

It amazes me that nullable references keep getting put into otherwise strongly statically typed languages. In 1965, it was completely understandable. By 1995, the harm should have been well understood.

AdmiralAsshat• 4 years ago

Did Meta release Nullsafe out to the world? Or is it an internal tool at the moment?

__float• 4 years ago

Does anyone have experience using this at Meta who can compare to https://github.com/uber/NullAway ?

Retrofitting null-safety onto Java at Meta

Related Stories