Go automatically downloads a newer toolchain if needed

https://kokada.capivaras.dev/blog/quick-bits-go-automatically-downloads-a-newer-toolchain-if-needed/

by pabs3 • 2 years ago

62 71 2 years ago

kokada.capivaras.dev

Sphax2• 2 years ago

I like this feature, I used this to be able to use iterators before Go was updated in Homebrew.

And yes you need to make sure GOTOOLCHAIN=auto, my colleague had to explicitly set it because it defaults to local on Gentoo.

Jyaif• 2 years ago

This sounds amazingly convenient! Language creators, please take note.

sgt• 2 years ago

Imagine if Python 2 automatically upgraded itself to Python 3.

iand• 2 years ago

Imagine if Python had as strong a compatibility promise as Go

IshKebab4• 2 years ago

Python 3 is not backwards compatible with Python 2 so why would it do that?

Maybe you meant "imagine if Python 3.6 automatically updated itself to Python 3.12"... Which would be amazing! Except that Python 3.12 is also not backwards compatible with Python 3.6. There's no saving Python.

trw99910192• 2 years ago

I don't like Python much, but they will definitely not pull off something like this (again), because they've actually learned from that mistake and broadly agree that it was one.

danslo• 2 years ago

After explicitly setting the python version to 3 in package.json? Yes please!

kokada8• 2 years ago

Author here.

This doesn't make sense. The toolchain will be updated only IF you say your project only runs in Go >= XX. This is what `go` directive in `go.mod` says: my projects doesn't work in versions of Go older than this. This is why it happened to me, I declared I need `go 1.23` because of range-over-func, and the toolchain respected my wishes.

This is not an "enforcing you to use the latest Go version, even if you don't wish for it".

chuckadams• 2 years ago

It’s more like if virtualenv looked in requirements.txt for the Python version and used that version locally. Other languages like Haskell and Scala have been doing that sort of thing for ages.

poincaredisk• 2 years ago

I'm not sure I like this feature. I prefer to be in control of the software that runs on my system, without any automated downloads. But I'm a nixos user so...

Sphax2• 2 years ago

Setting the GOTOOLCHAIN environment variable to local disables this. I don't use nix so I'm not sure if they set it by default but you can with set it with `go env`

srik• 2 years ago

I would've preferred that it be opt out or at least for it to be mentioned prominently enough at some point that it isn't a surprise.

doublerabbit3• 2 years ago

Setting the GOTOOLCHAIN environment variable to local disables this

I should be setting the GOTOCHAIN environment variable to enable this, opt-in not opt-out.

kokada8• 2 years ago

Author here, also a big user of NixOS.

But strangely I liked this feature, mostly because it allows me to use newer features without needing to wait nixpkgs to update the Go compiler to the version 1.23.

It is easy to disable if you don't like though, something like `environment.sessionVariables.GOTOOLCHAIN = "local"` or `home.sessionVariables.GOTOOLCHAIN = "local"` should do the trick.

kchr• 2 years ago

I can see the appeal, but hope they keep it opt-in so that people can choose their preferred workflow without surprises.

olivierduval• 2 years ago

Automatic updates are always a good idea... until they aren't ! It's an open door to problems ranging from subtle version semantic difference (try to find why what was working yesterday is not working today without any change) to hacking.

In corporate environment, it might trigger some unexpected alarms (or be blocked)

Gentoo got this right: by default, the "old" behaviour should be kept (no automatic update) and it should only be opt-in

jackielii• 2 years ago

It's not automatic though. You explicitly change your go.mod file and then it downloads the toolchains.

pjmlp5• 2 years ago

By the way, this only works well because Go binaries are static, one of the things that make the language reasonable good.

Only by default, as the Go toolchain supports dynamic linking for ages, and without third party dependencies called via cgo, or stuff like DNS resolution on Linux.

kokada8• 2 years ago

Author here.

What I meant is that the toolchain itself is linked statically, this is why it can be just downloaded from the internet and it will work without extra setup from the user.

Yes, Go will build the libraries once you call `go build`, and yes, they may link to the system C libraries for things like you said. But the `go` binary itself is static.

sestep3• 2 years ago

If I understand correctly, Rust acts similarly: if you have a `rust-toolchain.toml` file in your repo, any usages of `cargo` will automatically install and use that version of Rust. https://rust-lang.github.io/rustup/overrides.html

galangalalgol• 2 years ago

Yeah, I've run into that and don't love it. Build.rs files are probably a bigger concern from a supply chain attack standpoint.

cfreksen• 2 years ago

It seems like that will change in the (near) future according to the following github issue[0]. A quote from one of the developers, rami3l, in that thread[1]:

My current plan is indeed to remove implicit installations entirely.

[0] https://github.com/rust-lang/rustup/issues/3635

[1] https://github.com/rust-lang/rustup/issues/3635#issuecomment...

sestep3• 2 years ago

Oh sad! Dang I actually really liked the feature, it's super convenient for keeping developer environments in sync. I left a comment in that thread asking for clarification.

devsda• 2 years ago

Link to the proposal behind this change:

https://go.googlesource.com/proposal/+/master/design/57001-g...

Many people believe the go line in the go.mod file specifies which Go toolchain to use. This proposal would correct this widely held misunderstanding by making it reality.

That doesn't sound like a good reason to automatically download binaries and run them.

Is it difficult to update or install a new version of Go and are there frequent updates in Go spec introducing new features that it is necessary to auto install the compiler itself ?

Supply chain attacks are on rise and not a new concept and yet we see these changes.

This is not the first time Go lang has introduced a questionable opt-out feature [1]. They backed out but looks like there were no takeaways from that episode.

1. https://news.ycombinator.com/item?id=34771472

trw99910192• 2 years ago

Are you really surprised that people who explicitly refused to learn from 50 years of computer science and software engineering history are refusing to learn from their own mistakes, or even acknowledge them as such?

I'll be raising an issue with my distribution of choice to disable this behavior by default. They're the last line of defense for users' security and privacy, and we as an industry have been trying hard to circumvent them. Maybe precisely because of that reason, who knows.

teeray• 2 years ago

Is it difficult to update or install a new version of Go

Go is the easiest language I know to build from source, let alone run a binary distribution.

agwa2• 2 years ago

The binaries are fully reproducible from source[1] and are published in a transparency log to provide assurance that your go command is downloading the same binary as every other go command. This is state-of-the-art in preventing supply chain attacks and better than every other language.

[1] https://go.dev/rebuild

rjmill• 2 years ago

are there frequent updates in Go spec introducing new features that it is necessary to auto install the compiler itself ?

From what I've seen, the need for frequent updates is more about keeping security scans happy.

pjmlp5• 2 years ago

As usual, Go team shows how much they have learned from the history of computing.

matthartstonge• 2 years ago

Yeeeah, this made me lose half a day learning about and then unwinding “toolchain” being accidentally set across all our microservices…

Made a little harder to track down initially as we have a replace directive pointing to a local proto module in each service. It was here that toolchain was set which forced it to keep appearing in the top level module.

Should’ve been opt in behaviour by default not enforced.

We’ve decided to sit on n-1 of Go and I’ve already had an example last month of a well used OSS library setting toolchain to 1.22 as a patch release (which could be considered a breaking change) which fortunately was quickly reverted.

icholy6• 2 years ago

Should’ve been opt in behaviour by default not enforced.

It's not enforced, it's trivial to disable.

    go env -w GOTOOLCHAIN=local

mariusor• 2 years ago

Which parent argues that should be the default.

anotherhue• 2 years ago

Best worst thing about nixos is that none of these terrifying tricks work because the binaries can never find libc.

evomassiny• 2 years ago

I thought go binaries were purely static, aren't they ?

cpuguy832• 2 years ago

If you import net or os then you'll automatically link to libc unless you disable cgo or set the "netgo" and "osusergo" build tag respectively.

kokada8• 2 years ago

Author here and NixOS user.

This works even in NixOS, because `go` toolchain is statically compiled.

mseepgood• 2 years ago

The way to look at it: The Go toolchain is just another dependency to your project, declared in go.mod, just like the other modules, and handled accordingly (download, verification etc.)

seneca• 2 years ago

I'm broadly a Go fan, but this strikes me as a terrible decision. Critical tools should not be working through magic like this. Just fail with a clear error and let the user fix it as they see fit.

If they want magic like this, it should be opt-in, not opt-out.

tapirl4• 2 years ago

GoTV does similar things but plays it explicitly: https://go101.org/apps-and-libs/gotv.html

GoTV builds needed toolchain versions with the Go source code instead of downloading the pre-built toolchain packages.

igtztorrero• 2 years ago

It should be optional to activate this feature.

Go automatically downloads a newer toolchain if needed

Related Stories