Docker is deleting Open Source organisations - what you need to know

https://blog.alexellis.io/docker-is-deleting-open-source-images/

by alexellisuk • 3 years ago

1556 734 3 years ago

blog.alexellis.io

koolba• 3 years ago

Can we just get the big three cloud players to make a new public repo? They’ve got oodles of bandwidth and storage, plus the advantage that a lot of access would be local to their private networks.

Setup a non-profit, dedicate resources from each of them spendable as $X dollars of credits, and this problem is solved in a way that works for the real world. Not some federated mess that will never get off the ground.

pimterry3• 3 years ago

Consensus on a new repo for public community images would help, but it isn't the biggest problem (as the author notes, GHCR does that already, and GitHub seem pretty committed to free hosting for public data, and have the Microsoft money to keep doing so indefinitely if they like).

The issue I worry about is the millions of blog posts, CI builds, docker-compose files, tutorials & individual user scripts who all reference community images on Docker Hub, a huge percentage of which are about to disappear, apparently all at once 29 days from now.

From a business perspective particularly, this looks like suicide to me - if you teach everybody "oh this guide uses Docker commands, it must be outdated & broken like all the others" then you're paving a path for everybody to dump the technology entirely. It's the exact opposite of a sensible devrel strategy. And a huge number of their paying customers will be affected too! Most companies invested enough in Docker tech to be paying Docker Inc right now surely use >0 community images in their infrastructure, and they're going to see this breakage. Docker Inc even directly charge for pulling lots of images from Docker Hub right now, and this seems likely to actively stop people doing that (moving them all to GHCR etc) and thereby _reduce_ the offering they're charging for! It's bizarre.

Seems like a bad result for the industry in general, but an even worse result for Docker Inc.

JustBreath• 3 years ago

Yeah that's going to be the real issue, all the niche unmaintained images that no one is going to pick up the pieces for.

They're taking a big chunk of open source and tossing it in the garbage.

miyuru• 3 years ago

aws already have one https://gallery.ecr.aws/

schneems3• 3 years ago

My team used ECR for some stuff and it’s not great. We want to move on from it.

8organicbits• 3 years ago

Unlimited free downloads inside AWS. First 5TB of outbound transfer free. Then $0.09/GB for additional transfer.

https://aws.amazon.com/ecr/pricing/

remram4• 3 years ago

quay.io is a pretty popular general-purpose repo, it replaced docker.io for many projects when they started rate-limiting.

jacooper13• 3 years ago

There is no free tier https://quay.io/plans/

nikanj• 3 years ago

All free tiers everywhere are going away, because someone always figures out how to run a crypto miner on them.

fathyb• 3 years ago

How can you mine crypto using Docker Hub?

orangepurple• 3 years ago

Nice try

Tao3300• 3 years ago

https://youtu.be/1EF6kB9q4vg

madduci• 3 years ago

Storage ain't free as well, someone uploads huge images

pilif• 3 years ago

Traffic is probably the higher cost

Quekid5• 3 years ago

Are they actively trying to hasten their own demise?

I guess I'll be moving our (FOSS) container images to GitHub Registry... but what a pain :/

toastal• 3 years ago

Can you move to GitLab? GitHub is purely proprietary which doesn't match the spirit of FOSS. At least GitLab is open core.

martypitt2• 3 years ago

Gitlab have been increasing pricing heavily recently, especially in storage-heavy areas such as containers.

I don't think they're likely to be a cost-effective differentiator against what Docker are now charging.

jck• 3 years ago

iirc gitlab's free tier has a storage and bandwidth limit which applies to the container registry too. That makes gitlab a no go if you want to share containers widely.

candiddevmike• 3 years ago

Just did this yesterday, it was surprisingly easy. I used the HashiCorp Vault secrets plugin for GitHub and pushed containers via GitHub Actions, so for me it became more secure than storing and retrieving a docker hub API key.

davedx2• 3 years ago

Github also charges for usage. Are they also hastening their demise?

AWS also charges you to host things in S3. Is it hastening its demise too?

This argument is just weird to me.

nottorp2• 3 years ago

Hmm.

GitHub's Container Registry offers free storage for public images.

But for how long?

agumonkey• 3 years ago

But for how long?

the subtitle of the web era

preciousoo3• 3 years ago

The hippies were right

Kwpolska2• 3 years ago

They have a pretty good track record in code hosting (15 years), why would they ruin it for containers?

bflesch• 3 years ago

Github shutting down is not on my risk matrix. Hopefully it happens after I retire.

jnwatson• 3 years ago

The 6-8 orders of magnitude difference in storage required?

trey-jones2• 3 years ago

But for how long?

applies very much to Github, and for all of their services, not just containers. The elephant in the room, hardly needs to be mentioned I would think (Microsoft). We are only a few years into that regime.

pimterry3• 3 years ago

Unlike Docker Inc, GitHub (via Microsoft) do have very deep pockets & their own entire cloud platform, so they can afford to do this forever if they choose.

And their entire marketing strategy is built around free hosting for public data, so it'd take a major shift for this to disappear. Not to say it's impossible, but it seems like the best bet of the options available.

Is it practical to set up a redirect in front of a Docker registry? To make your images available at example.com/docker-images/abc, but just serve an HTTP redirect that sends clients to ghcr.io/example-corp/abc? That way you could pick a new host now, and avoid images breaking in future if they disappear or if you decide to change.

shaunn• 3 years ago

I like where your head is at; I found this [1] and it makes a case that an attack vector may be created.

[1] https://stackoverflow.com/a/67351972

nottorp2• 3 years ago

so they can afford to do this forever if they choose.

If they choose. It's in fashion right now to fire people and squeeze free tiers.

wvh2• 3 years ago

A simple vanity domain solution, like for Go packages, seems like it could work. Just redirect the whole URL directly to the actual registry's URL.

I don't know if container signature tools support multiple registry locations though.

cauthon• 3 years ago

And their entire marketing strategy is built around free hosting for public data

    1. Embrace
    2. Extend       <- you are here
    3. Extinguish

it's bonkers when people innocently trust Microsoft to do the right thing

jon-wood2• 3 years ago

Docker the tool has been a massive benefit to software development, every now and then I have a moan about the hassle of getting something bootstrapped to run on Docker, but it's still worlds better than the old ways of managing dependencies and making sure everyone on a project is aligned on what versions of things are installed.

Unfortunately Docker the company appears to be dying, this is the latest in a long line of decisions that are clearly being made because they can't work out how to build a business around what is at it's core a nice UI for Linux containers. My hope is that before the inevitable shuttering of Docker Inc another organisations (ideally a coop of some variety, but that's probably wishful thinking) pops up to take over the bits that matter, and then hopefully we can all stop trying to keep up with the latest way in which our workflows have been broken to try and make a few dollars.

frognumber• 3 years ago

Do you remember SCO?

emeraldd• 3 years ago

Honestly, this is reminding me of Oracle after buying Sun.

spicyusername• 3 years ago

    ideally a coop of some variety

This is the role I feel like podman, the tool developed by Red Hat, is filling.

quaintdev• 3 years ago

Podman is great and is first class citizen on Fedora. It also integrates nicely with SystemD. My only gripe with it is not many developers provide podman configuration on their install pages like they do with docker compose

jacooper13• 3 years ago

Its not as easy nor as simple as docker + docker compose.

raesene94• 3 years ago

This is more about Docker hub than Docker.

Image hosting is expensive at scale, and someone's got to pay for the compute/storage/network...

sofixa3• 3 years ago

How is a tool developed by, and strongly pushed by (to the point of strongarming customers to transition to their tool, features lacking be damned) a corporation, especially one owned by IBM, filling the role of a coop-developed tool?

londons_explore2• 3 years ago

Docker should have been a neat tool made by one enthusiast, just like curl is.

Instead it has a multi-million dollar company behind it, and VC's who demand profits from a thing that shouldn't have ever had a business plan.

aviramha2• 3 years ago

I think that Docker can have a viable business plan but they had terrible execution. At my previous position, I wanted to use DockerHub more heavily but the entire experience was like a bootstrap project someone did as a university assignment. Many advanced features for organizations were lacking (SSO/SAML) that we would have happily paid for.

pydry2• 3 years ago

I think they potentially could have made a decent business out of it but they made a lot of bad business decisions.

I find myself shaking my head at a lot of their technical decisions too.

Podman seems to me to be a case study for how to do this right.

voytec7• 3 years ago

Docker should have been a neat tool made by one enthusiast, just like curl is.

I have nothing but mad respect for Daniel Stenberg. 25 years of development of great software, for which he had been threatened[1] and had ridiculous US travel visa obtaining issues[2].

[1] https://daniel.haxx.se/blog/2021/02/19/i-will-slaughter-you/

[1] https://news.ycombinator.com/item?id=26192025

[2] https://daniel.haxx.se/blog/2020/11/09/a-us-visa-in-937-days...

wvh2• 3 years ago

In all fairness, curl is purely a software tool. Docker is arguably more like a service. As such, it creates costs for and direct dependency on the entity behind it.

steveBK123• 3 years ago

It seems like another symptom of zirp/cheap money.

Lots of ideas that could have been a neat feature or tool somehow ended up raising $500M of funding with no viable plan on ever monetizing.

The fact that the product is successful but after a decade they barely make $50M/year of revenue against $500M of lifetime funding is crazy. As a user, you can work at a company with a billion in revenue and barely owe them a few thousand/year. Or you might just use Podman for free, and prefer it due some of the design differences.

At the very least, a lot of these firms, with VC pressure, overstayed their welcome as private enterprises and should have sold themselves to a larger firm.

jzb• 3 years ago

Indeed. Docker should've been plumbing. They could've had a really nice business with developer tools on top of the core bits, but they decided to try to jump straight to enterprise and did a number of things to alienate partners and their broader community.

Instead of adding value to Docker they're just trying to find the right knobs to twist to force people into paying. And I think people should pay for value when they're using Docker substantially for business. But it seems like a very short-minded play for cash disregarding their long-term relationship with users and customers.

All that said: They have to find revenue to continue development of all the things people do like. I'd encourage people to ask if the things they've gotten for free do in fact have value, and if that's the case, maybe disregard the ham-fistedness and pony up if possible.

streetcat1• 3 years ago

what do you mean "demand profit"?

Last time I check rent is not free, food is not free, bus ticket is not free. No reason why software should be free.

Open source was invented by big co as a "marginalized your complement" strategy, not the ideal that is marketed as. As an evidence, I do not see any cloud vendor open source their code?

kajaktum• 3 years ago

Instead it has a multi-million dollar company behind it, and VC's who demand profits from a thing that shouldn't have ever had a business plan.

But you don't have to host curl? Who's gonna put the money to host all the images and bandwidths that ten of thousands of companies use but never pay?

codexb• 3 years ago

Yeah, but curl is used to access and download all sorts of data, which are all hosted by multi-million dollar companies. Just like git downloads and uploads data to git repositories. curl and git are valuable, but so is GitHub, and websites in general. The problem is that they haven't found a way to monetize docker hub.

krmboya• 3 years ago

The VCs offered free bandwidth and storage to gain market share.

Bandwidth and storage is not ultimately not free, it has to be paid for.

ollien• 3 years ago

FWIW, Docker was not intended originally to be a tool for commercialization; it grew out of dotCloud, which open sourced the tool as a last-ditch-effort of sorts, if memory serves.

waynesonfire• 3 years ago

Yeah! I should be able to get 50x value from software and not pay for it /s

The open source community that carrier Docker on its back and is now bending over. Let this be a lesson to you. If you're building open source, maybe stick to open source solutions in your tech stack and if it's not there build it. This is what Apache does for the Java ecosystem.

I don't have sympathy, the writing was on the wall and this isn't the first time it's happened to the community.

ihucos• 3 years ago

Coming trough: https://github.com/ihucos/plash 90% done and useful

wslh• 3 years ago

Yes, even when it was launched was obvious because used they packaged and configured existing solutions. It was like a company behind 'ls' (irony).

0xbadcafebee• 3 years ago

Docker was created by dotCloud for a different purpose than it ended up as. I think they are owed credit for what has become an incredibly elegant solution to many problems, and how great the user experience has always been.

Compare it to other corporate-managed tools like Terraform and Ansible. Both of them have horrible UX and really bad design decisions. Both make me hate doing my job, yet you can't not use them because they're so popular your company will standardize on them anyway. Docker, on the other hand, is a relative joy to use. It remains simple, intuitive, effective, and full of features, yet never seems to suffer from bloat. It just works well, on all platforms. There were a few years of pain on different platforms, but now it's rock solid.

And to be fair to them, their Moby project is pretty solidly open-source, and if Docker Inc dies, the project will continue.

bydlocoder4• 3 years ago

A co-op formed by big 3 cloud providers flush with cash and put in maintenance mode.

pmarreck7• 3 years ago

worlds better than the old ways of managing dependencies and making sure everyone on a project is aligned on what versions of things are installed.

And Nix is worlds better than even this. Imagine!

jon-wood2• 3 years ago

I’m yet to be entirely sold on that, mostly because I think Nix the language isn’t anywhere near as accessible as Dockerfiles, but I’ll be the first one cheering if Nix does manage to take over.

djbusby2• 3 years ago

How to run Nix on Gentoo and Debian?

Kinrany• 3 years ago

Does Nix have an equivalent of docker-compose yet?

nix-shell is amazing for installing binaries, but actually wiring up and running the services doesn't seem like a solved problem.

Unless Nix expects a separate tool to do this once binaries are installed, of course.

timcobb2• 3 years ago

Unfortunately Docker the company appears to be dying

Docker the company is crushing 2022-2023… record revenue and earnings

tyingq6• 3 years ago

Of course, I can do a lot of record revenue and earnings selling five dollar bills for $4. I'm curious what a path to profit would look like...is this kind of squeeze the only way to get there?

osigurdson6• 3 years ago

I'd like to see Docker succeed. They invented / formalized the space and deserve credit for that. They are probably doing the right thing with some of their development tooling (though maybe that should just be spun off to Microsoft) and ensuring images do not contain badware is something companies will pay for.

However, their core offering must be the leader if they want to survive. Devs must want to use "docker run" instead of "podman run" for example. Docker needs to be the obvious #1 for starting a container on a single machine.

wongarsu• 3 years ago

their core offering must be the leader if they want to survive. Devs must want to use "docker run" instead of "podman run"

If their core offering is container hosting, they should be able to make a company out of that even without the client. After all jfrog and cloudsmith are more per less just that, as is github.

gorjusborg• 3 years ago

I'd like to see Docker succeed. They invented / formalized the space and deserve credit for that.

If by succeed, you mean they deserve to have revenue, I disagree.

They spun some cool work out of dotCloud when it failed. They seemed to delay thinking about how they'd monetize the work, and sort of fell into charging for developer tooling after their orchestration play lost to kubernetes.

At this point, I think of Docker the company as a wannabe Oracle. They are desperate for money, and are hoping they can fool you into adopting their tech so they can ransom it from you once you rely on it. If that sounds appealing to you, I'd say go for it.

For me, that situation seems worse than what I do without containers at my disposal. In other words, the solution is worse overall than the problem.

voytec7• 3 years ago

they can't work out how to build a business around what is at it's core a nice UI for Linux containers.

It's quite a shame (for the lack of better wording) that the better, simpler and more intuitive a free product is, the harder it is to make money from it by selling support.

I think that the best way to go from here, would be building companion products and supporting the whole ecosystem. By companion products, I mean other standalone apps/services, not just GUI for existing one.

pjmlp• 3 years ago

It packaged containers that we already knew from other UNIX and mainframe/micros systems.

never_inline• 3 years ago

Sir computers nothing but we already known from Alen Turing.

JustSomeNobody• 3 years ago

Unfortunately Docker the company appears to be dying, this is the latest in a long line of decisions that are clearly being made because they can't work out how to build a business around what is at it's core a nice UI for Linux containers.

It should have been just a small company, doing this, and making some money for their trouble instead of whatever it is they're trying to be.

usrusr• 3 years ago

That might be possible for soemthing that "only" writes and distributes software (that can also work on levels of success much less stellar than sqlite), but when you try to pull off hosting services at internet scale like docker hub did, with some free tiers, it's just not feasible to include it all in the bills of user subset that is actually willing and able to pay. The playbook is always the same: "maybe we could do that high impact free tier if we had more funding?" and that "whatever it is they're trying to be" is an implicit consequence of that, that ambition comes with the funding. "just a small company" mean bootstrapped, and bulk hosting like docker hub would simply not happen.

user39393822• 3 years ago

I'd like to see something resembling the Linux model. In the case of Docker, a foundation built around a suite of open source tools that's contributed to by pledges from all the big companies that use the tool. Maybe that means podman has a reliable source of funds for maintenance and improvement.

What I don't like is having these critical tools directly in the hands of a single for-profit corporation, at least where it can be avoided.

Pet_Ant• 3 years ago

If we want that I feel like there should be a community buy out. Just so that they have something to return to investors. Just so that they have incentive to play nice and not Hudson-up the process. You shouldn't be able to build a critical piece of infrastructure and have nothing to show for it. Community buy-out should be a viable exit plan.

Iolaum• 3 years ago

Does that mean it would be a good idea to start moving to the podman ecosystem? RedHat/IBM seem to have this figured out better.

I m doing that personally but I m very hesitant about mentioning that to $job.

msla• 3 years ago

My hope is that before the inevitable shuttering of Docker Inc another organisations (ideally a coop of some variety, but that's probably wishful thinking)

Indeed. We should all be equal in that venture: Ain't nobody here but us chickens.

rendaw3• 3 years ago

Why didn't Docker ever offer managed container hosting? That seems like the obvious logical next step when you create a tool for easy deploys. Instead it's 2023 and we finally get that with Fly.io.

I must be missing something obvious, because otherwise I feel like I'm going insane.

nickstinemates5• 3 years ago

They did[1].

It was later shut down.

1: https://venturebeat.com/business/docker-acquires-container-d...

college_physics• 3 years ago

Can't comment specifically on this or that "dying company", but it is a bit disappointing that after, how many, four decades of open source? and the obvious utility of that paradigm, it still seems a major challenge to build sustainable open source ecosystems. This means we can't really move on and imagine grander things that might build on top of each other.

Its not clear if that is due to:

i) competition from proprietary business models

ii) more specifically the excessive concentration of said proprietary business models ("big tech")

iii) confusion from conflicting objectives and monetisation incentives (the various types of licenses etc)

iv) ill-adapted funding models (venture capital)

v) intrinsic to the concept and there is no solution

vi) just not having matured yet enough

What I am driving at is that building more complex structures requires some solid foundations and those typically require building blocks following some proven blueprint. Somehow much around open source is still precarious and made up. Ideally you'd want to walk into the chamber of commerce (or maybe the chamber of open source entities), pick a name, a legal entity type, a sector and get going. You focus on your solutions, not on how to survive in a world that doesn't quite know what to make of you.

Now, corporate structures and capital markets etc took hundreds of years to settle (and are still flawed in many ways) but we do live in accelerated times so maybe its just a matter of getting our act together?

trasz3• 3 years ago

It's because the incentives to make money quickly end up being stronger than incentives to build a sustainable open source ecosystems.

hot_gril2• 3 years ago

It's still doing better than it could be. Big tech companies have played way nicer than they had to, focusing more on vague long-term presence than on immediate profits, and imo continue to do so to a lesser extent. There always comes a point when the innovation is done and they lock things down again, but even then they have to fight their own employees.

zelphirkalt• 3 years ago

With lots of open source licenses, there is no copyleft. Without copyleft, for profit companies can simply take the hard work, add a little on top, make it proprietary, and sell it. Customer mentality is to use the most comfortable thing, without paying attention on whom they depend, often choosing the proprietary offer, because of feature X.

There are healthy ecosystems, even some partially replacing docker, some with more daily updates than I can process, but they have copyleft licenses in place and are free software, to ensure contributions flow back. Companies can still make profit, but not from adding a minimalistic thing and making it proprietary. They need to find other ways.

hot_gril2• 3 years ago

First time I saw Docker, I thought "that's great, but how do they make money?" They're selling a cloud containers service while also giving the software away to their direct competitors for free. Maybe I was too ignorant to understand their business model? But now I'm thinking I was right.

AtlasBarfed• 3 years ago

So.... podman?

No, I don't work for redhat. I'm glad a ... ?less? corporate entity / ?more? open source entity has pretty much gotten a replacement up.

alerighi• 3 years ago

To me is the opposite, Docker promotes bad software development practices that in the end will hurt you. In fact most of the time when you hear that you need Docker to run a software is because that software is so badly written that installing it on a system is too much complex.

Another bad use of Docker that I've seen is because people cannot figure out how to write systemd units, that is damn simple (just spend a day to read the documentation and learn the tools that you need). Of course that makes administering the system so much complex because you cannot use the benefits that systemd will give you (thus you start using iperoverengineered tools like kubernetes to just run a webserver and a database...).

I'm maybe oldschool but i use Dockers as a last resort, and prefer to have all the software installed properly on a server, with the use of Ansible as a configuration management tool. To me a system that uses Docker containers is much more difficult to manage in the long run, while a system that doesn't use it is more simple, thus less things that will break, thus if I need to make a fix in 10 years I ssh in the system, edit the program with vim, rebuild and restart the service, no complex deploy pipeline that break, depend on external sources that may be taken down (as is the case) and similar stuff.

orf3• 3 years ago

I think you are letting your specific feelings and head-canon about purity be mistaken for solid technical arguments.

If you’re sshing to boxes, editing things by hand and slinging ad-hoc commands around then your frame of reference is so far away from understanding it’s value proposition that it’s probably pointless to discuss it.

pxc7• 3 years ago

To me is the opposite, Docker promotes bad software development practices that in the end will hurt you. In fact most of the time when you hear that you need Docker to run a software is because that software is so badly written that installing it on a system is too much complex.

I think one reason you may be seeing downvotes here is that you have specific projects in mind, and without you naming them, others who haven't used such projects don't see how real the phenomenon is.

I was recently helping a friend work through some Nix configuration and he told me about a couple of different projects he used where deploying the software any way other than via Docker was treated as either officially or de facto unsupported. In some cases, dependencies are not even exhaustively named in the documentation. When users ask questions in community channels (often on Discord) about what the software's requirements are, they are (at least sometimes) directed to just use the pre-baked Docker images instead of receiving real answers to their questions.

This is second-hand info for me. I don't know how bad it really is, or how common, either. But that kind of thing a absolutely screams to me, too, 'very few of us actually know how this thing works'.

Still, sharing that sentiment without giving a specific account of software that you've seen fall into this trap is likely to be dismissed and downvoted. Maybe it would be helpful to give some concrete examples of what you brought all that to mind for you.

6nf• 3 years ago

Docker was always ‘oh we have too many moving parts in the deployment pipeline, let’s add Yet Another Thing. That will fix it’ It never fixed anything.

jszymborski• 3 years ago

Has there been any work on making these centralised public repositories distributed?

bandrami• 3 years ago

What work needs to be done? Provision a server somewhere and host it. AWS has one-click "give me a docker hub" and "give me a git hub" products.

alias_neo• 3 years ago

This has been happening recently with Helm chart repositories etc. Maybe it's time people started hosting their own Container Registries?

The huge bandwidth requirements are an incentive to keep images small.

angio• 3 years ago

Would it be possible to build something on top of DHT+Torrent? Main issue seems around image discovery.

qbasic_forever3• 3 years ago

Yes containerd supports pulling images from IPFS: https://github.com/containerd/nerdctl/blob/main/docs/ipfs.md

delfinom3• 3 years ago

Except you need to pay for pinning IPFS files to actually persist. And there-in lies the crux. People don't got money to pay for their small time docker containers.

The cost of some of the IPFS hosts that will give you a dropbox of sorts still end up costing roughly the $20/month similar to the Docker $25/month for a team account.

RobotToaster2• 3 years ago

I was just thinking this seems like an ideal use case for IPFS or bittorrent, since most users will already be running on a server.

It doesn't seem unreasonable to have the client automatically pin/seed the container it pulls.

anacrolix• 3 years ago

Hit me up if you want to discuss using BitTorrent to back images. https://github.com/anacrolix/torrent

jiggywiggy• 3 years ago

It was always unbelievable too me how much they hosted for free. I recklessly pushed over 100gbs of containers the last years, all free. Never made sense to me, even google doesn't do this anymore.

wenyuanyu2• 3 years ago

There are techniques to compress and dedup redundancies... I doubt it is real 100gbs on their disks...

1010112• 3 years ago

Even so...storage is not free.

Looking at the rates of enterprise storage costs compared to what Google or Apple charges consumers - I was surprised by how subsidized people's photo libraries are.

qbasic_forever3• 3 years ago

It's still 100gb over the wire and those bandwidth costs add up, especially if it's a popular image used by tons of projects.

remram4• 3 years ago

We are even using Docker Hub to store and distribute VM images... The so-called "container disk image" format is sticking a qcow2 file in a Docker image and storing it on a Docker registry.

https://github.com/kubevirt/kubevirt/blob/main/containerimag...

sneak2• 3 years ago

There is a way to go from that status quo to a new and sustainable one; it requires actual engineering of the state change.

That has not seemed to have happened here, or not happened well.

Nobody expected it to be free forever; I think we expected the transition to be a lot more orderly. There have been years to prepare.

lloydatkinson• 3 years ago

Is there any progress on podman for Windows or any other way of running containers on Windows? I cannot wait for the day the development community doesn't need to rely on anything from this company.

taspeotis• 3 years ago

I use podman and it works fine?

Podman Desktop runs podman machine for me at startup.

Containers set to restart automatically don’t restart across podman machine restarts but that hasn’t upset my workflow much (at all?). I just start containers as I need them.

girvo3• 3 years ago

I wonder if VSCode’s Remote Container stuff works with Podamn on windows. The Docker Desktop + VSCode + Remote Container Dev + the ESP-IDF tooling is the nicest way to do production ESP32 dev with multiple ESP-IDF version (and even without that is just much simpler to get up and running on Windows, despite the rube Goldberg machine-esque description of it)

orra• 3 years ago

Both Podman Desktop and Rancher Desktop are viable options for running Linux containers on Windows.

danielnesbitt• 3 years ago

Is there something in particular missing? I have been using Podman for Windows almost daily for the past six months. There is no management GUI built-in like Docker for Windows, but I have not found that be a problem at all.

FlyingSnake2• 3 years ago

Start publishing images to GitHub

And when GitHub starts similar shenanigans, move out to where? I am old enough to know the we can't trust BigTech and their unpredictable behaviors.

Eventually we need to start a Codeberg like alternative using Prototype funds to be self reliant.

1: https://codeberg.org/ 2: https://prototypefund.de/

davedx2• 3 years ago

It actually sounds reasonable to me? They have an open source program, the article says its open source definition is "too strict" because it says you must have "no pathway to commercialization".

I mean why should you expect someone to host gigabytes of docker images for you, for free?

JonChesterfield3• 3 years ago

Well, it's how they established themselves in the market. Without being friendly to open source projects they wouldn't have had that marketing and wouldn't exist as a company.

So now they destroy their foundations and learn whether they 10x or fold. Pretty standard VC playbook so I assume that's the driving force here.

lmm• 3 years ago

If you're going to call it "open source" that should mean what "open source" usually means, i.e. that e.g. RedHat is eligible.

rollcat• 3 years ago

the article says its open source definition is "too strict" because it says you must have "no pathway to commercialization"

What a load of crap. Free Software's "0th freedom" is the ability to use the program for whatever purpose you wish. The definition of Open Source is even looser than that. They are asking their "Open Source" users to make their software non-free, by restricting its use cases.

Anyway, the writing has been on the wall for a long while. If you haven't moved off Docker Hub yet, now is the time.

tommoor• 3 years ago

While I have no _expectations_ of free hosting, one example of a project that will be affected is mine – https://hub.docker.com/repository/docker/outlinewiki/outline

I have been building this for 5+ years, and offer a community edition for free while the hosted version is paid. Once the community edition starts costing money there will be even less reason to continue supporting it, it already causes a lot of extra work and problems that I'm otherwise uncompensated for.

gwd• 3 years ago

Gitlab's Open Source program has similar restrictions, and it's just kind of weird. Like, there are multiple companies actually making money off of Xen; but because Xen is owned by a non-profit foundation (with a six-digit yearly budget), and the foundation isn't trying to profit, it still qualifies. (As does, for instance, the GNOME project.)

OTOH, somewhere else in this context it was mentioned that curl is almost entirely maintained by one guy who makes money from consulting; and because of that, he wouldn't qualify.

So if you're either small enough to be a side hobby project, or large enough to have your own non-profit, you can get it for free; anywhere in between and you have to pay.

Personally I'd be happy for Xen to pay for Gitlab Ultimate, except that the price model doesn't really match an open-source project: we can't tell exactly how many people are going to show up and contribute, so how can we pay per-user?

BlueTemplar4• 3 years ago

Somewhat related : what is Docker's stance on the licenses that fail the first Open Souce test, those that forbid commercial use (NC) ?

JonChesterfield3• 3 years ago

We need to use a distributed system instead of a centralised one. Probably built on a source control system that can handle that.

RobotToaster2• 3 years ago

This seems like a perfect use case for IPFS.

mac-chaffee2• 3 years ago

May want to keep your eye on dragonfly, a P2P image distribution protocol: https://d7y.io/docs/

gooob• 3 years ago

a collection of distributed systems. start your own. connect with other enthusiasts in your area to get them connected.

dijit3• 3 years ago

What happened to the old mirror lists? The ones where apt/rpm package repositories tend to be hosted?

delfinom3• 3 years ago

Yea, people are really spoiled due to more than a decade of VC and general investing cashburn offering tons of services for free. But at the end of the day there are costs and companies will want to recoup their money.

The problem with just replacing GitHub isn't the source code hosting part. There's tons of alternatives both commercial and open source. The problem is the cost of CI infrastructure and CDN/content/release hosting.

Even moderating said CI infrastructure is a nightmare. freedesktop.org which uses a self-hosted gitlab instance recently had to shutdown CI for everything but official projects because the crypto mining bots attacked over the last few days hard and fast.

JeremyNT• 3 years ago

It actually seems pretty reasonable to let BigTech host stuff, so long as you know the rug pull is going to come. Let the VCs light money on fire hosting the stuff we use for free, then once they stop throwing money at it figure out a plan B. Of course you should have a sketch of your plan B ready from the start so you are prepared.

If you view all of this "free" VC subsidized stuff as temporary/ephemeral you can still have a healthy relationship with it.

5e92cb50239222b2• 3 years ago

This is how I've been living for many years and it has saved me many thousands of dollars, which is a significant amount of money here. The various "cloud" free tiers cost them at least $600 for the past year alone. Same for free CI offerings, etc. Thank you VCs and BigCo for not cutting out regions that are probably net negative for you overall (I guess it may be serious money for me, but doesn't even register on the radar at their scale).

user39393822• 3 years ago

It should use DHT/BitTorrent. Organizations could share magnet links to the official images. OS projects have been doing it for years with ISOs.

screamingninja• 3 years ago

BitTorrent will solve the distribution problem but not magically provide more storage. Someone still has to foot the bill for storing gigabytes (or terabytes) worth of docker images.

Szpadel• 3 years ago

I think something like IPFS would be perfect for that, you have some layers pulled in your storage anyways.

big projects could self host easily, as their popularity would quickly give them enough seeds to not need to provide much traffic themselves.

also I think adapting docker way of storing layers as tars is fundamentally broken, maybe with combination with something like ostree as a storage to decrease duplicates we could really cut a lot of storage.

imagine how much unique content does your average docker image have? 1 binary and maybe few text files? rest is probably os and deps anyways.

nindalf• 3 years ago

And when GitHub starts similar shenanigans

The difference between GitHub and Docker is that GitHub is profitable.

whydoyoucare• 3 years ago

Profitable today. Moving from docker to github is just kicking the can down the road.

vhanda• 3 years ago

Genuine question: Is Github profitable? I can't seem to figure out if it was before or after the acquisition from Microsoft.

FlyingSnake2• 3 years ago

So is Docker Inc. The last I heard it is profitable and is doing quite well

r3trohack3r4• 3 years ago

The economics of hosting an image registry are tough. Just mirroring the npm registry can cost $100s per month in storage for tiny little tarballs.

Hosting GB images in an append-only registry, some of which get published weekly or even daily, will burn an incredible amount of money in storage costs. And that’s before talking about ingress and egress.

There will also be a tonne of engineering costs for managing it, especially if you want to explore compression to push down storage costs. A lot of image layers share a lot of files, if you can store the decompressed tarballs in a chunk store with clever chunking you can probably reduce storage costs by an order of magnitude.

But, at the end of the day, expect costs for this to shoot into the 6-7 digit USD range per month in storage and bandwidth as a lower bound for your community hosted image registry.

ijaeifjzdi2• 3 years ago

you just have to host the recipe and the hash/meta-data

c'mon. This is not amateur hour. Hosting the whole thing only made sense for docker because their plan was always to do this microsoft style play.

If you assume you are either open source or fully closed enterprises, the problem is very, very easy to solve. and cheap. Just relinquish full control of being able to close all the doors for a fee, like they are doing now.

yrro7• 3 years ago

The funny thing is that I don't think anyone has ever had to pay a Linux distribution for hosting packages of their open source software... :thinking face:

maxloh• 3 years ago

I don't think we will receive enough donations to cover infrastructure costs, let alone maintainers' salaries.

Even core-js sole maintainer failed to raise enough donations to feed his own family, despite the library is used by at least half of the top 1000 Alexa websites. [0]

People (and also big-techs) just won't pay for anything they can get for free.

[0] https://github.com/zloirock/core-js/blob/master/docs/2023-02...

BlueTemplar4• 3 years ago

I guess the SQLite team managed to do it by using an even more permissive license than GPL, which attracted big companies into funding them ??

robotburrito• 3 years ago

Maybe we all start hosting this stuff via torrent or something?

syklep• 3 years ago

Codeburg is more strict for blocking projects at the moment. Wikiless is blocked by Codeburg for using the Wikipedia puzzle logo but is still up and unchanged on GitHub.

quickthrower2• 3 years ago

Hosting images can be doe via hypertext transfer protocol alone. Keep a local copy of your dependencies and back that up. Done.

preciousoo3• 3 years ago

Side note: what did happen to Travis? I was just googling them yesterday because they were everywhere. They even came with the GitHub education package.

Did GitHub just eat them?

tankerkiller2• 3 years ago

Seems based on some of my research that they've completely exited the free side of the business. All of their plans are now paid and the cheapest plan is $64/year

jacobwg• 3 years ago

Travis CI got acquired by Idera in 2019 (https://news.ycombinator.com/item?id=18978251) then a month later laid off senior engineer staff (https://news.ycombinator.com/item?id=19218036).

preciousoo3• 3 years ago

Damn. They were my introduction to CI/CD. Such is life

Xylakant• 3 years ago

Travis was acquired a few years ago and things went downhill from there on.

VWWHFSfQ3• 3 years ago

It wasn't a good business to be in anyway. I don't think any of these freebie devops businesses are all that smart. They're not a "business", they're a feature of someone else's business. And as soon as they catch up then you're done.

acdha2• 3 years ago

Also they’re surprisingly expensive: things like spam and cryptocurrency mean that you need a fairly large abuse prevention team which is expensive but has no customer-visible benefits. GitHub has that too but as you said they at least have the rest of the business with which to recoup that cost.

qbasic_forever3• 3 years ago

I suspect GitHub actions put a massive dent in their product usage. I seem to remember they started to cut costs and restrict free usage some years back too, and that was the beginning of the end.

JonChesterfield3• 3 years ago

My first thought on this was good riddance. The dev model of "we've lost track of our dependencies so ship Ubuntu and a load of state" never sat well.

However it looks like the main effect is going to be moving more of open source onto GitHub, aka under Microsoft's control, and the level of faith people have in Microsoft not destroying their competitor for profit is surreal.

Sebb7674• 3 years ago

The dev model of "we've lost track of our dependencies so ship Ubuntu and a load of state" never sat well.

Docker, the company, is failing. Docker, as in containerization technology, is alive and very well.

(edited for clarity)

osigurdson6• 3 years ago

> Docker, the company, is failing. Docker, the container technology, is alive and very well.

Is it though? Podman is more well liked (no daemon / non-root) and Kubernetes doesn't have direct support for it any more. I don't think it matters much that k8s uses CRI-O but docker needs to be #1 for running a container on a single machine. Yet, they seem to be letting that slip away because it is not directly monetizable. Software businesses need to be creative - invest a lot into free things, which support monetization of others. If you want low risk returns, buy T-bills.

CGamesPlay• 3 years ago

What state are you thinking of? The containers are ephemeral and the dependencies are well specified in it. You can complain about shipping Ubuntu, but the rest of this doesn’t make sense.

vilunov3• 3 years ago

Makes perfect sense to me, sadly. The dependencies are specified in excessively, that's why everyone is shipping Ubuntu. This is caused by and further facilitates the development style of "do not track what we use, just ship everything". Also, the dependencies are specified in container images, which themselves are derivative artifacts and not the original source code, and these dependencies often change in different container builds with no explicit relevant change.

There are three practical problems as a result: - huge image sizes with unused dependencies delivered as part of the artifact; - limited ability to share dependencies due to inheritance-based model of layers, instead of composition-based model of package managers; - non-reproducibility of docker images (not containers) due loosely specified build instructions.

Predicting future comments, nix mostly fixes these issues, but it has a bunch of issues of its own. Most importantly, nix is incredibly invasive in development process, adopting it requires heavy time investments. Containers also provide better isolation

hospitalJail• 3 years ago

The dev model of "we've lost track of our dependencies so ship Ubuntu and a load of state" never sat well.

This was my first thought when I learned of Docker.

I have a hard time calling myself an 'Engineer' when there are so many unknowns, that I'm merely playing around until something works. I insist on being called a Programmer. It pays better than 'real' engineering. Why not embrace it? (Credit toward safety critical C and assembly though, that's engineering)

EDIT: Programmer of 15 years here

Clubber2• 3 years ago

Developer/Programmer/Engineer titles are mostly meaningless because they mean different things at different companies. You can go wayback and call yourself a coder.

AlotOfReading• 3 years ago

As someone that does what you consider "engineering", my current project is containerized because I've lost too much of my life to debugging broken dev environments. The person who ships safety critical releases at most companies isn't a developer that's deeply familiar with the code, it's usually a distracted test engineer with other things to do that may not be very tech-saavy. Anything I can do to help them get a reproducible environment is great.

at_a_remove• 3 years ago

I remember back around 2009, our org had this horrible open source program, which shall not be named, foisted upon us by an excitable bloke who had heard wondrous things from the university where it had been developed. Well, we had a bitch of a time getting it running. Instructions and build scripts were minimal. We thrashed around.

I noted to someone that this felt less like a product and more like a website and set of scripts ripped from a working system. A few of us were shipped up to the originating university for a week to hob-knob with the people in charge of it. Toward the end, during the ritual inebriation phase, I managed to find out that they had never actually attempted to install it on a clean system. This had truly been ripped from a working system. And I thought to myself, "How horrible."

Now, I am admittedly pants at Linux. No good at all. But there is something about Docker and similar technologies that says, "Yes, we threw our hands in the air and stopped trying to make a decent installation system."

0xDEF• 3 years ago

I doubt the real™ engineers at NASA and SpaceX know everything their proprietary closed-source Matlab-to-FPGA tooling is actually doing under the hood.

tyingq6• 3 years ago

The only real moat they seem to have here is that "FROM" in a Dockerfile, "image:" in a docker-compose.yml file, and the docker command line default "somestring" as an image to "hub.docker.com:somestring".

They pushed that with the aggressive rate limiting first though, which caused a lot of people to now understand that paragraph above and use proxies, specify a different "hub", etc.

So this move, to me, has less leverage than they might have intended, since the previous move already educated people on how to work around docker hub.

At some point, they force everyone's hand and lose their moat.

remram4• 3 years ago

x/y expands to docker.io/x/y and z expands to docker.io/library/z

tyingq6• 3 years ago

Right, it's a little different than my summary, but the main point was that they educated everyone that there's a way around that with specific image names, or a proxy, etc. If they push hard enough, the internet will route around them, distros will ship a patched docker, preset environment variables, or a docker->podman alias, etc. They will lose control over the "root namespace".

numbsafari2• 3 years ago

Docker have a responsibility to _their_ customers, not just OpenFaas and other open source projects. _Docker's_ customers rely on them to provide a safe and reliable service. If Docker allows these projects to be taken over by nefarious actors, then the risk falls to _their_ customers, not the Open Source projects that they've broken with.

Ekaros• 3 years ago

It is kinda strange that developers who should be aware of their own costs think that these sort of services will be free forever or even available and never end. Specially when clearly there is not much monetization going on. Unlike when services are offered with adds.

Big players can carry the costs, but what can small ones really do once money runs out?

raesene94• 3 years ago

Image hosting at scale is an expensive business and Docker are not the only ones trying to manage the costs.

Kubernetes is currently in the process of changing the main repository for their images because, as I understand it, they're burning through their free GCP credits an an unsustainable rate.

Ideally Docker Hub would be an industry funded effort, but that would require co-operation and funding from the major tech. players, and I have a feeling that in an era of cost cutting, that might be harder to achieve than it was in the past.

adolph3• 3 years ago

Squatting and the effects of malware and poison images is my primary concern here.

One of the things the docker api has going for it is that it is hash based. Aside from the first time, it doesn’t seem far fetched for a docker api client to refuse or warn based on comparing the new download’s hash to the previous hash.

mac-chaffee2• 3 years ago

Not a lot of people pull by hash; they pull by tag. Tags are not immutable, so the image I get from "python:3.11" today will almost certainly change due to security updates and I will be none the wiser.

adolph3• 3 years ago

I can see that. A human specifiable name is important.

My proposal is that each time an image is pulled, the hash is recorded and retained even if the underlying container image is removed. When the same image is pulled again, if the files change from the previous hash, either fail or warn the user.

I can see how pinning to a specific patch version is not a great idea and that "python:3.11" keeps people from pinning to an insecure version.

justusw• 3 years ago

As long as we don’t share ownership in these platforms, nothing will ever truly belong to us. For Docker, the software, a Libre alternative is Podman. Instead of GitHub, use Codeberg, an open organization and service.

Now we need a Docker registry cooperative owned by everyone.

zoobab2• 3 years ago

"Now we need a Docker registry cooperative owned by everyone."

You can build a pyramid with a non-profit, but that does not mean it owned by everyone.

Pyramids are problematic, as governance are a nightmare.

prepend• 3 years ago

Is there any drop in replacement for dockerhub? I’m concerned about all my random oss containers (like ruby-latest) that may come from orgs that aren’t able or willing to pay.

Is there another container hosting site?

dijit3• 3 years ago

You could always try mirror.gcr.io

https://cloud.google.com/container-registry/docs/pulling-cac...

xrd2• 3 years ago

You can run the docker registry inside dokku. It's fantastic. I have an endpoint that is private for pushing with credentials, and a public endpoint that is open. This requires a little mucking around with the nginx configuration of the app, but totally doable.

gabeio• 3 years ago

Aws seems to have mirrored some of docker hub for us (docker/library images seem to be hosted clones) and has their own public repos as well (not just docker hub’s images):

https://gallery.ecr.aws/

edit: typo

remram4• 3 years ago

quay.io and ghcr.io are popular free options.

ridruejo• 3 years ago

Without entering into the specifics of this situation, I don’t understand the hate for Docker the company. They are providing a huge service for the community and looking for ways to make money from it to make it sustainable. I would give them a bit more empathy/benefit of the doubt as they iterate on their approach. Somewhere, somehow, someone has to pay for that storage and bandwidth whether directly or indirectly (I am old enough to remember what happened with sourceforge so I rather them find a model that works for everyone)

HelloNurse2• 3 years ago

If you inconvenience all users (by devastating the "ecosystem" of publicly available images) in order to extort money from a few users (some organizations will pay up, at least temporarily) you should expect hate.

The only benefit of doubt Docker deserves is on a psychological plane: evil or stupid?

NineStarPoint• 3 years ago

It’s a long standing hate for me that isn’t limited to just Docker, companies that used “we’re free” to obtain massive growth only to turn around and switch monetization models completely once they’ve become the dominant player in the market. It’s a massive distortion on the market, driving companies that tried to be fiscally sound from the start into irrelevancy while extremely inefficient ventures become the market leaders on account of superior funding.

Or to put it another way, Docker should have been focused on sustainability from the start and not dangled a price they knew couldn’t last in front of people to increase adoption.

armchairhacker• 3 years ago

I agree they deserve to get paid, but there are better ways than essentially holding customers’ data and URLs hostage. The problem is they are trying to extract money from other open-source developers who are at least as cash-strapped as them.

Plus, I doubt they will get many people to actually start paying. People will simply move to other storage (like Github) and switch the URLs. Docker is fully open-source and works without docker.io, they don’t really have a position here except owning the name.

IMO they just need to edit / clarify that open-source developers and organizations won’t need to pay, only those who presumably should have the funds. And take a more passive stance: bug people with annoying messages like Wikipedia does, and threaten shutting down docker.io altogether if they don’t somehow get funding (some people will complain about this too but more will understand and will be sympathetic). Wikimedia, Unix/Linux, Mozilla, etc. as well as Homebrew/cURL/Rust all seem to be doing fine as nonprofits without creating huge controversies like this.

Havoc2• 3 years ago

To me this smells of VC model issues.

Initially it's great if you can get all the FOSS to play in your technology walled garden. Subsidize it with VC cash.

Downside is it generates a ton of traffic that is hard to monetize. Sooner or later it reaches a point where it can't be subsidized and then you get pay up or get out decisions like this.

One question I haven't seen yet is 420 USD? Is that what it costs to serve the average FOSS project? Or is that number a bad Elon style joke? If they came out with "We've calculated X as actual costs. We're making no margin on this but can't free lunch this anymore" that would go down a lot better I think.

rad_gruchalski• 3 years ago

I just started paying for Docker. I use it daily, it makes my life easier so I decided to pay for it.

nrvn• 3 years ago

After Docker announced rate limiting for the hub this was an anticipated move. Was just the matter of time.

The only recommendation to everyone: move away or duplicate.

One of the strategies I am yet to test is the synchronization between gitlab and github for protected branches and tags and relying on their container registries. Thus (at least) you provide multiple ways to serve public images for free and with relatively low hassle.

And then for open source projects’ maintainers: provide a one command way to reproducibly build images from scratch to serve them from wherever users want. In production I don’t want to depend on public registries at all and if anything I must be able to build images on my own and expect them to be the same as their publicly built counterparts. Mirroring images is the primary way, reproducing is the fallback option and also helps to verify the integrity.

acdha2• 3 years ago

Mirroring images is the primary way, reproducing is the fallback option and also helps to verify the integrity.

I suspect the latter will become more common over time. I can count on no fingers the number of open source projects which I’ve encountered which have production-grade container images. Once you need to think about security you need to build your own containers anyway and once you’ve done that you’ve also removed the concern of a public registry having issues at an inopportune moment.

phillebaba• 3 years ago

Some self promotion but I have built a project that aims to solve some of these issues in Kubernetes. https://github.com/xenitAB/spegel

I have avoided a couple of incidents caused by images being removed or momentarily not reachable with it. It would at least mitigate any immediate issues caused by images being removed from Docker Hub.

Forestessential• 3 years ago

Its a business

ParetoOptimal10• 3 years ago

We'll see how the free market responds to them dropping (possibly) substantial open source project support.

xrd2• 3 years ago

I run multiple docker registries inside dokku. It's fantastic. $5/mo for peace of mind.

LoonyFruiter• 3 years ago

What alternatives to docker are there on ARM based devices like apple silicon macs? I'd like to get off of Docker in light of this change.

SparkyMcUnicorn• 3 years ago

I use Colima. https://github.com/abiosoft/colima

martypitt2• 3 years ago

I think it's an interesting challenge with FOSS infrastructure in general, and I'm surprised it isn't more of an issue.

Docker's storage is heavier than most, but what about other repositories like maven central, and npm? There must be significant costs associated with running those.

These tools are all the backbone of modern software dev, and need a business model. It's reasonable that consumers should pay for the benefit. I think Docker have screwed the execution of this transition, but the overall pattern of "someone has to pay" is one I support.

Personally, I pay for Docker. I use it every day, and get value from it.

The argument that the OP makes is really valid though - OSS needs distribution channels, which need to be funded - and expecting the publisher to pay for this isn't always appropriate.

I'd like to see something like the equivalent of CNCF which I can buy a subscription from, and it funnels money to the comapnies and developers that keep me in a job -- almost a Spotify model for OSS and it's supporting infrastructure.

schneems3• 3 years ago

RubyCentral pays the bills for rubygems.org. It’s a non-profit (501c3, not a 501c9). I believe you can buy an individual subscription to Ruby Central. Also profits from their conferences: RailsConf and RubyConf go towards funding infra (and paying people to wear the pager too).

jacques_chester• 3 years ago

That said, a lot of the true cost of rubygems.org is borne by Fastly, who provide the CDN.

Disclosure: I work at Shopify on a team that does work in and around RubyGems.

Symbiote• 3 years ago

https://mvnrepository.com/repos/central

That has Maven Central as 34TB, which seems reasonable.

Docker is extremely inefficient in comparison, it was 15PB two years ago: https://www.docker.com/blog/scaling-dockers-business-to-serv...

numbsafari2• 3 years ago

You don't go to war with the army you want, you go to war with a menagerie images built and generated by random strangers on the internet your team found in stack overflow posts.

pelasaco• 3 years ago

and now with some layer of chat gpt generated code/documentation/configuration and etc.

roydivision• 3 years ago

Docker should never have become a business. There’s virtually nothing there to make a business around, it’s a suite of useful utilities that should have remained a simple open source project. I switched to podman a while ago and haven’t looked back.

sirius87• 3 years ago

Docker Hub does host images running into several GBs for even small hobby projects, and they also bear network transfer costs. Even with podman, you're going to have to host your images somewhere, right?

Right now, the internet infrastructure heavily relies on the good graces of Microsoft (Github, npm), and storage space and network transfer charges are taken for granted.

manquer• 3 years ago

The design of docker distribution is poor because the company backing it wants to retain control .

Torrent based distribution for open source projects and other public initiatives were there long before docker .

Apt mirroring has also been there for a long long time . Checksum integrity verification of mirrors have well established workflows .

We don’t need good graces of any company to distribute assets .

Wronnay• 3 years ago

Gitea has support for packages, so if you don't want to use GitHub or something similar, you could simply self-host your packages.

verdverm2• 3 years ago

Then you have to pay for distribution, which is what people are using Docker Hub to avoid (grift off of?)

sam0x17• 3 years ago

They could probably have made more money from 1-line contextual advertising in image pull messages than this shit

chatmasta• 3 years ago

I think their goal, rather than making more money, is probably to stop spending money on resources belonging to "customers" who don't pay them any money.

As annoyed as I am with the change, I understand their motivation. It seems rather entitled to demand Docker continue offering me the free service of storing hundreds of gigabytes of redundant, poorly optimized image layers. The complaints seem to largely boil down to "This is outrageous! I will no longer consume your resources without paying you! GOODBYE, good sir!"

foepys2• 3 years ago

I posted this in the other thread already but will also add it here. https://news.ycombinator.com/item?id=35167136

---

In an ideal world every project had its own registry. Those centralized registries/package managers that are baked into tools are one of the reasons why hijacking namespaces (and typos of them) is even possible and so bad.

Externalizing hosting costs to other parties is very attractive but if you are truly open source you can tell everybody to build the packages themselves from source and provide a script (or in this case a large Dockerfile) for that. No hosting of binary images necessary for small projects.

Especially since a lot of open source projects are not used by other OSS but by large organizations I don't see the need to burden others with the costs for these businesses. Spinning this into "Docker hates Open Source" is absolutely missing the point.

Linux distributions figured out decades ago that universities are willing to help out with decentralized distribution of their binaries. Why shouldn't this work for other essential OSS as well?

nerdjon• 3 years ago

I am missing something and I can't find a concrete explanation anywhere.

What exactly does this mean as someone who pulls images but doesn't push to docker hub?

Within a month or so are we going to start getting failures trying to pull images or docker hub no longer being updated and needing to start pulling from somewhere else?

Riverheart• 3 years ago

It means the images you depend on may cease to exist failing your builds and at worst they'll be replaced by bad actors registering the free namespace so automated CI builds and unsuspecting users can pull their containers instead.

So depending on whether these open-source orgs pay up will determine whether you continue using Docker Hub or whatever registry they migrate to.

tolmasky• 3 years ago

Could IPFS possibly be a good distributed (and free?) storage backing for whatever replaces DockerHub for Open Source, as opposed to using something like GitHub? We'd still need a registry for mapping the image name to CID, along with users/teams/etc., but that simple database should be much cheaper to run than actually handling the actual storage of images and the bandwidth for downloading images.

kevincox• 3 years ago

Probably. You still need to store and serve the data somewhere of course but for even moderately successful open source organizations they will likely find volunteer mirrors. The nice thing about IPFS is that new people can start mirroring content without any risk or involvement, new mirrors are auto-discovered, like bittorrent.

It seems like the docker registry format isn't completely static so I don't think you can just use a regular HTTP gateway to access but there is https://github.com/ipdr/ipdr which seems to be a docker registry built on IPFS.

We'd still need a registry for mapping the image name to CID, along with users/teams/etc.

IPNS is fairly good for this. You can use a signing key to get a stable ID for your images or if you want a short memorable URL you can publish a DNS record and get /ipns/docker.you.example/.

Of course now you have pushed responsibility of access control to your DNS or by who has access to the signing key.

verdverm2• 3 years ago

IPFS is the same free that Docker provides. Someone, somewhere is paying for the storage and network. The public IPFS would not likely support the bandwidth, volume, and most CSOs.

ar9av• 3 years ago

Rock and a hard place. These organizations have large expenses and can't keep giving away so much for free.

OTOH there are many better ways this could have been handled. More notice, discount for existing orgs etc ..

OpenVS for life

Groxx• 3 years ago

Has Docker forgotten Remember leftpad?

Anyone who takes even a brief glance at the absurdly yolo identity, upgrade, and permissions model Docker encourages should be able to answer this with an immediate "obviously they don't care".

The faster this implodes, the faster we get a safer setup where we don't blindly trust everything.

dbingham4• 3 years ago

As an SRE Manager, this is causing me a hell of a headache this morning.

In 30 days a bunch of images we depend on may just disappear. We mostly depend on images from relatively large organizations (`alpine`, `node`, `golang`, etc), so one would want to believe that we'll be fine - they're all either in the open source program or will pay. But I can't hang my hat on that. If those images disappear, we lose the ability to release and that's not acceptable.

There's no way for us to see which organizations have paid and which haven't. Which are members of the open source program and which aren't. I can't even tell which images are likely at risk.

The best I can come up with, at the moment, is waiting for each organization to make some sort of announcement with one of "We've paid, don't worry", "We're migrating, here's where", or "We've applied to the open source program". And if organizations don't do that... I mean, 30 days isn't enough time to find alternatives and migrate.

So we're just left basically hoping that nothing blows up in 30 days.

And companies that do that to me give me a very strong incentive to never use their products and tools if I can avoid it.

ohgodplsno6• 3 years ago

Or you could, you know, host a Docker registry and reupload those images to something you control. Worst case scenario, in 30 days, nothing is gone from Docker and you can just spin it down.

Your job as an SRE is not to look at things and go "oh well, nothing we can do lol".

dbingham4• 3 years ago

Yes, that involves ripping out Docker Hub everywhere. It's a significant chunk of work, not something easily fit into 30 days on a team that is already strapped for resources with more work than we can do.

bushbaba• 3 years ago

That’s unplanned work. There’s other work needing to be done as well.

fieldcny• 3 years ago

i didn't read it as that, they were stating the realities, assumptions were made, those assumptions are now invalid, they are working on alternatives, 30days is a short deadline for something like this and docker as an organization is behaving poorly.

all of that seems pretty true, and frankly no one should support a company that does something like this. I get they need to figure out how to make money, but time has shown the worst way to do that is to screw over customers or potential customers.

I like the poster will never trust docker, and will never use their tooling or formats, pod-man all the way.

hoherd• 3 years ago

Imagine you shipped software that included references to docker hub images. That software will no longer work if any of the referenced images are deleted from docker hub. This will be the case with any helm charts that reference images that are deleted from docker hub.

Some of those charts will not have variables that let you override the docker images and tags, so some of those will not be usable without creating a new release.

This is one of the primary reasons to vendor your third party docker images into a docker registry that you control.

planetafro• 3 years ago

This exactly. We have pipelines designed specifically for this reason. We pull, patch, and perform minor edits to images we use. We then version lock all-the-things for consistency.

Not saying this is good news, but in the Enterprise, you have to plan for shit like this.

richardwhiuk• 3 years ago

You should be escrowing any Docker images you depend on, I'd have thought.

nine_k8• 3 years ago

Good thing is that setting up an image registry in AWS is so simple! (Ha-ha, only serious.)

petesergeant• 3 years ago

escrowing

Are you sure this is what you mean? Escrow is a type of contactual arrangement, one type of which is agreeing with a commercial partner that you get a copy of their source-code if they go broke.

I feel like you mean vendoring.

sergiotapia• 3 years ago

If your business is depending on these open source projects to exist, shouldn't you be paying them so they can then pay for Docker?

dahdum• 3 years ago

Not every open source project wants to deal with donations / payments that could force incorporation, tax filings, bank accounts, credit/debit cards, and other paperwork. I certainly wouldn't want to deal with that for a side project.

goodpoint4• 3 years ago

No, business love freeloading.

TimWolla2• 3 years ago

The images you mention (alpine, node, golang) are all so-called “Docker Official Images”. Those are all the ones without a slash as the namespace separator in them: https://hub.docker.com/search?q=&type=image&image_filter=off...

They are versioned and reviewed here: https://github.com/docker-library/official-images

I don't expect them to go away.

Disclosure: I maintain two of them (spiped, adminer).

karamanolev• 3 years ago

Useful information, bad look for Docker - "Oh, no slash as the namespace separator. Good and easy way to tell, that's how I would've done it!".

legohead• 3 years ago

"don't expect" or "for certain"? Can't really plan ahead without some kind of certainty.

_cenw2• 3 years ago

One should get in the habit of prefixing them with docker.io/library through, simply because docker's claim on being the default namespace is unacceptable (and also not true on RHEL-adjacent distros)

aprdm3• 3 years ago

You can vendor images. Never have your product depend on something that is in the internet. Spin up Harbour locally and put it in the middle to cache at the very least.

mc4ndr33• 3 years ago

Imagine if everyone actually did this. Then we would have a myriad of base images hiding even more malware than we do currently.

Not to mention vertically integrating the entire Docker layer set defeats the whole point of using Docker in the first place.

friendzis4• 3 years ago

If those images disappear, we lose the ability to release and that's not acceptable.

left-pad moment once again.

I mean, 30 days isn't enough time to find alternatives and migrate.

Maybe take control of mission critical dependencies and self-host?

Woodi• 3 years ago

Maybe take control of mission critical dependencies and self-host?

Last few years prove that this option is a no-go - they just don't do such things ! Independence ? Self-sufficiency ? Security ? Local, fast access ? Obviousness ? No payment required ? Avoid at all costs !

tyler33• 3 years ago

the bad thing about other computers, could happen to everybody, it is harder to use your machine but better long termn

softfalcon• 3 years ago

First of all, want to say, that sounds deeply frustrating.

Secondly, if this is a serious worry. I would recommend creating your own private docker registry.

https://docs.docker.com/registry/deploying/

Then I would download all current versions of the images you use within your org and push them up to said registry.

It’s not a perfect solution, but you’ll be able to pull the images if they disappear and considering this will take only a few minutes to set up somewhere, could be a life saver.

As well, I should note that most cloud providers also have a container registry service you can use instead of this. We use the google one to back up vital images to in case Docker Hub were to have issues.

Is this a massive pain in the butt? Yup! But it sure beats failed deploys! Good luck out there!

bravetraveler4• 3 years ago

As someone who maintains the registries we use globally at work, +1.

I know people groan at running infrastructure, but the registry software is really well documented and flexible.

If you don't need to 'push', but only pull - configuring them as pull through caches is nice for availability and reliability -- while also saving from nickle/diming.

They will get things from a configurable upstream, proxy.remoteurl.

Contrary to what the documentation says, this can work with anything speaking the API. Not just Dockerhub.

edit: My one criticism, it's not good from an HTTPS hardening perspective. It's functional, but audits find non-issues.

You'll want nginx or something in front to ensure good HSTS header coverage for non-actionable requests, for example.

the_jeremy2• 3 years ago

I would not recommend doing it through Docker, though, especially after this change. We use AWS's ECR, and you can set it to do pull-through caching of public images, so images you've already used will stick around even if Docker blows up, and you don't have to pull the images yourself, you just point everything in your environment to ECR and ~~ECR will pull from docker hub~~ (EDIT: it only supports quay.io, not docker hub) and start building its cache as you use the images.

KronisLV• 3 years ago

Secondly, if this is a serious worry. I would recommend creating your own private docker registry.

I've personally been using Sonatype Nexus for a few years with no issues - both for caching external images, as well as hosting my own custom ones. It has pretty good permissions management and cleanup policies.

Here's more info: https://blog.kronis.dev/tutorials/moving-from-gitlab-registr...

Edit: here's a link to the site of the product directly as well, in case anyone is interested in the self-hosted option: https://www.sonatype.com/products/nexus-repository

It's probably not for everyone, but only having to pay for the VPS (or host things on my homelab) feels both simpler and more cost effective in my case. I've also used it at work and there were very few issues across the years with it, mostly due to underestimating how much storage would be needed (e.g. going with 40 GB of storage for approx. 10 apps, each of which were in active development).

thinkmassive• 3 years ago

The regsync utility is very useful for this purpose. It’s part of this excellent regclient project:

https://github.com/regclient/regclient

nickcw• 3 years ago

Which are members of the open source program and which aren't.

You can tell which are members of the open source program if you go to their docker hub page and you'll see a banner "SPONSORED OSS"

Here is an example:

https://hub.docker.com/r/rclone/rclone

mc4ndr33• 3 years ago

That's a fair point, and when someone with a working brain mentions the fallout throughout the Internet that would result, I expect Docker Inc. will reverse course and embark on a PR campaign pretending it was all a mere tawdry joke.

cpitman• 3 years ago

Many of the responses here are talking about how to vendor/cache images instead of depending on an online registry, but remember that you also need access to a supply chain for these images. Base images will continue to be patched/updated, and you need those to keep your own images up to date. Unless the suggestion is to build all images, from the bottom up, from scratch.

sangnoir• 3 years ago

It's a stop-gap measure. There are dozens of companies chomping at the bit to replace Docker as THE docker registry: I'd bet someone at Github is very busy at this very moment.

web3-is-a-scam6• 3 years ago

Typically when you "cache" something, you're gonna expire it at some point...no? If the image is patched, it eventually gets refreshed in the mirror. If the image dissapears at least we still have it until we figure out where the heck it went.

ParetoOptimal10• 3 years ago

Base images will continue to be patched/updated, and you need those to keep your own images up to date. Unless the suggestion is to build all images, from the bottom up, from scratch.

If docker pushes people to that, hopefully more reproducible solutions like nix and it's ux friendly "porcelains" such as https://devenv.sh/ gain market share.

notatoad3• 3 years ago

Unless the suggestion is to build all images, from the bottom up, from scratch.

this doesn't really seem like an unreasonable suggestion.

just change your build process to pull from the source repository (almost always linked from the docker hub page) instead and eliminate one level of dependency from the chain. in an ideal world, docker hub would have been a more stable buffer inbetween the original source of the image and you, but as long as they are proving to be more of a liability than a source of stability, just cut them out.

leshenka• 3 years ago

How hard is it to spin your own registry and clone those images there? I’m not heavily invested in my company’s infrastructure but as far as I can tell we have our own docker and npm registries

maxyurk2• 3 years ago

Use JFrog Artifactory. If you're with self hosting there's a free JFrog Container Registry edition.

SergeAx• 3 years ago

Our organization currently caching all and every external dependencies we are using: Go, Python, npm and .NET packages, Docker images, Linux deb packages, so everything is contained inside our perimeter. We did that after one day our self-hosted Gitlab runners were throttled and then rate-limited by some package repository and all CI pipelines halted.

xyst• 3 years ago

if you haven't mirrored the docker images your application needs to a private registry, then you are doing it wrong.

jjav3• 3 years ago

If those images disappear, we lose the ability to release and that's not acceptable.

This shines light on why it is so risky (from both availability and security perspectives) to be dependent on any third party for the build pipeline of a product.

I have always insisted that all dependencies must be pulled from a local source even if the ultimate origin is upstream. I am continuously surprised how many groups simply rely on some third party service (or a dozen of them) to be always and perpetually available or their product build goes boom.

darkhelmet• 3 years ago

Likewise. I've always insisted on building from in-house copies of external dependencies for precisely this kind of scenario. It astonishes me the number of people who didn't get why. Having things like docker rate-limiting/shutdowns, regular supply chain attacks, etc has been helping though.

Slightly related: actually knowing for sure that you've got a handle on all of the external dependencies is sometimes harder than it should be. Building in an environment with no outbound network access turns up all sorts of terrible things - far more often than it should. The kind that worry me are supposedly self-contained packages that internally do a bunch of "curl | sudo bash" type processing in their pre/post-install scripts. Those are good to know about before it is too late.

SoftTalker• 3 years ago

I run a local Ubuntu mirror for the work systems I manage, for this reason.

dbingham4• 3 years ago

We can't go NIH for everything. If we do that we're back to baremetal in our own datacenters and that's expensive and (comparatively) low velocity. We have to pick and choose our dependencies and take the trade off of risk for velocity.

This is the tradeoff we made with the move to cloud. We run our workloads on AWS, GCP or Azure, use DataDog or New Relic for monitoring, use Github or GitLab for repos and pipelines, and so forth. Each speeds us up but is a risk. We hope they are relatively low risks and we work to ameliorate those risks as we can.

An organization like Docker should have been low risk. Clearly, it's not. So now it's a strong candidate for replacement with a local solution rather than a vendor to rely on.

flyinghamster• 3 years ago

Be sure to archive your development tools as well, just in case that rug gets pulled. You don't want to be in the position that you need v3.1415927 of FooWare X++ because version 4 dropped support for BazQuux™, only to find that it's no longer downloadable at any price.

jayp14183• 3 years ago

That's why better to have NetBSD + pkgsrc combo for servers.

drdaeman2• 3 years ago

You misspelled Nixpkgs ;-)

I'm kidding, of course, but IIRC pkgsrc (and alikes, such as APT) has a number of limitations, for example a very limited ability to have multiple versions of the same package installed, making it less than optimal replacement.

(I believe a lot of people depend on ability to spin up a new version while the old is running, then do the cutover and shut down the old one after it's not is use.)

BlueTemplar4• 3 years ago

How viable is it to fork Docker Hub ?

the8472• 3 years ago

Why not use your own registry with a pull-through cache?

salawat• 3 years ago

Time for you to locally clone the dockerfiles you're reliant on, build up your own in house repository, and then do what has been done since time immemorial.

Mirror the important shit. No excuses, just do. Yes, it's work. I guarantee though, you'll be less exposed to externally created drama.

Making sure your org stays up to date though, that's on you.

websap• 3 years ago

You probably wanna move to AWS Public ECR Gallery. They have a notion of official images.

AWS is in a better position to offer long term coverage.

owaislone• 3 years ago

Good time/opportunity to get your team/company to invest in a registry+proxy to host all images you depend on.

amouat• 3 years ago

Just to be clear, the official images are definitely not at risk, and I say that as a Docker captain.

Official images are hugely important to Docker, now and going forward.

phpisthebest• 3 years ago

Any organization that has the means to pay, should pay for another service that is not openly hostile to users...

caeril• 3 years ago

This whole thing is so weird. Why do so many organizations depend on the internet to function?

It wasn't too long ago that it was standard practice to vendor your dependencies; that is, dump your dependencies into a vendor/ directory and keep that directory updated and backed up.

But now, you all think it's 100% acceptable to just throw your hands up if github is down, or a maven repository is down, or docker hub makes a policy change?

Every year that goes by it becomes clear that we are actually regressing as a profession.

phaedrus• 3 years ago

There are some places that still work the old way, such as where I work - and we're finding we're increasingly out-of-touch with younger developers who grew up in a connected world. We had a recent college grad engineer (developer) who didn't work out as a hire. Some examples of the disconnect:

Try as I might, I couldn't get him to understand the difference between "git" the tool and "Github" the website. He kept making me nervous because he'd slip up and use the two terms interchangeably. (We have sensitive data that shouldn't be uploaded to the cloud.)

He didn't seem to completely understand files/folders and the desktop metaphor. He didn't seem to understand the difference between personal devices and work devices.

The last straw for our boss to let him go was he turned in a project that used a free web service on the cloud to upload data and get back the rows sorted. (Refer back to what I said above about: sensitive data.)

It didn't appear he was being obstinate, it was a tech-cultural difference. "Radical semantic disconnect" as I've seen the term used in science fiction.

heleninboodler2• 3 years ago

But now, you all think it's 100% acceptable to just

Who is this "all" you're talking to? Seems like most of the responses are suggesting vendoring too.

yenda• 3 years ago

Sounds like you could save yourself some time and budget by offering to pay for those images your are using?

ownagefool• 3 years ago

I mean, 30 days isn't enough time to find alternatives and migrate.

Write a script to iterate the images and push them to your own registry. This will buy you time in the event anything does dissapear.

web3-is-a-scam6• 3 years ago

This is why our team vendors the images we depend on into AWS Elastic Container Registry.

eecc• 3 years ago

If those images disappear, we lose the ability to release and that's not acceptable

It’s your responsibility to ensure your own business continuity. You should review how your build pipeline depends on resources outside of your org perimeter, and deploy a private registry under your own control.

btw, you could also contribute some mirroring bandwidth to the community. You must’ve heard that the cloud is just someone else’s computer.

agilob• 3 years ago

Very first thing you should have is a mirroring docker hub proxy. Im surprised SRE manager doesn't have it, why?

anecdotal1• 3 years ago

What's so hard about making your team build and host the images you rely on?

Install Gitlab, clone these projects onto it, it will usually detect and build the container images. You may have to manually fire off builds for older tags/branches but it will work

TeeMassive• 3 years ago

And this is why in many organizations I insist on mirroring every packages and images that we depend on.

vbezhenar• 3 years ago

I think that people should switch to GitHub container registry for free images. I don't like this kind of centralisation, but I least one can expect for Microsoft to have enough money to provide service for free without strings attached, just like they do with Git hosting.

millerm• 3 years ago

I suppose BitTorrent for Images should be a thing (again?)

Discussions of decentralization and redundancy always come up in software/system design and development, but we seem to always gravitate to bottlenecks and full dependency on single entities for the tools we "need".

Havoc2• 3 years ago

Good point. A dual system would be ideal. http to ensure coverage, bt to achieve network effects on the popular images.

Presumably a few - nginx and what not - have a high %

magicalhippo• 3 years ago

Many (most?) BitTorrent clients support web seeding[1] by a regular HTTP server.

A public registry could do web seeding, with bandwidth restrictions if needed, to ensure availability.

[1] https://en.wikipedia.org/wiki/BitTorrent#Web_seeding

bmitc• 3 years ago

It is my understanding that Microsoft has previously tried to purchase Docker. Despite me having problems with companies buying up each other, I wouldn't be surprised if Microsoft revisits, or already is revisiting, buying Docker.

Being a heavy Visual Studio Code user, I have centered my personal development around Docker containers using VS Code's Devcontainer feature, which is a very, very nice way of developing. All I need installed is VS Code and Docker, and I can pull down and start developing any project. (I'm not there yet for all my personal projects, but that's where I'm headed.)

boredumb3• 3 years ago

Docker had the ability to be baked into nearly every enterprise tech stack and extract money accordingly, instead they take time every 2 years to torment users. They will end up going down as one of the biggest missed plays in modern software.

abdellah123• 3 years ago

self hosting is the only alternative ... We cannot depend on the cloud anymore. This is not the first nor the last time something like this happens.

jmac01• 3 years ago

I havent used docker but my understanding is that dockerhub hosts docker images which are essentially just text files? Would that be something that cud just be migrated to another platform easily or does dockerhub do a lot of other things too?

albert_e• 3 years ago

dockerfile is a text file spec on how to build a docker image.

a container image (analogous to a VM snapshot) is built from a dockerfile.

but dockers hub contains the actual images (that run into MBs and GBs) not just dockerfiles.

most dockerfiles don't build an image from scratch. they start with a "FROM" keyword that references an existing pre-built image and then adds some layers of files and configuration on top.

everytime you build a containerized app, your build scripts first pull down the latest pre-built base image referenced by your app's dockerfile.

so a image registry like docker hub is core and essential for thousands of build pipelines and automation that run across thousands of companies globally.

there are some alternatives like Amazon ECR, and private registries hosted by big companies on their own.

but a lot of projects and pipelines still depend on public images of commonly used ones like Linux flavours and distros maintained by various teams.

paxys• 3 years ago

Dockerfiles are text files. Docker images are entire OS file system snapshots (not exactly but close enough) built from those Dockerfiles. A single one can be hundreds of MBs or even multiple GBs in size.

unilynx2• 3 years ago

It seems to me that the only thing more devastating for a lot of developers would be npmjs.com blowing up like this because they desperately needed funding. But they got acquired by parties in no rush to profit off them

Why didn't any of the big tech acquire Docker and their registry? They don't appear to be less interesting than npmjs considering technology or ecosystem. Did they resist being acquired and has the opportunity passed ?

ZunarJ52• 3 years ago

1. I just started teaching myself Docker for my home server. How will this affect me?

2. What is a good alternative to Docker?

Thanks!

marvinblum• 3 years ago

1. You probably won't be able to pull some of the images any longer and need to find an alternative host (ghcr.io instead of hub.docker.io for example)

2. Podman

ZunarJ52• 3 years ago

Thank you!

advancingu• 3 years ago

As I commented in yesterday's thread, this is not the first time Docker is pulling the plug on people with very short advance notice: See https://news.ycombinator.com/item?id=16665130 from 2018

Someone back then even wondered what would happen if such a change happened to Docker Hub https://news.ycombinator.com/item?id=16665340 and here we are today.

Liberonostrud• 3 years ago

What is the alternaive?

yrro7• 3 years ago

GitHub container registries are free for open source projects. For now...

roughly• 3 years ago

The ongoing Docker saga has convinced me I’m never, ever, ever making a product for developers.

delfinom3• 3 years ago

Welcome to the bane of targeting a educated or skilled audience.

nikolay• 3 years ago

They really want to go in vain, hated, and making people happy they're finally off their continuously degrading service and attitude! I get it, they are a commercial entity, but it seems that even they realize that they are in the mode of agony!

pmlnr• 3 years ago

if we don't pay up, systems will break for many free users.

Yep. These things happen, which is why hosting a copy on your own gitea, website, etc is so important.

Start publishing images to GitHub

Why, to have this repeated in a few years time?

judge2020• 3 years ago

Yep. These things happen, which is why hosting a copy on your own gitea, website, etc is so important.

...which involves an ongoing cost anyways. Docker is tired of free hosting to everyone (unless they're a vetted Open Source project), so you're going to see projects either move to the next free solution or solicit donations/more donations specifically to support hosting an open access registry.

irrational• 3 years ago

What is the best replacement or alternate to Docker?

akho• 3 years ago

The most Docker-shaped one is Podman. “Best” depends on what you do with it.

I think the bigger issue here is not the lack of replacements, but the end of oh-so-convenient centralization in DockerHub.

coding123• 3 years ago

There are other repositories to put your images and fetch images, right?

technick• 3 years ago

Docker is shooting itself in the foot. Oddly I decided to put on the docker shirt from one of their 2016 hackathons today before reading the news. I'm embarrassed to own this shirt and will throw it away after today.

RIP Docker, your former self will be missed while your current self will be loathed.

54656g3• 3 years ago

Well I am thankful because I don't use Docker in my professional life but do in my personal life. I guess I will rebuild my system without Docker. It made things easy but I have no interest in trying to track bullshit like this as an end user.

I guess Docker is as good as dead.

mc4ndr33• 3 years ago

I am confused by the meaning of Docker's announcement. They keep saying "organizations" will have Docker images deleted. Does that include personal FOSS images or not? Because the vast majority of Docker Hub images are uploaded by individual contributors, not "organizations."

Too bad about their poor relationship with the FOSS community. I've applied to them for years, and actually merged some minor patches to Docker to help resolve a go dependency fiasco. Zero offers.

I guess the next logical move is to republish any and all non-enterprise Docker images to a more flexible host like the GitHub registry.

bombolo2• 3 years ago

Cybersquat before a bad actor can

I don't see why the duty of preventing this falls on the shoulders of the maintainers that are being kicked out to begin with.

I don't publish on docker hub, but if they were in the process of kicking me out, I'd let the chip fall and let them deal with any eventual disaster.

ixtli• 3 years ago

Profit motive above all else is fundamentally incompatible with the social engine that powers the open source community. It always has been and always will be. I'm no longer surprised, but im still disappointed.

unity1001• 3 years ago

Profit motive above all else is fundamentally incompatible with the social engine that powers the open source community.

Its not. Freemium format works splendidly in various ecosystems, one of the biggest being WordPress. It enabled WP ecosystem to fund itself without VC or investor money and grow. Real indie growth. So its possible.

Without the funding from its own userbase to sustain itself, Open Source projects just flop eventually. Few remain if they are way too big or if they can get corporate sponsors. Thats not being 'free'. Real freedom is in Open Source being funded by its users without the unreliable mechanism of donations.

Animats• 3 years ago

What do we do when Microsoft/Github pulls a similar trick?

twblalock3• 3 years ago

It was sad to see people defending Docker Desktop changing from free to paid licenses. Now Docker is charging for even more things that used to be free.

The defenders are reaping what they have sown. Next time a company starts to charge for things that used to be free, remember not to encourage it, because that will only make it happen more.

People don't like this and many of them are not going to trust Docker in the future.

grumple2• 3 years ago

Nothing a company does is free to them. To expect them to provide a free service at all, let alone one with high costs associated with it, is not reasonable. They don't owe the world free service, same for any other company.

twblalock3• 3 years ago

When you offer something for free and then suddenly start charging for it, you are jerking around your customers -- especially when you do it with such confusing and incomplete communication as Docker has provided in this instance.

This is effectively a long-term bait and switch. This is not ok.

kuratkull• 3 years ago

If a company offers something for free, it's likely part of their business plan. However, if they start charging for that feature, users may feel uneasy. Digital Ocean offers a container repo for $5/month, so users may be upset about the perceived mismatch between price and features. Users not paying signals that the company has failed in its business strategy. Additionally, the repo hosting service has weak vendor lock-in, making it easy for users to switch providers. Businesses must be mindful of user feedback and work hard to retain customer loyalty in a competitive market.

muhehe• 3 years ago

Can someone recommend some simple "caching proxy for docker images"? Something where instead of docker/podman pull docker.io/something I would do docker/podman pull my.cache/something and it would do the rest. Official docker registry image is supposed to do this, but cannot mirror other repositories outside of dockerhub.

jhoelzel2• 3 years ago

Please dont forget that you can cache all these images in your own registry! you will still have to worry about how to get updates, but set up a private registry and deal with this on your on time!

As a side node, Rancher desktop is good enough. Docker has repeatedly demonstrated that they just where the first ones and not by any means the best ones.

rustyminnow• 3 years ago

Any tips to running your own registry? e.g. what registry software/package do you use?

I think when I looked into this in the past, I couldn't find anything suitable. A quick search now brings up https://hub.docker.com/_/registry, but considering the content of the article, not sure how I feel about it

jhoelzel2• 3 years ago

this depends on how your setup is!

for instance bigger clouds offer you private registry with your account usually.

Then there is the github container registry: https://github.blog/2020-09-01-introducing-github-container-...

one integrated for self hosted gitlab: https://docs.gitlab.com/ee/user/packages/container_registry/

and of couse docker themselves have a container for a private registy you can run.

the question is not so much how but where you want it to be and how big the images are that you are using!

Hope this helps otherwise hit me up on linkedin (in my profile) and we can figure something out

communism• 3 years ago

socialize Docker!

nathantotten• 3 years ago

Its pretty easy to setup a Github Action that mirrors images to a private registry. We use this to mirror to GCP Artifact registry: https://zuplo.com/blog/2023/03/15/mirroring-docker-images-wi...

zxcvbn40383• 3 years ago

The OP goes through a lot of trouble to obscure they are asking for $35 a month, which honestly I think most people can afford, even if its open source software they develop only out of kindness. So I'm not really buying that argument.

That said I don't really want to reward Docker for writing themselves in as the distribution hub for all things docker and then more or less extorting money from people.

I think the solution is don't give Docker a dime, just run your own registry in Digital Ocean, thats $5/m. If we can front the registry server with a CDN then its potentially free.

TheRealDunkirk• 3 years ago

I don't really want to reward Docker for writing themselves in as the distribution hub for all things docker and then more or less extorting money from people.

I finally bit the bullet and paid for YouTube Premium. The ad "experience" on YouTube has become so abysmal and unblockable, that to use it at all really forces you into it. I had a hard time expressing why this was so aggravating -- considering that I see literally 5 other streaming services I pay for on the same launch screen -- but this really nails it. It's the same monopolize-for-free-and-then-charge-and-extract-all-rent-forever move at the heart of every digital service now.

zxcvbn40383• 3 years ago

I just ignore the ads on youtube. I don't own any animals so the pet food ads are a miss. I don't take any medicines so I'm not buying those either. I work remotely so I'm not going to buy a new car. I already have a VPN so those are all pointless also. I think that is all they try to advertise to me.

Oh yeah! I will never play Rage Shadow Legends in this life or the next. Give it up!

999900000999• 3 years ago

Ehh.

How much money should I spend on my hobbies, 35$ isn't a lot, but I'd rather not waste it.

Plus instead of just grabbing the image I come up with to do x or y, you'll have to implement it yourself. Duplicate this hundreds or thousands of times.

zxcvbn40383• 3 years ago

I would spend $35/m on a hobby - but not on Docker. I wouldn't even spend $5 to host on my own registry. I think a dockerfile and a brew formula is all you'd get out of me.

PedroBatista• 3 years ago

The truth is Docker ( the company ) could never capitalize the success of their software. They clearly need the money and I have the impression things have not been "great" in the last couple of years. ( regardless of reasons )

The truth is also the fact that most people/organizations never paid a dime for the software or the service, and I'm talking about Billion dollar organizations that paid ridiculous amounts of money for both "DevOps Managers" and consultants but the actual source of the images they pull are either from "some dude" or some opensource orgs.

I get that there will be many "innocent victims" of the circumstances but most people who are crying now are the same ones who previously only took, never gave and are panicking because as Warren Buffett says: "Only when the tide goes out do you discover who's been swimming naked."

And there are a lot of engineering managers and organizations who like to brag with expressions like "Software supply chains" and we'll find out who has been swimming with their willy out.

foxandmouse• 3 years ago

I think it's also a product of the larger economic environment. The old model of grow now and profit later seems to be hitting a wall, leaving companies scrambling to find profit streams in their existing customer base not realizing that doing so will hinder their growth projection leading to more scrambling for profit.

It's a vicious cycle, but when you don't grow in a sustainable way it seems unavoidable.

projectazorian• 3 years ago

Sad but unsurprising; Docker has been slowly transforming into a traditional enterprise software company for quite a while now. They really squandered their potential.

sigwinch282• 3 years ago

This is incredibly frustrating to deal with because of how deeply the registry name is baked into Dockerfiles and image names. We end up "mirroring" our base images but there's some disconnect internally between "oh, yeah, our harbor.company/library/debian:bullseye is some random pull of library/debian:bullseye from the docker hub".

Imagine if you needed to change mirrors for `apt` and as part of that process you had to change all of the names of installed packages because the hostname of the mirror formed part of the package's identifier.

worik• 3 years ago

Sucked in.

I am sorry so many people got caught out by this. But it is an regular pattern in tech.

Cory Doctorow coined a word for this: enshittification. https://pluralistic.net/tag/enshittification/

matt3210• 3 years ago

Time to switch to Podman

tannhaeuser• 3 years ago

Or, you know, just install the fscking app on a Linux VM using the app's native installation method, and be done with it? Oh no, we can't have that, must be using k8s, downloading the Internet, and using a zoo of incidental tools and "registries" to download your base images over http (which are STILL either Debian- or RedHat-based, which is the entire reason of the distro abstraction circus to begin with) is SO MUCH EASIER lol.

didip• 3 years ago

Docker should have sold to Microsoft. Whatever they have been doing recently is a mistake.

VadimBauer• 3 years ago

Many people are quite upset. But on the other hand, how many years could this work? Petabytes of data and traffic.

When we started to offer an alternative to Docker Hub in 2015-2016 with container-registry.com, everyone was laughing at us. Why are you doing that, you are the only one, Docker Hub is free or almost free.

Owning your data and having full control over the distribution is crucial for every project, event open source.

MarkSweep• 3 years ago

One annoyance with how docker images are specified is they include the location where they are stored. So if you want to change where you store you image you break everyone.

I wonder if what regsitry.k8s.io does could be generalized:

https://github.com/kubernetes/registry.k8s.io/blob/main/cmd/...

The idea is the depending on which cloud you are pulling the image from, they will use the closest blob store to service the request. This also has the effect that you could change the source of truth for the registry without breaking all Dockerfiles.

synthc• 3 years ago

Another reminder that VC-funded hosting does not last forever. Self-host your mission critical dependencies folks!

rvz• 3 years ago

Tough. Should have self-hosted rather than get yourself locked in. (Again)

Start publishing images to GitHub
GitHub's Container Registry offers free storage for public images. It doesn't require service accounts or long-lived tokens to be stored as secrets in CI, because it can mint a short-lived token to access ghcr.io already.

Even after the montage of outages. Still suggesting to re-centralizing back to GitHub as the solution is really asking for more disappointment. [0]

Once again, we have learned nothing.

[0] https://news.ycombinator.com/item?id=34843847

[1] https://news.ycombinator.com/item?id=22867803

acd• 3 years ago

Since Silicon Valley bank with fdic takeover, the "free" services will go away. Free central bank money is gone due to higher inflation and interest rates. Free open source tiers are going away.

cookiengineer• 3 years ago

Does anybody know whether there could be something like an open/libre container registry?

Maybe the cloud native foundation or the linux foundation could provide something like this to prevent vendor lock-ins?

I was coincidentially trying out harbor again over the last days, and it seems nice as a managed or self-hosted alternative. [1] after some discussions we probably gonna go with that, because we want to prevent another potential lock-in with sonarpoint's nexus.

Does anybody have similar migration plans?

The thing that worries me the most is storage expectations, caching and purging unneeded cache entries.

I have no idea how large/huge a registry can get or what to expect. I imagine alpine images to be much smaller than say, the ubuntu images where the apt caches weren't removed afterwards.

[1] https://goharbor.io

jillesvangurp2• 3 years ago

It's all open source software. Stupidly simple and easy to host. It's a low value commodity thing without mucb value that anyone can trivially self host. All you need is a docker capable machine (any linux machine basically) and some disk space to host the images. And a bit of operational stuff like monitoring, backups, etc. So there's an argument to be made for using something that's there, convenient, and available but not too costly. Which until recently was dockerhub. But apparently they are happy to self destruct and leave that to others.

They should take a good look at Github. If only for the simple reason that it's a natural successor to what they are offering (a free hub to host software for the world). Github actually has a container registry (see above for why). And of course the vast majority of software projects already uses them for storing their source files. And they have github actions for building the docker images from those source files. Unlike dockerhub, it's a complete and fully integrated solution. And they are being very clever about their pricing. Which is mostly free and subsidized by paid features relevant to those who get the most value out of them.

I like free stuff of course. But I should point out that I was actually a paying Github user before they changed their pricing to be essentially free (for small companies and teams). I love that of course but I was paying for that before and I think they were worth the money. And yes, it was my call and I made that call at the time.

Also worth pointing out that Github actions builds on top of the whole docker eco system. It's a valuable service that is built on top of Docker. Hosting the docker images is the least valuable thing. And it's the only thing dockerhub was ever good for. Not anymore apparently. Unlike dockerhub, Github figured out how to create value here.

jeremyjh• 3 years ago

Yeah, honestly the answer is still Github. I entered this industry in 1998. I remember why people don't like Microsoft. I avoid Windows like the plague and have for many years. But Github has never been anything but good for developers and open source, and that hasn't seemed to change. Microsoft can afford to fund stuff like this for developer goodwill alone, and spends much more than that already for that reason. Its not the greatest model imaginable for the sustainability of open source, but there is absolutely no indication as of yet that it's a really bad one. I mean, who the hell wants to go back to sourceforge, even before it was bought and turned into a cesspool?

lima• 3 years ago

Stupidly simple and easy to host.

At scale, serving tens of thousands of users? Unfortunately not.

anon4242• 3 years ago

I wonder if this is related to the large number of more and more desperate sounding emails I've been getting from a Docker sales rep.

Are they bleeding and trying desperately to find additional revenue streams?

dadrian• 3 years ago

I migrated to Github Container Registry in less time than it took to read this blog.

yencabulator5• 3 years ago

Container images should just be files at a URL, anything else just leads to this sort of rent-seeking.

imhoguy• 3 years ago

IPFS or Torrents would be ideal way to host and distribute docker images as every layer is content addressable therefore read-only. I don't mind to seed any public images I keep running on my server.

mcdermott• 3 years ago

Switched to Multipass (https://multipass.run/) and run docker with portainer on my Mac there.

kazinator• 3 years ago

Ironically, Docker wouldn't exist without Linux kernel support for containers.

Simran-B• 3 years ago

Google's crane tool to efficiently copy images from Docker Hub to e.g. GitHub's Container Registry, in case you want to migrate: https://github.com/google/go-containerregistry/tree/main/cmd...

jiggawatts• 3 years ago

Microsoft’s Azure Container Registry now has a check box feature to “vendor” external container images. This takes more space obviously but will protect systems from critical base images suddenly disappearing or becoming rate-limited.

pxc7• 3 years ago

Maybe the process of rerunning upstream Dockerfiles in an attempt to recover lost images and then, in many cases, getting back substantially different images will help illustrate to more teams the difference between what Docker itself achieves (easily distributable disk images, which are very useful) and reproducibility.

(Hopefully most of those differences end up being unimportant and little to no tweaks to the tools using those images is required, though.)

thrashh• 3 years ago

You know what I love about Java’s library ecosystem?

You configure your project to use the de-facto ones…

OR you configure your project (not your system’s user profile!) to use your own internal registry/repo

And all the repo software supports pull-through from every other repo, so you cache all your dependencies however you want and have full control over everything.

Why does pretty much no other ecosystem do this?

mdavidn• 3 years ago

Cargo and NPM and PyPi and RubyGems and likely others all definitely support this too.

porker• 3 years ago

Tweet on the impact of this on the Elixir project: https://mobile.twitter.com/josevalim/status/1635767176769462...

sneak2• 3 years ago

The fact that this was allowed to be carried out in this manner means, in my view, that the capital-D Docker brand is basically toast now.

There is no set of subsequent decisions that allows serious people to trust them.

scoopr• 3 years ago

I'm not too familiar with docker infra at large, but could the docker hub in principle act just as a namespace such that the opensource projects could have the images hosted elsewhere, but docker hub just redirects them there, so saving on the bandwidth on those?

I suppose there are hand-wavy business reasons not to do that, but somehow I feel that would:

  1. Still keep themselves in the loop and relevant, owning the namespace/hub/main registry
  2. Offset the costs of those that they don't want to deal with (push them to ghcr or whatever)
  3. Preserve some notion of goodwill in not braking the whole dockerverse

tyranron• 3 years ago

From the article:

If you are able to completely delete your organisation, then you could re-create it as a free personal account. That should be enough to reserve the name to prevent hostile take-over. Has Docker forgotten Remember leftpad? > This is unlikely that large projects can simply delete their organisation and all its images. > If that's the case, and you can tolerate some downtime, you could try the following: - Create a new personal user account - Mirror all images and tags required to the new user account - Delete the organisation - Rename the personal user account to the name of the organisation

Seems like no?

We cannot rename personal accounts on Docker Hub in 2023. There is no such feature in account settings, and here is the related issue: https://github.com/docker/roadmap/issues/44

So, at the moment, any public organization images are doomed to be lost, if they won't pay.

sickill• 3 years ago

Yeah, the procedure outlined in the article is not going to work so don't even attempt it. No way to rename personal account right now.

aurelien• 3 years ago

This is cool and I explain why: - cool because advocate for a proprietary technology hidden behind Open Source and Free Software purism is a fallacy - cool because by theses actions you improve their business against users freedoms

And most of all, Cool because I hope you and other will take the time to ask yourself before advocating a technology:

Is this technology good for my rights and by the way for knowledge of all.

sigwinch282• 3 years ago

If you are using containerd then you might find the "hosts" section of the configuration file useful: https://github.com/containerd/containerd/blob/main/docs/host...

You could mirror the images to another registry of your choosing and then use containerd configuration as a stopgap until you update all of your `FROM`/`image` fields.

al_be_back• 3 years ago

sadly, cloud providers (GCE, AWS etc) make it too convenient and cheap to host/manage container/package registries. You also have github, gitlab offering similar services, free or cheap, eating further into Docker's market.

I don't see how they can offer Free accounts to Orgs unless they've a steady-growing paying-customer base to offset running costs.

Unless they massively restrict team size, image size, number of projects, throttle connections - costs would spiral out of control.

dnsmichi• 3 years ago

Thanks for the many insights in this discussion.

I've collected them and added more analysis, inventory and mitigation tips in a new blog post. (GitLab team member here)

https://about.gitlab.com/blog/2023/03/16/how-gitlab-can-help...

The blog post includes a Python script to programmatically search CI/CD configurations in GitLab, shows the Dependency Proxy and container registry possibilities, and discusses more ideas around custom Docker images (Dockerfile: FROM), Kubernetes/IaC image usage, policies and Observability. Hope sharing this knowledge helps :-)

Dowwie• 3 years ago

It's going to cost companies a lot more money to migrate their docker dependencies than simply pay their open source dependency maintainers to keep the org intact. Might as well become a project sponsor!

This could be the best thing to happen to open source projects if the argument is framed correctly. You don't solve a cost problem by pursuing a more costly alternative.

saddist0• 3 years ago

Discord pricing model has been so much cool when compared to these reckless move by Docker.

Imagine the following:

[1] Ultra-bad rate-limiting by default to all public images.

[2] X$/month "boost" to have relaxed rate-limit for yourself & contributing some to the community.

[3] The popular the repository is, the higher rate limit it get.

[4] After Y$/month, Docker just suggest other repositories to "support", so the $$ tickle down the whole ecosystem.

[5] Simple feature which allows you to quickly view public images you are dependent on. And which one you/your organisation should support to.

But hey wait... welcome to the world run by VCs and shitty PM roles. :)

xialvjun• 3 years ago

We need Nostr like protocols.

Docker is deleting Open Source organisations - what you need to know

Related Stories