Can't believe the sense of entitlement in this thread. I guess people think bandwidth grows on trees.
For residential usage, unless you're in an apartment tower where all your neighbors are software engineers and you're all behind a CGNAT, you can still do a pull here and there for learning and other hobbyist purposes, which for Docker is a marketing expense to encourage uptake in commercial settings.
If you're in an office, you have an employer, and you're using the registry for commercial purposes, you should be paying to help keep your dependencies running. If you don't expect your power plant to give you electricity for free, why would you expect a commercial company to give you containers for free?
Docker is a company, sure, and they’re entitled to compensation for their services, sure. That said, bandwidth is actually really cheap. Especially at that scale. Docker has publicly been struggling for cash for years. If they’re stuck on expensive clouds from a bygone VC era, that’s on them. Affordable bandwidth is available.
My main complaint is:
They built open source tools used all over the tech world. And within those tools they privileged their own container registry, and provided a decade or more of endless and free pulls. Countless other tools and workflows and experiences have been built on that free assumption of availability. Similarly, Linux distros have had built-in package management with free pulling for longer than I’ve been alive. To get that rug-pull for open-source software is deeply disappointing.
Not only that, but the actual software hosted on the platform is other people’s software. Being distributed for free. And now they’re rent-seeking on top of it and limiting access to it.
I assume most offices and large commercial businesses have cached and other tools built into their tools, but for indie developers and small businesses, storage of a ton of binary blobs starts to add up. That’s IF they can even get the blobs the first time, since I imagine they could experience contention and queuing if you use many packages.
And many people use docker who aren’t even really aware of what they’re doing - plenty of people (myself included) have a NAS or similar system with docker-wrapping GUI pre-installed. My NAS doesn’t even give me the opportunity to login to docker hub when pulling packages. It’s effectively broken now if I’m on a CGNAT.
> Countless other tools and workflows and experiences have been built on that free assumption of availability
Cannot help but notice that, had Microsoft offered such a sweet deal, this place would've been ablaze with cries of "Embrace, extend, extinguish" and suchlike. (This still regularly happens, e.g., when new Github features are announced). Perhaps even justifiably so, but the community has failed to apply that kind of critical thinking to any other company involved in open source. If your workflow is not agnostic wrt where you pull images from, it is kind of silly to blame it on Docker Inc.
Having said that, it is definitely a problem for many. I work at a technical university and I am sure colleges/research institutes will hit the limit repeatedly and easily.
Not a new concept. If you aren't paying you are the product. Increasingly capitalism demands the product also pay. Buckle up, late stage capitalism continues.
Sometimes you're the product, other times you're the raw material ready to be crushed up into a fine powder and mixed with 20 additives known only by numbers, and thoroughly homogenized to make the product.
"Late stage capitalism" doesn't exist. There are only consumers that exercise their rights as consumers (as in decades past) and those that don't (as now). If you don't do anything about companies treating you badly, then don't get surprised when they do.
I can't seem to parse this reply. I never said any of it was easy. Quite the opposite, in fact. The community was all too happy to embrace an ecosystem provided by a for-profit company just precisely because it made things easy (and the company was not Microsoft).
And I am not saying Docker is wrong to try and monetize. People have built entire business models on top of way more mundane things than the Docker Hub.
You can always rebuild the dockerfile, or create an account increasing your pull limit, or just host all the free infrastructure they’ve already built, or use competitors providing the same service with the risk of them doing exactly the same in the future.
The difference with docker hub and just mirroring a linux repo is docker made it really easy for people so they don’t need it to get into infra weeds but the complexity was always there.
I don't see how that particular issue is relevant here.
Add a port number to the reference, problem solved.
The reality is, DockerHub (though originally called the Docker Index), was the first Docker image registry to even exist, and it was the only one to exist when image references were created.
Now, I would say there are definitely some issues you could have referenced here that would be more relevant (e.g. mirrors only working for DockerHub).
They're OCI images now, and Docker was largely a stolen idea from UNIXes* (unices?), including the term containers. As much as I like what Podman to open it up using Containerfiles and not defaulting to this, it might as well go even farther tweak the standard a bit - provide something like Dockerfile that's less golang-inspired and more linux-inspired, and improve the format for images - so the industry can move on from Docker lock-in.
There is nothing Go-like about Dockerfiles. The file naming seems to be an offshoot of Heroku's "Procfile" convention. I have no idea where the syntax came from, but it's superficially a very simple "SHOUTYCOMMAND arguments...\n" grammar with no structure beyond sequence-of-lines. Perhaps it's the more imperative cousin of the ".env" files popular with web devs.
I guess it's kind of like assembly where COMMAND foo is like INSTRUCTION foo. Maybe it just clicked for me and I can't think of a syntax I'd prefer for it. Which now has me wondering about modeling syntaxes for other things on it.
For some reason it reminded me of the WAC model from WebAssembly component model https://component-model.bytecodealliance.org/creating-and-co... No particular comparison, but I'd like to understand how constructing a container image might compare to constructing a wasm module from components.
That's what makes it approachable (though I don't agree it has an semblance to go other than "FROM <ref>" being fully qualified (minus the carve-out for Hub), but even then it can absolutely act more like a local import if you have an image with that ref locally (or you can even override it in your build command in a couple of different ways).
Also note, Docker can build whatever format you want, it just defaults to the Dockerfile format, but you can give it whatever syntax parser you want.
And yet the Debian package manager has been there, provided free for so long that it was around to watch the dot-com boom and then the bust and then watch the rise of the public cloud afterwards.
Sure you can’t expect things to be available free forever, except many software package repositories have been available, free, forever. We have countless software package managers (apt, brew, pacman), countless language library managers (npm, maven, CPAN) and countless other tools that have had free and relatively unmetered access for literally a generation.
If anything, “Software X will always be a CLI installation away, free, forever” is an old Linux expectation that’s existed for over 30 years and not some “cloud” mentality.
My startup pays Docker for their registry hosting services, for our private registry. However, some of our production machines are not set up to authenticate towards our account, because they are only running public containers.
Because of this change, we now need to either make sure that every machine is authenticated, or take the risk of a production outage in case we do too many pulls at once.
If we had instead simply mirrored everything into a registry at a big cloud provider, we would never have paid docker a cent for the privilege of having unplanned work foisted upon us.
However, if you are using docker's registry without authentication and you don't want to go through the effort of adding the credentials you already have, you are essentially relying on a free service for production already, which may be pulled any time without prior notice. You are already taking the risk of a production outage. Now it's just formalized that your limit is 10 pulls per IP per hour. I don't really get how this can shift your evaluation from using (and paying for) docker's registry to paying for your own registry. It seems orthogonal to the evaluation itself.
The big problem is that the docker client makes it nearly impossible to audit a large deployment to make sure it’s not accidentally talking to docker hub.
This is by design, according to docker.
I’ve never encountered anyone at any of my employers that wanted to use docker hub for anything other than a one-time download of a base image like Ubuntu or Alpine.
I’ve also never seen a CD deployment that doesn’t repeatedly accidentally pull in a docker hub dependency, and then occasionally have outages because of it.
I have a vague memory of reading something to that effect on their bug tracker, but I always thought the reasoning was ok. IIRC it was something to the effect that the goal was to keep things simple for first time users. I think that's disservice to users, because you end up with many refusing to learn how things actually work, but I get the sentiment.
> I’ve also never seen a CD deployment that doesn’t repeatedly accidentally pull in a docker hub dependency, and then occasionally have outages because of it.
There's a point where developers need to take responsibility for some of those issues. The core systems don't prevent anyone from setting up durable build pipelines. Structure the build like this [1]. Set up a local container registry for any images that are required by the build and pull/push those images into a hosted repo. Use a pull through cache so you aren't pulling the same image over the internet 1000 times.
Basically, gate all registry access through something like Nexus. Don't set up the pull through cache as a mirror on local clients. Use a dedicated host name. I use 'xxcr.io' for my local Nexus and set up subdomains for different pull-through upstreams; 'hub.xxcr.io/ubuntu', 'ghcr.xxcr.io/group/project', etc..
Beyond having control over all the build infrastructure, it's also something that would have been considered good netiquette, at least 15-20 years ago. I'm always surprised to see people shocked that free services disappear when the stats quo seems to be to ignore efficiency as long as the cost of inefficiency is externalized to a free service somewhere.
> I'm always surprised to see people shocked that free services disappear when the stats quo seems to be to ignore efficiency as long as the cost of inefficiency is externalized to a free service somewhere.
Same. The “I don’t pay for it, why do I care” attitude is abundant, and it drives me nuts. Don’t bite the hand that feeds you, and make sure, regularly, that you’re not doing that by mistake. Else, you might find the hand biting you back.
That will most likely fail, since the daemon tries to connect to the registry with SSL and your registry will not have the same SSL certificate as Docker Hub. I don't know if a proxy could solve this.
This is supported in the client/daemon. You configure your client to use a self-hosted registry mirror (e.g. docker.io/distribution or zot) with your own TLS cert (or insecure without if you must) as pull-through cache (that's your search key word). This way it works "automagically" with existing docker.io/ image references now being proxied and cached via your mirror.
You would put this as a separate registry and storage from your actual self-hosted registry of explicitly pushed example.com/ images.
It's an extremely common use-case and well-documented if you try to RTFM instead of just throwing your hands in the air before speculating and posting about how hard or impossible this supposedly is.
You could fall back to DNS rewrite and front with your own trusted CA but I don't think that particular approach is generally advisable given how straightforward a pull-through cache is to set up and operate.
All the large objects in the OCI world are identified by their cryptographic hash. When you’re pulling things when building a Dockerfile or preparing to run a container, you are doing one of two things:
a) resolving a name (like ubuntu:latest or whatever)
b) downloading an object, possibly a quite large object, by hash
Part b may recurse in the sense that an object can reference other objects by hash.
In a sensible universe, we would describe the things we want to pull by name, pin hashes via a lock
file, and download the objects. And the only part that requires any sort of authentication of the server is the resolution of a name that is not in the lockfile to the corresponding hash.
Of course, the tooling doesn’t work like this, there usually aren’t lockfiles, and there is no effort made AFAICT to allow pulling an object with a known hash without dealing with the almost entirely pointless authentication of the source server.
Right but then you notice the failing CI job and fix it to correctly pull from your artifact repository. It's definitely doable. We require using an internal repo at my work where we run things like vulnerability scanners.
> since the daemon tries to connect to the registry with SSL
If you rewrite DNS, you should of course also have a custom CA trusted by your container engine as well as appropriate certificates and host configurations for your registry.
You'll always need to take these steps if you want to go the rewrite-DNS path for isolation from external services because some proprietary tool forces you to use those services.
I don't really get how this can shift your evaluation from using (and paying for) docker's registry to paying for your own registry
Announcing a new limitation that requires rolling out changes to prod with 1 week notice should absolutely shift your evaluation of whether you should pay for this company's services.
At Docker, our mission is to empower development teams by providing the tools they need to ship secure, high-quality apps — FAST. Over the past few years, we’ve continually added value for our customers, responding to the evolving needs of individual developers and organizations alike. Today, we’re excited to announce significant updates to our Docker subscription plans that will deliver even more value, flexibility, and power to your development workflows.
We’ve listened closely to our community, and the message is clear: Developers want tools that meet their current needs and evolve with new capabilities to meet their future needs.
That’s why we’ve revamped our plans to include access to ALL the tools our most successful customers are leveraging — Docker Desktop, Docker Hub, Docker Build Cloud, Docker Scout, and Testcontainers Cloud. Our new unified suite makes it easier for development teams to access everything they need under one subscription with included consumption for each new product and the ability to add more as they need it. This gives every paid user full access, including consumption-based options, allowing developers to scale resources as their needs evolve. Whether customers are individual developers, members of small teams, or work in large enterprises, the refreshed Docker Personal, Docker Pro, Docker Team, and Docker Business plans ensure developers have the right tools at their fingertips.
These changes increase access to Docker Hub across the board, bring more value into Docker Desktop, and grant access to the additional value and new capabilities we’ve delivered to development teams over the past few years. From Docker Scout’s advanced security and software supply chain insights to Docker Build Cloud’s productivity-generating cloud build capabilities, Docker provides developers with the tools to build, deploy, and verify applications faster and more efficiently.
Sorry, where in this hyped up marketingspeak walloftext does it say "WARNING we are rugging your pulls per IPv4"?
That's some cherry-picking right there. That is a small part of the announcement.
Right at the top of the page it says:
> consumption limits are coming March 1st, 2025.
Then further in the article it says:
> We’re introducing image pull and storage limits for Docker Hub.
Then at the bottom in the summary it says again:
> The Docker Hub plan limits will take effect on March 1, 2025
I think like everyone else is saying here, if you rely on a service for your production environments it is your responsibility to stay up to date on upcoming changes and plan for them appropriately.
If I were using a critical service, paid or otherwise, that said "limits are coming on this date" and it wasn't clear to me what those limits were, I certainly would not sit around waiting to find out. I would proactively investigate and plan for it.
The whole article is PR bs that makes it sound like they are introducing new features in the commercial plans and hiking up their prices accordingly to make up for the additional value of the plans.
I mean just starting with the title:
> Announcing Upgraded Docker Plans: Simpler, More Value, Better Development and Productivity
Wow great it's simpler, more value, better development and productivity!
Then somewhere in the middle of the 1500-word (!) PR fluff there is a paragraph with bullet points:
> With the rollout of our unified suites, we’re also updating our pricing to reflect the additional value. Here’s what’s changing at a high level:
> • Docker Business pricing stays the same but gains the additional value and features announced today.
> • Docker Personal remains — and will always remain — free. This plan will continue to be improved upon as we work to grant access to a container-first approach to software development for all developers.
> • Docker Pro will increase from $5/month to $9/month and Docker Team prices will increase from $9/user/month to $15/user/mo (annual discounts). Docker Business pricing remains the same.
And at that point if you're still reading this bullet point is coming:
> We’re introducing image pull and storage limits for Docker Hub. This will impact less than 3% of accounts, the highest commercial consumers.
Ah cool I guess we'll need to be careful how much storage we use for images pushed to our private registry on Docker Hub and how much we pull them.
Well it's an utter and complete lie because even non-commercial users are affected.
————
This super long article (1500 words) intentionally buries the lede because they are afraid of a backlash. But you can't reasonably say “I told u so” when you only mentioned in a bullet point somewhere in a PR article that there will be limits that impact the top 3% of commercial users, then 4 months later give a one week notice that images pulls will be capped to 10 pulls per hour LOL.
The least they could do is to introduce random pull failures with an increasing probability rate over time until it finally entirely fails. That's what everyone does with deprecated APIs. Some people are in for a big surprise when a production incident will cause all their images to be pulled again which will cascade in an even bigger failure.
None of this takes away from my point that the facts are in the article, if you read it.
If the PR stuff isn't for you, fine, ignore that. Take notes on the parts that do matter to you, and then validate those in whatever way you need to in order to assure the continuity of your business based on how you rely on Docker Hub.
Simply the phrase "consumption limits" should be a pretty clear indicator that you need to dig into that and find out more, if you rely on Docker in production.
I don't get everyone's refusal here to be responsible for their own shit, like Docker owes you some bespoke explanation or solution, when you are using their free tier.
How you chose to interpret the facts they shared, and what assumptions you made, and if you just sat around waiting for these additional details to come out, is on you.
They also link to an FAQ (to be fair we don't know when that was published or updated) with more of a Q&A format and the same information.
It's intentionally buried. The FAQ is significantly different in November; it does say that unauthenticated pulls will experience rate limits, but the documentation for the rate limits given doesn't offer the limit of 10/hour but instead talks about how to authenticate, how to read limits using API, etc.
The snippets about rate limiting give the impression that they're going to be at rates that don't affect most normal use. Lots of docker images have 15 layers; doesn't this mean you can't even pull one of these? In effect, there's not really an unauthenticated service at all anymore.
> “But the plans were on display…”
> “On display? I eventually had to go down to the cellar to find them.”
> “That’s the display department.”
> “With a flashlight.”
> “Ah, well, the lights had probably gone.”
> “So had the stairs.”
> “But look, you found the notice, didn’t you?”
> “Yes,” said Arthur, “yes I did. It was on display in the bottom of a locked filing cabinet stuck in a disused lavatory with a sign on the door saying ‘Beware of the Leopard.”
I'm certainly not trying to argue or challenge anyone's interpretations of motive or assumptions of intent (no matter how silly I find them - we're all entitled to our opinions).
I am saying that when change is coming, particularly ambiguous or unclear change like many people feel this is, it's no one's responsibility but yours to make sure your production systems are not negatively affected by the change.
That can mean everything from confirming data with the platform vendor, to changing platforms if you can't get the assurances you need.
Y'all seem to be fixated on complaining about Docker's motives and behaviour, but none of that fixes a production system that's built on the assumption that these changes aren't happening.
> but none of that fixes a production system that's built on the assumption that these changes aren't happening.
Somebody's going to have the same excuse when Google graveyards GCP. Till this change, was it obvious to anyone that you had to audit every PR fluff piece for major changes to the way Docker does business?
> was it obvious to anyone that you had to audit every PR fluff piece for major changes to the way Docker does business?
You seem(?) to be assuming this PR piece, that first announced the change back in Sept 2024, is the only communication they put out until this latest one?
That's not an assumption I would make, but to each their own.
This isn't exactly the same lesson, but I swore off Docker and friends ages ago, and I'm a bit allergic to all not-in-house dependencies for reasons like this. They always cost more than you think, so I like to think carefully before adopting them.
But Mr Dent, the plans have been available in the local planning office for the last nine months.”
“Oh yes, well as soon as I heard I went straight round to see them, yesterday afternoon. You hadn’t exactly gone out of your way to call attention to them, had you? I mean, like actually telling anybody or anything.”
“But the plans were on display …”
“On display? I eventually had to go down to the cellar to find them.”
“That’s the display department.”
“With a flashlight.”
“Ah, well the lights had probably gone.”
“So had the stairs.”
“But look, you found the notice didn’t you?”
“Yes,” said Arthur, “yes I did. It was on display in the bottom of a locked filing cabinet stuck in a disused lavatory with a sign on the door saying ‘Beware of the Leopard’.”
> I don't get everyone's refusal here to be responsible for their own shit
No kidding. Clashes with the “gotta hustle always” culture, I guess.
Or it means that they can’t hide their four full-time jobs from each of the four employers as easily while they fix this at all four places at the same time.
The “I am owed free services” mentality needs to be shot in the face at close range.
I haven't gone through my emails, but I assume there was email communication somewhere along the way. It's safe to assume there's been a good 2-3 months of communication, though it may not have been as granular or targeted as some would have liked.
I mean, there has never not been some issue with Docker Desktop that I have to remember to work around. We're all just collectively cargo culting that Docker containers are "the way" and putting up with these troubles is the price to pay.
If you offer a service, you have some responsibility towards your users. One of those responsibilities is to give enough notice about changes. IMO, this change doesn't provide enough notice. Why not making it a year, or at least a couple of months? Probably because they don't want people to have enough notice to force their hand.
What principal are you using to suggest that responsibility comes from?
I have a blog, do I have to give my readers notice before I turn off the service because I can't afford the next hosting charge?
Isn't this almost exclusively going to effect engineers? Isn't it more of the engineer's responsibility not to allow their mission critical software to have such a fragile signal point of failure?
> Probably because they don't want people to have enough notice to force their hand.
You don't. You have responsibility towards your owners/shareholders. You only have to worry about your customers if they are going to leave. Non-paying users not so much - you're just cutting costs now zirp isn't a thing.
If this was a public company I would put my tin foil hat and believe that it's a quick buck scheme to boost CEO pay. A short sighted action that is not in the shareholders interest. But I guess that's not the case? Who knows...
At this stage of the product lifecycle, free users are unlikely to ever give you money without some further "incentives". This shouldnt be news by now, especially on HN.
If you're production service is relying on a free-tier someone else provides, you must have some business continuity built in. These are not philanthropic organisations.
It's bait and switch that has the stakes of "adopt our new policy, that makes us money, that you never signed up for, or your business fails." That's a gun to the head.
Not an acceptable interaction. This will be the end of Docker Hub if they don't walk back.
Yes. But they are paying for this bandwidth, authenticated or not. This is just busy work, and I highly doubt it will make much of a difference. They should probably just charge more.
> take the risk of a production outage in case we do too many pulls at once.
And the exact time you have some production emergency is probably the exact time you have a lot of containers being pulled as every node rolls forward/back rapidly...
And then docker.io rate limits you and suddenly your 10 minute outage becomes a 1 hour outage whilst someone plays a wild goose chase trying to track down every docker hub reference and point it at some local mirror/cache.
> If we had instead simply mirrored everything into a registry at a big cloud provider, we would never have paid docker a cent for the privilege of having unplanned work foisted upon us.
Indeed, you’d be paying the big cloud provider instead, most likely more than you pay today. Go figure.
They should have provided more notice. Your case is simply prioritizing work that you would have wanted to complete anyway. As a paying customer you could check if your unauthenticated requests can go via specific outbound IP addresses that they can then whitelist? I’m not sure but they may be inclined to provide exceptions for paying customers - hopefully.
> It's busy-work that provides no business benefit, but-for our supplier's problems.
I dunno, if I were paying for a particular quality-of-service I'd want my requests authenticated so I can make claims if that QoS is breached. Relying on public pulls negates that.
Making sure you can hold your suppliers to contract terms is basic due diligence.
It is a trade-off. For many services I would absolutely agree with you, but for hosting public open-source binaries, well, that really should just work, and there's value in keeping our infrastructure simpler.
This sounds like its only talking about authenticated pulls:
> We’re introducing image pull and storage limits for Docker Hub. This will impact less than 3% of accounts, the highest commercial consumers. For many of our Docker Team and Docker Business customers with Service Accounts, the new higher image pull limits will eliminate previously incurred fees.
So it goes. You're a business, pay to make the changes. It's a business expense. Docker ain't doing anything that their agreements/licenses say they can't do.
It's not fair, people shout. Neither are second homes when people don't even have their first but that doesn't seem to be a popular opinion on here.
Why? We are running the exact same images that we would be mirroring into and pulling from our private registry if we were doing that, pinned to the sha256sum.
How can you make Docker pull debian:latest from your own registry instead of the official Docker registry, without explicitly specifying <my_registry>/debian:latest?
Wouldn't they get a choice as to what type of authentication they want to use then? I'd assume they could limit access in multiple ways, vs just the dockerhub way.
I just cannot imagine going into public and saying, roughly the equivalent of I want free unlimited bandwidth because I'm too lazy to do the very basics of managing my own infra.
> If we had instead simply mirrored everything into a registry at a big cloud provider, we would never have paid docker a cent for the privilege of having unplanned work foisted upon us.
I mean, if one is unwilling to bother to login to docker on their boxes, is this really even an actual option? Hm.
Hmm yes but if it is limited to 10 in an hour that could even be an issue for hobbyists if you update multiple dockers at the same time. For example the excellent matrix ansible playbook pulls numerous dockers in a single update run because every little feature is in a separate container. Same with home assistant add-ons. It's pretty easy to reach 10 in an hour. Even though you may not pull any for a whole month afterwards. I only do this once a month because most matrix bridges only get updates at that rate.
I have to say though, 90% of the dockers I use aren't on docker hub anymore. Most of them reside on the github docker repo now (ghcr.io). I don't know where the above playbook pulls from though as it's all automated in ansible.
And really docker is so popular because of its ecosystem. There are many other container management platforms. I think that they are undermining their own value this way. Hobbyists will never pay for docker pulls but they do generate a lot of goodwill as most of us also work in IT. This works the other way around too. If we get frustrated with docker and start finding alternatives it's only a matter of time until we adopt them at work too.
If they have an issue with bandwidth costs they could just use the infrastructure of the many public mirrors available that also host most Linux distros etc. I'm sure they'd be happy to add publicly available dockers.
Ironically, it's the paid rates that are being reduced more (though they don't have hourly limits still, so more flexibility, but the fair use thing might come up), as they were infinite previously, now Pro is 34 pulls/hour (on average, which is less than authenticated), Team is 138 pulls/hour (or 4 times Pro) and Business 1380 pulls/hour (40 times pro, 10 times team).
My feeling this is trying to get more people to create docker accounts, so the upsell can be more targeted.
This means there is a market for a docker proxy. Just install it, in a Docker container of course, and it caches the most common containers you use locally!
Yes, and we can agree I'm not going to participate and if they want to take away their service that's their business decision.
They're entitled to do what they want and implement any business model they want. They're not entitled to any business, to my data, nor their business model working.
The entitlement of ... the VC powered powerplant that reinvented and reimagined electricity, gave out free electricity and put all the comeptitors out of business, succeeded in monopolizing electrity then come looking for a payday so they can pad the accounts and 'exit' passing it off to the next round of suckers. Truly unbelieveable.
That's business, baby. We're all involved in it, like it or not. And especially American tech/developer culture seems to thrive on fyigm gated community stuff.
I couldn't give 2 shits whatever Docker does. They're a service, if I wanna use it I'll pay, if not then I'll use something else. Ez pz
> Does this commercial company expect volunteers to give them images for free which give their paid subscriptions value?
Yes, to an extent, because it costs money to store and serve data, no matter what kind of data it is or it's associated IP rights/licensing/ownership. Regardless, this isn't requiring people to buy a subscription or otherwise charging anyone to access the data. It's not even preventing unauthenticated users from accessing the data. It's reducing the rate at which that data can be ingested without ID/Auth to reduce the operational expense of making that data freely (as in money) and publicly available. Given the explosion in traffic (demand) and the ability to make those demands thanks to automation and AI relative to the operational expense of supplying it, rate limiting access to free and public data egress is not in and of itself unreasonable. Especially if those that are responsible for that increased OpEx aren't respecting fair use (legally or conceptually) and even potentially abusing the IP rights/licensing of "images [given] for free" to the "Library built on the back of volunteers".
To what extent that's happening, how relevant it is to docker, and how effective/reasonable Docker's response to it are all perfectly reasonable discussions to have. The entitlement is referring to those that explicitly or implicitly expect or demand such a service should be provided for free.
Note: you mentioned you don't use docker. a single docker pull can easily be 100's of MB's (official psql image is ~150MB for example) or even in some cases over a GB worth of network transfer depending on the image. Additionally, there is no restriction by docker/dockerhub that prevents or discourages people from linking to source code or alternative hosts of the data. Furthermore you don't have to do a pull everytime you wish to use an image, and caching/redistributing them within your LAN/Cluster is easy. Should also be mentioned Docker Hub is more than just a publicly accessible storage endpoint for a specific kind of data, and their subscription services provide more that just hosting/serving that data.
It's like GitHub limiting the number of checkout you can do each hour on public repos.
Unless you pay a sub to get rid of the limit.
So, yeah, they kind of taking advantage of people putting their work on DH to try&sell subs.
But nobody have to put their images on DH.
And to be honest, I don't think the discoverability factor is as important on DH that it is on GitHub.
So if people want to pay for they own registry to make it available for free for everyone, it's less an issue than hosting your repo on your own GitLab/Gitea instance.
From a security and reproducibility perspective, you, shouldn’t want to pull directly. I’ve used Artifactory in the past as a pass through cache that can “promote” image, making them available to test and production environments as they go through whatever validation process is required. Then you know images (or packages, or gems, or modules, or whatever you are deploying) has at least been tested and an unpinned dependency isn’t going to surprise you in production.
Five years ago when Docker changed a storage policy they said it would save 5PB. I can't find the current size of Docker Hub.
That's a huge cost to expect from a free mirror service, especially when a large fraction is of very limited interest, and unlike a Linux distribution Docker Hub isn't organized. (It's easy to only mirror the AMD64 packages for Debian, for example.)
The Docker client also isn't able to work with a partial mirror.
This is what I've seen (and done) at every place that used containers at any kind of scale. I'm frankly impressed with the people who can sleep at night with their production hosts pointed directly at docker hub.
Agreed, it seems like a bunch of people in this thread are scared of having to setup authentication and monitoring, but are not scared of chain attack in the latest docker image they never even looked at.
He could also hit the limit downloading a single image, if I'm understanding his situation.
If I'm an infrequent tinkerer that occasionally needs docker images, I'm not going to pay a monthly cost to download e.g. 1 image/month that happens to be hosted on Docker's registry.
(It sounds like you can create an account and do authenticated pulls; which is fine and pretty workable for a large subset of my above scenario; I'm just pointing out a reason paying dollars for occasional one-off downloads is unpopular)
No one is mad merely because there is a capped free service and an unlimited paid service offering.
The ire is because of the rug pull. (I presume) you know that. It’s predatory behavior to build an entire ecosystem around your free offering (on the backs of OSS developers) then do the good old switcheroo.
There’s plenty of folks behind a CGNAT, sometimes shared with thousands of others. And this is more common in regions where actually paying for this service is often too expensive.
I’ve also seen plenty of docker-compose files which pull out this amount of images (typically small images).
I’m not saying that Docker Inc should provide free bandwidth, but let’s not also pretend that this won’t be an issue for a lot of users.
If Docker explicitly offers a service for free, then users are well within their rights to use it for free. That’s not entitlement, that’s simply accepting an offer as it stands.
Of course, Docker has every right to change their pricing model at any time. But until they do, users are not wrong for expecting to continue using the service as advertised.
I've seen this "sense of entitlement" argument come up before, and to be clear: users expecting a company to honor its own offer isn’t entitlement, it’s just reasonable.
Is there an easy way of changing the default repository that's pulled from when you issue a 'docker pull <whatever>' command, or do you always have to make sure to execute 'docker pull <mycustomrepo.io/whatever>' explicitly?
> do you always have to make sure to execute 'docker pull <mycustomrepo.io/whatever>' explicitly
I started using explicit repository names for everything including Docker Hub 5+ years ago and I don't regret it. I haven't thought about mirrors since, and I find it easier to reason about everything. I use pull-through caches with dedicated namespaces for popular upstream registries.
- hub.example.com/ubuntu --> ubuntu from Docker Hub
- ghcr.example.com/org/projectA --> project from GHCR
I tried using mirrors at first, but it was a disaster with the shorthand notation because you can have namespace collisions. Consider:
That's not useful because your definitions are still ambiguous unless you go look at the mappings, so all you've done is add external config vs explicitly declaring the namespace.
Plus, you can set up a pull-through cache everywhere it makes sense.
I'd be interested to hear about scenarios where mirrors are more than a workaround for failing to understand the power of Docker's namespacing and defaulting to the shorthand notation for everything.
You're absolutely right, but explaining the cost to the employer and/or the client and getting approvals to even use Docker will be a PITA.
Currently for smaller clients of the software house I work for we (normal employees) were able to use Docker whenever we felt like without manager's approval to optimize the deployment and maintenance costs on our side.
> why would you expect a commercial company to give you containers for free
Because they did. But you're right—they have no obligation to continue doing so. Now that you mention it, it also reminds me that GitHub has no such obligation either.
In a way, expecting free container images is similar to how we can download packages from non-profit Linux distributions or how those distributions retrieve the kernel tarball from its official website. So, I’m not sure whether it’s better for everyone to start paying Docker Hub for bandwidth individually or for container images to be hosted by a non-profit, supported by donations from those willing to contribute.
There's already a rate limit on pulls. All this does is make that rate limit more inconvenient by making it hourly instead of allowing you to amortize it over 6 hours.
10 per hour is slightly lower than 100 per 6 hours, but not in any meaningful way from a bandwidth perspective, especially since image size isn't factored into these rate limits in any way.
If bandwidth is the real concern, why change to a more inconvenient time period for the rate limit rather than just lowering the existing rate limit to 60 per 6 hours?
If the electricity were generated by thousands of volunteers pedalling in their basement, then yes, I would expect the utility company not to be too greedy.
Not a huge fan of Docker as a company in general, but this is spot on- the DockerHub free tier is still quite generous for private/hobby usage actually - if you are a professional user, well you should very well be having your own professional solution, either your own internal registry or a commercial SaaS registry.
How many new distinct docker images do you get daily? I'd expect less then one, on average, with a occasional peak when you do exploration.
There is always a commercial subscription. You need only a single $9/mo account and you get 25,000 pulls/month.
And if you are not willing to pay $9/mo, then you should be OK with using free personal account for experiments, or to spread out your experiments over longer timeline.
It’s basically your apartment building example (esp. something like the STEM dorms)
When this stuff breaks in the hours leading up to a homework assignment being due, it’s going to discourage the next generation of engineers from using it.
Bandwidth is cheap, especially at scale, unless you're in one of the large clouds that make a shitload of money gouging their customers on egress fees.
I don't say that Docker Inc should foot the bill for other multibillion dollar companies, but the fact that even after 8 years it still is impossible to use authentication in the registry-mirrors option is mind-boggling.
I think Docker started the bloated image mess. Have you ever seen a project with <100MB in size?
Guess pack everything with gzip isn't a good idea when size matters.
Docker Hub have a traffic problem, so does every intranet image registry. It's slow. The culprit is Docker (and maybe ppl who won't bother to optimize)
Until you have this one weird buildpak thing that for some unfathomable reason keeps downloading all the toolchain layers all the time for each build of the app.
Then again, good that this measure forces fixing this bad behaviour, but as a user of buildpack you are not always in the know how to fix it.
It kind of depends. To a degree you are right, but not entirely. For the past two months for instance I've been making a huge push to de-cloud-ify myself entirely and self-host everything. I do have the bandwidth and I do have the hardware that is needed. Having said that, I am not making this whole thing little by little but whenever I have time. There were times when I was pulling 30 images/hour and it's clearly a one-off thing. While corporations are certainly abusing docker's generosity, in practice, the people that pull hundreds of images on hourly basis is astronomically low - most commonly one-off things, much like what I am doing. I've worked in similar environments and the abusers are the exception rather than the rule. The bottom line is, this genuinely feels like some half-assed biz-dev decision, promising to cut costs by 20%. Been there, done that. In the long run, those quick cuts ended up costing a lot more.
How much bandwidth do you suppose DockerHub uses? I can't see it being any less than 10gigabit, probably more like 100gigabit. Just the cost of that transit is likely in the $600-6,000/mo range. Then you need to factor in the additional costs for storage and compute to serve it, switching gear, and management and maintenance. That's probably at least as much as transit.
They aren't likely able to go for peering arrangements ("free" bandwidth) because their traffic is likely very asymmetric, and that doesn't save the management/storage/compute costs.
I don't know what Docker's financials are, but I can imagine, as a business owner myself, situations where it was lean enough that that sort of cost could mean the difference between running the service and not.
The bigger the service, the more financial incentive they have to be smart and not pay absurd prices for things, since they can give themselves higher profit margins by controlling their costs.
So yeah you can say it's entitlement but if you build your business in one way and then change the fundamental limits AFTER you've gotten market saturation you really shouldn't be shocked at complaints. It's their fault because they fostered the previous user behavior.
People understand that bandwidth costs money but that seems to have been priced in to their previous strategy or they did it knowingly as a loss leader to gain market share. If they knew this was a fundamental limitation they should have addressed it years ago.
Perhaps they should have started by putting "we will enforce limits soon" in all documentation.. and in a few years, starting enforcement but with pretty high limits? and then slowly dialing limits down over a few years?
That's exactly what they did. I remember setting up the docker proxy 4 years ago when we started getting first "rate limit" errors. And if someone was ignoring the news for all that time.. Well, tough for them, there was definitely enough notice.
Who knew that cramming 6 GB of Debian dependencies over the wire to run a Python script was a terrible idea? Who could have seen this coming? Maybe people wouldn't think bandwidth grows on trees if literally everyone in Silicon Valley hadn't developed software that way for the last 15 years.
But idk maybe Docker shouldn't have pulled a bait-and-switch, which is also classical known as a "dick move".
Literally every company offering a free service pulls a bait-and-switch. Fool me once, shame on you. Fool me three dozen times... you can't get fooled again. This was not unexpected. This was completely predictable from the moment anyone in the team asked "what does this command do?" and the answer was "it downloads something from Docker Hub"
We are all getting a little tired of “Come Into My Parlor,” Said the Spider to the Fly.
Of if you want it a little more colorfully: capturing a market with free candy to get us into your van.
Or more accurately, this “free until we need something from you” is the moral equivalent of a free meal or ski trip but you are locked into a room to watch a timeshare marketing deck.
Open Source is built on a gift economy. When you start charging you break the rules of that socioeconomic model. So most of us make tools that we have a hope of keeping running the way they were designed. Some of us stay at the edges because we know we won’t still be interested in this tool in two years and we don’t want to occupy other people’s head space unfairly or dishonestly. Some of us believe we can persevere and then we find there’s a new programming language that we like so much more than this one that we fuck off and leave a vacuum behind us that others have to scramble to fill (I’m talking about you, TJ).
And then there’s these kinds of people who hoover up giant sections of the mindshare using VC money and don’t ever find a new revenue stream, like Mozilla managed. And it’s getting really fucking old.
One of the problems with XML is that the schemas aren’t cached by default, and so a build system or a test harness that scans dozens or hundreds of these files per run, or twenty devs running their sandboxes in a tight loop, can eat up ridiculous amounts of bandwidth. But the important difference is that they architected a way to prevent that, it’s just that it is fiddly to get set up and nobody reads the instructions. I found an email chain with the W3c.org webmaster complaining about it, and myself and a couple other people tried to convince him that they needed to add a 100ms delay to all responses.
My reasoning was that a human loading a single XML file would never notice this increase, but a human running 100s of unit tests definitely would want to know why they suddenly got slower, and doing something about it wouldn’t just get back that extra 20 seconds, they’d get back 40 (or in our case 5-10minutes) by making the call one more time and putting it into a PR. We only noticed we were doing it because our builds stopped working one day when the internet went down.
There’s no build tool I know of that will deal with being 429’d at 10 requests. Especially if you know anything about how layers work. There are tons that would work just fine being traffic shaped to a server with a lower SLA. Dedicate half of your cluster to paying customers and half to the free tier. Or add 250ms delay. It’ll sort itself out. People will install Artifactory or some other caching proxy and you’ll still have the mindshare at a lower cost per use.
Entitlement like building a business on top of free open source tech, relying on free open source to build and support your userbase, then using your cheapest line item as a cudgel to coerce subscriptions?
For residential usage, unless you're in an apartment tower where all your neighbors are software engineers and you're all behind a CGNAT, you can still do a pull here and there for learning and other hobbyist purposes, which for Docker is a marketing expense to encourage uptake in commercial settings.
If you're in an office, you have an employer, and you're using the registry for commercial purposes, you should be paying to help keep your dependencies running. If you don't expect your power plant to give you electricity for free, why would you expect a commercial company to give you containers for free?