In his newsletter Ed Zitron hammered down the point that GPUs depreciate quickly, but these kind of reliability issues are shocking to read. The GPUs are so common to fail that they hang out in a 24/7 slack channel with customers like Meta (who apparently can't set up a cluster themselves..).
Ed Zitron also called out the business model of GPU-as-a-service middleman companies like modal deeply unsustainable, and I also don't see how they can make a profit if they are only reselling public clouds. Assuming they are VC funded the VCs need returns for their funds.
Unlike fiber cable during the dot com boom the currently used GPUs eventually end up in the trash bin. These GPUs are treated like toilet paper, you use them and throw them away, nothing you will give to the next generation.
Who will be the one who marks down these "assets"? Who is providing money to buy the next batch of GPUs, now that billions are already spent?
Maybe we'll see a wave of retirements soon.
> It’s underappreciated how unreliable GPUs are. NVIDIA’s hardware is a marvel, the FLOPs are absurd. But the reliability is a drag. A memorable illustration of how AI/ML development is hampered by reliability comes from Meta’s paper detailing the training process for the LLaMA 3 models: “GPU issues are the largest category, accounting for 58.7% of all unexpected issues.”
> Imagine the future we’ll enjoy when GPUs are as reliable as CPUs. The Llama3 team’s CPUs were the problem only 0.5% of the time. In my time at Modal we can’t remember finding a single degraded CPU core.
> For our Enterprise customers we use a shared private Slack channel with tight SLAs. Slack is connected to Pylon, tracking issues from creation to resolution. Because Modal is built on top of the cloud giants and designed for dynamic compute autoscaling, we can replace bad GPUs pretty fast!
>These GPUs are treated like toilet paper, you use them and throw them away, nothing you will give to the next generation.
I'm guessing this may be highly dependant on what the bathtub curve looks like, and how much the provider wants to spend on cooling.
Of course with Nvidia being a near monopoly here, they might just not give a fuck and will pump out cards/servers with shitty reliability rates simply because people keep buying them and they don't suffer any economic loss or have to sit in front of a judge.
Be interesting to see what the error rate per TFLOP (no /s, we're looking at operations not time) is compared to older generation cards.
I suppose NVidia could invest in making their GPUs more reliable? But then that'll make everything else even more expensive lol. If only one of the companies on the chain can take one for the team.
So if it is a european legal entity why they host it on .org which is only controlled by USA instead of using a .eu domain which is a ccTLD controlled by European Union?
When it comes to voluntary consumption or use, I always vote for freedom over restriction or censorship. Let people eat all the fried chicken they want if that is what they want to do with their hard-earned dollars.
We have deterrents in place for all of those actions. Freedom means freedom to perform an action, not freedom from it's consequences. What you advocate for is slavery. You may yet get it but it may not look like what you think it should.
People don't live alone they live in a society, even if you're above these temptations others aren't. This is why drugs are a problem, don't want tweakers in my train/bus/...
The human experience is defined by comparison. That's how our five senses work. While we can work for betterment of society as a whole, there will always be points of comparison between segments. Some segments will be better off than others. To deny that, to try to conform everyone to the same strata, that's contrary to our very nature.
Every so often we get a wave of socialism/communism hitting HN as a new generation is exposed to these ideas. It is a seductive idea, to be fair, has convinced many an intellectual over the years. It's very persuasive, especially on paper, yet it keeps failing in real life for one simple reason. It doesn't account for the human psyche, which is this:
People don't want to be equal. They want to be better. People are envious of success, they wish it upon themselves, not upon others. For, would you consider it success if everyone achieved what you did?
Except there's been way too much investment to trick the human brain into that consumption, so it is not a fair fight - that's true both for food and for tech. There are countless documentaries, books and articles on this. You're bringing a knife to a gunfight
You are free to sue Meta. If you can demonstrate harm, you will win. They may pay you off handsomely to avoid seeing the case go to trial. It's because we are free that the option exists. Is that not better than letting government decide which companies you can and cannot sue?
This assumes a perfect justice system, and that’s not the case.
Regardless, the judicial branch is a perfect example of the limits of freedom in practice, and the legislative does, in fact, decide who one can and cannot sue.
You'd have to have a real lock on it to get anybody but a shady lawyer to take it on. It's very unlikely that you'd be able to do that under almost any circumstance. Yes, the potential payout would be large, but the expenses the law firm would have to pay in the meantime, and carry for the many years it would take for the lawsuit to come to a conclusion, would also be very large.
It would be a huge gamble and reputable law firms would have to feel extremely confident that they wouldn't end up on the losing side. That's a big ask regardless of how good you case is. These tech companies have enormous warchests, can drag these things out essentially forever, and the odds they'll find a technicality that would blunt the lawsuit are very high.
I keep repeating myself: Back up your iCloud pictures via privacy.apple.com and move your mails away from M365. Worst case these things get blocked/deleted.
I moved my mails from M365 to a 2€ Hetzner cloud server in one day, it's quite easy with docker-compose apps like mailcow. Plus you have encryption and less errors when using thunderbird.
I call that "you are the garbage bin for other people's emotions". And once you realize this process you can't unsee it and re-evaluate some relationships. If it is each side taking turns being the "emotional garbage bin" then it's a healthy relationship.
But if people only reach out to drop their toxic waste and leave you without the chance to get rid of your own toxic waste you feel not good afterwards. Like where you have conversations and then afterwards notice that you were not able to actually speak about any of your own problems and worries.
That's what I really like about the kids and their words of the year: They used "aura" and at first I thought what a bullshit term is that, but after a while I came to understand it. It's totally fine to listen to your stomach feelings, if someone's aura is negative or their vibes are off you don't need to give them a reason why you stop interacting, you just leave.
We've been trained to be helpful and nice to everyone but then wonder why we feel drained at the end of the day. It's because we're spending emotional bandwidth on people and things that don't give us any energy back.
The word "aura" for all of this is extremely nice. If you see a spooky person approaching you on the street at night you also don't need to explain to them what exactly put you off about them - you just switch sides.
You're finding comforting explanations to allow you to act dismissively towards other people. I understand this is a strategy that is popular these days, but maybe consider how another fellow human will feel when you "don't give them a reason why you stop interacting, you just leave", and judge what they tell you as "toxic waste"; and how you might be the one to make it worse for them (and yourself). If you mentalize yourself into the other position, yours might appear arrogant and condecending if not psychologically violent from where you stand ("how's your aura looking?").
If you feel worn out after listening to other people, that's one way to avoid that, at the expense of human connection. There are other ways to not feel drained even after listening to the most horrible (or boring) stories that don't cut people (and thus yourself) off. You gain options, not lose any. You can learn to have more control over your own inner state without effort, and become more independent from what people around you are saying or doing, instead of turning your problem into their wrongdoing. Instead of having your world suddenly be full of energy vampires you need to protect yourself from.
> You're finding comforting explanations to allow you to act dismissively towards other people.
No, none of this is comforting. For some people it is a big step to not drop everything just because someone is waiting for the bus and wants to have a 15 minute phone call in order to de-stress their own day.
> If you feel worn out after listening to other people, that's one way to avoid that, at the expense of human connection.
Not every human connection is a net positive.
> You gain options, not lose any.
Please let me complete the options I already have before putting more options on my TODO list.
> You can learn to have more control over your own inner state without effort
That's ableism.
People are unique, and while I appreciate you taking the time to write these lines you might be coming from a very different place. To be a bit snarky, maybe you are more on the energy consumer side of things than on the energy producer side. People have a magic radar for others who make them feel heard, but there is a certain bandwidth and it's limits must be respected.
It's observable facts. They are rolling out the features now. So what changed in 2025? Is the present government more liberal than the past? Clearly not. More like this kind of feature will be ignored and irrelevant for 99% of users.
Why you still have the idea in your head that they play by the rules. With the current administration they have been empowered to extract maximum value from us.
In the early days of smartphone use, Google and Facebook uploaded contact lists of every single smartphone user to their servers.
Don't invest any second of your time into the US tech monopoly. That time is much better spent deploying non-US alternatives and backing up your data from US clouds, which could be blocked for us any moment.
Google is a rent-seeking parasitic middleman leeching off productive businesses, let them hang out with their best friends at the US administration.
Ed Zitron also called out the business model of GPU-as-a-service middleman companies like modal deeply unsustainable, and I also don't see how they can make a profit if they are only reselling public clouds. Assuming they are VC funded the VCs need returns for their funds.
Unlike fiber cable during the dot com boom the currently used GPUs eventually end up in the trash bin. These GPUs are treated like toilet paper, you use them and throw them away, nothing you will give to the next generation.
Who will be the one who marks down these "assets"? Who is providing money to buy the next batch of GPUs, now that billions are already spent?
Maybe we'll see a wave of retirements soon.
> It’s underappreciated how unreliable GPUs are. NVIDIA’s hardware is a marvel, the FLOPs are absurd. But the reliability is a drag. A memorable illustration of how AI/ML development is hampered by reliability comes from Meta’s paper detailing the training process for the LLaMA 3 models: “GPU issues are the largest category, accounting for 58.7% of all unexpected issues.” > Imagine the future we’ll enjoy when GPUs are as reliable as CPUs. The Llama3 team’s CPUs were the problem only 0.5% of the time. In my time at Modal we can’t remember finding a single degraded CPU core. > For our Enterprise customers we use a shared private Slack channel with tight SLAs. Slack is connected to Pylon, tracking issues from creation to resolution. Because Modal is built on top of the cloud giants and designed for dynamic compute autoscaling, we can replace bad GPUs pretty fast!
reply