Everyone always thinks this at least in big tech I’ve never heard a PM or exec say a market is not winner take all. It’s some weird corpo grift lang that nothing is worth doing unless its winner take all.
I’m amazed this is even viewed as a “hot take” tbh most of what he said here is pretty high level of abstraction and standard practice for custom hardware. In essence I feel like he’s saying nothing really controversial other than publicly calling out TT for too many abstraction layers (and tbh it’s just in a readme). This is completely fine, he’s a user and this is his experience.
I’m a dev working on torch.compile at meta (previously I worked on ML focused FPGAs) and the approach I would use is build a static graph compiler, use torch.compile (and probably JAX) as graph extraction front-ends and call it a day. I feel like hardware companies don’t know how to handle the flexibility of PyTorch and as a result develop their own APIs which is mistake #1 and virtually makes it impossible to get any market penetration once you head down that path because nobody will ever ever rewrite their models for your hardware when they don’t even know what perf they will get, the risk is just too high. As a result, hardware companies offer inference APIs which hide all of this behind a REST API to basically paper over the lack of generality of the software/hardware interface. This is convenient because then nobody actually knows the perf/$ and they can burn VC money for as long as they want. Whether this is a viable business model or not, we will have to wait until they go public to actually see what their true inference costs are.
To sum it up, start from PyTorch and work your way down to your hardware, this is the only general way if you want to actually sell chips and not just constantly port the model of the day to your hardware.
The idea that cutting research will make a dent in the budget is a fantasy. NSF has a budget of 10 billion. Stop rationalizing gutting crucial programs because of “the deficit” Medicare, social security and the military are the main costs in the US budget. Sure universities are bloated, tackle that problem separately then.
I used this to onboard to the PyTorch team a few years ago. It’s useful for understanding the key concepts of the framework. Torch.compile isn’t covered but the rest of it is still pretty relevant.
One of my friends stated this phenomenon very well “it’s a lever they can pull so they do it”. Once you’ve tied your career to a specific technology internally, there’s really only one option: keep pushing it regardless of any alternatives because your career depends on it. So that’s what they do.
Like idk how long I need to hear about all the great things in tech he’s done when I see his handiwork in an actual tech company over at X. Maybe he was a grifter all along?
This is chesterson’s fence playing out in real life with the major superpower of the world and it has real consequences for people. It should not be the insane fiasco that’s happening right now with an unelected foreign citizen admitting in a cabinet meeting that they accidentally stopped Ebola vaccine financing. If you seriously think this is going to turn out well after we saw trump’s first term I just don’t have anything to help you.the funniest shit is that all of this upheaval is literally < 1% of the actual budget and it’s going to be touted as billions in savings by musk the grifter in chief. I don’t need time to see that this is a dumpster fire.
reply