How to Buy AI Hardware Without Regret

If you want to see what anxiety looks like in spreadsheet form, ask a CTO to justify a big AI hardware purchase. It’s a peculiar kind of stress, sharper than with most tech investments, because the ground moves faster. AI hardware evolves so quickly that by the time you’ve unpacked your shiny new GPUs, the next model is already twice as fast and uses half the power. The classic trap: you finally hit the target, only to find the whole game has moved. Its even harder for individuals with normal household budgets.

Why is this so much harder than, say, buying servers or even software? Because you can patch software. Hardware is a bet you can’t take back. Once you’ve bought those racks, they’re yours, for better or worse, until they’re scrap. And AI is particularly cruel in this regard. The pace of change isn’t just fast—it’s unpredictable. You’re not just racing Moore’s Law; you’re racing a whole field of algorithms and architectures that might suddenly make your investment irrelevant.

The GPU treadmill is the best example. Nvidia, the current king of the hill, releases new architectures every year or so. Each leap is big enough to make last year’s flagship feel like a used Corolla. Worse, the requirements for AI models keep shifting too. Today it’s about compute; tomorrow it might be memory bandwidth (and probably will be), or some new specialized core. You thought you bought a racecar, but now the race is on water.

Take the acclaimed M-series that helped Apple bring its own brand of silicon into the mainstream. These chips are well-regarded for their integration of CPU, GPU, and “neural” dies on a single chip. But the Apple Neural Engine (ANE) was not positioned properly (too opaque to develop for) so it missed the entire LLM boat. And now the kind folks in the ANEMLL open source community are trying to rehabilitate that hardware for the core use case that it was ostensibly designed to satisfy.

Some companies try to dodge this scenario by making their hardware acquisitions entirely modular. Instead of building a solid block, you make something more like Lego: pull out the old GPUs, slide in the new. This helps, but only up to a point. The pain isn’t erased, just delayed. You still have to decide when to swap things out, and you’re still stuck with the sunk cost of whatever you bought last year. And for some of these decisions, you might be preserving a bottleneck (e.g., the speed of links between components) for multiple generations, placing your operations at a stunning disadvantage. Think about buying a new car but reusing the tires from two cars prior to save money.

Leasing is another option. Rent your hardware from a cloud vendor, and let them worry about obsolescence. This works especially well for startups experimenting with new models. If you need more juice, you scale up. If you hit a dead end, you scale down. The catch is that leasing is usually more expensive in the long run, and big companies often want to own their gear for control and cost reasons. So you trade flexibility for price.

There’s also the classic psychological trap: the sunk cost fallacy. Once a company spends millions on a cluster, it’s hard to admit it’s time to move on. People keep using old hardware long after it makes sense, just because it hurts to write off the investment. In reality, the best move is often to accept the loss and switch. The real question isn’t how to avoid obsolescence, but how to get comfortable with it.

This is just the latest round in a pattern that’s been repeating for decades. Mainframes gave way to minicomputers, then to commodity servers, then to the cloud. Each wave displaced the last, and each time, the winners were the ones who adapted fastest. The AI hardware cycle moves even faster. If you get caught on the wrong side, you don’t just fall behind—you get locked in.

So what should you do? The answer isn’t to make a perfect prediction. The answer is to build for change. Make your infrastructure as flexible as possible. Keep your budgets flexible. Don’t bet the company on a particular piece of hardware. If you have to buy, consider slightly older models—they’re cheaper, the bugs are known, and if you get stuck, the loss is smaller. If you’re trying to stock up before tariffs or the next crunch for chips manufacturing, look at the cards that were produced 3 years prior. If you can lease, lease. If you can’t, build in an escape route.

And most importantly, build a culture that expects to rebuild. Don’t treat hardware as a foundation, but as scaffolding. Assume you’ll replace it sooner than you’d like. The companies that win aren’t the ones who guess right—they’re the ones who can turn fast when the next leap arrives. Versatility and hybrid approaches will win the day.

As for me, I’m focused on three routes: (1) finding good used or deadstock deals on prior-gen Nvidia cards with high VRAM; (2) investing in Apple systems with high unified memory; and (3) working with AI accelerators in the cloud to find the best price-performance matches. It would be nice to buy a bunch of DGX Spark systems on day one, but that’s probably a decision for year three, at least in my world.