CPU vs GPU Battle
A recurring experiment: take a real problem, solve it first the easy, CPU-friendly way, then bring in the GPU and deep learning — and keep an honest scorecard each round. The problem changes from post to post; the tension between the mature CPU toolchain and the raw power of the GPU stays the same.
Start Here
This is an open-ended series about the battle itself, not any single problem. Start with Part 1 to see the format in action, and expect more problems to enter the ring over time.
Questions This Series Answers
- When is the GPU actually worth it, and when does the mature CPU toolchain win?
- How have coding agents changed the economics of GPU optimization?
- What does an honest, on-device scorecard look like for a real problem?
- Where does classical, CPU-friendly engineering still beat deep learning?
Key Themes
- CPU versus GPU tradeoffs
- On-device machine learning
- Classical methods versus deep learning
- Coding agents and the new cost of optimization
- Honest benchmarking
Why this battle is worth watching
In graduate school I once spent hours — days, really — hand-optimizing a small GPU program. Tiling, memory coalescing, occupancy, the whole ritual, chasing a kernel that would finally saturate the hardware. It was painstaking, specialist work, and getting it right felt like an achievement in itself.
That kind of optimization is exactly what coding agents now absorb. The tuning that used to eat a week gets handled, and we get to harvest the benefit instead of grinding for it. Reaching for the GPU has never been cheaper.
But here is the twist that makes the fight interesting: CPUs have been around far longer, and the compilers, libraries, and habits of a whole generation of engineers have a much deeper grasp of the CPU than the GPU. So on any given problem it is genuinely unclear which corner wins — the mature, well-understood CPU stack that runs anywhere, or the raw, newly-accessible power of the GPU.
This series runs that battle one real problem at a time. Each round: take a problem, solve it first the easy CPU-friendly way, then bring in the GPU and deep learning — and keep an honest scorecard, wins and losses. The problem changes from post to post; the tension does not.
Articles in This Series
- CPU vs GPU battle, Round 1: the agents brought the GPU
How well does a small prompted LLM grade e-commerce search relevance on a 0-3 scale, entirely on a laptop? We start the CPU-vs-GPU battle with the two easy arms - BM25 and a prompted local model - and find the honest surprise is not accuracy but calibration and circularity.