Researchers at the University of Oxford built a 128-GPU cluster from second-hand NVIDIA V100 processors, costing about $22,000, and used a custom Rust-based pipeline-first parallelization strategy to achieve competitive LLaMA-70B inference throughput. Their device-level pipelining treats individual GPUs as separate pipeline stages rather than grouping them into nodes, letting older, memory-limited hardware serve modern large language models. The setup is dramatically cheaper than current-generation systems like the B200 (about $22K versus $600K), but the authors find its environmental viability depends heavily on location, with operational carbon emissions ranging from negligible on renewable grids to roughly 40 times higher than new hardware on grid-average electricity.
