Four Tesla P100s. 128GB of DDR3. Models stored on Velociraptor HDDs. 33.4 tokens per second on Strata QWEN3.8-120b. Total cost: about a grand. Power: 100% solar, 85kWh of battery behind it.
See the rig NumbersProbably the world's first quad-GPU, DDR3-based rig that uses HDDs for LLM storage at this scale. It runs a 120-billion-parameter model on a platform the cloud vendors wrote off a decade ago.
No rack. No liquid cooling loop. No A100s. An Asus dual-socket workstation board, two Xeons, four datacenter accelerators pulled from the secondary market, and last-generation RAM — assembled, tuned, and put to work. The model lives on spinning metal and streams into 64GB of HBM2 at inference time.
Asus Z9PE-D16, 2x Xeon E5-2620 v2, 4x Tesla P100 16GB, 128GB DDR3, four WD Velociraptor 500GB drives for models.
Ubuntu 24.04.5 server, CUDA 12 kept alive on Pascal, Strata QWEN3.8-120b as the resident model.
33.4 tokens/sec sustained across four GPUs. Faster than a human reads, faster than the phone company charges for it.
Drives Hermes and Home Assistant. The model answers the door, watches the driveway, runs the house.
28.94kW PV array, 91 panels, 85kWh of LFP and AGM batteries, 1:1 net metering. Paid off Oct 2023. Details · Full system
V1: i7-3930K, 3x RTX 2080 Ti blower, 64GB DDR3. 91.5 tps on a 35B model, $771 out of pocket. Details · Build page