A current OS, a CUDA toolkit that officially moved on from Pascal, and a 120-billion-parameter model that fits because somebody made it fit.
| Layer | Component | Notes |
|---|---|---|
| OS | Ubuntu 24.04.5 server (amd64) | Headless. No desktop, no telemetry, no games preinstalled. |
| GPU runtime | CUDA 12 for Pascal | P100 is compute capability 6.0. CUDA 12 keeps Pascal compiling and running — the toolkit hasn't dropped it yet, so we run with it. |
| Model | Strata QWEN3.8-120b | 120B parameters, quantized to live across 64GB of VRAM on four GPUs. |
| Serving | Multi-GPU inference server | Tensor-parallel across the four P100s, OpenAI-compatible endpoint on the LAN. |
| Agents | Hermes Agent | Consumes the endpoint. Tools, memory, scheduled jobs, the works. |
| Home | Home Assistant | Automation layer the agent acts through. |
NVIDIA's support matrix says Pascal is legacy. The toolkit says it still compiles. That gap is where this machine lives: a modern OS, a modern CUDA major version, and accelerators from 2016 running FP16 tensor work at full rate. No new features, no new tricks — just the math, which is what inference is.
The 120B model is the other half of the trick. At 4-bit quantization the weights come down to roughly 60–70GB — which is exactly the envelope 64GB of HBM2 provides, split four ways. The quantized model is stored on the Velociraptor array and loaded into VRAM at service start.
# model lives on the velociraptor array $ ls -lh /mnt/models/strata-qwen3.8-120b/ -rw-r--r-- 1 solarai solarai 62G Sep 26 14:02 model-00001-of-00004.safetensors -rw-r--r-- 1 solarai solarai 62G Sep 26 14:04 model-00002-of-00004.safetensors -rw-r--r-- 1 solarai solarai 62G Sep 26 14:06 model-00003-of-00004.safetensors -rw-r--r-- 1 solarai solarai 61G Sep 26 14:08 model-00004-of-00004.safetensors # four-way tensor parallel, CUDA 12, Pascal sm_60 $ serve --model strata-qwen3.8-120b --tp 4 --port 8100 [ok] cuda 12.x / p100 x4 (sm_60) detected [ok] 64.0 GB vram total, weights resident [ok] endpoint live: http://192.168.0.2:8100/v1