Performance — SolarAI LLM Server V2

33.4 tps

Strata QWEN3.8-120b · 4x Tesla P100 · tensor parallel · CUDA 12

Average human reading speed is 200–300 words per minute — roughly 4 to 6 tokens per second. A 120B model generating 33 tokens per second outruns the reader, not the reader's patience. A paragraph of agent reasoning arrives in the time it takes to pour coffee. Long documents take minutes, not hours, and the machine is billing nobody for them.

Where the speed comes from

HBM2 bandwidth: each P100 moves ~700 GB/s against its own weights. Token generation is a memory-bandwidth problem, and Pascal's HBM is the honest part of this build.
Tensor parallelism: four GPUs split the layers. Each card computes a quarter of the matrix work per token; PCIe on the Z9PE-D16 carries the sync traffic.
4-bit quantization: the 120B model shrinks to fit 64GB. Less memory traffic per token, more context window per dollar.
Warm weights: once loaded, the model never touches the HDDs again. The Velociraptors matter at boot, not at token time.

The honest limits

Limit	Reality
No FP8 / new-format tricks	Pascal is FP16/FP32. Newer cards generate faster per dollar. This card generation is free-ish per dollar.
PCIe sync overhead	Four-way tensor parallel over PCIe costs a slice of throughput versus NVLink. Accepted at design time.
Model load time	Streaming ~250GB from HDD to VRAM takes minutes. Boots are scheduled, not casual.
Single-model residency	64GB holds one 120B model. Swapping models means a reload from the HDD array.
Power	~1.2–1.4 kW under load, 100% solar. It earns its electricity by replacing a cloud subscription, not by idling.

For comparison: at 33.4 tps, a 2,000-token response lands in about a minute. The electricity comes from a paid-off 28.94kW solar array with 85kWh of battery behind it, which makes the operating cost $0.00 per token — the number V1 was built to prove and V2 runs every day. A cloud API at $0.50–$1.50 per million output tokens would take months of heavy use to spend what these four GPUs cost. The speed isn't the point. The ownership is.