Blog · Hardware Building · September 20, 2026 · 8 min read
Building a Dual-GPU 48GB Rig for 70B Models: A Practical Guide
.webp)
One 48 GB workstation card costs as much as a used car down payment. Two 24 GB consumer cards cost a fraction of that and, for inference, behave like one big 48 GB memory pool. This is the most popular path to running 70B-class models at home — here is how it works and what to honestly expect.
How two cards become one pool
Inference frameworks (llama.cpp, Ollama, vLLM) split the model’s layers across the installed GPUs — a strategy called layer splitting. Half the layers live on card one, half on card two; tokens flow through both in sequence. Key properties:
- Capacity adds up: 24 + 24 GB fits a 70B model at Q3/Q4 with context to spare — the entire reason this rig exists.
- Speed does not add up. Tokens pass through both cards serially, so the generation speed is governed by the slower card’s bandwidth, roughly halved by the split. Expect the sum of the rig to feel like a bit slower than the single faster card, not twice as fast.
- Mismatched pairs are wasteful. Pair identical cards when possible. A fast card babysitting a slow one drags the whole pipeline down to the slow card’s pace.
The shopping list
- GPUs: two identical 24 GB cards (used RTX 3090s are the community favorite) — or 16+16 GB for 32B-class work. See what each pairing fits in the compatibility panel using the GPU-count multiplier.
- Motherboard: two physical x16-sized slots with real spacing — single-slot thickness cards or a riser cable if the board crowds them.
- Power: each 350 W-class card needs its own dedicated PSU cables (never one daisy-chained cable per card), and the PSU sizing rules double: two 3090s plus system means a 1200+ W quality unit.
- Cooling: the top card starves for air in most cases — plan front intake, consider blower-style cards, and give the rig room to breathe.
Honest expectations
| Setup | Feels like |
|---|---|
| 2× RTX 3090, 70B at Q4 | Roughly 8–12 t/s — slow-reading pace, genuinely usable |
| 2× RTX 3090, 32B at Q4 | Comfortable reading pace and lots of context headroom |
| Same rig, MoE models | Noticeably snappier — active parameters are small |
Set your expectations with the speed formula in our tokens-per-second guide: capacity doubled, bandwidth did not. If the goal is 70B quality at interactive speed on a budget, this rig delivers — and the electricity bill (see the crossover panel — both cards’ TDPs count) is the price of admission.