Why two cards make sense: how the model splits between them
Two graphics cards do not compute twice as fast — and that is not why you buy them. You buy them to add their memory together. Here is exactly what happens inside when a model spreads across two cards, and why this particular way of splitting wins on home hardware.
A model is a stack of layers, and layers can be dealt out
A language model is built from several dozen layers stacked one on top of another. Data enters the first, the result passes to the second, and so on to the end. Our configuration cuts this stack in half: the first card holds the early layers, the second card holds the rest.
What crosses the boundary? A single vector of numbers — the state after the first card’s last layer. That is a few kilobytes, sent once per pass. This is exactly why this way of splitting works on an ordinary motherboard, with no special bridges between the cards.
There is one price, and it has to be said plainly: the cards work in turn, not at the same time. While the first one computes, the second one waits. When generating text, two cards therefore do not give you double the speed — they give you double the capacity. Because both cards are identical, there is no weaker link to drag the whole thing down.
When reading in a long document, things look better: there is so much work that both cards genuinely share the computation. Hence the result of 7,000 tokens per second when reading in versus 200 tokens per second when writing — two different working modes of the same machine.
The ways of splitting we rejected
Before settling on the split by layers, we checked the alternatives. Both fell through for concrete, verifiable reasons.
Splitting by matrix rows no longer exists in the library we use. The option still appears in the program’s help text, but it has been removed from NVIDIA card support — trying to use it ends in an error before anything reaches the card’s memory. No parameter will bring it back.
Tensor splitting (each card computes a piece of the same layer at the same time) offers true parallelism in theory. In practice it breaks on two things. First, the working buffer is allocated at full size on each card — that is, twice over — and does not fit even with a modest context. Second, a failed allocation does not fall back gracefully to a simpler mode; it kills the process.
Even if it worked, this machine would be the worst possible ground for it: tensor splitting requires synchronizing the cards after every layer, and there is no fast bridge between our cards — communication goes over the motherboard’s ordinary bus. The split by layers sends one vector per pass, and that is why it wins.
One layer this way or that — and we gain more context
The split does not have to be exactly down the middle. Layers vary in weight, and extra components attach themselves to one of the cards (for example, the module responsible for understanding images). As a result, after the model loads, one card can be filled to the brim while the other still has headroom.
Moving a single layer from the tighter card to the looser one shifts about 266 MB of headroom. On the model with image support, that one move bought us 24,576 tokens of context — roughly a few dozen extra pages of text in a single conversation, for free.
It is worth knowing the limitation: the granularity is coarse. You move whole layers, not fractions, and on models with few heavy layers a single layer weighs too much for the maneuver to pay off. There is usually one sensible value, and its neighbors are worse than an even split.
A second model that helps the first
A separate trick, unrelated to splitting across cards but responsible for a good share of the speed: speculation. Alongside the big model works a tiny one that guesses the next few tokens. Instead of generating them one by one, the big model checks the whole proposal in a single move — and checking is much cheaper than generating.
On our machine this technique works in every model that supports it and speeds up writing by 1.3 to 2.4 times. The rule is the opposite of intuition: the slower the base model, the more guessing pays off, because the cost of one suggestion is nearly constant while the step it saves gets ever more expensive.
With two cards there is a condition you cannot see from the outside: the small model must sit on the same card as the big model’s final layers. Set up carelessly, it spreads across the cards and, instead of helping, it costs. We measured the difference: 27 versus 119 tokens per second on the same model. The entire difference comes down to one parameter saying where the helper model should live.
What it costs today
European retail prices, August 2026:
| Variant | Memory | Price |
|---|---|---|
| RTX 5090 — one card | 32 GB | €4,400–4,650 ($5,100–5,350) |
| RTX 5070 Ti — two cards | 16 GB + 16 GB | €2,100 ($2,400) |
The pair of cards costs 47% of the flagship’s price with identical memory capacity. Add one requirement that is easy to forget: the motherboard must have two slots with a sensible lane split, and the case must have room for two cards and airflow between them.
Treat these numbers as a snapshot, because the market is moving in one direction. NVIDIA has raised prices twice already in 2026 — by 10–15% in January across the whole RTX 50 line, and in May on the RTX 5090 alone — and a third increase of 20–30% has been announced. The cause is not demand for cards but memory: Samsung raised DRAM prices by about 20% in a quarter, and the memory shortage is forecast to last until at least 2028. Retail already feels it — the RTX 5090 was fetching about $4,329 in a US store in July 2026, with non-reference versions passing $5,000.
For a buyer, one thing follows: a percentage increase hits the more expensive variant harder. Thirty percent is about €630 ($720) on the pair of cards and close to €1,390 ($1,600) on the flagship. The price gap between these two paths is not shrinking — it is growing.
What this means when buying
Two mid-range cards give you the flagship’s capacity for less than half the price, but not its throughput. If what matters is a record in tokens per second on one big model — a single powerful card wins. If what matters is which model, and how long a context, will fit at all — two cards do exactly the same job, for less.
There is one condition: the cards should be identical. With the split by layers the cards work in turn, so the weaker one of the pair would set the pace for the whole machine.
All performance numbers come from measurements on our test machine: 2 × RTX 5070 Ti 16 GB, split by layers, one model at a time. Hardware prices: European retail, August 2026 — check current prices before buying.
Sources on the price increases: ThinkComputers — RTX 50 prices could jump another 30% · TechSpot — GPU pricing, mid-2026