NVIDIA announced support for clusters combining up to 8 DGX Spark systems. With 1TB of unified memory, developers can run larger models locally, operate multiple models together, or train small models.
1. Running large models with 1TB of memory
NVIDIA announced official support for 8-node clusters at GTC Berlin. Unified memory totals 128GB on 1 system, 512GB on 4, and 1TB on 8. It says this enables running Z.ai's full GLM 5.3 model (753B), which did not fit on 4 nodes.
According to the company, GLM 5.3 Flash generated about 80 tokens per second on 8 nodes. Separate measurements for the full GLM 5.3 model are shown below.
Table: Throughput of the full GLM 5.3 checkpoint
Conditions: 8-node DGX Spark, nvidia/GLM-5.3-NVFP4 (about 470GB), MTP3 enabled, input processing measured with an input length of 8K per request
Unit: TPS (tokens per second)
| Concurrent requests | Total input processing | Input processing per user | Total token generation | Token generation per user |
|---|---|---|---|---|
| 1 | 1,600 | 1,600 | 31 | 31 |
| 4 | 1,600 | 750 | 51 | 21 |
| 8 | 1,600 | 473 | 67 | 18 |
Increasing concurrent requests raises total token generation but lowers per-user speed. The operating choice depends on whether the priority is serving multiple users or responding quickly to one user.
2. How connectivity affects performance and cost
Splitting model computation across nodes requires data exchange between them. RoCEnante, evaluated by NVIDIA, exchanges data directly over a switched connection and sums it at each node to reduce communication steps.
The company reported the following increases in token generation performance for Qwen3.8-27B-NVFP4. Each multiplier uses a single node as its baseline.
Table: Token generation performance gains by communication method
Conditions: Switched connection, Qwen3.8-27B-NVFP4, speculative decoding disabled
Unit: Multiplier
| Nodes | With RoCEnante | Without RoCEnante |
|---|---|---|
| 1 | 1.00 | 1.00 |
| 4 | 3.06 | 2.56 |
| 8 | 4.43 | 3.05 |
Fewer communication steps mean more data transferred per node. This overhead grows with large messages, many nodes, or concurrent requests, so the company says the default communication method, NCCL, is more advantageous then.
3. Throughput gains from adding nodes
The company reported the following results by topology with the same model and input/output lengths. Multipliers use a single Spark at the same precision as the baseline. Switched setups used RoCEnante.
Table: Qwen3.8-27B performance by topology
Conditions: NVIDIA/Qwen3.8-27B(NVFP4), llama-benchy, 8192 input tokens and 256 output tokens, 1 concurrent request, speculative decoding and MTP disabled
Unit: Throughput in tokens per second
| Nodes | Topology | Input throughput | Input multiplier | Generation throughput | Generation multiplier |
|---|---|---|---|---|---|
| 1 | — | 2,635 | 1.00 | 11.86 | 1.00 |
| 2 | Direct connection | 3,285 | 1.25 | 21.71 | 1.838 |
| 4 | Physical ring | 4,446 | 1.69 | 32.82 | 2.77 |
| 4 | Switch (RoCEnante) | 4,556 | 1.73 | 36.34 | 3.06 |
| 8 | Physical ring | 4,962 | 1.88 | 40.11 | 3.38 |
| 8 | Switch (RoCEnante) | 5,105 | 1.94 | 52.58 | 4.43 |
The 8-node setup with a switch and RoCEnante delivered the highest generation throughput. A ring connecting nodes in a loop reduces cost and setup steps, while a switch is required to split nodes into independent groups.
4. Routing easy requests to smaller models
NVIDIA NeMo Switchyard separates easy and difficult requests and routes them to suitable models.
The company's prototype used OpenBMB's MiniCPM5 2B for easy tasks and Qwen3.8 Flash Next for difficult ones.
According to the company, this setup scored 91/100 on a tool-use evaluation with 92 cases, including difficult ones, matching Qwen alone.
Across 8 nodes, 3 Qwen replicas each received 2 nodes, and 2 MiniCPM replicas each received 1 node. The company reported about 1.5 times Qwen-only throughput when half the requests went to Qwen, and about 3 times when a quarter did. A switch is used to partition nodes by model this way.
5. Scaling small-model training locally
NVIDIA trained nanochat, with about 896.5M parameters, from scratch. Each node holds a model copy and synchronizes updates at every training step, which the company says makes throughput scale readily with added nodes.
Table: Scaling nanochat pretraining
Conditions: 1–8 DGX Spark nodes, nanochat with about 896.5M parameters, global batch of 524,288 tokens per step, 32 sequences per GPU microbatch
| Nodes | Throughput (tokens per second) | Speedup (times) | Scaling efficiency (%) |
|---|---|---|---|
| 1 | 8,741 | 1.00 | 100.0 |
| 2 | 17,202 | 1.97 | 98.4 |
| 4 | 33,640 | 3.85 | 96.2 |
| 8 | 68,079 | 7.79 | 97.4 |
According to the company, 8 nodes delivered 7.79 times the training speed of a single node.
Support for 8 nodes in the NVIDIA Sync management tool is scheduled to arrive by the end of 2026.
An 8-node DGX Spark expands the scope of large-model execution and local training, but actual gains depend on whether response speed, total throughput, or network cost takes priority.