NVIDIA GTC Berlin: Local AI devices, faster inference

1. From personal PCs to team systems

RTX Spark is a Windows personal PC platform launched on October 16. NVIDIA says it offers up to 128GB of unified memory and 1 petaflop of AI compute, a unit representing 1 quadrillion operations per second. The downloadable desktop agent app Underdog runs models on-device to keep user data from leaving the PC, according to the company.

The DGX Spark 64GB developer system will begin shipping through partners on October 23 and can run models such as Qwen3.8 27B.

DGX Spark can link multiple systems using built-in networking and connection tools. NVIDIA says a manually configured cluster of up to 8 systems provides 1TB of unified memory. This lets small teams share the same model while working on separate tasks.

NVIDIA says the team-oriented DGX Station has 748GB of unified memory and up to 20 petaflops of AI performance. With models such as GLM 5.3, it can respond to more than 30 users at speeds similar to cloud models.

2. Acceleration results for llama.cpp and vLLM

NVIDIA worked with the llama.cpp and vLLM open-source communities to accelerate inference for agent tasks. The key improvement is speculative decoding, which speeds generation by predicting upcoming output. vLLM also gained kernels, optimized code for specific computations, that accelerate input processing for Qwen models.

NVIDIA says Qwen 3.8 Flash Next agent tasks in llama.cpp ran up to 1.4 times faster on RTX PRO 6000 Blackwell. Task completion times were compared against the September 1 version using Windows and Q4_K_M quantization, a technique for representing numbers at lower precision.

NVIDIA says Qwen3.8 27B agent tasks in vLLM ran up to 1.3 times faster on DGX Spark. The figure compares completion times on a local agent benchmark against the September 1 configuration.

NVIDIA says GLM 5.3 Flash throughput, the amount of work processed over a given time, rose by up to 1.2 times on DGX Station. The result averages decoding throughput at 8·16·32 concurrent requests on a coding benchmark, relative to the September 1 configuration.

These three speedup factors use different models, devices, and metrics, so they cannot be read as a direct performance ranking.

3. Expanding local models and execution resources

Recursive Superintelligence unveiled Recursive Flint, a local model for coding tasks. Of its 40 billion parameters, the learned values that determine model behavior, only 2.5 billion are activated for computation per token. The company says it exceeded 220 tokens per second on a 64GB RTX Spark.

The company says dedicated kernels increased generation speed on RTX Spark by 55% without changing model weights, the internal values determined by training. The weights are released under the Apache 2.0 license and can run on NVIDIA local hardware from personal PCs to developer systems.

NVIDIA also released model weight packages with reduced memory requirements. Qwen3.8-Flash-Next-FP4-Local can run on a single DGX Spark 128GB. This approach reduces the model's memory footprint to leave room for processing conversational context.

4. Management features for multiple devices and models

NVIDIA Sync supports configurations of up to 8 DGX Spark systems and has added support for connecting two Linux-based DGX Station systems.

Model Launcher is scheduled to arrive later this month. It downloads models and containers suited to the system configuration and connects them to OpenCode. Containers bundle the programs and settings needed to run software. This reduces the steps developers take to select a model, prepare its runtime, and connect it to their work tools.

Early access to NVIDIA Cloud Functions is also planned for Linux-based DGX Station. Teams can register local model endpoints, addresses for accessing model capabilities, in a shared list and access them through a single OpenAI-compatible entry point. NVIDIA says members can use shared models without directly accessing devices and swap endpoint models without changing application code.

5. Shifting performance and operational burdens

Teams gain shared access to models while keeping data local, but take on manual cluster configuration and device management in return. Connection and model execution tools help reduce this work. The expansion of management features alongside acceleration also reflects an environment where multiple developers use the same local resources.

The announcement's focus is shifting from individuals running models to teams sharing and managing local inference resources.