DeepSeek-V4.1-Flash-1017: 1-bit local inference, lower memory use with SSDs

Quantization evaluation and results

The model hub currently lists 9 quantizations. Quantization reduces size by adjusting numerical precision, while GGUF is a file format for model weights. Unsloth compared each file with DeepSeek's official F32 (32-bit floating-point) weights. On wiki1100 and long17k, it measured KL, the difference between probability distributions, and top-1 agreement, the share of next-token predictions matching the original.

Table: File size and top-1 agreement by quantization

Conditions: Unsloth Dynamic GGUF versus official DeepSeek F32 weights. top-1 is the share of next-token predictions matching the original. Non-Engram weights exclude the Engram table and represent the absolute minimum memory requirement.

Units: Total size and non-Engram weights in GB; top-1 agreement in %

Source: Unsloth quantization analysis

1-bit IQ1_S is about 54% smaller than UD-Q8_K_XL, which Unsloth classifies as lossless. IQ1_M adds just 17.4GB, recovering 4.3 percentage points on wiki1100 and 3.1 on long17k. 4-bit Q4_K_XL is 7.7GB smaller than the same UD-Q8_K_XL, with agreement gaps of only 0.3 percentage points on wiki1100 and 0.2 on long17k.

Memory requirements

The 1-bit UD-IQ1_S download is 233.0GB, including a 110.6GB Engram table kept on SSD.

Table: Estimated memory requirements by bit level

Conditions: Unsloth's estimates by bit level, using RAM+VRAM or unified memory, with Engram kept on SSD. These allow about 5~10GB above non-Engram weights; context and runtime may require more.

Unit: GB

Memory requirements rise sharply with bit level. These estimates describe bit levels, not per-file minima, so the 1-bit lower bound does not apply to files with larger non-Engram weights, such as IQ1_M.

Official benchmarks

Table: Comparison of official DeepSeek benchmark scores

Conditions: Original model scores from DeepSeek's official model card. Terminal-Bench 4.0 uses the Claude Code harness. DeepSWE v1.1, a tool for testing bundled execution environments, uses the Mini-SWE-V2 harness.

Source: Official DeepSeek model card

For development work, the Terminal-Bench 4.0 gains support considering use in Claude Code, while DeepSWE v1.1 gains support considering Mini-SWE-V2 workflows. ProgramBench can inform coding candidate selection, and NL2Repo-Bench can inform selection for natural-language repository tasks. However, these are original model results, not evidence that 1-bit quantization preserves the same task scores.

Why these differences occur

The model has a 552B backbone, its core neural network, and a separate 196B-parameter Engram table, with about 16B active parameters participating in computation. Engram is conditional memory that combines up to 4 recent tokens to read just a few table rows. This enables SSD offloading: keeping the 110.6–202.8GB table on disk and loading only required rows into the OS cache. It saves RAM and VRAM, but disk usage stays the same.

Each quantization level treats MoE weights and other weights differently; MoE is a mixture-of-experts architecture that computes only active parameters. DeepSeek's original mixes MXFP4, FP8, and BF16. Unsloth's 4-bit and 8-bit levels retain MXFP4 experts and the same Engram tables. The 4-bit level keeps many non-expert weights in Q8_0, while 8-bit uses BF16, so level names do not describe every tensor's precision. The documentation says Unsloth Dynamic 3.0 performs post-training quantization using only calibration data.

DSpark boosts speed through accelerated decoding that proposes multiple tokens for the main model to verify. It is separate from Engram, allowing memory savings and faster decoding together. Unsloth reports about 1.3~1.5 times faster decoding on DSpark-supported setups. Drafter modules, auxiliary models that generate draft answers first, are available in Q8_0 at 8GB and BF16 at 8.5GB. Unsloth says its llama.cpp-based setup is about 3 times faster than other CPU, VRAM, and n-gram repositories.

DSpark is optional and requires a matching V4.1-1017 drafter, a DSpark-compatible runtime, and additional memory for the drafter.

Selection criteria

Based on Unsloth's estimates, 1-bit UD-IQ1_S is the first option for devices with around 128GB of memory. Unsloth says this setup runs on devices such as Macbook, DGX Spark, and RTX Spark. At around 256GB, 3-bit UD-IQ3_XXS is an option, while 4-bit UD-Q4_K_XL shows high agreement on both datasets.

Lowering Engram precision is one way to save disk space. Four variants, IQ4_NL, Q5_K, Q6_K, and MXFP8, are available as separate files ranging from 110.6GB to 202.8GB. Replacing only the 1-bit level's IQ4_NL table with MXFP8 is also possible, but increases the table size by 92.2GB.

Building it yourself requires Unsloth's llama.cpp PR.

The default reasoning effort is 75. The following is an example option appended to the run command in Unsloth's documentation.

--chat-template-kwargs '{"reasoning_effort":75}'

The weights and model card are distributed under the MIT license.

Choose 1-bit for lower memory and file size or 4-bit to preserve original predictions, then select a suitable file in Unsloth Chat or the model hub and keep Engram on SSD.