Boole-1.7B: 1-bit GGUF at 292.1MB, average score 8.28 points below BF16

1. Weight data and download files differ

Precision Numerics released Boole-1.7B on October 11, 2026. Based on Qwen3-1.7B, it targets English answer generation with the separate thinking mode disabled.

The announcement lists 247.0MB of weight data, 13.9 times smaller than the 3,441.1MB BF16 original, a 16-bit floating-point format. Each bit retains a weight’s positive or negative sign. Shared scale factors supply magnitudes and also occupy storage.

Those sizes cover weight data, excluding headers, metadata, and the tokenizer. The actual safetensors file is 247.1MB; the GGUF file, which also runs on CPUs, is 292.1MB. The former shares sign bits between the input embedding matrix and the output computation matrix, while the latter stores the two separately.

2. Average declines differ by capability

The company evaluation weights ten tasks equally, with no examples and thinking mode disabled. Generation normally selects the most probable answer. The exception, GPQA-Diamond, averages ten generations with temperature set to 0.6 and the cumulative probability cutoff for narrowing candidates set to 0.95.

The average fell by 8.28 points, from the original’s 58.83 to 50.55. The announced 85.9% is the ratio of these averages, not an accuracy rate for arbitrary questions or equal retention across capabilities.

In the same evaluation, the mathematics task MATH-500 fell from 74.20 to 49.80. By contrast, the instruction-following task IFBench rose slightly from 17.67 to 18.25. This shows why the average decline cannot reliably predict losses on individual tasks.

Bonsai-1.7B, compared by the same company under the same evaluation framework, averaged 44.97 at 242.4MB. Ternary-Bonsai-1.7B averaged 51.57 with about 457MB of weights. It was about 1.9 times larger than Boole-1.7B but scored 1.02 points higher on average. These results cannot be mixed with scores that the respective developers published separately using different task sets.

Precision Numerics evaluation. The average is an equal-weight mean of ten tasks; storage is weight data including quantization scales, excluding file headers, metadata and tokenizers. Runtime memory and latency were not measured.

3. Match execution to the chosen format

The example repository requires Python 3.10 or later. The following original CUDA GPU example runs in an environment with the download tool installed. It sets the context length to 4,096 tokens and disables thinking mode.

git clone https://github.com/PrecisionNumerics/boole-demo.git && cd boole-demo
hf download precision-numerics/Boole-1.7B-GGUF Boole-1.7B-Q1_0.gguf --local-dir model
bash scripts/setup_llama_cpp.sh
third_party/llama.cpp/build/bin/llama-server \
    -m model/Boole-1.7B-Q1_0.gguf -ngl 99 -c 4096 \
    --reasoning off

GGUF runs through llama.cpp on CPUs or supported GPUs. For CPU-only execution, follow the original instructions to build with GPU support disabled and set the number of layers the server offloads to the GPU to 0.

4. Distinguish storage savings from runtime efficiency

The benefit is reduced storage; the confirmed cost is lower evaluation quality. Examples for the separate vLLM and Humming GPU paths request 16-bit data types. Thus, 1-bit storage should not be read as 1-bit computation throughout inference.

No measurements were provided for peak runtime memory, time to first token, or generation speed. A small file alone therefore guarantees neither low-memory nor fast execution. Before adoption, separately measure resource use at the intended context length and reliability on required tasks on the target device.

Boole-1.7B is a candidate when storage is constrained, but actual tasks should determine whether its average loss of 8.28 points against the original is acceptable.