llama.cpp b11716/b11717: Hexagon 1-bit and 2-bit support, row extraction and Hadamard operations

1. Support completed across two releases

b11716 was released on October 17, 2026, and b11717 on October 18, in UTC. The earlier release added low-bit matrix multiplication, while the follow-up added conversion and row extraction operations.

The newly supported formats are Q1_0 for 1-bit quantization and Q2_0 for 2-bit quantization. Quantization represents model weights using fewer bits. The first change also repacks weight layouts when loading a model and supports rereading arbitrary byte ranges.

Matrix multiplication is supported on both the HVX vector processing unit and the HMX matrix accelerator. Beyond simply reading files, this change connects weights in the new formats to actual computation paths.

2. Row extraction and Hadamard transforms added

The follow-up change added GET_ROWS, which reads only selected rows, for both flat and tiled layouts. A tiled layout divides a matrix into small blocks stored contiguously, making it easier for the accelerator to fetch data for computation.

Hadamard matrix multiplication was also added. This transform mixes input values through addition and subtraction. This path requires a processing length that is a power of 2 and at least 32. Weights and outputs must be 32-bit floating point, while inputs must be 32-bit or 16-bit floating point.

Each input row and the output must be contiguous, and the input and output must have the same size. Support checks also consider whether they fit in the device's internal working memory. The new low-bit weight support and the data type requirements for Hadamard operations should therefore be read separately.

3. Storage savings and runtime conversion costs

The author of the follow-up proposal said Bonsai-2's 1-bit and 2-bit paths are now fully usable. However, further optimization is needed to better exploit the smaller main-memory footprint compared with formats of 4 bits or more. Being usable does not necessarily mean optimal execution efficiency.

Developer documentation says the matrix accelerator path expands a 576-byte 1-bit tile at runtime into four 16-bit floating-point tiles totaling 8192 bytes, held in the device's internal memory. A 576-byte 2-bit tile expands into two tiles totaling 4096 bytes. Thus, a smaller storage representation still requires conversion work and intermediate storage.

4. Limits of validation and performance evidence

The first proposal records successful builds across several architectures and testing with v81 on an Android SM8850 device. Both 1-bit and 2-bit support passed 128 out of 128 tests each. This validates the first change, not the follow-up change to the same extent.

Neither proposal provides measurements comparing processing speed or actual memory savings for the new formats. The number of passing tests therefore cannot serve as evidence of a speed improvement.

5. Getting started with release builds and operation tests

The releases include Linux and Android arm64 builds for Snapdragon and links to installation instructions. These builds include CPU, Adreno GPU, and Hexagon NPU support. For those building from source, the guide's unified build tool uses Docker containers and presets to assist compilation and deployment.

The following reproduces the matrix multiplication test command from the saved Snapdragon guide verbatim. It tests an operation on the specified NPU; it is not a command to run the Bonsai-2 model or a performance measurement result.

~/src/llama.cpp$ ./scripts/snapdragon/run.py --target adb --devices HTP0:0 -- test-backend-ops -b HTP0:0 -o MUL_MAT

To run a model, select the NPU device in the execution tool and pass the model path. However, the provided materials do not give specific conversion commands for creating models in the new formats.

These changes expand the groundwork for testing ultra-low-bit models on NPUs, but adoption decisions should assess operation support and execution efficiency separately.