NeuronDump 0.1.3: Exploring model structure, shrinking GGUF files

1. From inspecting model structure to saving a new file

Introduced by developer aymen94 on October 26, NeuronDump 0.1.3 bundles a command-line tool and a local web interface in one executable. It reads the local model format GGUF directly and embeds its interface for offline use.

The graph displays the model as a layer-by-layer blueprint. Users can inspect weight arrays called tensors, quantization formats that store numbers at lower precision, vocabulary, and chat templates. Opening a layer reveals heads that examine input relationships and groups of computational units called neurons.

Structural analysis and file size calculations read the file without running the model. When installing from source or a package, enabling execution tracing runs the model on the CPU to compare tokens and probabilities before and after edits.

A simple workflow follows. These commands are examples from the repository README.

neurondump list                          # every local model
neurondump inspect llama3.2:1b           # what's inside a model
neurondump vocab llama3.2:1b --search hello
neurondump template llama3.2:1b          # the chat template as a tree
neurondump analyze llama3.2:1b --out plan.json
neurondump slim llama3.2:1b --plan plan.json --only vocab --out small.gguf
neurondump tokens llama3.2:1b "fn main() {" --chat     # tokens, template tokens, nearest tokens
neurondump web --open                    # the same, in your browser

2. Finding where predictions diverge within the same structure

The repository compares Qwen3 8B with DeepSeek R1 0528 Qwen3 8B: all 399 tensors share names, shapes, and types, but none match byte for byte. It interprets this as fine-tuning all weights rather than adding new layers.

Given an incomplete English sentence asking for the star Earth orbits, both models first predicted the Sun at layer 29. The trace reads intermediate layer states as next-token candidates; the probabilities below come from the repository.

Table: Prediction probabilities by layer for the same input

Conditions: NeuronDump 0.1.3, candle, CPU, both models in Q4_K_M, the same 9-token input asking for the star Earth orbits

The repository interprets the answer emerging in intermediate layers, followed by an English definite article at the end, as fact retrieval followed by phrasing selection. Developers can inspect where candidates change, rather than just the final output, and compare the effects of editing those parts.

3. Evaluating size savings alongside quality changes

In the repository's llama3.2:1b test plan, reducing vocabulary to 570 entries using a sample corpus cut file size by 21.6%, from 1.23 GiB to 987 MiB. GGUF validation confirmed 147 tensors and 16 layers. A smaller vocabulary also limits the tokens available for output.

The effects of layer removal were compared using a corpus, a fixed set of evaluation texts. Perplexity below measures how difficult those texts are for the model to predict; lower is better.

Table: Perplexity after layer removal

Conditions: candle, CPU, llama3.2:1b, the same corpus of 5 English evaluation texts

Removing the first or last layer hurt prediction quality more than removing a middle layer in this evaluation. Size savings come at the cost of output quality and exploration time. The test plan's search evaluated removing layers one at a time, taking about 4 minutes on the CPU for 85 model computations across 96 positions in 5 samples.

Edits are compared in memory without touching the original, then saved to a new file. Developers can select removal candidates, evaluate quality on their intended texts, and then create a smaller file.

4. Choosing experiments within execution support limits

Execution experiments are limited to CPU processing for listed model families. Running MoE models that select experts for computation or hybrids that mix layer approaches is unsupported, as is shrinking files by cutting out heads, neurons, or experts.

The repository labels this an early version. Its documentation says the new 3D data flow view and token and editing views have not yet been verified by running them in a browser.

NeuronDump's value lies not just in making models smaller, but in helping users decide what to remove by examining structure alongside changes in output.