FlashProve 0.16.0: Inference verification, open-source prototype

1. Checking whether an answer came from the promised model

An AI answer alone rarely reveals which model actually produced it. The project targets hard-to-detect cases where a routing service switches to a smaller model to cut costs or a server intruder manipulates output tokens.

The provider first creates a commitment, a short fingerprint of model weights that cryptographically fixes them while hiding the originals. It then generates a proof file for the input and answer tokens. Users check the file against the commitment to verify that the answer came from those weights. Proofs are designed to conceal weights, intermediate inference values, and temporary values produced during computation.

2. Testing large-model proofs with batching

Anthropic measured an integer model converted from Llama 4 Maverick's public 4-bit weights. This mixture-of-experts model activates only some computational experts, using about 17 billion of its 400 billion parameters per token. The research post says proving work scales with the roughly 17 billion active parameters, not all 400 billion. This structure reduces the proving burden for large models.

Section 6 of the technical report gives these results for single-request and 24-request batch proving. Throughput counts all request tokens during prefill, the stage that processes input tokens first, rather than only verified output tokens.

Table: Proof generation results for single requests and 24-request batches

Conditions: Integer model from Llama 4 Maverick's public 4-bit weights; one H100 80GB and 24 CPU cores; 1,024 tokens per request; one next token verified; 128-bit security target; company-reported results

Units: Time in seconds; throughput in tokens/second across the full request; proof size per request in MB

Chart: Proving throughput by batch size. Conditions: Same integer-model measurements as the table above, counting all request tokens. Unit: tokens/second

Single request    ███▌                   17.6
24-request batch  ██████████████████████ 110

These figures show proving speed when recomputing long inputs. Report Section 6 reports no measured speed difference in a separate test verifying every answer token. Counting only verified tokens, throughput was 39.1 tokens per second.

The research post says proof verification took 17 seconds on 24 CPU threads and 21 seconds on a single thread.

In the research post, the company says a weight commitment is generated once per model, takes about 39 minutes, and can be reused for about 16 million requests.

The vLLM baseline ran the same model's FP8 version, an 8-bit floating-point format, on eight H100 GPUs. Per the research post, FlashProve's 24-request batch proofs took about 72x as long as same-size vLLM prefill, normalized to one GPU. The range across runs was 66–77x. Against vLLM's highest tested load, the ratio was about 94x.

3. Proving after inference separates the real-time burden

FlashProve delivers the response first, then proves it by recomputing inference from recorded inputs and outputs on a separate server. This is post-hoc processing, performed separately after the response. Instead of proving each token as it is generated, it batches requests and parallelizes GPU work to run computations concurrently. It splits model layers into segments, loading only part of the computation onto the GPU at once and reading weights from host memory, the CPU's main memory.

This keeps proof generation from blocking the inference service's immediate response path. However, the company reports that proving itself uses about 70–95x more GPU time than ordinary inference. The report appendix estimates that including recomputation adds about 1–1.5% to the proving workload. Per report Section 6, individual requests may wait about 5 minutes for a batch proof to finish. Processing requests individually produces a proof in 58 seconds but lowers throughput to 17.6 tokens per second.

4. The prototype's verification scope and usage

The research post says security properties were formalized as logical statements in Lean 4, a tool for machine-checking mathematical propositions. The repository includes a fast C++ verifier and a slower Lean verifier with formal guarantees.

Report Section 7 says neither the fast code's equivalence to the Lean definitions nor batch proofs' concealment of weight information has been formally proven. The research post describes verification of an integer model choosing the highest-scoring token in contexts of up to 1,024 tokens. Floating-point computation or answers sampled probabilistically may diverge from these results.

Developers can follow the repository instructions to build the verifier and test it using a saved proof for a small model with synthetic test weights. The commands below are an example. For actual proof verification, users must obtain the commitment ID through a trusted channel.

bin/cp_verify examples/accepted-proof/config examples/accepted-proof/ivk examples/accepted-proof/example.req examples/accepted-proof/example.y examples/accepted-proof/example.proof -id "$(cat examples/accepted-proof/ID.txt)"

FlashProve is a research prototype showing that inference can be verified on a large model without disclosing its weights, though computational costs and verification scope remain key to practical use.