SLP Verifiable Inference: Proof-of-Concept Experiment Report

Summary

1. The protocol works end to end. On GPT-2 (124M), TinyLlama-1.1B and Llama-2-70B, the seal-then-sample layerwise proof completed sealing, proving and verification. Adversarial cases — tampered output, tampered commitment manifest, tampered boundary commitment, missing proof, substituted model, mismatched sampling policy, mismatched beacon — were all rejected by the verifier.
2. 70B is reachable. Llama-2-70B ran end to end on a CPU server with 2 TB of memory: sealing 163 chunk boundaries and proving 5 of them took 21 minutes, the proof is 4.34 MiB, and verification takes 46 seconds without access to the model weights. Peak memory stayed under 2 TB, whereas the same pipeline without streaming is estimated to need about 2.8 TB.
3. Proving cost is driven by model size and is nearly independent of token count. On TinyLlama, raising the context from 8 to 16 tokens changed the proving time from 94.8 s to 99.0 s; packing 12 concurrent requests into one proof took 181.9 s in the proving stage, about 6.5× less than proving them separately at the measured single-proof cost.
4. Fidelity of the quantized model to the original is measurable and has improved substantially. Over 334,705 predicted positions of the WikiText-2 test split, the quantized model agrees with the f32 model on the top-1 token at 84.8%–84.9% of positions, with perplexity 12.94–12.95 (f32: 11.94) and a KL divergence of 0.118 nat; the pre-fix quantization path reached only 13.7% agreement on a comparable evaluation.
5. Detection power of sampling equals coverage. For a forgery confined to unsampled chunks, the single-audit detection probability is the sampling coverage (1.9% at 70B with 3 sampled chunks); repeated audits and economic mechanisms must amplify it. This is an inherent property of the protocol and is reported as such.
6. Pure Fiat–Shamir sampling can be ground. An attacker who forges a boundary can re-seal repeatedly until the sample avoids the forged chunks; each attempt costs 12.5 ms on GPT-2 and the mean number of attempts matches the theoretical 1/q. With a beacon introduced after sealing, the escape fraction returns to the theoretical q.
7. Model versions reproduce deterministically; inference execution does not yet. Ten independent registrations produced identical fingerprints; for the same quantized model and input, 16 of 20 repeated GPU inferences were token-for-token identical.

1. Background and Objectives

SLP is a verifiable-computation protocol for large-language-model inference. An inference is partitioned into layer groups (chunks); the activations at every chunk boundary are committed with polynomial commitments; all commitments together with the claimed input and output are written into the Fiat–Shamir transcript before the chunks to be proven are drawn. Chunks touching the model's input or output are always proven, binding the proof to the specific prompt and answer. The verifier can escalate to a full proof over the same commitments. The experiments answer five questions:

• Does the protocol accept honest proofs and reject tampering on real models (Section 4.1)?
• Does it scale to 70B, and at what cost (Section 4.2)?
• How does proving cost vary with model width, sequence length and concurrency (Sections 4.3, 4.4, 4.5, 4.7)?
• How close is the proven quantized model to the original floating-point model (Section 4.6)?
• Where are the security limits of sampling (Sections 4.8, 4.9)?

2. Experimental Setup

Common configuration: 12-bit activation quantization with the residual stream kept at about 24 bits; KZG-family polynomial commitments over the BN254 curve; Blake3 transcript hashing; LLM chunking cuts after every residual addition plus a standalone embedding chunk and a standalone final-projection chunk, giving 2L+3 chunks (GPT-2: 27, TinyLlama: 47, Llama-2-70B: 163); the two end chunks are anchors and are always proven; default sample sizes are 3 for GPT-2 and Llama-2 and 2 for TinyLlama. Unless stated otherwise, every figure is a single measured run.

3. Experiment Index

4. Results

4.1 Protocol Correctness and Adversarial Negatives (E01, E06, test suite)

GPT-2 end-to-end demo (Mac, re-measured 2026-09-17):

Adversarial negatives in the test suite (all rejected): missing chunk proof, tampered commitment manifest, tampered boundary commitment, tampered input/output, verification under a different model context, mismatched sampling policy, mismatched beacon. Positive cases cover model depth {8, 12} × chunking {default, 2, 4, 6} × sample size {0, 1, 3, all}, plus a serialization round trip of the verifier context. In the service simulation (E06, Section 4.5), the window in which the worker tampered with one generated token before publishing was rejected.

4.2 Scale: Llama-2-70B End to End (E02)

Single inference, context 8, 3 sampled chunks plus 2 anchors, integer weights persisted to disk and weight polynomials committed in streaming fashion.

The same pipeline without streaming is estimated to need about 2.8 TB; an earlier non-streamed commitment attempt held 557 GB resident and had not finished after 5.5 hours (raw log run70b.log). Inference accounts for 70% of the total time and is the current bottleneck; it is a runtime-implementation issue, not a protocol issue.

4.3 Cost Curves (E03, E04, E04b, GPU inference acceleration)

Single-layer cost (E03, Mac). Synthetic feed-forward block (two linear layers, 4× expansion, non-linearity), sequence length 128:

Proving time grows sublinearly in the parameter count (local exponents between adjacent points range from 0.2 to 0.7; the end-to-end fit is about 0.66); proof size grows slowly with d.

Attention and MLP chunks against sequence length (E04, GPT-2, Mac):

The quadratic sequence-length term of the attention chunk becomes visible from 256 tokens. Longer contexts were not measured in this round and are a priority for the next phase.

GPU inference and proving cost (TinyLlama, context 8). Before and after the device-side lookup-table optimization: quantization 170.2 s → 17.9 s, inference 101.2 s → 14.2 s; proving (seal 47 boundaries + prove 4 chunks) 95.2 s → 100.5 s (proving runs on the host CPU and is unaffected). Context 8 → 16: inference 12.8 s → 105.1 s (the latter from a pre-optimization run), proving 94.8 s → 99.0 s, proof size 0.43 MiB → 1.41 MiB. Proving-stage breakdown (E04b): constraint-claim generation 40.2%, witness-context construction 20.5%, proof generation 18.5%, witness polynomial commitments 8.8%, lookup-table claims 6.3%, challenge storage 5.7%.

4.4 Packed Batch Proving (E05, TinyLlama, slot length 16, 4 generated tokens per slot, 2 samples)

Each additional concurrent inference adds about 7 s to the proving stage. Leak check: after replacing the contents of all neighbouring slots, the outputs of the three slots were token-for-token unchanged; the block-diagonal mask leaks nothing across slots. Equivalence check: slot 0 matches a standalone inference token for token; slots 1 and 2 do not, because packing shifts their absolute positions and the quantized rounding of the rotary position encoding changes accordingly. Packed inference is therefore not token-for-token equivalent to standalone inference outside slot 0; this is a known limitation.

4.5 Verifiable Inference Service Simulation (E06, TinyLlama)

Twelve requests arrive in three windows of [4, 6, 2]; slot length 16, 4 generated tokens per request, 2 samples; in the last window the worker tampers with one answer token before publishing.

One-time registration (deterministic calibration, quantization, weight commitments, publication of a 118 KiB verifier context): 417 s. Proving total 366.9 s, i.e. 30.6 s per request on average; verification total 7.6 s.

4.6 Quantization Fidelity (E07, E08, E09, TinyLlama)

Generation level (E07): 16 prompts not used in calibration, 8 tokens generated greedily per prompt, compared with the f32 model.

Conclusion: fidelity is determined by the observer (the bit width of the residual stream); the choice of calibration corpus matters little; percentile clipping is unusable for LLMs. Two re-runs of the same configuration gave per-token / full-sentence matches of 53% / 31% and 49% / 25%; the difference comes from GPU inference non-determinism (Section 4.8).

Standard corpus (E09): WikiText-2 test split, 655 non-overlapping windows of 512 tokens, teacher forcing, 334,705 predicted positions.

The cumulative metrics stabilise from window 30 onwards (agreement 84.7%–85.3%); the two calibrations are indistinguishable over 335 thousand positions. An earlier evaluation on 1,984 positions of the train split (E08: agreement 76.4%–77.3%, pre-fix generic observer 13.7%) is superseded by E09 and kept only for comparison.

4.7 Full Proof vs. Sampled Proof (E10, TinyLlama, context 8)

Same trace and same weight commitments, proven with increasing sample sizes; times include sealing all 47 boundaries.

At s = 5, proving time is 22% of the full proof, proof size 6.8%, and verification time 9.6%. The chunked full proof (tier T3) itself serves as the escalation baseline.

4.8 Determinism (E11, TinyLlama)

Ten independent processes each performed calibration and registration and then repeated inference on the same input twice in-process.

The model version (weight fingerprint + scale fingerprint) reproduces deterministically; the non-determinism lies in GPU inference execution itself, pointing to floating-point reduction order, and is to be eliminated in the next phase.

4.9 Security Limits of Sampling (E12 and detection probability)

Detection probability (computed). With M non-anchor chunks, b forged chunks and s sampled chunks, the single-audit detection probability is P = 1 − C(M−b, s) / C(M, s); for b = 1, P = s / M.

For a forgery confined to unsampled chunks, single-audit detection equals coverage and cannot be amplified by the proof system itself; repeated audits, escalation to a full proof, and stake-and-penalty mechanisms must carry it.

Grinding attack and beacon (E12). The attacker forges the boundary shared by two adjacent non-anchor chunks (b = 2) and, under the beacon-less pure Fiat–Shamir schedule, re-seals repeatedly until the sample avoids the forged chunks; 200 repetitions per s.

Without a beacon, the cost of an attack is one boundary re-commitment per attempt (milliseconds), not a model re-execution; with a beacon outside the attacker's control introduced after sealing, the escape fraction returns to the theoretical q.

5. Limitations and Notes

• Except for E09 and E12, all figures are single measured runs; no variance across repeated runs is given.
• The 70B inference stage is single-threaded integer execution (13.5 hours); peak memory of the proving stage was not recorded separately; the CPU model of that host was not recorded.
• TinyLlama proving and verification ran on the host CPU of the GPU server; the GPU handled only inference and calibration. The "GPU box" figures should be read accordingly.
• All 70B and TinyLlama experiments use context lengths of 8–16 (E09 is a 512-token teacher-forced evaluation without proving); proving cost at production context lengths has not been measured.
• Packed inference is not token-for-token equivalent to standalone inference outside slot 0 (Section 4.4).
• The single-transcript full-proof baseline in E10 did not complete because of a prover stack overflow; the chunked full proof is used as the comparison.
• The proven object is a 12-bit quantized canonical model, not the floating-point serving model; the difference is measured by the metrics of Section 4.6.
• The detection-probability table is analytical; the security of sampling is economic, not cryptographic.

6. Data and Reproduction

Raw logs and data tables are in the accompanying repository:

• raw/mac-m3pro/, raw/gpu-rtx4090/, raw/cpu-2tb-70b/: raw run logs (terminal colour codes removed and local paths redacted; otherwise verbatim).
• data/E01_* … data/E12_*: CSV tables extracted from the logs; column descriptions in README.md.
• tools/build_dataset.py: script that rebuilds every data table from the archived logs.

All models and datasets are public releases: GPT-2 (Q8_0 GGUF), TinyLlama/TinyLlama-1.1B-Chat-v1.0, Llama-2-70B, WikiText-2 raw v1 (test split for evaluation, train split for calibration). The experiment code will be open-sourced in a later phase.

Appendix: Terminology

• Chunk: a contiguous subgraph of the model's computation graph; the unit of proving and commitment.
• Sealing: committing to the activations at every chunk boundary and writing them, together with the claimed input and output, into the transcript.
• Anchor: a chunk touching the model's input or output; always proven.
• Sample size s: the number of non-anchor chunks drawn and proven per audit.
• Tiers T0–T3: four audit strengths — hash chain, tolerance check, sampled proof, full proof.
• Beacon: a public random value outside the prover's control, introduced after sealing and before sampling.
• Canonical model: the quantized model version fixed at registration and identified by its weight fingerprint and scale fingerprint.