TrueOpen Papers & Experiments
Research overviews with source citations and experiment limits: English · 中文 · 한국어.
CitationThese experiments, figures and data files are published by the TrueOpen.ai team. When citing or redistributing them, in whole or in part, state that the source is the TrueOpen.ai team and link to www.trueopen.ai. The same requirement is repeated in the download bundle's README, in CITATION.txt, and in the header of every CSV file.
Qwen2.5-7B: Verification Compute Cost
This input-length experiment measured the compute-time ratio between three full teacher-forced prefill passes and one generation run. The ratio increased with input length while output length stayed fixed.
Measured Results
| Input tokens | 3 × full-prefill time / generation time |
|---|---|
| 50 | 1.9% |
| 200 | 2.1% |
| 1,000 | 3.3% |
| 4,000 | 8.2% |
| 8,000 | 14.8% |
Test Configuration
- Model: Qwen2.5-7B base, FP16.
- Hardware: one RTX PRO 6000 Blackwell GPU, 96 GB.
- Runtime: vLLM 0.20.1.
- Batch size: 1.
- Output length: fixed at 1,000 tokens.
What This Shows
Under these conditions, the compute time for three verification replays was substantially lower than the time required to generate the output. It demonstrates a possible compute-cost advantage of prefill-based verification; it does not measure model-substitution detection accuracy.
Measurement Scope
The ratio is 3 × tf_full / gen. It excludes networking, gas, queuing, evidence handling, and settlement. It is a batch-size-1 computation benchmark, not an end-to-end three-node service benchmark or evidence of deployed multi-task batch verification.
The preserved summary does not specify repeat counts or absolute timings. Complete logs are not available in the current website files.
Source: Historical evidence record EXP-02; “Step 2 Supplement”, §1, input-length sweep. This page reproduces the preserved experiment summary.
Qwen3-8B and Qwen3-32B: Worker vs. Verifier Latency Benchmark
- Test Setup
- Two runs with the same script and settings: Qwen3-8B on one RTX 4090, and Qwen3-32B on one H100 NVL. Both in bfloat16, tensor_parallel_size 1, vLLM 0.29.0 offline
LLMwith the V1 engine, batch size 1, top-k logprobs 20. Six input lengths (128 to 8,192) crossed with eight output lengths (1 to 8,192) give 48 combinations per run, each with 2 warmup passes and 5 repeats, reported as medians. - Method
- Within each run both paths share one vLLM instance, so the model and GPU state are identical. The worker generates normally — one prefill over the input, then serial decode of the output, with the output length pinned exactly. The verifier runs a single prefill over the worker’s input and output token ids to recover per-position top-k logprobs.
- Key Results
- Worker time tracks output length at roughly 17.5 ms per token on the 8B and 22.7 ms on the 32B, and barely moves with input length; verifier time tracks total tokens at a few thousand tokens per second plus a fixed overhead. For 1,024-token outputs the worker/verifier ratio runs from 103.78× down to 12.19× on the 8B, and from 104.79× down to 11.88× on the 32B, as input grows from 128 to 8,192. The highest ratios measured are 113.56× (8B) and 117.05× (32B). Despite being about four times larger, the 32B is only about 30% slower per decode token, because the H100 NVL’s higher memory bandwidth absorbs most of the extra weight traffic.
- Scope
- This is single-request latency at batch size 1; a production server that batches generation and verification will see different absolute numbers, though the parallel-prefill against serial-decode gap remains. The two runs use different GPUs, so per-model absolute latencies are not directly comparable — what the report compares is the ratio within each run. The ratio drops below 1× for single-token outputs, where per-position logprob work outweighs the decode it saves. The TTFT and decode split is not available from either run, and a few individual cells are noisy (notably the 32B 128/2048 cell). The ratio counts one verification pass, so it is not directly comparable with the Qwen2.5-7B experiment above, which reports three replays.
Dense Models: Logprob Verification on Different GPU Models and Devices
- Test Setup
- Qwen3-32B was tested with 300 prompts across six input-length ranges; Qwen3-8B used 200 prompts across four ranges. Tests compared BF16, FP8, AWQ, and smaller-model substitution on configurations including 6000ws / H100 NVL and RTX 4090 / L4.
- Method
- The Verifier does not regenerate the answer. It runs prefill on the original input and generated tokens, then compares their Logprobs, rankings, and Top-K distributions.
- Key Results
- In the 32B results, the P99 absolute Logprob deviation was 0.1140–0.1167 for same-model BF16, 0.2997–0.4021 for FP8, and 3.9261 / 5.8105 for substitution with 14B / 8B models. These values describe probability deviations, not task-level cheating detection rates.
- Scope
- Under these test configurations, quantization and model substitution caused larger deviations than running the same model across GPUs. Longer outputs, other hardware, and different runtime parameters require separate calibration.
MoE Models: Verification with Logprobs and Expert Routing
- Test Setup
- BF16 / FP8 comparisons used Qwen3.6-35B-A3B. Logprob experiments included 300-sample and 5,000-sample tests. Each expert-routing group used 1,000 prompts, with same-GPU and cross-GPU comparisons on 6000ws and H100 hardware.
- Method
- The experiments compare Logprobs and routed_experts—the selected expert IDs at each layer—between generation and prefill recomputation. They examine both routing drift across all layers and differences at layer 0.
- Key Results
- Even the same MoE model can show large Logprob deviations. In one layer-0 comparison, expert IDs differed by 4.92% for the same FP8 model and by 11.34% between BF16 and FP8. These are differences in routing entries, not order error rates.
- Scope
- Layer 0 provided a clearer model-identity signal in these tests. Checking this layer alone does not prove that later layers or final tokens were computed with the target model.
Sampled Layerwise Proofs: Cryptographic Verification Proof of Concept
- Test Setup
- GPT-2 (124M) end to end on a laptop CPU, TinyLlama-1.1B with RTX 4090 inference and host-CPU proving, and Llama-2-70B on a 256-thread CPU server with 2 TB of memory. 12-bit activation quantization, KZG-family commitments over BN254, and 2L+3 chunks per model (27, 47 and 163 respectively).
- Method
- Every chunk boundary is committed, and all commitments together with the claimed input and output are written into the transcript before the chunks to be proven are drawn. The chunks touching the input and output are always proven. The verifier can escalate to a full proof over the same commitments.
- Key Results
- On Llama-2-70B, sealing 163 boundaries and proving 5 of them took 1,259 s; the proof is 4.34 MiB and verification took 46.3 s without access to the model weights. Packing 12 concurrent requests into one proof took 181.9 s in the proving stage. All seven adversarial cases in the test suite were rejected.
- Scope
- For a forgery confined to unsampled chunks, single-audit detection equals sampling coverage — 1.9% at 70B with 3 sampled chunks — so repeated audits and economic mechanisms must carry the rest. Without a beacon, pure Fiat–Shamir sampling can be ground at 12.5 ms per attempt. Except for two experiments, every figure is a single measured run.
RELATED WORK
Community Papers
Research on model identity, computation proofs, and open inference infrastructure, organized by topic.
These papers are by external authors. They are not TrueOpen publications, and their inclusion does not imply endorsement. Each entry explains the technical connection and its scope.
Model and Inference Auditing
Are You Getting What You Pay For? Auditing Model Substitution in LLM APIs
This paper examines API providers substituting smaller or quantized models for advertised models. It compares auditing methods based on output text, log probabilities, and trusted execution environments.
The study helps identify signals of model or precision substitution. It also shows how nondeterministic inference can affect auditing; it does not establish that Logprob checks are reliable under every condition.
IMMACULATE: A Practical LLM Auditing Framework via Verifiable Computation
The framework uses verifiable computation to audit a subset of requests for model substitution, unauthorized quantization, and token overbilling, amortizing the cost of audits.
Its API-auditing methods, per-request checks, and cost analysis provide useful comparisons. Its guarantees and costs depend on its own assumptions and are not TrueOpen deployment parameters or measurements.
TOPLOC: A Locality Sensitive Hashing Scheme for Trustless Verifiable Inference
TOPLOC uses locality-sensitive hashes of intermediate activations to create compact fingerprints. It studies how to detect changes in models, inputs, or precision and how to verify across hardware.
It offers a comparison between activation fingerprints and output-probability checks. Matching fingerprints do not by themselves prove the token-generation process. Citing TOPLOC does not mean TrueOpen has adopted it.
Computation Proofs
Gaussian Variant of Freivalds' Algorithm for Efficient and Reliable Matrix Product Verification
The paper uses Gaussian random-vector projections to check matrix products and examines detection under exact arithmetic and floating-point rounding errors.
It provides mathematical background for FP16/BF16 matrix checks. Numerical fault analysis does not by itself establish security against adversarial inference; matrix commitments and a specific error model are also needed.
Slalom: Fast, Verifiable and Private Execution of Neural Networks in Trusted Hardware
Slalom splits neural-network computation between a trusted environment and an untrusted accelerator. The trusted component verifies outsourced linear-layer computations.
It provides a reference for checking linear computation and dividing work between operators. Slalom relies on a trusted execution environment (TEE); its assumptions do not directly establish security for TrueOpen without a TEE.
zkLLM: Zero Knowledge Proofs for Large Language Models
zkLLM constructs zero-knowledge proofs for LLMs. It introduces tlookup for non-arithmetic tensor operations and zkAttn for attention, with CUDA-parallelized computation.
It provides research background for sumcheck / GKR, lookup arguments, and attention proofs in sampled layerwise cryptographic verification. It is not direct evidence of TrueOpen's implementation, supported models, or performance.
Performance and Data Availability
Efficient Memory Management for Large Language Model Serving with PagedAttention
This paper studies KV-cache memory management for LLM serving. Paged memory allocation reduces waste and enables more efficient request processing.
It provides background on GPU serving efficiency, memory use, and inference runtimes. Runtime batching is distinct from protocol-level batch verification and does not directly measure TrueOpen's verification costs.
Narwhal and Tusk: A DAG-based Mempool and Efficient BFT Consensus
Narwhal and Tusk separate reliable data dissemination from transaction ordering, using a DAG mempool and BFT consensus to address throughput, availability, and network failures.
The architecture offers a comparison for separating task-material distribution from consensus ordering. It does not imply that TrueOpen uses this consensus or establish TrueOpen's throughput.
Interpreting the Results
Low false-positive rates or high detection rates in one experiment do not automatically apply to other models and hardware. No observed errors in a sample does not mean the true error rate is zero. Results should be read alongside profile thresholds, test configurations, and verification-parameter versions.

