TRUEOPEN
Trueopen.AI

An Open GPU Network

for Verifiable and Private AI Inference

← Back to chatTRUEOPEN / DOCUMENTS
← Back to Home

TrueOpen Papers & Experiments

Research overviews with source citations and experiment limits: English · 中文 · 한국어.

CitationThese experiments, figures and data files are published by the TrueOpen.ai team. When citing or redistributing them, in whole or in part, state that the source is the TrueOpen.ai team and link to www.trueopen.ai. The same requirement is repeated in the download bundle's README, in CITATION.txt, and in the header of every CSV file.

Qwen2.5-7B: Verification Compute Cost

This input-length experiment measured the compute-time ratio between three full teacher-forced prefill passes and one generation run. The ratio increased with input length while output length stayed fixed.

Measured Results

Input tokens3 × full-prefill time / generation time
501.9%
2002.1%
1,0003.3%
4,0008.2%
8,00014.8%

Test Configuration

  • Model: Qwen2.5-7B base, FP16.
  • Hardware: one RTX PRO 6000 Blackwell GPU, 96 GB.
  • Runtime: vLLM 0.20.1.
  • Batch size: 1.
  • Output length: fixed at 1,000 tokens.

What This Shows

Under these conditions, the compute time for three verification replays was substantially lower than the time required to generate the output. It demonstrates a possible compute-cost advantage of prefill-based verification; it does not measure model-substitution detection accuracy.

Measurement Scope

The ratio is 3 × tf_full / gen. It excludes networking, gas, queuing, evidence handling, and settlement. It is a batch-size-1 computation benchmark, not an end-to-end three-node service benchmark or evidence of deployed multi-task batch verification.

The preserved summary does not specify repeat counts or absolute timings. Complete logs are not available in the current website files.

Source: Historical evidence record EXP-02; “Step 2 Supplement”, §1, input-length sweep. This page reproduces the preserved experiment summary.

Qwen3-8B and Qwen3-32B: Worker vs. Verifier Latency Benchmark

Test Setup
Two runs with the same script and settings: Qwen3-8B on one RTX 4090, and Qwen3-32B on one H100 NVL. Both in bfloat16, tensor_parallel_size 1, vLLM 0.29.0 offline LLM with the V1 engine, batch size 1, top-k logprobs 20. Six input lengths (128 to 8,192) crossed with eight output lengths (1 to 8,192) give 48 combinations per run, each with 2 warmup passes and 5 repeats, reported as medians.
Method
Within each run both paths share one vLLM instance, so the model and GPU state are identical. The worker generates normally — one prefill over the input, then serial decode of the output, with the output length pinned exactly. The verifier runs a single prefill over the worker’s input and output token ids to recover per-position top-k logprobs.
Key Results
Worker time tracks output length at roughly 17.5 ms per token on the 8B and 22.7 ms on the 32B, and barely moves with input length; verifier time tracks total tokens at a few thousand tokens per second plus a fixed overhead. For 1,024-token outputs the worker/verifier ratio runs from 103.78× down to 12.19× on the 8B, and from 104.79× down to 11.88× on the 32B, as input grows from 128 to 8,192. The highest ratios measured are 113.56× (8B) and 117.05× (32B). Despite being about four times larger, the 32B is only about 30% slower per decode token, because the H100 NVL’s higher memory bandwidth absorbs most of the extra weight traffic.
Scope
This is single-request latency at batch size 1; a production server that batches generation and verification will see different absolute numbers, though the parallel-prefill against serial-decode gap remains. The two runs use different GPUs, so per-model absolute latencies are not directly comparable — what the report compares is the ratio within each run. The ratio drops below 1× for single-token outputs, where per-position logprob work outweighs the decode it saves. The TTFT and decode split is not available from either run, and a few individual cells are noisy (notably the 32B 128/2048 cell). The ratio counts one verification pass, so it is not directly comparable with the Qwen2.5-7B experiment above, which reports three replays.

Dense Models: Logprob Verification on Different GPU Models and Devices

Test Setup
Qwen3-32B was tested with 300 prompts across six input-length ranges; Qwen3-8B used 200 prompts across four ranges. Tests compared BF16, FP8, AWQ, and smaller-model substitution on configurations including 6000ws / H100 NVL and RTX 4090 / L4.
Method
The Verifier does not regenerate the answer. It runs prefill on the original input and generated tokens, then compares their Logprobs, rankings, and Top-K distributions.
Key Results
In the 32B results, the P99 absolute Logprob deviation was 0.1140–0.1167 for same-model BF16, 0.2997–0.4021 for FP8, and 3.9261 / 5.8105 for substitution with 14B / 8B models. These values describe probability deviations, not task-level cheating detection rates.
Scope
Under these test configurations, quantization and model substitution caused larger deviations than running the same model across GPUs. Longer outputs, other hardware, and different runtime parameters require separate calibration.

MoE Models: Verification with Logprobs and Expert Routing

Test Setup
BF16 / FP8 comparisons used Qwen3.6-35B-A3B. Logprob experiments included 300-sample and 5,000-sample tests. Each expert-routing group used 1,000 prompts, with same-GPU and cross-GPU comparisons on 6000ws and H100 hardware.
Method
The experiments compare Logprobs and routed_experts—the selected expert IDs at each layer—between generation and prefill recomputation. They examine both routing drift across all layers and differences at layer 0.
Key Results
Even the same MoE model can show large Logprob deviations. In one layer-0 comparison, expert IDs differed by 4.92% for the same FP8 model and by 11.34% between BF16 and FP8. These are differences in routing entries, not order error rates.
Scope
Layer 0 provided a clearer model-identity signal in these tests. Checking this layer alone does not prove that later layers or final tokens were computed with the target model.

Sampled Layerwise Proofs: Cryptographic Verification Proof of Concept

Test Setup
GPT-2 (124M) end to end on a laptop CPU, TinyLlama-1.1B with RTX 4090 inference and host-CPU proving, and Llama-2-70B on a 256-thread CPU server with 2 TB of memory. 12-bit activation quantization, KZG-family commitments over BN254, and 2L+3 chunks per model (27, 47 and 163 respectively).
Method
Every chunk boundary is committed, and all commitments together with the claimed input and output are written into the transcript before the chunks to be proven are drawn. The chunks touching the input and output are always proven. The verifier can escalate to a full proof over the same commitments.
Key Results
On Llama-2-70B, sealing 163 boundaries and proving 5 of them took 1,259 s; the proof is 4.34 MiB and verification took 46.3 s without access to the model weights. Packing 12 concurrent requests into one proof took 181.9 s in the proving stage. All seven adversarial cases in the test suite were rejected.
Scope
For a forgery confined to unsampled chunks, single-audit detection equals sampling coverage — 1.9% at 70B with 3 sampled chunks — so repeated audits and economic mechanisms must carry the rest. Without a beacon, pure Fiat–Shamir sampling can be ground at 12.5 ms per attempt. Except for two experiments, every figure is a single measured run.

RELATED WORK

Community Papers

Research on model identity, computation proofs, and open inference infrastructure, organized by topic.

These papers are by external authors. They are not TrueOpen publications, and their inclusion does not imply endorsement. Each entry explains the technical connection and its scope.

Model and Inference Auditing

2025 · arXiv:2504.04715 · v2

Are You Getting What You Pay For? Auditing Model Substitution in LLM APIs

Will Cai, Tianneng Shi, Xuandong Zhao, Dawn Song

This paper examines API providers substituting smaller or quantized models for advertised models. It compares auditing methods based on output text, log probabilities, and trusted execution environments.

Relevance to TrueOpen

The study helps identify signals of model or precision substitution. It also shows how nondeterministic inference can affect auditing; it does not establish that Logprob checks are reliable under every condition.

Read the Paper ↗

2026 · arXiv:2602.22700

IMMACULATE: A Practical LLM Auditing Framework via Verifiable Computation

Yanpei Guo et al.; coauthors include Dawn Song and Jiaheng Zhang

The framework uses verifiable computation to audit a subset of requests for model substitution, unauthorized quantization, and token overbilling, amortizing the cost of audits.

Relevance to TrueOpen

Its API-auditing methods, per-request checks, and cost analysis provide useful comparisons. Its guarantees and costs depend on its own assumptions and are not TrueOpen deployment parameters or measurements.

Read the Paper ↗

2025 · arXiv:2501.16007 · v2

TOPLOC: A Locality Sensitive Hashing Scheme for Trustless Verifiable Inference

Jack Min Ong et al.

TOPLOC uses locality-sensitive hashes of intermediate activations to create compact fingerprints. It studies how to detect changes in models, inputs, or precision and how to verify across hardware.

Relevance to TrueOpen

It offers a comparison between activation fingerprints and output-probability checks. Matching fingerprints do not by themselves prove the token-generation process. Citing TOPLOC does not mean TrueOpen has adopted it.

Read the Paper ↗

Computation Proofs

2017 · NIST Publication Record

Gaussian Variant of Freivalds' Algorithm for Efficient and Reliable Matrix Product Verification

Michael V. Mascagni, Hao Ji, Yaohang Li

The paper uses Gaussian random-vector projections to check matrix products and examines detection under exact arithmetic and floating-point rounding errors.

Relevance to TrueOpen

It provides mathematical background for FP16/BF16 matrix checks. Numerical fault analysis does not by itself establish security against adversarial inference; matrix commitments and a specific error model are also needed.

Read the Paper ↗

2019 · ICLR 2019 · arXiv:1806.03287

Slalom: Fast, Verifiable and Private Execution of Neural Networks in Trusted Hardware

Florian Tramèr, Dan Boneh

Slalom splits neural-network computation between a trusted environment and an untrusted accelerator. The trusted component verifies outsourced linear-layer computations.

Relevance to TrueOpen

It provides a reference for checking linear computation and dividing work between operators. Slalom relies on a trusted execution environment (TEE); its assumptions do not directly establish security for TrueOpen without a TEE.

Read the Paper ↗

2024 · ACM CCS 2024 · arXiv:2404.16109

zkLLM: Zero Knowledge Proofs for Large Language Models

Haochen Sun, Jason Li, Hongyang Zhang

zkLLM constructs zero-knowledge proofs for LLMs. It introduces tlookup for non-arithmetic tensor operations and zkAttn for attention, with CUDA-parallelized computation.

Relevance to TrueOpen

It provides research background for sumcheck / GKR, lookup arguments, and attention proofs in sampled layerwise cryptographic verification. It is not direct evidence of TrueOpen's implementation, supported models, or performance.

Read the Paper ↗

Performance and Data Availability

2023 · arXiv:2309.06180

Efficient Memory Management for Large Language Model Serving with PagedAttention

Woosuk Kwon et al.

This paper studies KV-cache memory management for LLM serving. Paged memory allocation reduces waste and enables more efficient request processing.

Relevance to TrueOpen

It provides background on GPU serving efficiency, memory use, and inference runtimes. Runtime batching is distinct from protocol-level batch verification and does not directly measure TrueOpen's verification costs.

Read the Paper ↗

2022 · arXiv:2105.11827 · v4

Narwhal and Tusk: A DAG-based Mempool and Efficient BFT Consensus

George Danezis, Eleftherios Kokoris Kogias, Alberto Sonnino, Alexander Spiegelman

Narwhal and Tusk separate reliable data dissemination from transaction ordering, using a DAG mempool and BFT consensus to address throughput, availability, and network failures.

Relevance to TrueOpen

The architecture offers a comparison for separating task-material distribution from consensus ordering. It does not imply that TrueOpen uses this consensus or establish TrueOpen's throughput.

Read the Paper ↗

Interpreting the Results

Low false-positive rates or high detection rates in one experiment do not automatically apply to other models and hardware. No observed errors in a sample does not mean the true error rate is zero. Results should be read alongside profile thresholds, test configurations, and verification-parameter versions.

AN OPEN NETWORK FOR VERIFIABLE & PRIVATE AI INFERENCEBuilt in the open ↗
TRUEOPEN / RESOURCES