Worker-Verifier Model Identity Verification with Logprobs and Routed Experts: An Experimental Study

Abstract

This paper studies whether token-level logprob traces recorded during worker generation can be used for effective model identity verification under a worker-verifier architecture. The experiments cover two model families: dense Transformer models and Mixture-of-Experts (MoE) models. For dense models, we use Qwen3-32B BF16 and Qwen3-8B BF16 as the main targets, and compare same-model BF16 verifiers, FP8/AWQ quantized verifiers, and smaller-model verifiers using selected/top-k logprob differences. The results show that dense models have a narrow and stable same-model logprob baseline, while FP8, AWQ, and smaller models separate clearly from that baseline. Therefore, logprob fingerprints can effectively distinguish dense model identities.

For MoE models, we use Qwen3.6-35B-A3B and compare BF16 workers against BF16/FP8 verifiers, as well as cross-verification between BF16 and FP8 workers. The experiments show that MoE logprob differences do contain a statistical right shift from BF16 to FP8, but the same-model BF16 baseline itself has sparse, position-sensitive extreme spikes. As a result, token-level logprob alone cannot form a clean and robust hard decision boundary. Further routed experts experiments show that expert routing traces provide a more direct structural fingerprint for MoE models: same-model and cross-model cases differ consistently in entry mismatch, token-layer set match, and early-layer route match metrics, and these differences remain stable across 6000ws/H100 cross-GPU comparisons. A particularly important finding is that the layer-0 route fingerprint is clearer than aggregating all layers, because it has not yet been diluted by later-layer routing error accumulation and path branching. Therefore, dense models can rely primarily on logprob verification, while MoE models should use routed_experts, especially the layer-0 route fingerprint, as the main verification path.

Keywords: worker-verifier, logprob fingerprint, MoE, routed experts, model identity verification, vLLM

1. Introduction

In open inference networks or decentralized compute networks, verifying whether a worker actually ran the claimed model is a core requirement for trustworthy model services. Recomputing full outputs is expensive, while checking only the final text is insufficient to distinguish models from the same family, quantized variants, or smaller models pretending to be larger ones. A natural approach is to require the worker to submit a logprob trace along the generated path, and then let the verifier perform teacher-forced replay on the same input_ids + output_ids path. The verifier then compares per-token logprobs, ranks, and top-k distributions.

This paper focuses on two questions:

1. For dense Transformer models, is the logprob trace sufficient as a model identity fingerprint?
2. For MoE models, is the logprob trace equally effective? If not, can expert routing information provide a better verifier?

Our conclusions are architecture-dependent:

• Dense models: logprob fingerprints are effective. The same-model BF16 baseline is stable, while FP8, AWQ, and smaller models deviate clearly.
• MoE models: logprob fingerprints contain signal, but are not clean enough. The same-model baseline has heavy-tailed spikes, making single-token or simple percentile thresholds prone to error.
• MoE routed_experts: expert routing traces provide a structural fingerprint that can distinguish model or quantization variants more directly than logprobs.

2. Worker-Verifier Method

2.1 Worker Evidence

The worker performs normal decode generation and records the following evidence:

input_ids
output_ids
worker_selected_logprob[d]
worker_selected_rank[d]
worker_topk_logprobs[d]

Here, d is the output token depth. The worker generation stage can use greedy decoding or sampling. In the main experiments, we use:

temperature = 0.0
top_p = 1.0
top_k = 0
max_new_tokens = 128

2.2 Verifier Replay

The verifier does not regenerate text. Instead, it fixes the path submitted by the worker:

full_prompt_ids = input_ids + output_ids

It then uses vLLM prompt logprobs to perform full-prefill replay. For output token output_ids[d], the verifier alignment position is:

prompt_pos = len(input_ids) + d

Therefore, the comparison is between the model distributions produced by the worker and verifier on the exact same token path, rather than a text similarity comparison between two freely generated outputs.

2.3 Logprob Metrics

This paper mainly uses the following metrics:

It is important to emphasize that these experiments compare selected/top-k logprobs exposed by vLLM, not full-vocabulary raw logits.

3. Dense Model Experiments

For dense models, the GPU comparison is mainly in the logprob verifier dimension: Qwen3-32B covers 6000ws and H100 NVL verifiers, while Qwen3-8B covers 4090 and L4 verifiers. Therefore, the dense-model conclusion compares not only model and quantization differences, but also whether the same-model logprob baseline remains stable across GPUs.

3.1 Qwen3-32B Setup

The Qwen3-32B dense experiment uses 300 prompts covering the following input token buckets:

32, 128, 512, 2048, 8192, 16384

Each bucket has 50 prompts. The worker is:

Qwen/Qwen3-32B BF16
GPU: 6000ws
backend: vLLM 0.23.0
batch size: 8
logprobs: 64
output rows: 36882

The verifiers cover:

• Qwen3-32B BF16
• Qwen3-32B-FP8
• Qwen3-32B-AWQ
• Qwen3-14B BF16
• Qwen3-8B BF16
• verifier GPUs: 6000ws and H100 NVL

3.2 Qwen3-32B Results

Using 6000ws worker -> 6000ws Qwen3-32B BF16 verifier as the baseline:

The results show that same-model BF16 forms a narrow valid drift envelope on both 6000ws and H100 NVL. FP8 already deviates significantly, while AWQ and smaller models deviate even more strongly. Therefore, for dense models such as Qwen3-32B, selected/top-k logprob traces form an effective model identity fingerprint.

3.3 Qwen3-8B Results

The Qwen3-8B experiment uses 200 prompts covering the following input buckets:

32, 128, 512, 2048

The worker is Qwen3-8B BF16 on a 4090 GPU. The verifiers cover 4090 and L4, and include Qwen3-8B BF16, Qwen3-8B-FP8, Qwen3-8B-AWQ, and Qwen2/Qwen2.5-7B models.

The Qwen3-8B results further confirm that dense models have a narrow same-model BF16 baseline, small cross-GPU differences, and clear deviations for FP8/AWQ/other models. Based on these results, reasonably clear batch-level pass/reject thresholds can be defined, for example:

BF16 pass:
mean_abs_logprob_diff <= 0.015
p95_abs_logprob_diff <= 0.090
p99_abs_logprob_diff <= 0.200
rank_delta_rate <= 0.012
topk_jaccard_mean >= 0.940
union_js_p99 <= 0.012

Reject:
mean_abs_logprob_diff > 0.020
p95_abs_logprob_diff > 0.120
p99_abs_logprob_diff > 0.250
rank_delta_rate > 0.030
topk_jaccard_mean < 0.920
union_js_p99 > 0.020

4. MoE Logprob Experiments

4.1 Qwen3.6-35B-A3B Setup

The MoE experiments use Qwen/Qwen3.6-35B-A3B and include two main datasets:

1. A 300-sample experiment for detailed analysis of BF16 baseline spikes, FP8 right shift, and window-level decision metrics.
2. A 5000-sample moe_1 experiment to confirm BF16/FP8 worker-verifier directionality at larger scale.

4.2 300-Sample MoE Results

From mean, p95, p99, rank_delta_rate, and union_js_p99, the FP8 verifier does show a statistical right shift relative to the BF16 verifier. However, the MoE same-model BF16 baseline already has an extreme heavy tail:

BF16 p50 approx. 5e-4
BF16 p99 approx. 0.36 - 0.43
BF16 p999 approx. 10 - 12
BF16 max approx. 24 - 26

In other words, most tokens have very small differences, but a small number of tokens can produce extremely large logprob differences. This makes it difficult for a MoE logprob verifier to form a clean hard decision boundary using single-token thresholds or simple percentiles.

4.3 MoE Spike Pattern

The four BF16 baselines show:

These spikes are not purely random noise. Using (sample_id, depth) as the location, among spike locations with abs_diff > 1:

This suggests that spikes are tied to specific tokens or contexts, likely related to MoE routing boundaries, decode/prefill computation path differences, or kernel scheduling. Regardless of the root cause, they destroy the clean decision boundary required by a logprob verifier.

4.4 Window-Level Statistics Help, But Do Not Fully Solve the Problem

Using window medians can reduce the BF16 tail:

When using the pooled BF16 window p99 as the threshold:

This shows that window-level statistics can reveal the systematic FP8 shift. However, this remains a statistical separation rather than a clean and stable model identity boundary like the one observed for dense models. MoE logprobs provide useful evidence, but should not be used as the sole final identity verifier.

4.5 5000-Sample MoE Results

The large-scale moe_1 experiment contains 5000 prompts with the following input buckets:

32, 128, 512, 2048, 8192

Each bucket has 1000 prompts. Both worker and verifier run on 6000ws, with vLLM version 0.27.1 and logprobs/top-k set to 64.

These results reinforce the earlier conclusion: MoE logprobs can reveal an overall distributional difference between BF16 and FP8, but the same-model BF16 baseline already has large p99/p999/max values. As a result, FP8 and BF16 do not separate as cleanly as they do for dense models.

5. Routed Experts Experiments

MoE also has experiments comparing different GPUs, but the core evidence comes from routed_experts rather than the 5000-sample logprob overall results above. The route reports cover BF16/FP8 same-model and cross-model comparisons across 6000ws and H100, and are used to verify whether expert routing fingerprints remain distinguishable across GPUs.

5.1 Method

In addition to the output token distribution, a MoE model selects routed experts for each token at each MoE layer. When vLLM is enabled with:

--enable-return-routed-experts

it can return the expert routing result for each token/layer. The experiment compares:

decode-time routed_experts
vs
prefill-time routed_experts

and computes:

5.2 Routed Experts Results

The all-layer route results show that MoE routed_experts already covers same-model and cross-model comparisons across 6000ws/H100:

• BF16 same-model: entry mismatch is about 0.189-0.192
• FP8 same-model: entry mismatch is about 0.266-0.273
• BF16/FP8 cross-model: entry mismatch is about 0.292-0.294

Therefore, routed experts can distinguish BF16 same-model from BF16/FP8 cross-model cases. FP8 same-model and cross-model are closer under all-layer metrics, but can be further separated using early-layer metrics.

5.3 Early-Layer Route Fingerprint

When observing only the first few layers, same-model and cross-model cases separate more clearly:

At layer 0, FP8 same-model has an entry mismatch of 0.0492, while BF16/FP8 cross-model has 0.1134; token-layer set match also drops from 0.9504 to 0.8871. This shows that the early-layer routed_experts route can serve as a stronger structural fingerprint.

More importantly, layer 0 is not merely a reduced-layer approximation that happens to work; in the current data, it is better suited for MoE identity verification than all-layer aggregation. In all-layer metrics, FP8 same-model entry mismatch is about 0.266-0.273, while BF16/FP8 cross-model is about 0.292-0.294, a gap of only about 0.02. At layer 0, however, FP8 same-model is 0.0492 and BF16/FP8 cross-model is 0.1134, increasing the absolute gap to 0.0642, with a cross/same ratio of about 2.30x. The BF16 verifier layer-0 cross-GPU check shows the same trend: BF16 positives have single-sample means around 0.024-0.029, while FP8 negatives are around 0.115-0.117, producing a much clearer separation.

The cross-GPU random-5 threshold test further supports this conclusion. Using only layer 0 and aggregating entry mismatch over 5 randomly sampled prompts each time, the BF16 verifier with threshold 0.060 achieves a combined TPR of 1.000000 and a negative false pass rate of 0.000000. The FP8 verifier with threshold 0.080 achieves a TPR of 0.999453, while the negative false pass rate remains 0.000000. This shows that the layer-0 route fingerprint is not an artifact of a single report, but remains stable under 6000ws/H100 cross-GPU comparisons.

5.4 Why Layer 0 Outperforms All Layers

At first glance, using all MoE layers might appear to contain more information. However, the experiments show that all-layer aggregation mixes identity signal with path noise. There are three main reasons.

First, the routing input to later MoE layers has already been affected by earlier layer outputs. Even for the same model, decode-time versus prefill-time execution, small hardware/kernel numerical differences, batch composition, and similar factors can move some tokens across a router decision boundary in later layers. These later-layer mismatches raise the same-model baseline and increase the all-layer entry mismatch.

Second, averaging over all layers dilutes the most discriminative early-layer differences. Layer 0 acts directly on the embedding and original context representation, before multiple rounds of expert selection have propagated errors. It is therefore closer to an initial routing signature: same-model layer-0 routes are more stable, while BF16/FP8 or different-model router boundary differences are already visible.

Third, cross-layer token-level exact match is an overly strict metric. As the number of layers grows, a boundary flip in any single layer can reduce token_match_rate, quickly pushing all-layer token_match into a low-value region and reducing discriminative power. In contrast, layer-0 entry mismatch and token-layer set match preserve a larger dynamic range, making thresholds easier to set.

Therefore, for MoE routed_experts verifiers, we recommend using layer 0 as the primary criterion or a strong feature. All-layer route diff is better suited as an auxiliary diagnostic for understanding the source of routing drift, rather than as a replacement for the layer-0 identity fingerprint.

5.5 Why Routed Experts Are Better Suited for MoE

MoE output logprobs are affected by expert routing, batch composition, and decode/prefill path differences. Even when the model weights are identical, if some tokens lie near a routing boundary, the selected logprob can produce extreme spikes. Logprob is the final projection of the output distribution and cannot directly explain whether these spikes come from model differences, path differences, or routing-sensitive points.

routed_experts directly exposes a structural intermediate state of the MoE model:

input/context -> router -> selected experts

If two models, two quantization settings, or two execution paths differ in expert selection, this difference appears before the final logprob. Thus, routed_experts is a more natural verifier for MoE identity: it verifies the key internal decision of the MoE architecture, rather than only the resulting output probabilities.

6. Discussion

6.1 The Fundamental Difference Between Dense and MoE Models

In dense models, every token passes through the same dense layers. Numerical differences mainly come from GPU, kernel, dtype, or quantization effects. Therefore, the same-model BF16 logprob replay baseline is usually narrow, while smaller models, quantization changes, or weight changes cause stable distribution shifts over many tokens.

In MoE models, each token also goes through a router that selects experts. Even when the final output token path is the same, decode-time versus prefill-time routing, batch composition, or numerical path differences can move a small number of tokens into different expert sets, producing local logprob spikes. This makes the MoE logprob baseline naturally wider and more heavy-tailed.

6.2 Logprobs Are Still Useful, But Not the Main MoE Criterion

The MoE logprob experiment is not useless. FP8 shows statistical shifts relative to BF16 in mean, p95, p99, rank_delta_rate, union_js_p99, and bad-window-rate. It can serve as an auxiliary signal, especially for window-level and batch-level risk assessment.

However, if the goal is clean identity verification in production, MoE logprobs have several issues:

1. The same-model baseline itself has an extreme long tail.
2. Large-diff locations recur, indicating that they are not simple random noise.
3. FP8/BF16 differences are partially masked by the same-model spike envelope.
4. Single-token and simple percentile thresholds can easily oscillate between false reject and false accept.

Therefore, MoE should not rely only on a logprob verifier.

6.3 Recommended Verification Strategy

For dense models:

Use selected/top-k logprob replay.
Build a multi-metric envelope using mean/p95/p99/rank_delta/topk_jaccard/union_js.

For MoE models:

Primary criterion: layer-0 routed_experts route fingerprint.
Auxiliary diagnostic: all-layer route drift.
Secondary signal: window-level logprob statistics.

A two-stage workflow can be used:

1. The worker submits input_ids + output_ids + logprob trace + routed_experts trace.
2. The verifier performs prefill replay on the same path and compares:
3. layer-0 routed_experts mismatch / set match / random-5 aggregate score;
4. all-layer routed_experts route set / Jaccard / mismatch rate;
5. window-level drift of selected/top-k logprobs;
6. missing selected tokens and rank drift.

7. Limitations

1. The logprob experiments use selected/top-k logprobs exposed by vLLM, not full-vocabulary raw logits.
2. The output length limit is mainly 128 tokens; long-output settings require separate calibration.
3. The main Dense 32B experiment uses 6000ws worker evidence and does not cover every worker/verifier direction.
4. MoE logprobs already fail to form a clean and robust decision boundary in the same-GPU setting, so cross-GPU logprob verification is not a key missing experiment required for the paper's conclusion. MoE cross-GPU stability should mainly be verified through routed_experts, especially the layer-0 route fingerprint.
5. routed_experts experiments already cover 6000ws/H100 cross-GPU comparisons. For production deployment, the vLLM version, prefix cache, batching policy, and routed_experts return format should still be fixed or calibrated.
6. This paper studies statistical identity verification, not a cryptographic proof.

8. Conclusion

The core conclusions of this paper are:

Dense models can be effectively distinguished using worker-verifier logprob fingerprints;
MoE models cannot be cleanly and robustly distinguished using logprobs alone;
For MoE models, layer-0 routed_experts provides a more effective structural model fingerprint.

Experimentally, the Qwen3-32B and Qwen3-8B dense results show that the same-model BF16 baseline is narrow and stable across GPUs, while FP8, AWQ, and smaller models separate clearly from the baseline. In contrast, the Qwen3.6-35B-A3B MoE results show that the same-model BF16 baseline already has heavy-tailed spikes. FP8 has a statistical right shift, but the separation is not as clean as in dense models. Further routed_experts experiments show observable structural differences between same-model and cross-model routing traces, and this conclusion remains valid under 6000ws/H100 cross-GPU route comparisons. More importantly, layer 0 has a higher signal-to-noise ratio than all-layer aggregation: it preserves early routing differences while avoiding later-layer path branching and accumulated mismatches that raise the same-model baseline.

Therefore, a worker-verifier system should be designed by model architecture: dense models should primarily use logprob fingerprints, while MoE models should primarily use layer-0 routed_experts fingerprints, with all-layer route drift and logprob window statistics as auxiliary signals.

Appendix: Main Data Sources