MoE Model Verification

Worker-Verifier Method

Worker Evidence

The worker performs normal decode generation and records the following evidence:

input_ids
output_ids
worker_selected_logprob[d]
worker_selected_rank[d]
worker_topk_logprobs[d]

Here, d is the output token depth. The worker generation stage can use greedy decoding or sampling. In the main experiments, we use:

temperature = 0.0
top_p = 1.0
top_k = 0
max_new_tokens = 128

Verifier Replay

The verifier does not regenerate text. Instead, it fixes the path submitted by the worker:

full_prompt_ids = input_ids + output_ids

It then uses vLLM prompt logprobs to perform full-prefill replay. For output token output_ids[d], the verifier alignment position is:

prompt_pos = len(input_ids) + d

Therefore, the comparison is between the model distributions produced by the worker and verifier on the exact same token path, rather than a text similarity comparison between two freely generated outputs.

Logprob Metrics

This paper mainly uses the following metrics:

It is important to emphasize that these experiments compare selected/top-k logprobs exposed by vLLM, not full-vocabulary raw logits.

MoE Logprob Experiments

Qwen3.6-35B-A3B Setup

The MoE experiments use Qwen/Qwen3.6-35B-A3B and include two main datasets:

1. A 300-sample experiment for detailed analysis of BF16 baseline spikes, FP8 right shift, and window-level decision metrics.
2. A 5000-sample moe_1 experiment to confirm BF16/FP8 worker-verifier directionality at larger scale.

300-Sample MoE Results

From mean, p95, p99, rank_delta_rate, and union_js_p99, the FP8 verifier does show a statistical right shift relative to the BF16 verifier. However, the MoE same-model BF16 baseline already has an extreme heavy tail:

BF16 p50 approx. 5e-4
BF16 p99 approx. 0.36 - 0.43
BF16 p999 approx. 10 - 12
BF16 max approx. 24 - 26

In other words, most tokens have very small differences, but a small number of tokens can produce extremely large logprob differences. This makes it difficult for a MoE logprob verifier to form a clean hard decision boundary using single-token thresholds or simple percentiles.

MoE Spike Pattern

The four BF16 baselines show:

These spikes are not purely random noise. Using (sample_id, depth) as the location, among spike locations with abs_diff > 1:

This suggests that spikes are tied to specific tokens or contexts, likely related to MoE routing boundaries, decode/prefill computation path differences, or kernel scheduling. Regardless of the root cause, they destroy the clean decision boundary required by a logprob verifier.

Window-Level Statistics Help, But Do Not Fully Solve the Problem

Using window medians can reduce the BF16 tail:

When using the pooled BF16 window p99 as the threshold:

This shows that window-level statistics can reveal the systematic FP8 shift. However, this remains a statistical separation rather than a clean and stable model identity boundary like the one observed for dense models. MoE logprobs provide useful evidence, but should not be used as the sole final identity verifier.

5000-Sample MoE Results

The large-scale moe_1 experiment contains 5000 prompts with the following input buckets:

32, 128, 512, 2048, 8192

Each bucket has 1000 prompts. Both worker and verifier run on 6000ws, with vLLM version 0.27.1 and logprobs/top-k set to 64.

These results reinforce the earlier conclusion: MoE logprobs can reveal an overall distributional difference between BF16 and FP8, but the same-model BF16 baseline already has large p99/p999/max values. As a result, FP8 and BF16 do not separate as cleanly as they do for dense models.

Routed Experts Experiments

MoE also has experiments comparing different GPUs, but the core evidence comes from routed_experts rather than the 5000-sample logprob overall results above. The route reports cover BF16/FP8 same-model and cross-model comparisons across 6000ws and H100, and are used to verify whether expert routing fingerprints remain distinguishable across GPUs.

Method

In addition to the output token distribution, a MoE model selects routed experts for each token at each MoE layer. When vLLM is enabled with:

--enable-return-routed-experts

it can return the expert routing result for each token/layer. The experiment compares:

decode-time routed_experts
vs
prefill-time routed_experts

and computes:

Routed Experts Results

The all-layer route results show that MoE routed_experts already covers same-model and cross-model comparisons across 6000ws/H100:

• BF16 same-model: entry mismatch is about 0.189-0.192
• FP8 same-model: entry mismatch is about 0.266-0.273
• BF16/FP8 cross-model: entry mismatch is about 0.292-0.294

Therefore, routed experts can distinguish BF16 same-model from BF16/FP8 cross-model cases. FP8 same-model and cross-model are closer under all-layer metrics, but can be further separated using early-layer metrics.

Early-Layer Route Fingerprint

When observing only the first few layers, same-model and cross-model cases separate more clearly:

At layer 0, FP8 same-model has an entry mismatch of 0.0492, while BF16/FP8 cross-model has 0.1134; token-layer set match also drops from 0.9504 to 0.8871. This shows that the early-layer routed_experts route can serve as a stronger structural fingerprint.

More importantly, layer 0 is not merely a reduced-layer approximation that happens to work; in the current data, it is better suited for MoE identity verification than all-layer aggregation. In all-layer metrics, FP8 same-model entry mismatch is about 0.266-0.273, while BF16/FP8 cross-model is about 0.292-0.294, a gap of only about 0.02. At layer 0, however, FP8 same-model is 0.0492 and BF16/FP8 cross-model is 0.1134, increasing the absolute gap to 0.0642, with a cross/same ratio of about 2.30x. The BF16 verifier layer-0 cross-GPU check shows the same trend: BF16 positives have single-sample means around 0.024-0.029, while FP8 negatives are around 0.115-0.117, producing a much clearer separation.

The cross-GPU random-5 threshold test further supports this conclusion. Using only layer 0 and aggregating entry mismatch over 5 randomly sampled prompts each time, the BF16 verifier with threshold 0.060 achieves a combined TPR of 1.000000 and a negative false pass rate of 0.000000. The FP8 verifier with threshold 0.080 achieves a TPR of 0.999453, while the negative false pass rate remains 0.000000. This shows that the layer-0 route fingerprint is not an artifact of a single report, but remains stable under 6000ws/H100 cross-GPU comparisons.

Why Layer 0 Outperforms All Layers

At first glance, using all MoE layers might appear to contain more information. However, the experiments show that all-layer aggregation mixes identity signal with path noise. There are three main reasons.

First, the routing input to later MoE layers has already been affected by earlier layer outputs. Even for the same model, decode-time versus prefill-time execution, small hardware/kernel numerical differences, batch composition, and similar factors can move some tokens across a router decision boundary in later layers. These later-layer mismatches raise the same-model baseline and increase the all-layer entry mismatch.

Second, averaging over all layers dilutes the most discriminative early-layer differences. Layer 0 acts directly on the embedding and original context representation, before multiple rounds of expert selection have propagated errors. It is therefore closer to an initial routing signature: same-model layer-0 routes are more stable, while BF16/FP8 or different-model router boundary differences are already visible.

Third, cross-layer token-level exact match is an overly strict metric. As the number of layers grows, a boundary flip in any single layer can reduce token_match_rate, quickly pushing all-layer token_match into a low-value region and reducing discriminative power. In contrast, layer-0 entry mismatch and token-layer set match preserve a larger dynamic range, making thresholds easier to set.

Therefore, for MoE routed_experts verifiers, we recommend using layer 0 as the primary criterion or a strong feature. All-layer route diff is better suited as an auxiliary diagnostic for understanding the source of routing drift, rather than as a replacement for the layer-0 identity fingerprint.

Why Routed Experts Are Better Suited for MoE

MoE output logprobs are affected by expert routing, batch composition, and decode/prefill path differences. Even when the model weights are identical, if some tokens lie near a routing boundary, the selected logprob can produce extreme spikes. Logprob is the final projection of the output distribution and cannot directly explain whether these spikes come from model differences, path differences, or routing-sensitive points.

routed_experts directly exposes a structural intermediate state of the MoE model:

input/context -> router -> selected experts

If two models, two quantization settings, or two execution paths differ in expert selection, this difference appears before the final logprob. Thus, routed_experts is a more natural verifier for MoE identity: it verifies the key internal decision of the MoE architecture, rather than only the resulting output probabilities.

Limitations

1. The logprob experiments use selected/top-k logprobs exposed by vLLM, not full-vocabulary raw logits.
2. The output length limit is mainly 128 tokens; long-output settings require separate calibration.
3. The main Dense 32B experiment uses 6000ws worker evidence and does not cover every worker/verifier direction.
4. MoE logprobs already fail to form a clean and robust decision boundary in the same-GPU setting, so cross-GPU logprob verification is not a key missing experiment required for the paper's conclusion. MoE cross-GPU stability should mainly be verified through routed_experts, especially the layer-0 route fingerprint.
5. routed_experts experiments already cover 6000ws/H100 cross-GPU comparisons. For production deployment, the vLLM version, prefix cache, batching policy, and routed_experts return format should still be fixed or calibrated.
6. This paper studies statistical identity verification, not a cryptographic proof.