SGLang MTP / NEXTN Explained
Separate speculative decoding concepts from model- and release-specific launch options.
On this page
Problem
A launch command mentioning MTP or NEXTN is easy to copy but may refer to a different model architecture, backend or release.
Conclusion
Speculative decoding uses candidate generation followed by target-model verification. MTP support and algorithm labels are implementation-specific. Check the exact model and SGLang release documentation; this lab has not validated a universal NEXTN command.
Environment
Record target checkpoint revision, any draft checkpoint, SGLang commit, GPU backend, precision and all speculative settings. Keep a non-speculative baseline with otherwise matching settings.
Symptoms
A configuration may be rejected, use an unsupported path, or add overhead that outweighs accepted draft tokens. No measured acceptance rate or speedup is available here.
Cause
Drafting and verification both cost time. The balance depends on the model, request distribution, concurrency and how many proposed tokens are accepted. The name of an algorithm is not a guarantee of acceleration.
Solution
First confirm ordinary decoding works. Read the speculative-decoding section for the pinned release and the selected model. Enable only the documented compatible path. Change one speculative setting at a time and keep an experiment log.
{
"status": "not_run",
"baseline": { "speculation": false },
"candidate": { "algorithm": null },
"target_revision": null,
"draft_revision": null,
"acceptance_metrics": null
}
Compare end-to-end latency and throughput as well as acceptance behavior. Include drafting overhead and failed requests. Keep correctness and sampling conditions aligned with your application.
Verification
This article provides a verification procedure; no hardware measurements are claimed.
Publish a recommendation only with the complete launch commands, raw measurements and stated compatibility scope.
Caveats
MTP and a separate draft model are not interchangeable configuration choices. NEXTN naming in one backend or release should not be generalized to all SGLang deployments.
Separate the proposal from the result
Think of one decoding cycle as candidate generation, target verification and state update. More candidate tokens can mean more useful work per cycle, but also more work discarded when proposals are rejected. Acceptance alone is therefore an incomplete performance measure.
An illustrative comparison makes the tradeoff visible: if ordinary decoding produces one token in 10 ms, its rate is 100 tokens/s. If a speculative cycle produces three usable tokens in 24 ms, the corresponding rate is 125 tokens/s. If the same cycle takes 36 ms, it is about 83 tokens/s. These invented values demonstrate arithmetic only; they are not SGLang measurements or a prediction for your hardware.
A small experiment matrix
| Case | What remains fixed | What it answers |
|---|---|---|
| Ordinary decoding | Model, prompts, precision and sampling | What is the baseline? |
| Documented speculative path | Same workload and quality criteria | Does the added work help? |
| Higher concurrency | Same model and algorithm settings | Does the benefit survive contention? |
| Different prompt group | Same serving configuration | Is the gain workload-dependent? |
Use only algorithm names and arguments documented for the pinned release. Save any draft-model revision separately from the target revision. Compare final output quality as well as usable tokens per second and request latency. A change in sampling behavior or output length can invalidate a speed comparison even when every request returns successfully.
Keep ordinary decoding as the fallback until both compatibility and acceptance tests pass. The engine comparison guide explains how to define those gates.
References
Revision note
September 14, 2026: added a worked decision framework and explicit verification limits. Original publication date retained.