Inference
Understanding FP8 KV Cache in vLLM
Understand what KV cache quantization changes and build a compatibility and quality check before enabling it.
Documentation-based guide FIELD NOTES FOR AI ENGINEERS
LLM Infrastructure,
Explained Clearly.
Practical guides, benchmarks, and troubleshooting for LLM inference, GPUs, vLLM, SGLang and production AI infrastructure.
Reproducible by design. Transparent about the evidence.
● ● ● inference-lab / research notes
THE INFERENCE STACK
01 MODELQwen · DeepSeek · Llama
↓
02 ENGINEvLLM · SGLang · llama.cpp
↓
03 HARDWAREGPU · CUDA · Interconnect
› Understand. Deploy. Measure._
Inference
Engines, serving & optimization
02 ↗GPU & Hardware
Memory, drivers & compatibility
03 ↗Troubleshooting
From error log to root cause
04 ↗Benchmarks
Measurements with context
THE LAB NOTEBOOK
Engineering, beyond the demo.
Deployment guides and technical notes with their evidence status clearly marked.
INFERENCE · EDITOR'S PICK
vLLM vs SGLang:
Choosing an Inference Engine
Start with your workload, not a leaderboard. A practical framework for evaluating compatibility, latency and operational complexity.
Read the guide ↗vLLMvsSGLangONE WORKLOAD. A FAIR COMPARISON.
FRESH FROM THE NOTEBOOK
Latest articles
Inference
Prefix Cache vs KV Cache in LLM Inference
Distinguish the attention state stored during generation from reuse of that state across requests.
Documentation-based guideInference
Qwen3.8-27B on 4×RTX 4090: Deployment Guide
A deployment validation plan for Qwen3.8-27B, covering hardware inventory, version pinning and a reproducible acceptance test.
Documentation-based guideInference
SGLang MTP / NEXTN Explained
Separate speculative decoding concepts from model- and release-specific launch options.
Documentation-based guideTroubleshooting
NVIDIA Xid 79: GPU Has Fallen Off the Bus
Preserve evidence, isolate the affected GPU and investigate PCIe accessibility without assuming a single root cause.
Documentation-based guideTROUBLESHOOTING
When the GPU disappears,
start with the evidence.
A diagnostic approach to NVIDIA Xid 79, with read-only commands and a clear recovery checklist.
Investigate Xid 79 ↗BENCHMARK NOTEBOOK
No numbers without
the experiment.
Our first results are awaiting real-world testing. See the measurement standards behind every future report.
Explore benchmark methodology ↗