The guided series
Each recipe depends only on the ones before it. Build a trustworthy instrument first, then use it to make decisions.
Measure first
Serve the model, benchmark it honestly, and set up the decision task every later recipe measures.
7 of 7 available
Getting started with self-hosting Qwen
Serve Qwen3.5-4B behind a local OpenAI-compatible endpoint on one NVIDIA GPU, with pinned software and a clear VRAM budget.
How to benchmark
Build a repeatable serving baseline by controlling prompt length, output length, caching, concurrency, and the tokenizer used to count tokens.
Reading the curves: TTFT, ITL, and throughput
Understand what happens during prefill and decode, then connect those phases to the latency and throughput curves you measure.
Where the GPU memory goes
Calculate the memory needed for model weights and active requests, then use vLLM's logs and metrics to see what fits on an A10.
Turn Qwen into a classifier without training
Give Qwen a decision to make, restrict its answer to your options, and read the model's preference for each one.
Evaluate your classifier with OpenJudge
Test Qwen on JevBench, count the correct decisions, and inspect the mistakes.
Build a dataset for your classifier
Combine public training examples with Qwen-generated documents, then check the answers before training.
Train and ship a classifier
Fine-tune with LoRA, check the probabilities, and serve what you trained.
1 of 3 available
Use Qwen in applications
Thinking modes, tool calls, an AgentScope harness, long context and prefix caching.
5 planned
Measure speculative decoding
Lossless speedups — and the experiment that proves nothing changed.
4 planned
Compress and measure the tradeoff
Quantize your own checkpoint and measure what it costs in quality.
5 planned
Improve decisions, close the loop
Probability training, RL, robustness and a draft trained on your own traffic.
4 planned