⚡ Performance and Efficiency Benchmarks
This section reports the performance of Qwen 3.8 on NPU with FastFlowLM (FLM).
Note:
- Results are based on FastFlowLM v1.0.7.
- Under FLM’s default NPU power mode (Performance)
- Newer versions may deliver improved performance.
- Fine-tuned models show performance comparable to their base models.
qwen3.8-mtp:27bdecodes with its built-in MTP draft head, so decoding speed is prompt-dependent — predictable text (code, structured output, long reasoning chains) accepts more drafts and decodes faster than highly creative text. See the model card for how speculation works.- Benchmarks for this model were run up to 8k context. The model itself takes up to 16k (16,377 tokens).
Test System 1:
AMD Ryzen™ AI 5 340 (Kraken Point) with 64 GB DRAM; performance is comparable to other Kraken Point systems.
🚀 Decoding Speed (TPS, or Tokens per Second, starting @ different context lengths)
| Model | HW | 1k | 2k | 4k | 8k |
|---|---|---|---|---|---|
| Qwen3.8-27B | XDNA2 | 2.04 | 2.23 | 2.70 | 2.31 |
🚀 Prefill Speed (TPS, or Tokens per Second, with different prompt lengths)
| Model | HW | 1k | 2k | 4k | 8k |
|---|---|---|---|---|---|
| Qwen3.8-27B | XDNA2 | 104.94 | 110.56 | 113.05 | 112.81 |
🚀 Time to First Token (TTFT, Seconds, with different prompt lengths)
| Model | HW | 1k | 2k | 4k | 8k |
|---|---|---|---|---|---|
| Qwen3.8-27B | XDNA2 | 9.35 | 17.62 | 34.34 | 68.71 |
🚀 Prefill TTFT with Image Input (Seconds)
Prefill time-to-first-token (TTFT) for Qwen3.8-27B on NPU (FastFlowLM) with different image resolutions.
Mid Resolution Images:
| Model | HW | 720p (1280×720) | 1080p (1920×1080) |
|---|---|---|---|
| Qwen3.8-27B | XDNA2 | 15.4 | 34.7 |
This test uses a short prompt: “Describe this image.”
🚀 DFlash 2 draft: decode speed against the MTP head (TPS, greedy, 160 new tokens, short prompts)
qwen3.8-dflash2:27b is the same model decoded with z-lab’s DFlash 2 draft instead of the built-in MTP head. Both rows of a prompt ran the same engine build and kernels on the same machine, one request at a time, thinking off.
| Prompt | qwen3.8-mtp:27b |
qwen3.8-dflash2:27b |
Speedup |
|---|---|---|---|
| Hash map explanation + Python code | 2.95 | 3.09 | 1.05x |
| TCP vs UDP | 3.39 | 3.48 | 1.03x |
| Short story (creative) | 1.71 | 1.96 | 1.14x |
| Train catch-up word problem | 4.12 | 4.91 | 1.19x |
| French Revolution bullets | 2.42 | 2.89 | 1.19x |
| SQL query + explanation | 4.10 | 4.42 | 1.08x |
| C linked-list reversal | 3.57 | 5.25 | 1.47x |
| Grade-school math word problem | 3.40 | 4.24 | 1.25x |
| Translation to French and German (60 tokens) | 2.65 | 2.30 | 0.87x |
| JSON object (three planets) | 3.98 | 4.87 | 1.22x |
| Mean | 3.23 | 3.74 | 1.16x |
On a short code prompt the draft keeps about 6.6 of every 7 drafted tokens (the model then emits 7.6 tokens per step); on open prose it keeps about 3. One step costs about 1.2 s with either draft: 0.9 s to check the 8 positions and about 0.27 s to draft. Gains come from the draft predicting more of what the model will say, so they follow the text, as with the MTP head.
Decode speed after a long prompt (TPS, greedy, 96 new tokens, 8k context)
One run per cell: a run of 96 tokens is only about a dozen speculative steps, so differences of a few tenths of a token per second between the two tags are within the run-to-run scatter. The draft attends to the last 2048 tokens only, so its cost stops growing once the context passes that.
| Prompt length | Answer | qwen3.8-mtp:27b |
qwen3.8-dflash2:27b |
|---|---|---|---|
| ~1k | code | 5.02 | 4.41 |
| ~1k | prose | 1.70 | 1.90 |
| ~2k | code | 4.95 | 4.52 |
| ~2k | prose | 1.61 | 1.44 |
| ~4k | code | 4.79 | 4.91 |
| ~4k | prose | 2.07 | 2.08 |
| ~7k | code | 4.09 | 3.98 |
| ~7k | prose | 1.56 | 1.63 |
With a long context qwen3.8-dflash2:27b is about level with qwen3.8-mtp:27b: its draft costs about 0.32 s per step at a full window (host attention over 2048 rows), against about 0.27 s on a short prompt, which cancels most of its higher acceptance. Time to first token is the same for both tags.