⚡ Performance and Efficiency Benchmarks

This section reports the performance on NPU with FastFlowLM (FLM).

Note:

• Results are based on FastFlowLM v1.0.5.
• Under FLM’s default NPU power mode (Performance)
• Newer versions may deliver improved performance.
• Hy-MT2 is a dedicated translation model tuned for short, single-turn requests, so it is benchmarked up to 16k max context length rather than the longer sweeps used for general chat models.

Test System:

AMD Ryzen™ AI 9 370 (Strix Point) with 32 GB DRAM; performance is comparable to other Strix Point and Strix Halo systems.

🚀 Decoding Speed (TPS, or Tokens per Second, starting @ different context lengths)

🚀 Prefill Speed (TPS, or Tokens per Second, with different prompt lengths)