⚡ Performance and Efficiency Benchmarks

This section reports the performance on NPU with FastFlowLM (FLM).

Note:

• Results are based on FastFlowLM v0.9.38.
• Under FLM’s default NPU power mode (Performance)
• Newer versions may deliver improved performance.
• Fine-tuned models show performance comparable to their base models.

Test System 1:

AMD Ryzen™ AI 7 350 (Kraken Point) with 32 GB DRAM; performance is comparable to other Kraken Point systems.

🚀 Decoding Speed (TPS, or Tokens per Second, starting @ different context lengths)

🚀 Prefill Speed (TPS, or Tokens per Second, with different prompt lengths)