🧩 Model Card: d3-lite

• Type: Decision model (typed choices with a probability for every option; no text generation)
• Think: No
• Tool Calling Support: No
• Base Model: vllm-sr/d3-lite v3.1.3 (Qwen3.5-0.8B backbone, bidirectional full attention, 255-way code readout)
• Quantization: Q4_K
• Max input: 32,768 tokens per question (longer prompts get max_length_exceeded; nothing is truncated). One question takes about 0.4 s at 1,024 tokens, 1.6 s at 4,096 and 23 s at 32,768; the NPU serves one request at a time
• Inputs: text and images (video: not supported)
• FastFlowLM: 1.0.11 or later

🧩 Model Card: d3-nano

• Type: Decision model (typed choices with a probability for every option; no text generation)
• Think: No
• Tool Calling Support: No
• Base Model: vllm-sr/d3-nano v3.1.3 (Qwen3.5-2B backbone, bidirectional full attention, 255-way code readout)
• Quantization: Q4_K
• Max input: 32,768 tokens per question, as for d3-lite. One question takes about 0.6 s at 1,024 tokens, 2.5 s at 4,096 and 28 s at 32,768; the NPU serves one request at a time. About 2.0 GB of weights (0.66 GB of them the vision tower); the engine peaks at about 5.3 GB of memory at 32,768 tokens
• Inputs: text and images (video: not supported)

🧩 Model Card: d3-mini

• Type: Decision model (typed choices with a probability for every option; no text generation)
• Think: No
• Tool Calling Support: No
• Base Model: vllm-sr/d3-mini v3.1.3 (Qwen3.5-4B backbone, bidirectional full attention, 255-way code readout)
• Quantization: Q4_K
• Max input: 32,768 tokens per question, as for d3-lite. One question takes about 1.3 s at 1,024 tokens, 5.3 s at 4,096 and 66 s at 32,768; the NPU serves one request at a time. About 3.4 GB of weights (0.67 GB of them the vision tower); the engine peaks at about 8.6 GB of memory at 32,768 tokens
• Inputs: text and images (video: not supported)

d3 answers System One requests: a state (any JSON value) and one or more questions, each a choice (up to 255 options), a noul (no / yes) or a score (2–10 ordered levels). Every option gets a probability. It is the decision model vLLM Semantic Router calls for routing, guardrails and tool selection, and FastFlowLM serves the same API.

▶️ Run with FastFlowLM:

A decision model is served next to (or without) a chat model, in Server Mode only.

Server Mode

flm serve --decision d3:lite                 # decision model only
flm serve qwen3.5:4b --decision d3:lite      # next to a chat model
flm serve --decision d3:nano                 # the larger model (2B backbone)
flm serve --decision d3:mini                 # the largest model (4B backbone)

d3:nano and d3:mini have the same API and limits as d3:lite; d3:nano is about 1.7× slower per short question and d3:mini about 3.4× slower (see the benchmark page).

Send requests to POST /v1/systemone (alias /v1/decisions).

Latency next to a chat model. The NPU runs one request at a time, so a System One request that arrives while the server is generating a chat completion waits for that completion to finish. Measured with qwen3.5:0.8b and d3:lite in one server: a decision takes 146 ms alone and 12.7 s when it arrives during a 600-token completion. A second flm serve for the decision model does not avoid this: the two processes contend for the NPU and both slow down. Where decisions gate chat (routing in front of a chat model), keep chat completions short or run the decision model on a machine of its own.

Example: curl

curl -s http://127.0.0.1:52625/v1/systemone -H "Content-Type: application/json" -d '{
  "state": "The order arrived damaged yesterday. The customer has a receipt and asks for a replacement today.",
  "questions": {
    "route":   {"type": "choice", "instructions": "Which team should handle this?",
                "criteria": {"returns": "Refunds, replacements and damaged deliveries",
                             "billing": "Payments, invoices and charges",
                             "general": "Anything else"}},
    "urgent":  {"type": "noul", "instructions": "Does this need a reply today?"},
    "anger":   {"type": "score", "instructions": "How upset is the customer?", "criteria": ["calm", "annoyed", "angry"]}
  }
}'

The reply on a Ryzen AI MAX+ 395 (numbers rounded to 3 places here; the server sends full precision):

{"model":"d3-lite","answers":{
  "route":{"type":"choice","choice":"returns","probabilities":{"returns":0.996,"billing":0.001,"general":0.003},"confidence":0.977},
  "urgent":{"type":"noul","noul":0.766},
  "anger":{"type":"score","score":1.057,"probabilities":{"0":0.209,"1":0.525,"2":0.266},"confidence":0.073,"legend":{"0":"calm","1":"annoyed","2":"angry"}}},
 "usage":{"input_tokens":342,"output_tokens":0}}

Smoke test

Five commands check a new install end to end, text and images:

flm pull d3:lite                         # downloads D3-Lite-NPU2 (about 0.8 GB, vision weights included)
flm serve --decision d3:lite             # wait for "Decision model ready: d3-lite (d3:lite), max input 32768 tokens"

In a second terminal:

curl -s http://127.0.0.1:52625/v1/systemone/models
# {"object":"list","data":[{"id":"d3:lite","object":"model","api":"systemone","max_input_tokens":32768,
#  "modalities":["text","image"],"image_max_pixels":1638400}]}

curl -s http://127.0.0.1:52625/v1/systemone -H "Content-Type: application/json" \
  -d '{"state": "The order arrived damaged.", "questions": {"route": {"type": "choice",
       "criteria": {"returns": "Refunds and replacements", "billing": "Payments", "general": null}}}}'
# {"model":"d3-lite","answers":{"route":{"type":"choice","choice":"returns","probabilities":{"returns":0.98,...

A healthy install answers "choice":"returns" with returns near 0.98 in about 0.1 s.

Then an image (Linux shell; the image is a 639 × 480 PNG from this repository):

curl -sLo panda.png https://raw.githubusercontent.com/ROCm/FastFlowLM/main/src/tb_files/panda.png
printf '{"questions": {"animal": {"type": "choice", "instructions": "Which animal is in the photo?",
  "criteria": {"dog": "A dog", "panda": "A panda", "cat": "A cat"}}},
  "images": ["data:image/png;base64,%s"]}' "$(base64 -w0 panda.png)" > image_request.json
curl -s http://127.0.0.1:52625/v1/systemone -H "Content-Type: application/json" -d @image_request.json
# {"model":"d3-lite","answers":{"animal":{"type":"choice","choice":"panda","probabilities":{"dog":0.0004,"panda":0.999,...
#  "usage":{"input_tokens":404,"output_tokens":0}}

On Windows (PowerShell):

Invoke-WebRequest https://raw.githubusercontent.com/ROCm/FastFlowLM/main/src/tb_files/panda.png -OutFile panda.png
$img = [Convert]::ToBase64String([IO.File]::ReadAllBytes("panda.png"))
$body = '{"questions": {"animal": {"type": "choice", "instructions": "Which animal is in the photo?", "criteria": {"dog": "A dog", "panda": "A panda", "cat": "A cat"}}}, "images": ["data:image/png;base64,' + $img + '"]}'
Invoke-RestMethod http://127.0.0.1:52625/v1/systemone -Method Post -ContentType "application/json" -Body $body | ConvertTo-Json -Depth 5

A healthy install answers "choice":"panda" with panda near 0.999 and 404 input tokens, in about 0.4 s (the first image request also loads the vision tower). flm run d3:lite does not start a chat; it prints how to serve the model.

Request and response

Images. Every question sees the request’s images, in order, before its text:

import base64, requests
image = "data:image/jpeg;base64," + base64.b64encode(open("receipt.jpg", "rb").read()).decode()
reply = requests.post("http://127.0.0.1:52625/v1/systemone", json={
    "state": "A customer sent this photo with a refund request.",
    "questions": {"damaged": {"type": "noul", "instructions": "Does the item in the photo look damaged?"}},
    "images": [image],
}).json()

The first request with images loads the vision tower (two more NPU hardware contexts); text-only use never does.

Errors follow the upstream server exactly: a body that is not JSON, unknown fields or malformed questions get 422 {"detail": ...} ({"detail": {"invalid_questions": {...}}} for questions) and nothing runs; a question whose prompt is over the limit (image tokens included) is answered with {"type", "error": "max_length_exceeded", "message"} inside a 200 reply. Images are checked as upstream does: a bad one is refused with 422 {"detail": "images[i]: ..."}.

Accuracy and latency

Prompts, token ids and answer objects are byte-identical to the upstream runtime (checked on every build against the upstream package). Accuracy against the upstream BF16 model and latency on this NPU are in the d3 benchmark page.