🧩 Model Card: NVIDIA-Nemotron-3.5-Bolt-E4B with the DFlash draft

• Type: Any-to-Text, with speculative decoding
• What it is: the same model as nemotron3.5-bolt:e4b, decoded faster: NVIDIA’s DFlash draft proposes up to 7 tokens at a time and the NPU checks all of them in one pass
• Speed: 1.3x to 1.9x the decode speed of nemotron3.5-bolt:e4b, from short prompts to 16k tokens of context (benchmarks)
• Think / Conversational mode / Tool Calling / Vision / Audio: as nemotron3.5-bolt:e4b
• Packages: two downloads. The main weights are the nemotron3.5-bolt:e4b package (Nemotron-3.5-Bolt-E4B-NPU2, about 9 GB); the draft is Nemotron-3.5-Bolt-E4B-DFlash-NPU2 (0.27 GB). flm pull nemotron3.5-bolt-dflash:e4b downloads whichever is missing
• Draft: 5 layers, 0.36B parameters, reads features of 5 layers of the main model; Q4_1 with its feature projection in BF16
• Max Context Length: 64k tokens
• Default Context Length: 32k tokens (change default)
• License: OpenMDW-1.1
• FastFlowLM: 1.0.12 or later

▶️ Run with FastFlowLM in PowerShell:

flm run nemotron3.5-bolt-dflash:e4b

Server mode:

flm serve nemotron3.5-bolt-dflash:e4b

🧪 Smoke test (one clear answer, long enough for many speculative steps):

flm run nemotron3.5-bolt-dflash:e4b
>>> List the integers from 1 to 40 separated by commas, then write the word done.

Expected: 1, 2, 3, ..., 40 followed by done, at a visibly higher speed than with nemotron3.5-bolt:e4b.

📝 Note:

• Every step drafts 7 tokens and checks them together; the model keeps the longest prefix it agrees with plus one token of its own. Sampling works as usual: each checked position is sampled with your settings (temperature, top_k, top_p, penalties) and compared with the draft, so the output follows the main model’s distribution.
• The gain depends on how predictable the text is: code, lists and structured answers gain the most (up to about 3x), open-ended prose the least. Thinking on ("think": true) gains more than thinking off.
• Greedy output can differ from nemotron3.5-bolt:e4b at near-ties: the NPU checks 8 positions in one pass, which rounds differently from decoding one token at a time. 97% of checked positions pick exactly the same token as token-by-token decoding (99.5% when the two best tokens are not within a hair of each other).
• Speculation works on both NPU runtimes (XRT and HRX) with the same speed and output.
• FLM_DFLASH=0 in the environment turns speculation off: the tag then decodes exactly like nemotron3.5-bolt:e4b.
• Speculation pauses while a tool-call or JSON grammar constrains the output, and whenever it would not be faster (for example at very long contexts); it resumes by itself.
• flm remove nemotron3.5-bolt-dflash:e4b removes only the draft package; nemotron3.5-bolt:e4b stays installed.

🧩 Model Card: NVIDIA-Nemotron-3.5-Bolt-E4B — DFlash Flash

• Type: Any-to-Text (image, audio), single-turn, with speculative decoding
• What it is: nemotron3.5-bolt-dflash:e4b built for the Flash serving style: the fast short-prompt prefill of gemma4e-flash:e4b together with DFlash speculative decoding. Every request starts from a clean state; the system prompt is pinned and reused (how flash models work in server mode)
• Time to first token: the same as gemma4e-flash:e4b, about 0.3 s for a short prompt, against 1.2 s for nemotron3.5-bolt-dflash:e4b
• Decode speed: 33 to 40 tokens/s on code and structured answers, 19 to 21 tokens/s on open prose, against 13 for gemma4e-flash:e4b
• Think / Conversational mode / Vision / Audio: as nemotron3.5-bolt:e4b. Tool calling is not supported (the tools field is dropped). Images default to the smallest visual token budget (70) and only the first 30 seconds of an audio clip are used
• Packages: the same two downloads as nemotron3.5-bolt-dflash:e4b (Nemotron-3.5-Bolt-E4B-NPU2, about 9 GB, and the 0.27 GB draft Nemotron-3.5-Bolt-E4B-DFlash-NPU2). Nothing new to download if you already have that tag
• Max / Default Context Length: 1k tokens (fixed, like every flash model)
• License: OpenMDW-1.1
• FastFlowLM: 1.0.13 or later

▶️ Run with FastFlowLM in PowerShell:

flm run nemotron3.5-bolt-dflash-flash:e4b

Server mode:

flm serve nemotron3.5-bolt-dflash-flash:e4b

📊 Measured (greedy, 18-token pinned system prompt, NPU, quiet machine):

📝 Note:

• Flash prefill is tuned for short prompts: it is the faster choice up to a prompt of roughly 700 tokens. For longer prompts use nemotron3.5-bolt-dflash:e4b, which also has a longer context.
• A long, fixed system prompt (instructions, examples) is where this tag gains most: it is prefilled once and the draft model sees it on every request, which keeps the speculative steps long. Changing the system prompt between requests discards the pin.
• Sampling, FLM_DFLASH=0 and the other behaviours of speculation are as on the nemotron3.5-bolt-dflash:e4b card. Greedy output can differ from nemotron3.5-bolt:e4b at near-ties.
• Accuracy (50-question samples, XRT and HRX give the same results): GSM8K 92%, the same as nemotron3.5-bolt-dflash:e4b and gemma4e-flash:e4b. Document reading (DocVQA, ANLS) is 45, against 76 for nemotron3.5-bolt-dflash:e4b: flash models look at an image with the smallest visual token budget (70 instead of 280), which is enough for captions and simple scenes but not for small text. For documents and screenshots use nemotron3.5-bolt-dflash:e4b.
• Needs 9 of the NPU’s 16 hardware contexts (vision and audio included), so other models can still be loaded next to it.
• flm remove nemotron3.5-bolt-dflash-flash:e4b removes only the draft package, shared with nemotron3.5-bolt-dflash:e4b: remove both tags’ use of it first if you want the space back.