🧩 Model Card: gemma-3-1b-it
- Type: Text-to-Text
- Think: No
- Tool Calling Support: No
- Prefix cache (server mode): Yes; Gemma 3 puts the system prompt in the first user turn, so each conversation reuses its own earlier turns (see Prefix Cache (Prompt Reuse))
- Base Model: google/gemma-3-1b-it
- Quantization: Q4_1
- Max Context Length: 32k tokens
- Default Context Length: 32k tokens (change default)
- Set Context Length at Launch
▶️ Run with FastFlowLM in PowerShell:
flm run gemma3:1b
🧩 Model Card: gemma-3-4b-it
- Type: Image-Text-to-Text
- Think: No
- Tool Calling Support: No
- Prefix cache (server mode): Yes; Gemma 3 puts the system prompt in the first user turn, so each conversation reuses its own earlier turns (see Prefix Cache (Prompt Reuse))
- Base Model: google/gemma-3-4b-it
- Quantization: Q4_1
- Max Context Length: 128k tokens
- Default Context Length: 64k tokens (change default)
- Set Context Length at Launch
▶️ Run with FastFlowLM in PowerShell:
flm run gemma3:4b
📝 Note: In CLI mode, attach an image with:
/input "file/to/image.jpg" describe this image.
🧩 Model Card: gemma-4-E2B-it
- Type: Any-to-Text
- Think: Toggleable
- Tool Calling Support: Yes
- Prefix cache (server mode): Yes for text prompts; a prompt with a picture or audio keeps the older single-checkpoint reuse (see Prefix Cache (Prompt Reuse))
- Base Model: google/gemma-4-E2B-it
- Quantization: Q4_1
- Max Context Length: 128k tokens
- Default Context Length: 64k tokens (change default)
- Set Context Length at Launch
▶️ Run with FastFlowLM in PowerShell:
flm run gemma4-it:e2b
🖼️ Note: In CLI mode, attach an image with:
/input "file/to/image.jpg" describe this image.
🗣️ Note: In CLI mode, attach an audio with:
/input "file/to/audio.mp3" summarize this audio.
📝 Note:
-
In server mode, Gemma 4 supports multimodal input with text, images, and audio. See the OpenAI API multimodal example.
-
Change the visual token budget for images with the
image-max-tokensparameter for different tasks. For more details, see the Open WebUI custom parameters example.
🧩 Model Card: gemma-4-E4B-it
- Type: Any-to-Text
- Think: Toggleable
- Tool Calling Support: Yes
- Prefix cache (server mode): Yes for text prompts; a prompt with a picture or audio keeps the older single-checkpoint reuse (see Prefix Cache (Prompt Reuse))
- Base Model: google/gemma-4-E4B-it
- Quantization: Q4_1
- Max Context Length: 128k tokens
- Default Context Length: 64k tokens (change default)
- Set Context Length at Launch
▶️ Run with FastFlowLM in PowerShell:
flm run gemma4-it:e4b
🖼️ Note: In CLI mode, attach an image with:
/input "file/to/image.jpg" describe this image.
🗣️ Note: In CLI mode, attach an audio with:
/input "file/to/audio.mp3" summarize this audio.
📝 Note:
-
In server mode, Gemma 4 supports multimodal input with text, images, and audio. See the OpenAI API multimodal example.
-
Change the visual token budget for images with the
image-max-tokensparameter for different tasks. For more details, see the Open WebUI custom parameters example.
🧩 Model Card: gemma-4-12B-it
- Type: Any-to-Text
- Think: Toggleable
- Tool Calling Support: Yes
- Prefix cache (server mode): Yes for text prompts; a prompt with a picture or audio keeps the older single-checkpoint reuse (see Prefix Cache (Prompt Reuse))
- Base Model: google/gemma-4-12B-it
- Quantization: Q4_0
- Max Context Length: 128k tokens
- Default Context Length: 64k tokens (change default)
- Set Context Length at Launch
▶️ Run with FastFlowLM in PowerShell:
flm run gemma4-it:12b
🖼️ Note: In CLI mode, attach an image with:
/input "file/to/image.jpg" describe this image.
🗣️ Note: In CLI mode, attach an audio with:
/input "file/to/audio.mp3" summarize this audio.
📝 Note:
-
In server mode, Gemma 4 supports multimodal input with text, images, and audio. See the OpenAI API multimodal example.
-
Change the visual token budget for images with the
image-max-tokensparameter for different tasks. For more details, see the Open WebUI custom parameters example.
🧩 Model Card: gemma-4-E2B-it — Flash
- Type: Any-to-Text
- Think: Toggleable
- Tool Calling Support: No
- Base Model: google/gemma-4-E2B-it
- Quantization: Q4_1
- Max Context Length: 1k tokens
- Default Context Length: 1k tokens (fixed, see note below)
▶️ Run with FastFlowLM in PowerShell:
flm run gemma4e-flash:e2b
🖼️ Note: In CLI mode, attach an image with:
/input "file/to/image.jpg" describe this image.
🗣️ Note: In CLI mode, attach an audio with:
/input "file/to/audio.mp3" summarize this audio.
⚡ Note — what “Flash” changes:
- Same checkpoint as
gemma4-it:e2b, served by a different engine. Prefill runs on a single fused NPU overlay instead of swapping overlays layer by layer, which cuts time-to-first-token on short prompts. - Prefill speed: ~390 tokens/s at a 128-token prompt — the short-prompt case Flash is tuned for.
- Decode speed is unchanged. Flash only changes how prefill runs, so generation tokens/s matches
gemma4-it:e2b— see the Gemma 4 benchmarks. - Single-turn. Every request starts from a clean KV state — earlier turns are not carried over. A system prompt is the one exception: it is pinned on first use and reused on later requests, so it is not re-prefilled on every call.
- Context length is fixed at 1k tokens. Context-length overrides are ignored for this model.
- Tool calling is not supported. Any tools passed with the request are dropped.
- The visual token budget defaults to 70.
- Audio is truncated to the first 30 seconds of the clip so it fits the short context.
- Example usage: Flash models in server mode
🧩 Model Card: gemma-4-E4B-it — Flash
- Type: Any-to-Text
- Think: Toggleable
- Tool Calling Support: No
- Base Model: google/gemma-4-E4B-it
- Quantization: Q4_1
- Max Context Length: 1k tokens
- Default Context Length: 1k tokens (fixed, see note below)
▶️ Run with FastFlowLM in PowerShell:
flm run gemma4e-flash:e4b
🖼️ Note: In CLI mode, attach an image with:
/input "file/to/image.jpg" describe this image.
🗣️ Note: In CLI mode, attach an audio with:
/input "file/to/audio.mp3" summarize this audio.
⚡ Note — what “Flash” changes:
- Same checkpoint as
gemma4-it:e4b, served by a different engine. Prefill runs on a single fused NPU overlay instead of swapping overlays layer by layer, which cuts time-to-first-token on short prompts. - Prefill speed: ~256 tokens/s at a 128-token prompt — the short-prompt case Flash is tuned for.
- Decode speed is unchanged. Flash only changes how prefill runs, so generation tokens/s matches
gemma4-it:e4b— see the Gemma 4 benchmarks. - Single-turn. Every request starts from a clean KV state — earlier turns are not carried over. A system prompt is the one exception: it is pinned on first use and reused on later requests, so it is not re-prefilled on every call.
- Context length is fixed at 1k tokens. Context-length overrides are ignored for this model.
- Tool calling is not supported. Any tools passed with the request are dropped.
- The visual token budget defaults to 70.
- Audio is truncated to the first 30 seconds of the clip so it fits the short context.
- Example usage: Flash models in server mode