🧩 Model Card: r2t2:1.7b
- Type: Speech-to-Text (ASR: Automatic Speech Recognition)
- Think: No
- Tool Calling Support: No
- Base Model: netease-youdao/Confucius4-R2T2 (fine-tuned from Qwen/Qwen3-ASR-1.7B)
- Parameters: 2B (0.3B audio encoder + 1.7B text decoder)
- Quantization: decoder Q8_0 for the prompt (CPU), mixed Q4_1 / Q8_0 for the NPU decode (Q8_0 for v_proj and down_proj), audio encoder BF16
- Languages: 30 (best at Chinese and English)
- Max / Default Context Length: 2k tokens (12.5 audio tokens per second), enough for one piece of about a minute and its text. Requests have no length limit: audio longer than a minute is transcribed in pieces of about a minute, cut at quiet moments (CLI
/input, chat completions with audio andv1/audio/transcriptionsalike). Streaming requests have no length limit either (see Streaming mode).
Confucius4-R2T2 (“Real Real-Time Transcription”) is NetEase Youdao’s speech recognition model, trained for true streaming: it can transcribe while the speaker is still talking and never edits text it has already written. FastFlowLM transcribes whole audio files, either in one go or in the model’s streaming mode (text appears piece by piece, append-only). Live microphone streaming is not exposed yet.
▶️ Run with FastFlowLM in PowerShell:
flm run r2t2:1.7b
Then, type (replace audio.wav with your audio file path; wav, flac, mp3 and ogg are supported, any sample rate or channel count):
/input "path\to\audio.wav"
Text after the file name is used as context / hot words for the model (names, terms, topic):
/input "path\to\meeting.wav" Acme, Zhang Wei, quarterly roadmap
Server Mode
flm serve r2t2:1.7b
curl http://localhost:52625/v1/audio/transcriptions \
-F file=@audio.wav -F model=r2t2:1.7b
The OpenAI-compatible v1/audio/transcriptions endpoint accepts response_format = json, text, verbose_json, srt or vtt, language (one of the model’s 30 languages by name, e.g. English or Chinese, or its ISO 639-1 code, e.g. en or zh; empty = automatic detection; anything else is answered 400 with the supported list) and prompt (context / hot words). Requesting r2t2:1.7b loads it as the server’s model (unless it runs as the --asr model, see below); any other model name is served by the Whisper sidecar (--asr 1) as before. Audio that cannot be decoded or does not fit in the context window is answered with HTTP 400.
The
srt/vtt/verbose_jsonformats carry one segment for the whole file; R2T2 does not produce word timestamps.
Streaming mode
Add stream=true to receive the text while the file is being processed, as server-sent events (OpenAI’s transcription events):
curl -N http://localhost:52625/v1/audio/transcriptions \
-F file=@audio.wav -F model=r2t2:1.7b -F stream=true -F chunk_size_ms=1000
data: {"type":"transcript.text.delta","delta":"之前有顾客"}
data: {"type":"transcript.text.delta","delta":"自己带酒水,"}
...
data: {"type":"transcript.text.done","text":"之前有顾客自己带酒水,也没加收钱或者不让喝。"}
data: [DONE]
The audio is fed to the model in chunks (chunk_size_ms, 80 ms to 2000 ms, default 1000). After every chunk the model is asked again, with all the audio so far, to continue the text; everything except its last token is final and sent at once (unfixed_token_num, default 1, sets how many trailing tokens stay open). The text you receive never changes. The result equals the reference streaming implementation of the model.
Speed: each step only computes what is new: the encoder output of the finished parts of the recording is kept, the next step’s audio is encoded while the NPU decodes the current one, and one streaming context covers 30 to 45 s (it ends at the first quiet chunk end after 30 s; a final step finishes its text and the next chunk starts a fresh context, so long recordings do not slow down). Measured on this hardware (time to process / length of the audio, below 1 keeps up with live speech): 160 ms chunks (the model paper’s) 1.05, 200 ms about 0.9, 240 ms 0.75, 320 ms 0.6, 480 ms 0.45; a 205 s recording at 480 ms chunks 0.56, at 2 s chunks 0.24. For the lowest latency that keeps up, use 200 ms chunks or more.
As the ASR model (-a)
R2T2 can also be the server’s speech-to-text model, the way --asr whisper-v3:turbo is, so that a chat model and R2T2 are both loaded and a request for either one never reloads the other:
flm serve -a r2t2:1.7b # speech-to-text only
flm serve qwen3:0.6b -a r2t2:1.7b # a chat model and R2T2 side by side
With -a r2t2:1.7b every v1/audio/transcriptions request is answered by R2T2, whatever model it names. The -a option of flm run accepts Whisper only; use the CLI of r2t2:1.7b itself (above) for R2T2 on the command line.
A minimal client with the OpenAI Python SDK (pip install openai), against flm serve -a r2t2:1.7b:
from openai import OpenAI
client = OpenAI(base_url="http://127.0.0.1:52625/v1", api_key="flm") # the key is not checked
# Whole file at once
with open("audio.wav", "rb") as f:
resp = client.audio.transcriptions.create(model="r2t2:1.7b", file=f)
print(resp.text)
# Text as it is recognised (server-sent events); the text you receive never changes
with open("audio.wav", "rb") as f:
stream = client.audio.transcriptions.create(model="r2t2:1.7b", file=f, stream=True)
for event in stream:
if event.type == "transcript.text.delta":
print(event.delta, end="", flush=True)
print()
With the sample clip of the smoke test below this prints the sentence twice; the streamed text can differ from the whole-file text in punctuation (here it has none).
Voice to text to answer, with a chat model next to R2T2 (flm serve qwen3:0.6b -a r2t2:1.7b):
from openai import OpenAI
client = OpenAI(base_url="http://127.0.0.1:52625/v1", api_key="flm")
# Voice -> text (r2t2) -> answer (the chat model the server was started with)
with open("question.wav", "rb") as f:
question = client.audio.transcriptions.create(model="r2t2:1.7b", file=f).text
print("You said:", question)
reply = client.chat.completions.create(
model="qwen3:0.6b",
messages=[{"role": "user", "content": question}],
)
print(reply.choices[0].message.content)
Both scripts were run as shown against a server started with the commands above.
Smoke test
The sample clip of the model’s repository (test.wav, 6.7 s of Mandarin) must give:
之前有顾客自己带酒水,也没加收钱或者不让喝。
Notes
- Every request is its own transcription: nothing is kept between requests.
- The model reports a language (shown as
languageinverbose_json); for some Mandarin clips it reports none, which is normal. - Long recordings: a normal request (and the CLI, and a chat completion with audio) splits audio longer than a minute at quiet moments (within 5 s of each minute) and transcribes the pieces one by one, the way the reference
qwen_asrpackage splits long audio. One-pass decoding of several minutes is avoided: at long contexts the NPU decode can end a transcript early. A streaming request uses contexts of 30 to 45 s for the same reason, with no limit on the total length. -
Memory (Linux, resident memory of the
flmprocess):after load after the first request peak, 5.5 min recording flm serve r2t2:1.7borflm serve -a r2t2:1.7b2.4 GB 4.2 GB 4.7 GB offline, 5.0 GB streaming flm serve qwen3:0.6b -a r2t2:1.7b6.8 GB 8.6 GB 8.7 GB (2 min recording) Of R2T2’s 4.2 GB, 1.6 GB are the weights the NPU decode reads (pinned in system memory), 0.7 GB is working memory, and 1.8 GB is the model file mapped from disk for the CPU’s prompt processing, which the system can reclaim under memory pressure (the audio encoder and the prompt processing run on the CPU, the decoder on the NPU). With a chat model next to R2T2 the figures add up; the chat model’s own share is that of
flm serve <model>alone.
License
The model is released by NetEase Youdao under the NetEase Youdao Model Use License Agreement (code: Apache 2.0). The license text ships with the model files (MODEL_LICENSE) and must stay with any copy. Users whose organization, or an affiliate, has more than 100 million monthly active users or more than RMB 1 billion in annual revenue need a separate license from NetEase Youdao.
Any modifications made to the original model in this Derivative Work are not endorsed, warranted, or guaranteed by the original right-holder of the original model, and the original right-holder disclaims all liability related to this Derivative Work.
The FastFlowLM package is a converted copy (quantized decoder, re-ordered encoder weights) of the original weights. 4-bit quantization of every decoder matrix breaks this model; the NPU decode keeps v_proj and down_proj at 8 bit, which leaves its transcripts as accurate as the original’s (see the benchmarks).