MLX Audio 0.6.2 expands model options for speech generation, recognition, and spoken conversation. Developers on Apple Silicon can load the new models through the same library and optionally accelerate transcription.
1. Three new families span synthesis and multilingual recognition
Ming 3.0 is a model suite for speech recognition, synthesis, and conversation. Its recognition model supports English and Chinese. Its conversational model combines audio understanding and speech generation, loading the latter only for speech output.
IBM Granite Speech 5.0 2B offers speech recognition and translation. It transcribes English, French, German, Spanish, Portuguese, and Japanese, and also translates English audio into Italian and Chinese. It supports punctuation, language hints, and up to 2,000 keywords.
Fermion Research's Phonon 2.5 automatically detects and recognizes 25 European languages. Its English-only variant and the earlier Phonon 2 use the same API, with no separate Fermion runtime required.
2. Granite accelerates transcription by reviewing drafts
Granite's speculative decoding has a language model review text drafts from the speech processor in batches and accept matching prefixes. It checks multiple tokens together rather than generating them one at a time. It is off by default. It requires temperature 0, no repetition penalty, and auxiliary files from the checkpoint that stores model state.
The documentation example enables speculative decoding for transcription in the original language and sets the draft length to 16 tokens.
mlx_audio.stt.generate \
--model ibm-granite/granite-speech-5.0-2b \
--audio audio.wav --output-path transcript \
--gen-kwargs '{"language": null, "speculative": true, "num_draft_tokens": 16}'These representative inputs come from local measurements in newly added project documentation. Unlike the example, drafts use 32 tokens. Each call includes audio feature extraction through text conversion. Inputs are public English speech test data, concatenated recordings of that data, and IBM multilingual samples.
Table: Granite Speech 5.0 2B transcription times in BF16, a 16-bit floating-point format
Conditions: Apple M5 Max, 128 GB unified memory, MLX 0.32.4, BF16. Median of 5 calls after warm-up; default and decoding GPU streams synchronized. Speculation: 32-token drafts, default transcription prompt. Audio I/O and weight loading excluded.
Units: time in seconds, speedup as a multiplier
| Audio input | Standard time | Speculative time | Speedup |
|---|---|---|---|
| 5.9 seconds | 0.241 | 0.090 | 2.67 |
| 24.9 seconds, English and French | 0.830 | 0.230 | 3.61 |
| 109.8 seconds, concatenated speech | 3.168 | 2.879 | 1.10 |
The documented speedup varied by input: 3.61 times for 24.9 seconds of audio and 1.10 times for 109.8 seconds.
3. Gains also vary by input at 8-bit precision
The same document also measured an 8-bit checkpoint. The results below compare standard and speculative runs using the same inputs and procedure.
Table: Granite Speech 5.0 2B 8-bit transcription times
Conditions: Apple M5 Max, 128 GB unified memory, MLX 0.32.4, 8-bit. Median of 5 calls after warm-up; default and decoding GPU streams synchronized. Speculation: 32-token drafts, default transcription prompt. Audio I/O and weight loading excluded.
Units: time in seconds, speedup as a multiplier
| Audio input | Standard time | Speculative time | Speedup |
|---|---|---|---|
| 5.9 seconds | 0.215 | 0.097 | 2.23 |
| 24.9 seconds, English and French | 0.680 | 0.244 | 2.78 |
| 109.8 seconds, concatenated speech | 2.512 | 2.518 | 1.00 |
All 8-bit transcripts matched the standard method. BF16 changed the spelling of one proper noun in the 62.5-second input. Word errors against the reference transcript were 33/522 (6.32%) for both methods and both precisions. The document used these figures, including overlapping inputs, to check behavior before and after the change.
Benefits and costs depend on the task. Batching verification saves time when transcription drafts match well. However, the document reports that speculative decoding took about 14~15% longer to translate the same 24.9-second sample into French. For translation, drafts are written in the source language and may not match the target text.
4. Ming outputs chunks, while Phonon reuses its cache
The separate Ming-3.0-tts-0.5B synthesis family now returns audio for sentence chunks as they finish when streaming is enabled. Previously, it waited for the entire paragraph to finish.
On first load, Phonon 2.5 downloads an archive, verifies its checksum to check for corruption, and extracts the MLX checkpoint into a cache. After the initial download and extraction, it reuses that temporary storage for reusable data.
5. Default installation now includes packages needed for conversational models
This release requires transformers[chat-template] and sets the minimum jinja2 version to 3.1.6. Including packages that format conversations as model input in the default installation fixes missing dependencies for Ming voice conversations.
MLX Audio 0.6.2's practical value lies in expanding speech model choices while making transcription acceleration optional for each task.