A stronger local baseline. Still room to improve.
Useful improvement, uneven quality
Crane reduced Swahili WER from Whisper’s 38.0% to 24.3%, and Luganda from 108.9% to 61.7%. Median clips completed in about 0.69 seconds. Those improvements made it a stronger regional baseline on this sample.
Luganda’s remaining error rate was too high to describe as reliable. Word and character error tell different stories, especially with spelling and word-boundary differences. Neither should be replaced by an unsupported overall accuracy claim.
What came next
Crane moved forward for integration. The subsequent Sunbird comparison tested whether a language-focused model could improve transcription further, using the same clip set. This study evaluates ASR only; it does not validate a Crane text-to-speech model.
Results, in context.
| Model | LG WER / CER | SW WER / CER | LG median (s) | SW median (s) |
|---|---|---|---|---|
| Crane | 61.7% / 13.9% | 24.3% / 10.4% | 0.69 | 0.68 |
| Whisper | 108.9% / 30.7% | 38.0% / 13.1% | 1.57 | 1.17 |
The original Crane experiment. Later reruns have slightly different timings and are kept separate.
How we ran it.
- Cape Town T4, Crane FP32 (637,997,088 parameters), compared with Whisper large-v3 CUDA INT8_FLOAT16, beam five and VAD. Twelve matched FLEURS recordings each for Luganda and Swahili, about 190 seconds per language.
- Crane used a pinned revision. Swahili was explicit for Whisper; Luganda used autodetection because the original Whisper vocabulary has no Luganda language token. Text normalisation used NFKC, lowercase, punctuation and spaces.
The limits matter.
- Twelve read-speech clips per language; possible training overlap. Speaker independence, phone audio, code-switching and real background noise were not established.
- Different architectures and precisions; timing includes their practical runtimes. No general African-language ranking is implied.
Evidence & sources.
The public extract contains the methods, selected results and caveats in this note. It is a curated summary, not the full raw run. Fingerprints identify the retained source artifacts; gated audio and private operational records are not included.
Download results extractSource artifact fingerprints · SHA-256
67604230afec67073cd087e471f19ea5243c6971b888171fa70bb25953e062ca215892fec3dcc85fd7242f8a0fe1019337daa7359139b139a83657e37cb9c09fSuggested citation: Embiro Labs Research. “Testing Crane on Luganda and Swahili.” Published 12 Oct 2026. embiro.ai/research/crane-luganda-swahili-asr
Report a correction
