{
  "formatVersion": 1,
  "scope": "Curated public research extract; not the full raw run.",
  "canonical": "https://embiro.ai/research/crane-luganda-swahili-asr",
  "fieldNote": 10,
  "slug": "crane-luganda-swahili-asr",
  "title": "Testing Crane on Luganda and Swahili.",
  "description": "A matched recording comparison shows a meaningful improvement over Whisper, and why Luganda still needs care.",
  "category": "Speech",
  "art": "speech-wave",
  "thumbnail": {
    "src": "/research/thumbnails/crane-luganda-swahili-asr.webp",
    "alt": "Two speech pathways merging into an open transcription book."
  },
  "date": "2026-10-11",
  "published": "2026-10-12",
  "updated": "2026-10-12",
  "author": "Embiro Labs Research",
  "status": "Exploratory benchmark",
  "headline": "A stronger local baseline. Still room to improve.",
  "metric": "24.3%",
  "metricLabel": "Crane Swahili WER · 12 clips",
  "method": [
    "Cape Town T4, Crane FP32 (637,997,088 parameters), compared with Whisper large-v3 CUDA INT8_FLOAT16, beam five and VAD. Twelve matched FLEURS recordings each for Luganda and Swahili, about 190 seconds per language.",
    "Crane used a pinned revision. Swahili was explicit for Whisper; Luganda used autodetection because the original Whisper vocabulary has no Luganda language token. Text normalisation used NFKC, lowercase, punctuation and spaces."
  ],
  "sections": [
    {
      "title": "Useful improvement, uneven quality",
      "paragraphs": [
        "Crane reduced Swahili WER from Whisper’s 38.0% to 24.3%, and Luganda from 108.9% to 61.7%. Median clips completed in about 0.69 seconds. Those improvements made it a stronger regional baseline on this sample.",
        "Luganda’s remaining error rate was too high to describe as reliable. Word and character error tell different stories, especially with spelling and word-boundary differences. Neither should be replaced by an unsupported overall accuracy claim."
      ]
    },
    {
      "title": "What came next",
      "paragraphs": [
        "Crane moved forward for integration. The subsequent Sunbird comparison tested whether a language-focused model could improve transcription further, using the same clip set. This study evaluates ASR only; it does not validate a Crane text-to-speech model."
      ]
    }
  ],
  "limitations": [
    "Twelve read-speech clips per language; possible training overlap. Speaker independence, phone audio, code-switching and real background noise were not established.",
    "Different architectures and precisions; timing includes their practical runtimes. No general African-language ranking is implied."
  ],
  "tables": [
    {
      "title": "Matched FLEURS results",
      "columns": [
        "Model",
        "LG WER / CER",
        "SW WER / CER",
        "LG median (s)",
        "SW median (s)"
      ],
      "rows": [
        [
          "Crane",
          "61.7% / 13.9%",
          "24.3% / 10.4%",
          "0.69",
          "0.68"
        ],
        [
          "Whisper",
          "108.9% / 30.7%",
          "38.0% / 13.1%",
          "1.57",
          "1.17"
        ]
      ],
      "note": "The original Crane experiment. Later reruns have slightly different timings and are kept separate."
    }
  ],
  "references": [
    {
      "label": "Google FLEURS · dataset and attribution",
      "url": "https://huggingface.co/datasets/google/fleurs"
    }
  ],
  "audio": [],
  "evidence": [
    {
      "file": "README.md",
      "sha256": "67604230afec67073cd087e471f19ea5243c6971b888171fa70bb25953e062ca"
    },
    {
      "file": "summary.json",
      "sha256": "215892fec3dcc85fd7242f8a0fe1019337daa7359139b139a83657e37cb9c09f"
    }
  ]
}