{
  "formatVersion": 1,
  "scope": "Curated public research extract; not the full raw run.",
  "canonical": "https://embiro.ai/research/african-inference-on-cpu",
  "fieldNote": 15,
  "slug": "african-inference-on-cpu",
  "title": "Does every AI workload need a GPU?",
  "description": "Kokoro, embeddings and three transcription models on Azure CPUs in South Africa, compared with our AWS Cape Town baseline.",
  "category": "Infrastructure",
  "art": "open-intelligence",
  "thumbnail": {
    "src": "/research/thumbnails/african-inference-on-cpu.webp",
    "alt": "An ivory CPU and mint GPU balanced on a sculptural beam."
  },
  "date": "2026-10-12",
  "published": "2026-10-12",
  "updated": "2026-10-12",
  "author": "Embiro Labs Research",
  "status": "Exploratory benchmark",
  "headline": "Start with the workload, not the GPU.",
  "metric": "1.44s",
  "metricLabel": "Kokoro EN median on Azure D4",
  "method": [
    "Azure South Africa North: D4s v5 (4 logical CPUs, 16 GiB) and D16s v5 (16 logical CPUs, 64 GiB), tested sequentially. Both reported Intel Xeon Platinum 8573C. AWS baseline: Cape Town g4dn.xlarge, one NVIDIA T4, four vCPUs, 16 GiB RAM.",
    "Kokoro: nine matched prompts across af_heart, bf_emma and ff_siwis; three Azure repeats and two fresh AWS API repeats, same FP32 weights and speed 1. Retrieval: six documents and six known-answer queries, CPU FP32 versus GPU FP16. ASR: twelve matched FLEURS clips per language, plus separate noise and silence checks.",
    "Azure timings are direct synthesis/model execution. AWS Kokoro and embeddings use the public API, including HTTP and admission; historical AWS Swahili/Luganda ASR uses direct execution and English includes API overhead. These are deployment-path observations, not isolated hardware speedups."
  ],
  "sections": [
    {
      "title": "What we learned",
      "paragraphs": [
        "Kokoro was the clearest CPU candidate: the small D4 completed English speech at a 1.44-second median and French at 1.30 seconds. A larger D16 lowered those to 0.85 and 0.77 seconds, but increased allocated compute cost per character. Bigger was faster; it was not cheaper.",
        "Embeddings worked on both CPU sizes and preserved all six expected search matches. The AWS GPU was faster and cheaper on the measured batch. Crane was a plausible CPU fallback. Sunflower and Whisper ran successfully, but the GPU remained preferable for interactive transcription."
      ]
    },
    {
      "title": "Same model, measured differences",
      "paragraphs": [
        "The nine paired Kokoro recordings had exactly matching durations and waveform correlations of 0.993–0.997. ASR round-trip WER was 14.8% for AWS and 13.9% for Azure across 108 reference words. These proxies cannot prove equal naturalness or pronunciation. Listen to the samples below.",
        "Crane retained all 36 clean-clip transcripts on D4. CPU INT8 Whisper/Sunflower can produce different words from GPU INT8_FLOAT16. Small WER movements on twelve clips do not establish a quality improvement."
      ]
    },
    {
      "title": "What this changes",
      "paragraphs": [
        "Evaluate an isolated Kokoro CPU canary before considering a production move. Keep GPU transcription for interactive work; consider CPU embeddings for background indexing or fallback. No production routing changed in this experiment.",
        "Kokoro already used CPU on the shared AWS GPU host. Moving it only reduces that AWS bill if the host can also be stopped or resized. Azure credit eligibility does not change economic unit cost."
      ]
    }
  ],
  "limitations": [
    "Small diagnostic samples, not a production SLA or a population accuracy estimate. Different output lengths affect completion time.",
    "Sequential single-model tests do not establish co-resident memory or concurrent throughput. Disk cache was not controlled for load-time comparisons. FLEURS may overlap training data. No East African end-user network latency was measured.",
    "Orpheus and image generation were not CPU-tested here. No blind human listening panel was conducted.",
    "Busy-compute estimates allocate the whole host rate to the measured operation. They exclude idle time, startup, storage, network and taxes; they are not customer API prices or marginal savings. Rates are the recorded October 2026 inputs, not a current quote."
  ],
  "tables": [
    {
      "title": "Kokoro: latency and allocated compute",
      "columns": [
        "Host / path",
        "EN median (s)",
        "FR median (s)",
        "EN USD / 1M chars",
        "FR USD / 1M chars"
      ],
      "rows": [
        [
          "Azure D4 · model",
          "1.44",
          "1.3",
          "1.52",
          "1.35"
        ],
        [
          "Azure D16 · model",
          "0.85",
          "0.77",
          "3.63",
          "3.22"
        ],
        [
          "AWS · public API",
          "2.49",
          "2.1",
          "6.94",
          "5.9"
        ]
      ],
      "note": "AWS includes HTTP/admission; Azure does not. AWS Kokoro also executes on CPU. Busy-compute estimates allocate the whole host rate to the measured operation. They exclude idle time, startup, storage, network and taxes; they are not customer API prices or marginal savings. Rates are the recorded October 2026 inputs, not a current quote."
    },
    {
      "title": "Qwen3 Embedding 0.6B",
      "columns": [
        "Host",
        "Six-document median (s)",
        "USD / 1M tokens",
        "Expected matches"
      ],
      "rows": [
        [
          "Azure D4",
          "0.419",
          "0.448",
          "6 / 6"
        ],
        [
          "Azure D16",
          "0.149",
          "0.637",
          "6 / 6"
        ],
        [
          "AWS API",
          "0.109",
          "0.314",
          "6 / 6"
        ]
      ],
      "note": "All vectors were finite and 1,024-dimensional. Maximum cosine-similarity difference was approximately 0.0005. This is a small synthetic retrieval diagnostic."
    },
    {
      "title": "Transcription: matched clean recordings",
      "columns": [
        "Model",
        "Language",
        "Host",
        "WER ↓",
        "Median (s) ↓",
        "USD / audio min"
      ],
      "rows": [
        [
          "Crane",
          "EN",
          "AWS baseline",
          "20.16%",
          "0.667",
          "0.00092"
        ],
        [
          "Crane",
          "EN",
          "Azure D4",
          "20.16%",
          "1.426",
          "0.00062"
        ],
        [
          "Crane",
          "EN",
          "Azure D16",
          "20.16%",
          "0.791",
          "0.00154"
        ],
        [
          "Crane",
          "SW",
          "AWS baseline",
          "24.31%",
          "0.692",
          "0.00059"
        ],
        [
          "Crane",
          "SW",
          "Azure D4",
          "24.31%",
          "1.997",
          "0.00053"
        ],
        [
          "Crane",
          "SW",
          "Azure D16",
          "24.31%",
          "0.967",
          "0.00120"
        ],
        [
          "Crane",
          "LG",
          "AWS baseline",
          "61.68%",
          "0.675",
          "0.00056"
        ],
        [
          "Crane",
          "LG",
          "Azure D4",
          "61.68%",
          "1.721",
          "0.00053"
        ],
        [
          "Crane",
          "LG",
          "Azure D16",
          "61.68%",
          "0.893",
          "0.00121"
        ],
        [
          "Sunflower",
          "SW",
          "AWS baseline",
          "12.55%",
          "1.140",
          "0.00084"
        ],
        [
          "Sunflower",
          "SW",
          "Azure D4",
          "12.16%",
          "9.888",
          "0.00265"
        ],
        [
          "Sunflower",
          "SW",
          "Azure D16",
          "12.16%",
          "4.084",
          "0.00442"
        ],
        [
          "Whisper",
          "EN",
          "AWS baseline",
          "6.45%",
          "0.889",
          "0.00100"
        ],
        [
          "Whisper",
          "EN",
          "Azure D4",
          "4.44%",
          "8.157",
          "0.00339"
        ],
        [
          "Whisper",
          "EN",
          "Azure D16",
          "4.44%",
          "3.239",
          "0.00544"
        ],
        [
          "Whisper",
          "SW",
          "AWS baseline",
          "38.04%",
          "1.186",
          "0.00083"
        ],
        [
          "Whisper",
          "SW",
          "Azure D4",
          "38.04%",
          "10.059",
          "0.00262"
        ],
        [
          "Whisper",
          "SW",
          "Azure D16",
          "38.04%",
          "4.156",
          "0.00435"
        ],
        [
          "Whisper",
          "LG",
          "AWS baseline",
          "108.88%",
          "1.533",
          "0.00109"
        ],
        [
          "Whisper",
          "LG",
          "Azure D4",
          "107.48%",
          "16.357",
          "0.00429"
        ],
        [
          "Whisper",
          "LG",
          "Azure D16",
          "107.48%",
          "6.553",
          "0.00684"
        ]
      ],
      "note": "Twelve clips per language. EN = English, SW = Swahili, LG = Luganda. WER can exceed 100% through insertions. Timing boundaries differ as described in the method."
    },
    {
      "title": "Recorded host rates",
      "columns": [
        "Host",
        "USD / hour"
      ],
      "rows": [
        [
          "Azure D4s v5",
          "0.254"
        ],
        [
          "Azure D16s v5",
          "1.016"
        ],
        [
          "AWS g4dn.xlarge",
          "0.698"
        ]
      ],
      "note": "Busy-compute estimates allocate the whole host rate to the measured operation. They exclude idle time, startup, storage, network and taxes; they are not customer API prices or marginal savings. Rates are the recorded October 2026 inputs, not a current quote."
    }
  ],
  "references": [
    {
      "label": "Google FLEURS · dataset and attribution",
      "url": "https://huggingface.co/datasets/google/fleurs"
    },
    {
      "label": "Kokoro · model card",
      "url": "https://huggingface.co/hexgrad/Kokoro-82M"
    },
    {
      "label": "SunflowerASR · model card",
      "url": "https://huggingface.co/Sunbird/SunflowerASR-51-african-languages"
    },
    {
      "label": "Azure Retail Prices API",
      "url": "https://learn.microsoft.com/en-us/rest/api/cost-management/retail-prices/azure-retail-prices"
    }
  ],
  "audio": [
    {
      "title": "Kokoro · English collection message",
      "text": "Your order is ready for collection. The office closes at five p.m.",
      "note": "Synthetic test prompt, not a customer recording or live transaction. Unmodified WAVs; the same voice, weights and speed setting. This is a labelled listening comparison, not a blinded listening study.",
      "variants": [
        {
          "label": "AWS reference",
          "src": "/research/audio/kokoro-aws-en-0.wav"
        },
        {
          "label": "Azure D4 CPU",
          "src": "/research/audio/kokoro-azure-en-0.wav"
        }
      ]
    },
    {
      "title": "Kokoro · English place names",
      "text": "Embiro helps businesses in Nairobi, Dar es Salaam and Kigali use artificial intelligence.",
      "note": "Synthetic test prompt, not a customer recording or live transaction. Unmodified WAVs; the same voice, weights and speed setting. This is a labelled listening comparison, not a blinded listening study.",
      "variants": [
        {
          "label": "AWS reference",
          "src": "/research/audio/kokoro-aws-en-2.wav"
        },
        {
          "label": "Azure D4 CPU",
          "src": "/research/audio/kokoro-azure-en-2.wav"
        }
      ]
    },
    {
      "title": "Kokoro · French payment message",
      "text": "Votre paiement a bien été reçu. Merci de votre confiance.",
      "note": "Synthetic test prompt, not a customer recording or live transaction. Unmodified WAVs; the same voice, weights and speed setting. This is a labelled listening comparison, not a blinded listening study.",
      "variants": [
        {
          "label": "AWS reference",
          "src": "/research/audio/kokoro-aws-fr-1.wav"
        },
        {
          "label": "Azure D4 CPU",
          "src": "/research/audio/kokoro-azure-fr-1.wav"
        }
      ]
    }
  ],
  "evidence": [
    {
      "file": "REPORT.md",
      "sha256": "28d7aa75fa8dd3170e55523122982593a1a0aaae6007e1c931f679f0697bab6a"
    },
    {
      "file": "summary.json",
      "sha256": "d4a5e78d615978fca2c5688b74c0efbdb21dbd620aec571c0a1e3237fcd69fbd"
    },
    {
      "file": "summary-d4.json",
      "sha256": "77cae0fc1522d28118db65ec612dcd262a4cc076e524d33b3f941726b8887154"
    }
  ]
}