Start with the workload, not the GPU.
What we learned
Kokoro was the clearest CPU candidate: the small D4 completed English speech at a 1.44-second median and French at 1.30 seconds. A larger D16 lowered those to 0.85 and 0.77 seconds, but increased allocated compute cost per character. Bigger was faster; it was not cheaper.
Embeddings worked on both CPU sizes and preserved all six expected search matches. The AWS GPU was faster and cheaper on the measured batch. Crane was a plausible CPU fallback. Sunflower and Whisper ran successfully, but the GPU remained preferable for interactive transcription.
Same model, measured differences
The nine paired Kokoro recordings had exactly matching durations and waveform correlations of 0.993–0.997. ASR round-trip WER was 14.8% for AWS and 13.9% for Azure across 108 reference words. These proxies cannot prove equal naturalness or pronunciation. Listen to the samples below.
Crane retained all 36 clean-clip transcripts on D4. CPU INT8 Whisper/Sunflower can produce different words from GPU INT8_FLOAT16. Small WER movements on twelve clips do not establish a quality improvement.
What this changes
Evaluate an isolated Kokoro CPU canary before considering a production move. Keep GPU transcription for interactive work; consider CPU embeddings for background indexing or fallback. No production routing changed in this experiment.
Kokoro already used CPU on the shared AWS GPU host. Moving it only reduces that AWS bill if the host can also be stopped or resized. Azure credit eligibility does not change economic unit cost.
Results, in context.
Faster. Cheaper. Sometimes both.
Switch workloads to compare the measured paths. Lower is better for both metrics.
AWS includes API and admission overhead; Azure measures synthesis directly. Kokoro runs on CPU on both clouds. Cost excludes idle time, storage and transfer. Shared-host allocation is not marginal savings. Each chart starts at zero.
| Host / path | EN median (s) | FR median (s) | EN USD / 1M chars | FR USD / 1M chars |
|---|---|---|---|---|
| Azure D4 · model | 1.44 | 1.3 | 1.52 | 1.35 |
| Azure D16 · model | 0.85 | 0.77 | 3.63 | 3.22 |
| AWS · public API | 2.49 | 2.1 | 6.94 | 5.9 |
AWS includes HTTP/admission; Azure does not. AWS Kokoro also executes on CPU. Busy-compute estimates allocate the whole host rate to the measured operation. They exclude idle time, startup, storage, network and taxes; they are not customer API prices or marginal savings. Rates are the recorded October 2026 inputs, not a current quote.
| Host | Six-document median (s) | USD / 1M tokens | Expected matches |
|---|---|---|---|
| Azure D4 | 0.419 | 0.448 | 6 / 6 |
| Azure D16 | 0.149 | 0.637 | 6 / 6 |
| AWS API | 0.109 | 0.314 | 6 / 6 |
All vectors were finite and 1,024-dimensional. Maximum cosine-similarity difference was approximately 0.0005. This is a small synthetic retrieval diagnostic.
| Model | Language | Host | WER ↓ | Median (s) ↓ | USD / audio min |
|---|---|---|---|---|---|
| Crane | EN | AWS baseline | 20.16% | 0.667 | 0.00092 |
| Crane | EN | Azure D4 | 20.16% | 1.426 | 0.00062 |
| Crane | EN | Azure D16 | 20.16% | 0.791 | 0.00154 |
| Crane | SW | AWS baseline | 24.31% | 0.692 | 0.00059 |
| Crane | SW | Azure D4 | 24.31% | 1.997 | 0.00053 |
| Crane | SW | Azure D16 | 24.31% | 0.967 | 0.00120 |
| Crane | LG | AWS baseline | 61.68% | 0.675 | 0.00056 |
| Crane | LG | Azure D4 | 61.68% | 1.721 | 0.00053 |
| Crane | LG | Azure D16 | 61.68% | 0.893 | 0.00121 |
| Sunflower | SW | AWS baseline | 12.55% | 1.140 | 0.00084 |
| Sunflower | SW | Azure D4 | 12.16% | 9.888 | 0.00265 |
| Sunflower | SW | Azure D16 | 12.16% | 4.084 | 0.00442 |
| Whisper | EN | AWS baseline | 6.45% | 0.889 | 0.00100 |
| Whisper | EN | Azure D4 | 4.44% | 8.157 | 0.00339 |
| Whisper | EN | Azure D16 | 4.44% | 3.239 | 0.00544 |
| Whisper | SW | AWS baseline | 38.04% | 1.186 | 0.00083 |
| Whisper | SW | Azure D4 | 38.04% | 10.059 | 0.00262 |
| Whisper | SW | Azure D16 | 38.04% | 4.156 | 0.00435 |
| Whisper | LG | AWS baseline | 108.88% | 1.533 | 0.00109 |
| Whisper | LG | Azure D4 | 107.48% | 16.357 | 0.00429 |
| Whisper | LG | Azure D16 | 107.48% | 6.553 | 0.00684 |
Twelve clips per language. EN = English, SW = Swahili, LG = Luganda. WER can exceed 100% through insertions. Timing boundaries differ as described in the method.
| Host | USD / hour |
|---|---|
| Azure D4s v5 | 0.254 |
| Azure D16s v5 | 1.016 |
| AWS g4dn.xlarge | 0.698 |
Busy-compute estimates allocate the whole host rate to the measured operation. They exclude idle time, startup, storage, network and taxes; they are not customer API prices or marginal savings. Rates are the recorded October 2026 inputs, not a current quote.
How we ran it.
- Azure South Africa North: D4s v5 (4 logical CPUs, 16 GiB) and D16s v5 (16 logical CPUs, 64 GiB), tested sequentially. Both reported Intel Xeon Platinum 8573C. AWS baseline: Cape Town g4dn.xlarge, one NVIDIA T4, four vCPUs, 16 GiB RAM.
- Kokoro: nine matched prompts across af_heart, bf_emma and ff_siwis; three Azure repeats and two fresh AWS API repeats, same FP32 weights and speed 1. Retrieval: six documents and six known-answer queries, CPU FP32 versus GPU FP16. ASR: twelve matched FLEURS clips per language, plus separate noise and silence checks.
- Azure timings are direct synthesis/model execution. AWS Kokoro and embeddings use the public API, including HTTP and admission; historical AWS Swahili/Luganda ASR uses direct execution and English includes API overhead. These are deployment-path observations, not isolated hardware speedups.
Hear the tradeoffs.
AI-generated samples. The prompt is the intended transcript; listen for omissions, pronunciation, pauses and voice character.
Kokoro · English collection message
Your order is ready for collection. The office closes at five p.m.
Synthetic test prompt, not a customer recording or live transaction. Unmodified WAVs; the same voice, weights and speed setting. This is a labelled listening comparison, not a blinded listening study.
Kokoro · English place names
Embiro helps businesses in Nairobi, Dar es Salaam and Kigali use artificial intelligence.
Synthetic test prompt, not a customer recording or live transaction. Unmodified WAVs; the same voice, weights and speed setting. This is a labelled listening comparison, not a blinded listening study.
Kokoro · French payment message
Votre paiement a bien été reçu. Merci de votre confiance.
Synthetic test prompt, not a customer recording or live transaction. Unmodified WAVs; the same voice, weights and speed setting. This is a labelled listening comparison, not a blinded listening study.
The limits matter.
- Small diagnostic samples, not a production SLA or a population accuracy estimate. Different output lengths affect completion time.
- Sequential single-model tests do not establish co-resident memory or concurrent throughput. Disk cache was not controlled for load-time comparisons. FLEURS may overlap training data. No East African end-user network latency was measured.
- Orpheus and image generation were not CPU-tested here. No blind human listening panel was conducted.
- Busy-compute estimates allocate the whole host rate to the measured operation. They exclude idle time, startup, storage, network and taxes; they are not customer API prices or marginal savings. Rates are the recorded October 2026 inputs, not a current quote.
Evidence & sources.
The public extract contains the methods, selected results and caveats in this note. It is a curated summary, not the full raw run. Fingerprints identify the retained source artifacts; gated audio and private operational records are not included.
Download results extract- Google FLEURS · dataset and attribution
- Kokoro · model card
- SunflowerASR · model card
- Azure Retail Prices API
Source artifact fingerprints · SHA-256
28d7aa75fa8dd3170e55523122982593a1a0aaae6007e1c931f679f0697bab6ad4a5e78d615978fca2c5688b74c0efbdb21dbd620aec571c0a1e3237fcd69fbd77cae0fc1522d28118db65ec612dcd262a4cc076e524d33b3f941726b8887154Suggested citation: Embiro Labs Research. “Does every AI workload need a GPU?” Published 12 Oct 2026. embiro.ai/research/african-inference-on-cpu
Report a correction
