A fast first sentence matters more than a crowded server.
The first sentence is the product
A short acknowledgement reached first audio in a 0.19-second median inside Azure and 0.45 seconds from the Nairobi client. That is fast enough for a conversational opening while the language model prepares the next complete sentence.
Long answers should still be broken at sentence boundaries. The 1,479-character test produced 88.85 seconds of audio and completed synthesis in 6.63 seconds on D32, but first audio took 2.11 seconds. Streaming improves perceived latency; it does not make a large first synthesis unit disappear.
More workers changed the service
Two workers were the best fit for one or two active conversations. Four workers handled four callers more evenly but raised individual latency. Eight workers completed the eight-caller sample, yet p95 first audio reached 4.73 seconds. That is useful batch throughput, not a responsive voice agent.
The production design therefore admits two active speech generations on one D32 and sends overflow to another warm target or the existing AWS service. A bigger queue is not a substitute for a quick response.
Failover worked, detection was the slow part
An injected Azure 503 fell back to AWS and returned a valid WAV with one billing receipt in 4.58 seconds. An unreachable port took 14.81 seconds because the experimental client waited ten seconds before declaring the primary unavailable. Production health and connection deadlines need to fail closed in roughly one to two seconds.
The direct Nairobi-to-VM path also produced intermittent transport timeouts that were absent from Azure-local tests. No production traffic moved in this experiment. The next step is a TLS-terminated canary on the real routing path, verified from multiple East African probes before traffic increases.
Why choose D32
D32 reduced streamed first-audio latency by roughly 27 to 32 percent over D16 across the three measured input sizes. Its recorded hourly rate was twice as high, so the reason to use it is responsiveness and two-call capacity rather than cheaper synthesis.
A conservative 1,500-character limit is supported for streamed narration or asynchronous jobs. For live agents, short complete sentences remain the better interface.
Results, in context.
| Input | D16 first audio | D32 first audio | D32 complete |
|---|---|---|---|
| 135 characters | 1.07 s | 0.72 s | 1.05 s |
| 570 characters | 3.00 s | 2.18 s | 2.71 s |
| 1,479 characters | 2.90 s | 2.11 s | 6.63 s |
Medians from the warm direct synthesis harness. The long output was 88.85 seconds of audio.
| Turn | One caller median | Two callers p95 | Four callers p95 |
|---|---|---|---|
| 41-character acknowledgement | 0.19 s | 1.22 s | 1.79 s |
| 79-character brief answer | 0.84 s | 0.96 s | 2.66 s |
| 135-character short answer | 1.19 s | 2.63 s | 4.23 s |
Azure-local test, five calls per turn size and caller at each concurrency level. The Nairobi single-caller acknowledgement median was 0.45 seconds.
| Layout | Test load | Completed | p95 first audio |
|---|---|---|---|
| 2 workers × 16 threads | 2 callers | 6 / 6 | 2.22 s |
| 4 workers × 8 threads | 4 callers | 12 / 12 | 2.33 s |
| 8 workers × 4 threads | 8 callers | 24 / 24 | 4.73 s |
The separate load harness used a 135-character prompt. Higher completion capacity came with slower individual responses.
| Scenario | Selected service | Result | Total |
|---|---|---|---|
| Healthy primary | Azure | Valid WAV | 1.17 s |
| Injected HTTP 503 | AWS | Valid WAV · one receipt | 4.58 s |
| Unreachable port · 10 s timeout | AWS | Valid WAV · one receipt | 14.81 s |
The ten-second connection timeout dominated recovery time and is intentionally not the production recommendation.
| VM | USD / hour | 730-hour month | 8 hours/day × 30 |
|---|---|---|---|
| Standard D16s v5 | $1.016 | $741.68 | $243.84 |
| Standard D32s v5 | $2.032 | $1,483.36 | $487.68 |
Dated Linux pay-as-you-go inputs from the Azure Retail Prices API. Storage, network and tax are excluded.
How we ran it.
- Azure South Africa North: Standard D16s v5 and D32s v5 tested sequentially with Kokoro 0.9.4, the same Kokoro 82M checkpoint, af_heart voice, PyTorch 2.8 CPU and synthetic English prompts. The selected D32 layout used two workers with sixteen threads each.
- Latency covered 135, 570 and 1,479 character inputs. Conversation-shaped tests used 41, 79 and 135 character turns with one, two and four concurrent callers. Separate load tests compared one, two, four and eight workers. Warm calls excluded model installation and VM provisioning.
- Failover used a temporary Embiro API key restricted to Kokoro and a small spending cap. Tests covered a healthy Azure response, an injected HTTP 503 and an unreachable port, then checked the returned WAV and billing receipts. The key and policy were revoked after the run.
The limits matter.
- This was an isolated canary with synthetic English prompts, not a production SLA or a population listening study. Voice naturalness was not rescored in this run.
- The direct test service used an operator-restricted endpoint rather than the production TLS and load-balancer path. Some Nairobi calls timed out; public-path reliability was therefore not established.
- Samples were small and warm. Provisioning, cold starts, storage, network, idle time and taxes are excluded from the timing and cost view.
- Recorded cloud list prices are dated inputs. Credits reduce pilot cash spend but do not change the opportunity cost of the compute.
Evidence & sources.
The public extract contains the methods, selected results and caveats in this note. It is a curated summary, not the full raw run. Fingerprints identify the retained source artifacts; gated audio and private operational records are not included.
Download results extractSource artifact fingerprints · SHA-256
5c6810593ed80feffe180b58d14ee57047644a4486cf6b782b51e1a5ff9ee4ba42191a65899d5a998b790bcc3dc8a9a5aae86b318ec6d247ba93a383ee3a58a8fdcf4b120594209b3b681ccc8f72c311208dced05d0ab610865bc6cdef3b8052Suggested citation: Embiro Labs Research. “Can a CPU keep up with a conversation?” Published 13 Oct 2026. embiro.ai/research/kokoro-conversation-canary
Report a correction
