{
  "formatVersion": 1,
  "scope": "Curated public research extract; not the full raw run.",
  "canonical": "https://embiro.ai/research/local-reasoning-baseline",
  "fieldNote": 1,
  "slug": "local-reasoning-baseline",
  "title": "Our first benchmark: fast tokens, unfinished answers.",
  "description": "A local DeepSeek test established the gateway and accounting baseline before cloud comparisons began.",
  "category": "Serving",
  "art": "open-intelligence",
  "thumbnail": {
    "src": "/research/thumbnails/local-reasoning-baseline.webp",
    "alt": "A ceramic laptop showing an unfinished copper reasoning maze."
  },
  "date": "2026-10-10",
  "published": "2026-10-12",
  "updated": "2026-10-12",
  "author": "Embiro Labs Research",
  "status": "Exploratory benchmark",
  "headline": "Count useful answers, not just tokens.",
  "metric": "10 / 10",
  "metricLabel": "short runs hit the output cap",
  "method": [
    "Local Apple M1 MacBook Pro, 16 GiB unified memory, CPU-only, four threads; not an African cloud deployment. Metal initialization failed in this execution environment; this does not establish general GPU incompatibility.",
    "DeepSeek R1 Distill Qwen 1.5B via a pinned third-party Q4_K_M GGUF, llama.cpp b11429. A 2,048-token context, temperature 0.6, top-p 0.95, min-p 0.05. Five direct and five gateway requests alternated after warm-up; the same 22-token prompt and 128-output-token cap, no fixed seed."
  ],
  "sections": [
    {
      "title": "The useful warning",
      "paragraphs": [
        "Median streamed throughput looked encouraging, but all ten short runs reached the output limit before closing their reasoning segment. The first streamed token was reasoning/output, not necessarily useful answer text. This became an early reason to track final-answer completion separately."
      ]
    },
    {
      "title": "What the gateway did prove",
      "paragraphs": [
        "A later arithmetic probe answered 391 correctly but ignored the integer-only format; a summary returned two sentences instead of one. The async addition job completed. Nine measured gateway receipts totalled 1,829 tokens with no unresolved holds. This verified a development integration, not durable cloud operation."
      ]
    },
    {
      "title": "Why we moved beyond a laptop",
      "paragraphs": [
        "The gateway appeared faster than direct calls in this tiny sample, which is scheduling variation rather than evidence of acceleration. A separate CPU microbenchmark was slower. Background applications and thermal conditions were uncontrolled, so this was a feasibility baseline for the Cape Town tests."
      ]
    }
  ],
  "limitations": [
    "Five samples per path, shared laptop, unseeded sampling, uncontrolled file cache and background load. No reliable tail latency, gateway overhead or sustained-capacity claim.",
    "No cloud meter was involved; local electricity was not measured."
  ],
  "tables": [
    {
      "title": "Local HTTP baseline",
      "columns": [
        "Measurement",
        "Direct",
        "Gateway"
      ],
      "rows": [
        [
          "Median first output (s)",
          "0.144",
          "0.171"
        ],
        [
          "Median completion (s)",
          "3.866",
          "3.625"
        ],
        [
          "Median end-to-end tokens/s",
          "33.11",
          "35.31"
        ],
        [
          "Output-cap finishes",
          "5 / 5",
          "5 / 5"
        ]
      ],
      "note": "Every short run ended before completing its reasoning. Lower latency here is not a completed-answer quality result."
    }
  ],
  "references": [],
  "audio": [],
  "evidence": [
    {
      "file": "summary.json",
      "sha256": "32d0f695805999597d8c129d5764e27779abb8302d63824c06792c989b49f3cf"
    },
    {
      "file": "results.json",
      "sha256": "bc1c71eecedeea065a66de79e441a6c6f8517ebac10239e04cab76cb5b11ef59"
    },
    {
      "file": "cpu-microbenchmark.json",
      "sha256": "44dc98fe58e3affe53383ae88f78eb986f1ec6a51c71a83c5f342eada93518ac"
    }
  ]
}