{
  "formatVersion": 1,
  "scope": "Curated public research extract; not the full raw run.",
  "canonical": "https://embiro.ai/research/seven-small-model-families",
  "fieldNote": 6,
  "slug": "seven-small-model-families",
  "title": "Seven small models. No universal winner.",
  "description": "Gemma, Qwen, Llama, GLM, Ministral and Kimi on one T4, with language and reasoning tradeoffs exposed.",
  "category": "Models",
  "art": "open-intelligence",
  "thumbnail": {
    "src": "/research/thumbnails/seven-small-model-families.webp",
    "alt": "Seven distinct model sculptures arranged on one curved display shelf."
  },
  "date": "2026-10-10",
  "published": "2026-10-12",
  "updated": "2026-10-12",
  "author": "Embiro Labs Research",
  "status": "Exploratory benchmark",
  "headline": "Model choice is more than response speed.",
  "metric": "7",
  "metricLabel": "serving profiles screened",
  "method": [
    "Each model ran separately on one Cape Town T4 using pinned llama.cpp b11429 and Q4_K_M weights; F16 projectors where supported. One 4,096-token slot, four CPU threads, 2,048 output tokens and a requested 512-token thinking cap.",
    "Seven English chat, nine Swahili, four reasoning, two French and five vision fixtures (vision-capable models only). Model-specific sampling, one seed. Seven separate cold requests and 184 measured requests. Kimi’s reasoning delimiters were not parsed, so its requested cap did not apply."
  ],
  "sections": [
    {
      "title": "Why Gemma moved forward",
      "paragraphs": [
        "Gemma 4 E4B combined a 1.16-second English median with correct outcomes on all five image fixtures and a modest memory footprint. It still invented shipment status, failed a Swahili clock translation and needed language review. It was a candidate for the next integration test, not an accuracy winner.",
        "Llama 3.1 8B was a useful fast English alternative. Qwen3.5-9B handled the image fixtures well but its thinking profile was slower. Every model missed the full Swahili clock translation in this screen."
      ]
    },
    {
      "title": "Runtime compatibility is part of quality",
      "paragraphs": [
        "All streams completed, yet eleven replies were truncated and seven Kimi requests lacked a final answer after separating its native reasoning markers. A 200 response cannot stand in for task completion. Kimi’s configuration was not an equivalent controlled thinking-budget test."
      ]
    },
    {
      "title": "What we cannot rank",
      "paragraphs": [
        "Four reasoning questions and two French questions do not establish advanced reasoning or multilingual ability. Human-like fluency, short completion times and small active-parameter counts do not prove task quality or total weight-memory requirements."
      ]
    }
  ],
  "limitations": [
    "Small diagnostic samples, not a production SLA or a population accuracy estimate. Different output lengths affect completion time.",
    "Assistant review of synthetic fixtures, not blinded or native-speaker validation. Answer lengths and sampling differ. Vision includes the first image request. Memory readings were snapshots, not peak co-residency measurements.",
    "Kimi cap/parser incompatibility prevents an equivalent reasoning comparison. No tool-call contract or long-context accuracy was tested."
  ],
  "tables": [
    {
      "title": "Median completion time",
      "columns": [
        "Profile",
        "English (s)",
        "Swahili (s)",
        "Reasoning (s)",
        "French (s)",
        "Vision (s)"
      ],
      "rows": [
        [
          "Qwen3.5-4B",
          "10.25",
          "12.32",
          "13.84",
          "10.78",
          "8.29"
        ],
        [
          "Qwen3.5-9B",
          "16.42",
          "16.29",
          "20.77",
          "16.62",
          "14.70"
        ],
        [
          "Gemma 4 E4B",
          "1.16",
          "10.62",
          "9.69",
          "0.35",
          "5.77"
        ],
        [
          "Ministral 3 8B",
          "7.99",
          "15.39",
          "22.86",
          "12.20",
          "12.61"
        ],
        [
          "GLM-4.6V-Flash",
          "7.28",
          "18.29",
          "15.00",
          "13.66",
          "3.29"
        ],
        [
          "Llama 3.1 8B",
          "1.23",
          "1.43",
          "2.47",
          "1.59",
          "Not tested"
        ],
        [
          "Kimi-VL-A3B",
          "3.70",
          "24.98",
          "17.70",
          "15.26",
          "4.78"
        ]
      ],
      "note": "Llama is text-only; Kimi’s requested reasoning cap was ineffective. Compare practical configurations, not theoretical model capability."
    }
  ],
  "references": [],
  "audio": [],
  "evidence": [
    {
      "file": "README.md",
      "sha256": "bca9ec8bc569aff8622a3188520c8a690da552fec07e05983dcb8d20d4a63d57"
    },
    {
      "file": "summary.json",
      "sha256": "8be6b3a2774e4cb1169e604b881c2e2a5cb5b52dd5fcb05726ebdfd9254137fd"
    },
    {
      "file": "quality-review.json",
      "sha256": "e3370982319ea9d79962ee9f4b47076cbabb9a5a2011cba75a1fb290b8d63529"
    }
  ]
}