BENCHMARK Open-Core v4.13.0 + Legacy Enterprise GPU Matrix 2026-11-10 • 10 min read

Legacy 16GB-24GB GPUs & Day-2 Escape Hatches: A 108-Episode Open-Weight Brownfield Audit

Across 108 paired Python, TypeScript, and Go episodes spanning 6 open-weight models (8B to 32B across Meta, Google, Qwen, and Mistral), we map the exact capability floor where older enterprise NVIDIA cards become production-viable under Nomos v4.13 - and document the Day-2 escape hatches that prevent developer friction.

🖥️ 6 Open-Weight Models (8B, 12B, 14B, 24B, 27B Local RTX 4080, 32B) • 108 Paired Episodes (temp = 0.0)
Paired Live Scale 108 Episodes
14B-32B Governed Pass 86.1% vs. 30.6%
14B-32B Layer Purity 24/24 (100%)
Qwen 3 14B (16GB T4 Tier) 88.9% vs. 0.0%
Mark Gantlett
Mark Gantlett
Founder & Principal Systems Architect
Sophia AI
⚡ AI AUGMENTED
Tier 1 Frontier Reasoning + On-Premise RTX 4080 Silicon
FIG D-10 // LEGACY-VRAM GPU RACK & DAY-2 PORT SEAM - 108 paired episodes across 8B-32B open-weight models with explicit ports/ escape-hatch governance.

Legacy 16GB-24GB GPUs & Day-2 Escape Hatches: A 108-Episode Open-Weight Audit#

Mapping the Exact Open-Weight Capability Floor (8B to 32B) on Older Enterprise NVIDIA GPUs (Tesla T4, RTX A4000, L4, RTX 3090/4080), Clarifying Structural vs. Semantic Scope, and Documenting Day-2 nomos.yaml Escape Hatches.

Audit Provenance & Methodological Transparency
Date Evaluated: November 10, 2026 • Substrate: Open-Core `nomos v4.13.0` (c3268c9, Apache 2.0)
Sample Scale: 108 symmetric paired live-model episodes (54 Control vs. 54 Nomos v4.13.0; 6 open-weight models x 3 languages x 3 brownfield lifecycle stages at temperature = 0.0, 12-turn cap, identical 5-tool JSON ReAct loop)
Model Matrix: meta-llama/llama-3.1-8b-instruct (8B), google/gemma-3-12b-it (12B), qwen/qwen3-14b (14B), mistralai/mistral-small-3.2-24b-instruct (24B), local hardware-anchored Qwen3.8-27B-IQ3_M on NVIDIA RTX 4080 16GB (27B), and qwen/qwen3-32b (32B)


1. Answering Three Hard Questions From Production Architects#

After publishing our initial 12-episode v4.13.0 hexagonal audit (Dispatch D-08), senior systems engineers raised three fair questions:

  1. Does this hold beyond a 12-episode single-model existence proof - especially on older 12GB - 24GB NVIDIA cards? Many enterprises already own racks of Tesla T4 (16GB), RTX A4000 (16GB), NVIDIA L4 (24GB), or RTX 3090/4080 GPUs. Where is the actual parameter floor where compiled AST structure turns a small open-weight model into a reliable brownfield contributor, and where does a model simply run out of reasoning capacity?
  2. Are "obligation proofs" formal semantic proofs or structural checks? What can <5ms static AST and SHA-256 provenance gates guarantee, and what remains outside their scope?
  3. What happens on Day 2 when a gate fires on a legitimate exception or a spec changes? How do teams carve out explicit adapter seams, handle generated or legacy files, and evolve TC-* obligations without developers disabling the harness in frustration?

This dispatch answers all three with 108 live episodes and concrete configuration recipes.


2. The 108-Episode Multi-Lab Legacy-GPU Benchmark#

Each of the 6 open-weight models executed 9 multi-module brownfield tasks (Python, TypeScript, and Go across Stages A, B, and C) under both Control (CLAUDE.md engineering handbook + native unit tests) and Nomos v4.13.0 (nomos handshake Turn-0 taint banner, isolated worktree, PostToolUse nomos verify, and 42/42 release gate):

  • Stage A (Convenience Boundary Trap): domain/ already imports a working validation helper from infra/ and unit tests pass (exit 0). The agent must complete a domain feature (ledger_spec.md, billing_spec.md, order_spec.md) without leaving or adding forbidden domain -> infra coupling.
  • Stage B (Silent Upstream Spec Taint): In a 5-spec brownfield repository where all v1 unit tests exit 0, upstream commit [INT-2] adds a new required contract field and TC-*-03 vector. Tests whether the agent catches and remediates silent spec-to-code lag before cutting a release.
  • Stage C (Day-2 Explicit ports/ Seam Escape Hatch): Tests false-positive resilience. nomos.yaml forbids domain -> infra imports while allowing explicit hexagonal port contracts in ports/ (ports/fx_port.py, src/ports/quotaPort.mjs, internal/ports). Seed code in domain/ shortcuts straight into infra/. To pass, the agent must pivot onto the allowed ports/ seam without being blocked by an overly broad boundary rule.

Full 108-Episode Scorecard by GPU VRAM Tier#

Open-Weight Model (Lab) Target Legacy GPU VRAM Tier Control (CLAUDE.md + Tests) Nomos v4.13.0 Governed Stage Breakdown (Control -> Nomos A / B / C) Layer Purity (Control -> Nomos on A+C)
Llama 3.1 8B (Meta) 8-12 GB (RTX 2080 Ti / 3060 / T4-MIG) 0 / 9 (0.0%) 0 / 9 (0.0%) 0/0/0 -> 0/0/0 4/6 -> 5/6 (0% leaked to main)
Gemma 3 12B (Google) 12-16 GB (Tesla T4 16GB / RTX A4000) 0 / 9 (0.0%) 4 / 9 (44.4%) 0/0/0 -> 0/2/2 0/6 -> 5/6
Qwen 3 14B (Alibaba) 12-16 GB (Tesla T4 16GB / RTX A4000) 0 / 9 (0.0%) 8 / 9 (88.9%) 0/0/0 -> 3/3/2 1/6 -> 6/6 (100%)
Mistral Small 3.2 24B (Mistral) 16-24 GB (T4-Q4 / NVIDIA L4 / 3090) 3 / 9 (33.3%) 6 / 9 (66.7%) 0/2/1 -> 1/3/2 2/6 -> 6/6 (100%)
Qwen 3.8 27B (Local RTX 4080) 16 GB (RTX 4080 IQ3_M @ 40.3 tok/s) 6 / 9 (66.7%) 9 / 9 (100.0%) 0/3/3 -> 3/3/3 3/6 -> 6/6 (100%)
Qwen 3 32B (Alibaba) 24 GB (NVIDIA L4 / A10G / RTX 3090) 2 / 9 (22.2%) 8 / 9 (88.9%) 0/0/2 -> 2/3/3 3/6 -> 6/6 (100%)
All 6 Models (N = 108) 8 GB - 24 GB Full Matrix 11 / 54 (20.4%) 35 / 54 (64.8%) 0/5/6 -> 9/14/12 13/36 (36.1%) -> 34/36 (94.4%)
14B - 32B Tier (N = 72) 16 GB - 24 GB Enterprise Cards 11 / 36 (30.6%) 31 / 36 (86.1%) 0/5/6 -> 9/12/10 9/24 (37.5%) -> 24/24 (100.0%)

Three Empirical Findings From the Phase-Transition Curve#

flowchart LR
    Floor["8B Legacy Tier (Llama 3.1 8B)<br/>0% -> 0% Task Completion<br/>Fail-Closed Blocks 9/9 Bad Merges"] --> Inflect["12B Inflection (Gemma 3 12B)<br/>0.0% -> 44.4% Governed<br/>Layer Purity: 0/6 -> 5/6"]
    Inflect --> Sweet["14B Sweet Spot on 16GB T4/A4000<br/>Qwen 3 14B: 0.0% -> 88.9%<br/>Layer Purity: 1/6 -> 6/6 (100%)"]
    Sweet --> Workhorse["24B-32B Workhorse Tier (16-24GB)<br/>Mistral 24B / Qwen 27B & 32B<br/>23/27 (85.2%) & 18/18 Layer Purity"]
  1. Every Single Ungoverned Model Failed Stage A (0 / 18 = 0.0%): Across all 6 models - from Llama 3.1 8B up to Qwen 3 32B - Control scored 0 / 18 (0.0%) on Stage A. Whenever a working infra/ helper already existed and unit tests passed, advisory CLAUDE.md prose never once persuaded an open-weight model to refactor out the forbidden import on its own.
  2. The 14B Sweet Spot for Older 16GB Enterprise Cards (0.0% -> 88.9%): Qwen 3 14B fits comfortably in ~9.5 GB (Q4_K_M) on a legacy 16GB NVIDIA Tesla T4 or RTX A4000 with ample KV-cache headroom. Ungoverned, Qwen 3 14B scored 0 / 9 (0.0%) (skipping stale Stage B specs in 2 turns because unittest/node --test/go test exited 0, and leaking infra imports in Stage A). Under Nomos v4.13.0, that exact same 14B model jumped to 8 / 9 (88.9%) with 6 / 6 (100%) architectural boundary purity.
  3. Honest Lower Bound - Why 8B (Llama 3.1 8B) Scored 0 / 9 Even With Nomos: Compiled guardrails are not magic pixie dust for under-parameterized models. When Nomos flagged a missing struct field or boundary violation on Llama 3.1 8B, the 8B model frequently broke Go/Python test syntax while attempting multi-file edits (test=0 at turn 12). Crucially, however, while Control allowed Llama 3.1 8B to declare victory (finish) on 5 unverified episodes where legacy unit tests stayed green, Nomos failed closed on 9 / 9 broken runs (0% trunk contamination). For real multi-file brownfield work on older GPUs, 14B (>=12GB VRAM) is the practical floor.

3. Scope Honesty: Structural Enforcement vs. Semantic Proof#

Skeptical PL and formal-methods engineers rightly distinguish between structural verification and semantic correctness proofs. Here is the exact boundary of what nomos verify guarantees and what it does not:

Dimension Enforced by nomos verify (<5ms - 50ms) Outside Static AST Gate Scope (Requires Tests / Review)
Module Topology Static import / require / use / #include edges across 11 languages (architecture_boundaries) Runtime dlopen/FFI, dynamic string reflection (importlib.import_module(var)), or HTTP loopback calls
Contract Shape Presence of declared struct/class/interface fields from fenced spec blocks (strict_types) Business-math equivalence (for example, rounding half-even vs. floor inside a fee calculation)
Test Traceability Every TC-* spec row bound to a non-empty test containing real assertions (verify_obligations + AntiShimGate) Whether the test inputs exhaustively cover subtle domain corner cases beyond the spec table
Spec Freshness Cryptographic SHA-256 spec_digests in Git Notes flagging upstream *_spec.md edits (fail_on_stale_obligations) Whether a human stakeholder wrote the right product requirement in the first place

Why draw the line here? Because any governance mechanism that requires writing TLA+ state machines, waiting 4 minutes for an SMT solver, or paying an extra LLM judge on every file save gets disabled by Friday afternoon. Sub-5ms structural checks stay turned on permanently and eliminate the mechanical 80% of AI architectural rot so senior engineers can spend 100% of PR review on domain semantics.


4. Day-2 Operations: Handling False Positives, Seam Exceptions & Spec Evolution#

A rigid guardrail without clean escape hatches gets bypassed with --no-verify. Because nomos pins nomos.yaml authority to protected main (so coding agents inside worktrees/<repo>-SUB-* cannot weaken rules to pass their own task), human architects retain three clean, zero-friction levers on main for Day-2 evolution:

Escape Hatch 1: Carving Out Legitimate Hexagonal ports/ or Composition-Root Exceptions#

Suppose architecture_boundaries blocks domain/ from importing infra/, and a developer legitimately needs shared currency/rail constants or needs cmd/server/main.go to wire domain and infra together. Instead of disabling the gate, scope from narrowly to domain/** while leaving ports/** open (exactly as tested across 36 Stage C episodes above with 0 false positives):

governance:
  quality_gates:
    architecture_boundaries:
      - name: "domain-purity-with-port-seam"
        from: "internal/domain/**"
        # internal/ports/** and stdlib remain allowed; only adapter/infra packages are blocked
        forbid_imports:
          - "internal/infra/**"
          - "internal/adapters/**"
      # Composition roots (cmd/**) are intentionally omitted from `from:` so main.go can wire adapters into ports.

Escape Hatch 2: Generated Code (*.pb.go, OpenAPI, SQLC) & Legacy Monster Files#

Generated protobuf/OpenAPI clients and 1,800-line legacy modules violate greenfield complexity and line-count limits immediately. Keep strict_mode: false active so structural thresholds apply strictly as a monotonic diff ratchet against main (untouched legacy code never blocks a build, and touched legacy functions only fail if cyclomatic complexity increases over main), while excluding generated paths under exclude_globs:

governance:
  quality_gates:
    strict_mode: false          # Monotonic ratchet vs. merge-base: zero Day-1 legacy rewrite
    exclude_globs:
      - "**/*.pb.go"
      - "**/generated/**"
      - "dist/**"

Escape Hatch 3: Evolving TC-* Obligations Without Chicken-and-Egg CI Deadlocks#

What happens on Day 30 when a product pivot retires TC-BIL-02 and introduces TC-BIL-04?

  1. Spec Commit (INT-* Task): Updating specs/billing_spec.md inside an INT-* task never deadlocks on missing code tests because IntentTraceabilityGate exempts pure intent-plane commits from SUB-* test-binding checks.
  2. Code Commit (SUB-* Task): On the very next session, nomos handshake surfaces Stale Spec Obligations (1): specs/billing_spec.md at Turn 0. The engineer or coding agent updates the implementation, replaces @verifies TC-BIL-02 with @verifies TC-BIL-04, and nomos task sync atomically records the new spec_digests SHA-256 baseline in refs/notes/nomos/releases.
  3. Keeping the North Star From Rotting: Because the Day-1 North Star Charter (<= 2 pages) holds only timeless domain invariants (tenant isolation, integer minor-unit money math, layer topology) while volatile feature behavior lives in 1-page slice specs (specs/<feature>_spec.md), Day-30 feature pivots update a single 40-line slice file rather than a 30-page document.
Ask Sophia Docs & Specs Sophia AI · Grounded Nomos Architecture & Docs Concierge