# Full Vault — Agent Primer — The Anatomy of an Agent Harness > **Single-fetch comprehensive vault.** Contains the agent primer + map-of-content + glossary + speakers + every note inline. Use this file for agents that cannot follow embedded links (e.g., URL-provenance-restricted fetchers). For agents that can follow links, prefer `_AGENT_PRIMER.md` for progressive disclosure with on-demand drill-down. > *All wikilinks resolve to within-document anchors (e.g. `[concept-foo](#concept-foo)`). The vault contains 36 notes total.* --- ## Agent Primer > **Read me first.** This document primes a downstream AI agent to act as a subject-matter expert on the source. Read this in full before consulting individual notes. **Source**: [The Anatomy of an Agent Harness](https://www.langchain.com/blog/the-anatomy-of-an-agent-harness) **Words**: ~2,465 **Author(s)**: Vivek Trivedy **Domains**: `agent-architecture`, `harness-engineering`, `context-management`, `llm-orchestration`, `autonomous-agents`, `tool-use` **Vault slug**: `anatomy-of-agent-harness` **Generated**: 2026-06-07T08:53:23.819Z --- # _AGENT_PRIMER — The Anatomy of an Agent Harness You are about to act as a subject-matter expert on a single article: **“The Anatomy of an Agent Harness”** by **Vivek Trivedy** (see [entity-vivek-trivedy](#entity-vivek-trivedy)), published on the [LangChain](#entity-langchain) blog. This primer gives you the full distillation needed to answer roughly 80% of expected questions without leaving this note. --- ## 1. The Source In One Paragraph An autonomous agent is fundamentally composed of a **raw intelligence model** and a surrounding **“harness”** — the code, infrastructure, and orchestration logic that makes the model useful. Models are text-in / text-out intelligence; harnesses provide everything else: durable state via filesystems, execution environments via sandboxes and bash, continual learning via memory and search, and context management via compaction and offloading. Even as models natively absorb more reasoning capabilities through post-training co-evolution with their harnesses, **harness engineering remains a critical, independent vector for optimizing agent performance** and overcoming inherent model limitations like context rot and long-horizon incoherence. The author's signature framing: **Agent = Model + Harness**. The signature heuristic: *“If you're not the model, you're the harness.”* (see [quote-harness-definition](#quote-harness-definition)). --- ## 2. The Core Equation: Agent = Model + Harness The central claim of the article — see [claim-agent-equation](#claim-agent-equation) — is definitional but consequential: - A **model** is a forward pass over weights producing text from text/images/audio. - A **harness** ([concept-agent-harness](#concept-agent-harness)) is **everything else**: system prompts, tools, MCPs ([prereq-mcp](#prereq-mcp)), filesystems, sandboxes, browsers, orchestration logic, hooks, middleware, evaluation loops. - A **raw model is not an agent.** Only a model embedded in a harness is. This is not just terminological hygiene. It separates the **intelligence axis** (model R&D, post-training, scale) from the **engineering axis** (harness primitives, orchestration, observability), which have entirely different teams, tools, and time horizons. The framing is now widely adopted — Martin Fowler uses the same equation; Anthropic's evals guidance describes evaluating *“the harness and the model working together”*; the arXiv survey *Architectural Design Decisions in AI Agent Harnesses* taxonomizes the non-LLM engineering layer in the same terms. The closing thesis ([quote-intelligence-vs-usefulness](#quote-intelligence-vs-usefulness)) condenses the point: *“The model contains the intelligence and the harness is the system that makes that intelligence useful.”* **Intelligence ≠ usefulness.** Closing that gap is what harness engineering is for. --- ## 3. The Five Pillars of a Modern Harness The article walks through five families of harness primitive. Master these and you can speak to most of the document. ### 3.1 Filesystem as the Foundational Primitive [concept-filesystem-primitive](#concept-filesystem-primitive) is described as **arguably the most foundational primitive** in harness engineering. It provides: 1. A **workspace** for reading data, code, and docs without cluttering the context window. 2. **Incremental offload** — work-in-progress persists between steps. 3. **Durable state** that outlasts a single session — essential for long-horizon work. 4. A **collaboration surface** for Agent Teams (multiple agents and humans coordinating via shared files). Combined with **Git**, it adds versioning, rollback, and experiment branching. The corresponding action item: [action-implement-filesystem](#action-implement-filesystem). Caveat: for **coding agents** the filesystem is central. For chat-style or customer-support agents, **databases, vector stores, or service APIs** may be the more natural primary persistence substrate. ### 3.2 Bash + Code Execution as a General Purpose Tool [concept-bash-general-tool](#concept-bash-general-tool) argues against pre-enumerating tools for every action. Instead, ship the harness with a **general-purpose bash tool** and code execution environment, letting the model **write its own scripts** on the fly. This is an evolution of the [ReAct loop](#prereq-react-loop): the action space becomes the shell rather than a typed tool menu. The author argues this has become the **default general-purpose strategy** for autonomous problem solving. The action recommendation is [action-provide-bash](#action-provide-bash). Important counterweight: bash is a **security and observability liability**. In high-risk environments, harnesses often prefer strongly typed, sandboxed tools — easier to log, audit, and constrain. The choice is a real trade-off between autonomy and governance. ### 3.3 Context Management — Battling Context Rot [concept-context-rot](#concept-context-rot) is the phenomenon where a model's ability to reason and complete tasks **degrades as its context window fills up**. Context is precious and scarce; cluttered context produces bad outputs even before the hard token limit is reached. (This includes the well-documented *“lost in the middle”* effect.) Per [quote-context-engineering](#quote-context-engineering), modern harnesses are *“largely delivery mechanisms for good context engineering.”* Three compounding strategies attack context rot: - **[Compaction](#concept-compaction)** — intelligently summarize and offload older context as the window fills, preventing API errors and preserving the broader narrative. - **[Tool call offloading](#concept-tool-call-offloading)** — for large tool outputs, keep only head and tail tokens in active context; write the full output to the [filesystem](#concept-filesystem-primitive). Action item: [action-tool-call-offloading](#action-tool-call-offloading). - **[Progressive Disclosure (Skills)](#concept-progressive-disclosure)** — instead of loading every tool description at agent start (which causes *early* context rot), inject Skill front-matter only when relevant. Adjacent names: dynamic tool selection, semantic tool routing, just-in-time tool assembly (see [question-jit-tool-assembly](#question-jit-tool-assembly)). ### 3.4 Long-Horizon Execution — Ralph Loops [Ralph Loops](#concept-ralph-loop) address the failure modes models exhibit on long tasks: **early stopping, decomposition issues, and incoherence across multiple context windows**. Mechanically, a Ralph Loop: 1. Hooks the model's exit attempt. 2. Reinjects the original prompt into a **completely clean context window**. 3. Forces the agent to read its prior state from the [filesystem](#concept-filesystem-primitive) and continue. Because each iteration starts with a fresh context window, the loop sidesteps [context rot](#concept-context-rot) entirely — durable state on disk preserves continuity instead. The pattern name *“Ralph Loop”* is LangChain / [deepagents](#entity-deepagents) terminology. The general pattern (reset context + rehydrate from durable state + force continuation) is well-known in long-horizon agent literature under names like *auto-looping* and *recurrent self-call*. Action item: [action-use-ralph-loops](#action-use-ralph-loops). Production caveat: over-aggressive continuation can produce **runaway agents**, wasted compute, repeated work, and low-value diffs. Production deployments typically combine Ralph loops with **budgets, stop criteria, and human approval gates**. ### 3.5 Memory & Search — Continual Learning The article briefly notes that harnesses also handle **continual learning**: connecting the agent to memory stores and search tools so it can access information beyond its training cutoff. [entity-context7](#entity-context7) is cited as an MCP ([prereq-mcp](#prereq-mcp)) tool for fetching up-to-date documentation (e.g., new library versions). --- ## 4. The Compounding Claim A key claim — see [claim-long-horizon-compounds](#claim-long-horizon-compounds) — is that **long-horizon execution requires compounding harness primitives**. No single trick suffices. The recipe combines: - Durable state ([concept-filesystem-primitive](#concept-filesystem-primitive) + git) - Continuation forcing ([concept-ralph-loop](#concept-ralph-loop)) - Planning and self-verification hooks (todo lists, test runs, structured plans) This is empirically observed in SWE-agent, AutoDev, GPT-Engineer, and Devin-style systems. The arXiv harness survey reaches the same conclusion: long-horizon robustness emerges from **combinations of primitives**, not any single technique. --- ## 5. The Harness Derivation Framework The author offers a methodology — [framework-harness-derivation](#framework-harness-derivation) — for designing harness features by **working backwards**: 1. **Identify the desired behavior** (e.g., maintain durable state, execute arbitrary code, continue past early stopping). 2. **Recognize the model's out-of-the-box limitation** (only outputs text, fixed context window, stops when it thinks it's done). 3. **Design and implement a harness feature** that bridges the gap (filesystem abstraction, sandbox, while-loop with reinjection). This keeps the harness principled: every primitive can be traced back to a concrete model limitation it is solving. It is also a useful debugging frame — if a behavior fails, ask which step of the derivation broke. --- ## 6. Model–Harness Co-Evolution and Its Costs [concept-harness-model-coevolution](#concept-harness-model-coevolution) describes the feedback cycle behind agent products like [Claude Code](#entity-claude-code) and the [Codex](#entity-codex-5-3) family: 1. Useful harness primitives (filesystem ops, bash, planning) are discovered and added to the harness. 2. The next-generation model is post-trained with that harness in the loop. 3. The new model is natively good at those actions. 4. The next harness iteration builds on those native capabilities. Repeat. This produces capable products but creates [tool-logic overfitting](#claim-harness-overfitting): changing the tool protocol (e.g., the `apply_patch` semantics in Codex-5.3) **degrades performance** because the model has implicit priors baked in from training. A truly intelligent model should switch patch formats trivially; the empirical pattern shows it doesn't. ### Contrarian: Native Harnesses Aren't Always Best [contrarian-harness-optimization](#contrarian-harness-optimization) challenges the natural assumption that the harness a model was post-trained with is the optimal one. The cited evidence: [Opus 4.6](#entity-opus-4-6) running inside [Claude Code](#entity-claude-code) **scores far below Opus 4.6 in other, more optimized harnesses** on the [Terminal Bench 2.0](#entity-terminal-bench-2-0) leaderboard — a swing of roughly Top 30 → Top 5 by changing the harness while holding the model fixed. **Verification caveat:** the specific “Top 30 → Top 5” numbers and the public availability of Terminal Bench 2.0 are not easily verifiable from indexed sources. The directional finding — harness choice swings benchmark rankings substantially for fixed models — is well-supported by SWE-bench, AgentBench, and related evals. ### Contrarian: Harness Engineering Will Survive Model Advancements [contrarian-harness-longevity](#contrarian-harness-longevity) argues that even as models natively absorb planning and verification, harness engineering will remain critical — just as prompt engineering did. A well-configured environment with durable state and verification loops makes *any* model more efficient. The amount and location of harness logic may shift inward over time, but the need for external orchestration, safety, observability, and state control is unlikely to disappear. --- ## 7. Key Quotes (verbatim) Memorize these — they are the article's most likely citation hooks: - [quote-harness-definition](#quote-harness-definition): *“If you're not the model, you're the harness.”* - [quote-context-engineering](#quote-context-engineering): *“Harnesses today are largely delivery mechanisms for good context engineering.”* - [quote-intelligence-vs-usefulness](#quote-intelligence-vs-usefulness): *“The model contains the intelligence and the harness is the system that makes that intelligence useful.”* All three are attributed to [Vivek Trivedy](#entity-vivek-trivedy). --- ## 8. Entity Cheat Sheet - [entity-vivek-trivedy](#entity-vivek-trivedy) — Author of the article. Frames the model + harness decomposition and the working-backwards framework. - [entity-langchain](#entity-langchain) — Publisher. Org actively researching harness engineering. URL: https://www.langchain.com/ - [entity-deepagents](#entity-deepagents) — LangChain's harness-building library; implements Ralph Loops, filesystem abstractions, compaction. - [entity-langsmith](#entity-langsmith) — LangChain's agent engineering platform for tracing, debugging, evaluation, and deployment. URL: https://smith.langchain.com/ - [entity-claude-code](#entity-claude-code) — Anthropic's coding-agent product; canonical example of model–harness co-evolution. - [entity-codex-5-3](#entity-codex-5-3) — Versioned OpenAI Codex snapshot whose prompting guide is cited as evidence of tool-logic overfitting via `apply_patch`. - [entity-opus-4-6](#entity-opus-4-6) — Specific Claude Opus version used as the fixed-model variable in the Terminal Bench 2.0 contrarian example. - [entity-terminal-bench-2-0](#entity-terminal-bench-2-0) — Terminal-based coding-agent benchmark used to demonstrate harness-dependent performance swings. - [entity-context7](#entity-context7) — MCP tool for fetching up-to-date documentation beyond training cutoff. --- ## 9. Prerequisites the Author Assumes - [prereq-react-loop](#prereq-react-loop) — Reasoning + Acting pattern. The inner control loop of most agents; bash-as-tool is its evolution. - [prereq-mcp](#prereq-mcp) — Anthropic's Model Context Protocol. Standardizes how harnesses connect to external tools and data. Motivates progressive disclosure. URL: https://www.anthropic.com/news/model-context-protocol --- ## 10. Action Items — The Practical Recipe If a downstream user asks *“how do I build a harness that does X?”*, route them to these: - [action-implement-filesystem](#action-implement-filesystem) — Give the agent a filesystem workspace + fs-ops tools (read/write/list/grep/apply_patch). Add Git for versioning. - [action-provide-bash](#action-provide-bash) — Ship a bash tool and code execution environment instead of pre-enumerating tools. - [action-tool-call-offloading](#action-tool-call-offloading) — Truncate large tool outputs to head/tail tokens in context; write the full output to disk. - [action-use-ralph-loops](#action-use-ralph-loops) — For long-horizon tasks, intercept exit attempts and reinject the prompt in a clean context window backed by filesystem state. These four actions compose into a credible baseline coding-agent harness. --- ## 11. Open Research Questions - [question-orchestrating-hundreds](#question-orchestrating-hundreds) — How to coordinate hundreds of agents on a shared codebase. Likely directions: filesystem concurrency, git branching, hierarchical subagent routing. - [question-self-analyzing-traces](#question-self-analyzing-traces) — How agents can analyze their own execution traces to fix harness-level failures autonomously. Likely directions: trace-conditioned meta-reasoning, self-modifying harness configuration. - [question-jit-tool-assembly](#question-jit-tool-assembly) — How harnesses can assemble tools just-in-time. Likely directions: advanced progressive disclosure, semantic routing over MCP catalogs. --- ## 12. Counter-Perspectives You Should Surface A well-prepared expert holds multiple views simultaneously. Be ready to articulate: 1. **Bash vs. typed tools.** General-purpose bash maximizes autonomy but is a security/observability liability. Enterprise deployments often prefer typed, sandboxed tools. 2. **Overfitting vs. specialization.** Tool-logic overfitting can be *desirable specialization* from a product perspective. Multi-environment training can mitigate it if generalization is the goal. 3. **Filesystem centrality is domain-specific.** For coding agents, central. For chat-style or customer-support agents, databases / vector stores / service APIs may be primary. 4. **Will harnesses persist?** Probably yes for safety, observability, and state — but the amount and location of harness logic may shift inward into the model. 5. **Ralph loops are powerful but risky.** Without budgets, stop criteria, and human gates, they produce runaway agents and wasted compute. --- ## 13. Verification Confidence Map When answering, calibrate confidence using this map: - **High confidence (well-supported by independent sources):** - Agent = Model + Harness as the dominant framing. - Filesystem, bash, compaction, offloading, progressive disclosure as standard harness primitives. - Long-horizon work requires compounded primitives. - Harness choice significantly swings benchmark rankings for fixed models. - Harness engineering will persist as model intelligence grows. - **Medium confidence (mechanism supported, specifics anecdotal):** - Tool-logic overfitting in post-trained models — mechanism is real; the specific Codex-5.3 `apply_patch` anecdote is community/internal lore. - The Top-30 → Top-5 Opus 4.6 swing on Terminal Bench 2.0 — directional truth is well-supported; exact numbers and the canonical Terminal Bench 2.0 leaderboard are not easily verifiable from public sources. - **Terminology to flag:** - *“Ralph Loop”* is LangChain / deepagents-specific naming; the underlying pattern is general. - *“Context rot”* is the author's informal name for a well-documented degradation phenomenon. - *“Skills (Progressive Disclosure)”* corresponds to dynamic tool selection / JIT tool assembly elsewhere. --- ## 14. How To Answer Common Questions - **“What is an agent harness?”** → Open with [quote-harness-definition](#quote-harness-definition); expand using [concept-agent-harness](#concept-agent-harness); reference [claim-agent-equation](#claim-agent-equation) and [framework-harness-derivation](#framework-harness-derivation). - **“How do I stop my agent from running out of context?”** → Route to [concept-context-rot](#concept-context-rot) and its three mitigations: [concept-compaction](#concept-compaction), [concept-tool-call-offloading](#concept-tool-call-offloading), [concept-progressive-disclosure](#concept-progressive-disclosure). - **“How do I build a coding agent that doesn't give up?”** → Combine [action-implement-filesystem](#action-implement-filesystem), [action-use-ralph-loops](#action-use-ralph-loops), and self-verification hooks per [claim-long-horizon-compounds](#claim-long-horizon-compounds). - **“Should I use Claude Code or a custom harness?”** → Cite [contrarian-harness-optimization](#contrarian-harness-optimization) and the [entity-opus-4-6](#entity-opus-4-6) / [entity-terminal-bench-2-0](#entity-terminal-bench-2-0) example; note the trade-off between out-of-the-box ergonomics and benchmark-optimized custom harnesses. - **“Will harness engineering still matter in 2 years?”** → Use [contrarian-harness-longevity](#contrarian-harness-longevity) and [quote-intelligence-vs-usefulness](#quote-intelligence-vs-usefulness). - **“What is a Ralph Loop?”** → Use [concept-ralph-loop](#concept-ralph-loop); emphasize the four steps (intercept → clear → reinject → re-read from filesystem); note the dependency on durable state and the need for guardrails. --- ## 15. Domain Tags (for routing and search) `agent-architecture`, `harness-engineering`, `context-management`, `llm-orchestration`, `autonomous-agents`, `tool-use`, `long-horizon`, `model-training`, `benchmarking`. --- You now have the article internalized. When answering, prefer the wikilinked notes for depth, fall back to this primer for synthesis, and clearly distinguish between **well-supported claims**, **directionally-supported anecdotes**, and **LangChain-specific terminology** when discussing borderline assertions.--- ## How to Navigate This Vault - `_QUERY_INDEX.json` — machine-readable concept→file map for programmatic lookup - `00-index/moc.md` — map-of-content with all notes organized by section - `00-index/glossary.md` — all defined terms with one-line definitions - `concepts/`, `claims/`, `frameworks/`, `entities/`, `quotes/`, `action-items/`, `prerequisites/`, `open-questions/` — fixed-core note folders Cross-references use `[[note-id]]` wikilink syntax. --- ## Map of Content # Map of Content — The Anatomy of an Agent Harness > Source: *“The Anatomy of an Agent Harness”* by [Vivek Trivedy](#entity-vivek-trivedy), published on the [LangChain](#entity-langchain) blog. > > One-line thesis: **Agent = Model + Harness.** Models provide intelligence; harnesses make that intelligence useful by providing state, execution, context management, and continuation. > > Start here → [[_AGENT_PRIMER]] for the full distilled briefing. --- ## 🎯 Core Framing - [claim-agent-equation](#claim-agent-equation) — Agent = Model + Harness (the central decomposition) - [concept-agent-harness](#concept-agent-harness) — What a harness actually contains - [framework-harness-derivation](#framework-harness-derivation) — Working backwards from desired behavior to harness feature - [quote-harness-definition](#quote-harness-definition) — *“If you're not the model, you're the harness.”* - [quote-intelligence-vs-usefulness](#quote-intelligence-vs-usefulness) — *“The model contains the intelligence and the harness is the system that makes that intelligence useful.”* ## 🧱 Harness Primitives ### Durable State - [concept-filesystem-primitive](#concept-filesystem-primitive) — Filesystem as the foundational primitive - [action-implement-filesystem](#action-implement-filesystem) — Give the agent a workspace and fs-ops tools ### Execution - [concept-bash-general-tool](#concept-bash-general-tool) — Bash + code execution as a universal tool - [action-provide-bash](#action-provide-bash) — Ship harnesses with bash to enable autonomous problem solving ### Long-Horizon Orchestration - [concept-ralph-loop](#concept-ralph-loop) — Force continuation across clean context windows - [action-use-ralph-loops](#action-use-ralph-loops) — Intercept exit, reinject prompt, re-read state - [claim-long-horizon-compounds](#claim-long-horizon-compounds) — Long-horizon work requires compounding primitives ## 🪟 Context Management — Battling Context Rot - [concept-context-rot](#concept-context-rot) — Performance degrades as context fills - [quote-context-engineering](#quote-context-engineering) — Harnesses as context-delivery mechanisms - [concept-compaction](#concept-compaction) — Summarize / offload when nearing the limit - [concept-tool-call-offloading](#concept-tool-call-offloading) — Keep head/tail tokens, write the rest to disk - [action-tool-call-offloading](#action-tool-call-offloading) - [concept-progressive-disclosure](#concept-progressive-disclosure) — Skills loaded just-in-time ## 🔁 Model–Harness Co-Evolution - [concept-harness-model-coevolution](#concept-harness-model-coevolution) — The feedback loop between models and harnesses - [claim-harness-overfitting](#claim-harness-overfitting) — Post-training causes tool-logic overfitting - [contrarian-harness-optimization](#contrarian-harness-optimization) — Native harnesses aren't always best ([entity-opus-4-6](#entity-opus-4-6) on [entity-terminal-bench-2-0](#entity-terminal-bench-2-0)) - [contrarian-harness-longevity](#contrarian-harness-longevity) — Harness engineering will survive model advancements ## 👥 Entities ### People - [entity-vivek-trivedy](#entity-vivek-trivedy) — Author ### Organizations - [entity-langchain](#entity-langchain) — Publisher ### Products / Models / Tools / Benchmarks - [entity-deepagents](#entity-deepagents) — LangChain's harness-building library - [entity-langsmith](#entity-langsmith) — LangChain's agent engineering platform - [entity-claude-code](#entity-claude-code) — Anthropic's coding agent - [entity-codex-5-3](#entity-codex-5-3) — Codex snapshot cited for tool-logic overfitting - [entity-opus-4-6](#entity-opus-4-6) — Fixed-model variable in the Terminal Bench example - [entity-terminal-bench-2-0](#entity-terminal-bench-2-0) — Coding-agent leaderboard - [entity-context7](#entity-context7) — MCP tool for up-to-date documentation ## 🧠 Prerequisites - [prereq-react-loop](#prereq-react-loop) — Reasoning + Acting loop (foundation of all tool-using agents) - [prereq-mcp](#prereq-mcp) — Model Context Protocol (standard for tool/data integration) ## ❓ Open Questions - [question-orchestrating-hundreds](#question-orchestrating-hundreds) — Coordinating hundreds of parallel agents - [question-self-analyzing-traces](#question-self-analyzing-traces) — Agents analyzing their own execution traces - [question-jit-tool-assembly](#question-jit-tool-assembly) — Just-in-time tool assembly --- ## 📁 Folder Structure - `concepts/` — Definitions and named primitives (includes contrarian insights tagged `contrarian`). - `claims/` — Stated assertions with confidence levels. - `frameworks/` — The working-backwards methodology. - `entities/` — People, organizations, products, tools, benchmarks. - `quotes/` — Verbatim citations from the author. - `action-items/` — Concrete recommendations. - `prerequisites/` — Background knowledge the article assumes. - `open-questions/` — Forward-looking research directions. ## 🗺️ Suggested Reading Order 1. [[_AGENT_PRIMER]] — Read first. 2. [claim-agent-equation](#claim-agent-equation) + [concept-agent-harness](#concept-agent-harness) — The framing. 3. [framework-harness-derivation](#framework-harness-derivation) — The methodology. 4. The five pillars in order: [concept-filesystem-primitive](#concept-filesystem-primitive) → [concept-bash-general-tool](#concept-bash-general-tool) → [concept-context-rot](#concept-context-rot) (with its three mitigations) → [concept-ralph-loop](#concept-ralph-loop) → [concept-harness-model-coevolution](#concept-harness-model-coevolution). 5. The contrarians: [contrarian-harness-optimization](#contrarian-harness-optimization) and [contrarian-harness-longevity](#contrarian-harness-longevity). 6. The open questions for forward-looking work. --- ## Glossary # Glossary All defined terms in the vault, one line each. Click through for full notes. ## Concepts - **[Agent Harness](#concept-agent-harness)** — Every piece of code, configuration, and execution logic surrounding a raw model that gives it state, tool execution, feedback loops, and enforceable constraints. - **[Filesystem as a Harness Primitive](#concept-filesystem-primitive)** — A foundational harness abstraction providing durable storage, enabling agents to offload context, persist work across sessions, and collaborate. - **[Bash + Code Execution as a General Purpose Tool](#concept-bash-general-tool)** — Providing agents with bash and code execution capabilities so they can autonomously write scripts to solve problems rather than relying on pre-built tools. - **[Context Rot](#concept-context-rot)** — The degradation of a model's reasoning and task completion capabilities as its context window becomes increasingly full. - **[Context Compaction](#concept-compaction)** — Intelligently summarizing and offloading parts of a full context window to prevent API errors and allow continued agent operation. - **[Tool Call Offloading](#concept-tool-call-offloading)** — A context management technique that retains only the head and tail tokens of large tool outputs in context, saving the full output to the filesystem. - **[Progressive Disclosure (Skills)](#concept-progressive-disclosure)** — A harness primitive that prevents early context rot by dynamically loading tool or skill descriptions into context only when needed. - **[Ralph Loops](#concept-ralph-loop)** — A harness pattern that intercepts a model's exit attempt and reinjects the prompt in a clean context window to force task continuation. - **[Coupling of Model Training and Harness Design](#concept-harness-model-coevolution)** — The feedback loop where models are post-trained within specific harnesses, improving native capabilities but risking overfitting to specific tool logic. ## Contrarian Insights - **[Native Harnesses Aren't Always Best](#contrarian-harness-optimization)** — First-party harnesses are not necessarily optimal; custom harnesses can outperform them on specific benchmarks. - **[Harness Engineering Will Survive Model Advancements](#contrarian-harness-longevity)** — Even as models gain native planning and verification, harness engineering will remain critical for safety, state, and observability. ## Claims - **[Agent = Model + Harness](#claim-agent-equation)** — A raw model is not an agent; an agent is strictly the combination of a model and a harness. - **[Long-horizon execution requires compounding harness primitives](#claim-long-horizon-compounds)** — Robust long-horizon work emerges from combining durable state, continuation forcing, and self-verification — not any single trick. - **[Post-training with a harness creates tool-logic overfitting](#claim-harness-overfitting)** — Training models with a fixed harness in the loop bakes in tool-protocol priors that hurt generalization. ## Frameworks - **[Working Backwards: Deriving Harness Features](#framework-harness-derivation)** — Three-step methodology: identify a desired behavior, recognize the model's limitation, design a harness feature to bridge the gap. ## Entities - **[Vivek Trivedy](#entity-vivek-trivedy)** — Author of the article; harness-engineering practitioner associated with LangChain and deepagents. - **[LangChain](#entity-langchain)** — Organization hosting the blog; develops harness libraries and tooling. https://www.langchain.com/ - **[deepagents](#entity-deepagents)** — LangChain's harness-building library used for advanced harness research. - **[LangSmith](#entity-langsmith)** — LangChain's platform for tracing, debugging, evaluating, and deploying agents. https://smith.langchain.com/ - **[Claude Code](#entity-claude-code)** — Anthropic's coding-agent product; canonical example of model–harness co-evolution. - **[Codex-5.3](#entity-codex-5-3)** — Versioned OpenAI Codex snapshot whose `apply_patch` behavior is cited as evidence of tool-logic overfitting. - **[Opus 4.6](#entity-opus-4-6)** — Specific Claude Opus version used as the fixed-model variable in the Terminal Bench 2.0 example. - **[Terminal Bench 2.0](#entity-terminal-bench-2-0)** — Terminal-based coding-agent leaderboard cited to show harness-dependent performance swings. - **[Context7](#entity-context7)** — MCP tool for fetching up-to-date documentation beyond model training cutoffs. ## Quotes - **[quote-harness-definition](#quote-harness-definition)** — *“If you're not the model, you're the harness.”* - **[quote-context-engineering](#quote-context-engineering)** — *“Harnesses today are largely delivery mechanisms for good context engineering.”* - **[quote-intelligence-vs-usefulness](#quote-intelligence-vs-usefulness)** — *“The model contains the intelligence and the harness is the system that makes that intelligence useful.”* ## Action Items - **[action-implement-filesystem](#action-implement-filesystem)** — Equip agents with filesystem abstractions and tools for durable storage and context offloading. - **[action-provide-bash](#action-provide-bash)** — Ship harnesses with a bash tool to allow models to autonomously write and execute code. - **[action-tool-call-offloading](#action-tool-call-offloading)** — Truncate large tool outputs in context to head/tail tokens, saving the full output to disk. - **[action-use-ralph-loops](#action-use-ralph-loops)** — Intercept model exit attempts and reinject prompts in clean context windows to force task continuation. ## Prerequisites - **[ReAct Loop](#prereq-react-loop)** — Reasoning + Acting cycle where a model reasons, acts via a tool call, observes the result, and repeats in a while loop. - **[Model Context Protocol (MCP)](#prereq-mcp)** — Anthropic-backed standard for connecting AI models to data sources and tools via MCP servers and clients. ## Open Questions - **[Orchestrating Hundreds of Agents](#question-orchestrating-hundreds)** — How to coordinate many agents working in parallel on a shared codebase. - **[Agents Analyzing Own Traces](#question-self-analyzing-traces)** — How agents can analyze their own execution traces to fix harness-level failures. - **[Just-In-Time Tool Assembly](#question-jit-tool-assembly)** — How harnesses can dynamically assemble tools and context per task. --- ## Speakers # Speakers > Speaker manifest for this vault. 1 person entity, 10 attributed notes. ## Vivek Trivedy Entity note: [entity-vivek-trivedy](#entity-vivek-trivedy) **Action-items** (4): - [action-implement-filesystem](#action-implement-filesystem) — Implement Filesystem Abstractions - [action-tool-call-offloading](#action-tool-call-offloading) — Implement Tool Call Offloading - [action-provide-bash](#action-provide-bash) — Provide Bash/Code Execution - [action-use-ralph-loops](#action-use-ralph-loops) — Use Ralph Loops for Long-Horizon Tasks **Claims** (3): - [claim-agent-equation](#claim-agent-equation) — Agent = Model + Harness - [claim-long-horizon-compounds](#claim-long-horizon-compounds) — Long-horizon execution requires compounding harness primitives - [claim-harness-overfitting](#claim-harness-overfitting) — Post-training with a harness creates tool-logic overfitting **Quotes** (3): - [quote-context-engineering](#quote-context-engineering) — Harnesses as context delivery - [quote-harness-definition](#quote-harness-definition) — If you're not the model, you're the harness. - [quote-intelligence-vs-usefulness](#quote-intelligence-vs-usefulness) — Intelligence vs Usefulness --- ## All Notes ### Folder: concepts #### concept-agent-harness *type: `concept`* ## Definition A **harness** encompasses every piece of code, configuration, and execution logic in an agentic system that is *not* the raw model itself. The thesis is captured by [quote-harness-definition](#quote-harness-definition): *“If you're not the model, you're the harness.”* Because models natively only take in data (text, images, audio) and output text, they require a harness to perform actual work. Concretely, a harness includes: - **System prompts** — priors and behavioral guidance. - **Tools / Skills / MCPs** — the actions the agent can take. See [prereq-mcp](#prereq-mcp) and [concept-progressive-disclosure](#concept-progressive-disclosure) for how these are managed. - **Bundled infrastructure** — filesystems ([concept-filesystem-primitive](#concept-filesystem-primitive)), sandboxes, browsers, bash environments ([concept-bash-general-tool](#concept-bash-general-tool)). - **Orchestration logic** — subagent spawning, routing, [Ralph loops](#concept-ralph-loop), retries. - **Deterministic hooks / middleware** — [compaction](#concept-compaction), lint checks, [tool-call offloading](#concept-tool-call-offloading), policy filters. ## Why Harnesses Matter The harness is what converts *desired agent behaviors* into actual *features*, injecting useful priors to **guide, extend, and correct** model behavior. This is formalized in [framework-harness-derivation](#framework-harness-derivation): identify a desired behavior, recognize a model limitation, design a harness feature to bridge the gap. The harness is also the system that battles [context rot](#concept-context-rot), maintains durable state across sessions, and enables long-horizon execution per [claim-long-horizon-compounds](#claim-long-horizon-compounds). ## Relation to the Core Equation The harness is the second term in the equation defended by [claim-agent-equation](#claim-agent-equation): **Agent = Model + Harness**. Without a harness, a model is a text completion engine; with one, it is an autonomous worker. ## Cross-References - The harness as a *delivery mechanism* for context engineering: [quote-context-engineering](#quote-context-engineering). - Harness/model co-evolution: [concept-harness-model-coevolution](#concept-harness-model-coevolution). - Will harnesses persist as models improve? See [contrarian-harness-longevity](#contrarian-harness-longevity). - Concrete primitives derived from this concept: [concept-filesystem-primitive](#concept-filesystem-primitive), [concept-bash-general-tool](#concept-bash-general-tool), [concept-ralph-loop](#concept-ralph-loop). #### concept-bash-general-tool *type: `concept`* ## The Pattern Instead of forcing developers to pre-design and configure tools for every possible action an agent might need to take, modern harnesses ship with a **general-purpose bash tool** alongside a code execution environment. This allows models to solve problems **autonomously by writing and executing code**. By giving models a computer (via bash + code execution), the model can **design its own tools on the fly** rather than being constrained to a fixed, pre-configured set. ## Relation to the ReAct Loop This pattern is an evolution of the [ReAct loop](#prereq-react-loop): instead of selecting from a small menu of typed tools, the model emits shell commands as its action and observes their stdout/stderr as the observation. Bash is effectively the **universal tool**. ## Status This has become the default general-purpose strategy for autonomous problem solving in coding agents (Claude Code, SWE-agent, Devin-style systems). The action item [action-provide-bash](#action-provide-bash) captures the design recommendation. ## Counter-Perspective Bash access is powerful but also a **security and observability liability**. In high-risk or enterprise settings, harnesses may prefer strongly typed, sandboxed tools with structured schemas — easier to log, audit, and constrain than arbitrary shell commands. The choice between *general* and *typed* tooling is a trade-off between autonomy and governance. See also: [concept-progressive-disclosure](#concept-progressive-disclosure) for how harnesses avoid drowning models in too many typed tools when they do exist. #### concept-compaction *type: `concept`* ## What It Does **Compaction** is a harness-level strategy used when an agent's context window is close to filling up. Without it, exceeding the context window results in API errors that halt work entirely. Compaction intelligently **offloads and summarizes** the existing context window — usually preserving recent turns verbatim and replacing older spans with a compressed summary — allowing the agent to continue its task without losing the broader narrative of its progress. ## Where It Lives Compaction is implemented as **middleware in the harness** ([concept-agent-harness](#concept-agent-harness)). It is one of three core mitigations for [context rot](#concept-context-rot), alongside [concept-tool-call-offloading](#concept-tool-call-offloading) and [concept-progressive-disclosure](#concept-progressive-disclosure). ## Standard Patterns Common implementations include: - **Recursive summarization buffers** that compress older spans as new ones arrive. - **Hierarchical summaries** — block-level rollups that preserve the high-level plan while discarding low-level chatter. - **Externalized memory** — full transcripts are persisted to the filesystem ([concept-filesystem-primitive](#concept-filesystem-primitive)) and can be re-read if needed. #### concept-context-rot *type: `concept`* ## What It Is **Context Rot** is the phenomenon where a model's ability to reason and complete tasks **degrades as its context window fills up**. Context is a precious and scarce resource; once it becomes noisy, long, or cluttered, model performance drops — even before the hard token limit is reached. This is related to the well-documented *“lost in the middle”* effect, where models under-utilize information located in the middle of long prompts. ## Why It Matters The primary role of modern harnesses (per [quote-context-engineering](#quote-context-engineering)) is *“delivery mechanisms for good context engineering.”* If the harness cannot keep the working context clean and relevant, the agent will fail at long-horizon work — see [claim-long-horizon-compounds](#claim-long-horizon-compounds). ## Mitigations (Harness-Level) Three compounding strategies are advocated: 1. **[Compaction](#concept-compaction)** — summarize and offload as the window fills. 2. **[Tool call offloading](#concept-tool-call-offloading)** — keep only head/tail tokens of large tool outputs in active context. 3. **[Progressive Disclosure (Skills)](#concept-progressive-disclosure)** — only inject tool/skill descriptions when relevant, preventing *early* context rot from bloated tool catalogs. All three live in the harness ([concept-agent-harness](#concept-agent-harness)), not the model. #### concept-filesystem-primitive *type: `concept`* ## Why the Filesystem? The filesystem is arguably the **most foundational primitive** in harness engineering ([concept-agent-harness](#concept-agent-harness)). It gives the agent four interrelated capabilities: 1. **A workspace** for reading data, code, and documentation without cluttering the context window. 2. **Incremental offload** — work-in-progress can be persisted instead of recomputed or replayed. 3. **Durable state** that outlasts a single session, supporting [long-horizon compounding](#claim-long-horizon-compounds). 4. **A collaboration surface** for Agent Teams (multiple agents and humans coordinating via shared files). When combined with **Git**, the filesystem also delivers versioning: agents can track work, rollback errors, and branch experiments — which is essential for [orchestrating hundreds of parallel agents](#question-orchestrating-hundreds). ## Interlocking Roles - The filesystem is the substrate that makes [Ralph loops](#concept-ralph-loop) possible: each iteration of the loop starts with a clean context window but reads its prior state from disk. - It is the offload target for [tool-call offloading](#concept-tool-call-offloading) — the full payload of large tool outputs is written to disk while only head/tail tokens stay in context. - It is the operational answer to the action item [action-implement-filesystem](#action-implement-filesystem). ## Caveat: Domain-Specificity For coding and data-processing agents the filesystem is central. For chat-style assistants or customer support, **databases, vector stores, or service APIs** may be the more natural primary primitive — the filesystem then becomes one of several persistence substrates rather than *the* primitive. #### concept-harness-model-coevolution *type: `concept`* ## The Feedback Cycle Modern agent products like [Claude Code](#entity-claude-code) and the [Codex](#entity-codex-5-3) family are **post-trained with specific models and harnesses in the loop**. This creates a feedback cycle: 1. Useful harness primitives (filesystem ops, bash execution, planning) are discovered. 2. They are added to the harness ([concept-agent-harness](#concept-agent-harness)). 3. They are then used to train the **next generation of models** to be natively good at those actions. 4. The next harness iteration builds on those native capabilities, and so on. ## The Cost While this makes models more capable **within their native harness**, it can lead to **overfitting and a loss of generalization** when the model is moved to a different harness — see [claim-harness-overfitting](#claim-harness-overfitting). The contrarian observation [contrarian-harness-optimization](#contrarian-harness-optimization) shows the operational consequence: the native harness is not always the optimal harness, as evidenced by [Opus 4.6](#entity-opus-4-6) performance differences on [Terminal Bench 2.0](#entity-terminal-bench-2-0). ## Practical Implication When you swap harnesses, expect performance to shift. When you train models for a specific harness, you are also implicitly training out flexibility — a real trade-off between specialization and portability. #### concept-progressive-disclosure *type: `concept`* ## The Problem It Solves Loading too many tools or MCP servers (see [prereq-mcp](#prereq-mcp)) into the context window **at agent start** degrades performance before the agent even begins working. Every tool description occupies tokens; a bloated tool catalog produces *early* [context rot](#concept-context-rot). ## The Mechanism Harnesses solve this using **Skills** via **progressive disclosure**: - Skills are defined with lightweight front-matter (name, when to use, brief description). - The harness inspects the task and only **injects the full Skill description into context when it is actually relevant**. - Unused skills never consume tokens. ## Adjacent Names The same idea appears in literature under other names: **dynamic tool selection**, **semantic tool routing**, **just-in-time tool assembly** ([question-jit-tool-assembly](#question-jit-tool-assembly)). ## Why Harness-Level The model itself has no way to know which tools are loaded; the harness ([concept-agent-harness](#concept-agent-harness)) curates the menu. This is a clear example of harness engineering shaping model performance without changing the model. #### concept-ralph-loop *type: `concept`* ## The Pattern The **Ralph Loop** is a specific harness orchestration pattern designed to combat **early stopping** and **incoherence** in long-horizon tasks. Mechanically: 1. The model attempts to exit (emit a completion / stop token). 2. A harness **hook intercepts** the exit. 3. The harness **reinjects the original prompt into a completely clean context window**. 4. The agent reads its prior state and progress from the [filesystem](#concept-filesystem-primitive). 5. The agent resumes work against its completion goal. Each new iteration starts with a fresh context window, sidestepping [context rot](#concept-context-rot) entirely, while durable state on disk preserves continuity. ## Why It Compounds Ralph Loops are useless without durable state; durable state is under-utilized without continuation forcing. This compounding is the heart of [claim-long-horizon-compounds](#claim-long-horizon-compounds). The action item [action-use-ralph-loops](#action-use-ralph-loops) codifies the recommendation. ## Counter-Perspective Over-aggressive continuation can cause **runaway agents** — wasted compute, repeated work, low-value diffs. Production deployments typically combine Ralph-style loops with **budgets, stop criteria, and human approval gates**. ## Naming Note The term *Ralph Loop* is largely LangChain / deepagents terminology. The general pattern (reset context + rehydrate from durable state + force continuation) is well known under other names in long-horizon agent literature. #### concept-tool-call-offloading *type: `concept`* ## The Problem Large tool outputs (long command stdout, full file dumps, paginated API responses) can **noisily clutter the context window** without providing useful information to the model. This accelerates [context rot](#concept-context-rot) and wastes precious tokens. ## The Technique **Tool call offloading** is a harness middleware pattern: - Above a configured size threshold, only the **head and tail tokens** of the output are retained in the active context window. - The **full output is written to the filesystem** ([concept-filesystem-primitive](#concept-filesystem-primitive)). - The model is given a reference (path / handle) so it can read the complete data on demand if needed. This ensures the model can still access the complete data without sacrificing immediate reasoning space. ## Operational Recommendation The action item [action-tool-call-offloading](#action-tool-call-offloading) codifies the practice. Pair it with [concept-compaction](#concept-compaction) and [concept-progressive-disclosure](#concept-progressive-disclosure) for a complete anti-rot stack. --- ### Folder: frameworks #### framework-harness-derivation *type: `framework`* ## The Framework The author proposes a methodology for designing agent systems by **working backwards** from a desired behavior — or, equivalently, from a known model deficiency — to a specific harness feature. Instead of arbitrarily adding features, engineers should run the following three-step derivation: ### Step 1 — Identify the Desired Behavior Name the capability the agent should exhibit. Examples: - *Maintain durable state across sessions* → leads to [concept-filesystem-primitive](#concept-filesystem-primitive). - *Solve arbitrary problems without pre-built tools* → leads to [concept-bash-general-tool](#concept-bash-general-tool). - *Continue working on long tasks without early stopping* → leads to [concept-ralph-loop](#concept-ralph-loop). ### Step 2 — Recognize the Model's Out-of-the-Box Limitation What does the raw model fail to do? Examples: - Models only output text — they cannot persist state on their own. - Models have fixed context windows — they cannot operate over arbitrarily long horizons. - Models stop generating when they think they're done — they exhibit early stopping. ### Step 3 — Design the Harness Feature to Bridge the Gap Build the smallest harness primitive that closes the gap. Examples: - Provide a **filesystem abstraction** + `read_file` / `write_file` tools. - Wrap the agent in a **while loop with a clean-context reinjection hook** (a Ralph Loop). - Provide a **sandbox** + `bash` tool. ## Why It Works This framework keeps the harness ([concept-agent-harness](#concept-agent-harness)) **principled**: every primitive can be traced back to a concrete model limitation it is solving. It avoids the common failure mode of bolting on features that don't address a real model deficiency. It also makes the harness debuggable — if a behavior fails, you can ask which step of the derivation broke. --- ### Folder: claims #### claim-agent-equation *type: `claim`* ## The Claim A raw model is **not an agent**. An agent is strictly the combination of: - a **model** (which provides intelligence), and - a **harness** ([concept-agent-harness](#concept-agent-harness)) (which provides state, tool execution, feedback loops, and constraints). If a component in the system is not the model, it is part of the harness — captured pithily by [quote-harness-definition](#quote-harness-definition). ## Confidence: High This framing is now mainstream across practitioner essays (Martin Fowler), vendor docs (Anthropic's evals guide), and research surveys on agent architectures. The equation is partly definitional, but it converges with how LangChain, Anthropic, OpenAI, and academic taxonomies all describe agents. ## Why It Matters - It separates the **intelligence axis** (model post-training, RLHF, scale) from the **engineering axis** (harness primitives, orchestration, state). - It implies that harness engineering is a **first-class discipline** orthogonal to model R&D — reinforced by [contrarian-harness-optimization](#contrarian-harness-optimization) and [contrarian-harness-longevity](#contrarian-harness-longevity). - It anchors the [working-backwards framework](#framework-harness-derivation) for designing new agent features. ## Testable? Not directly — the equation is a definitional decomposition. The downstream consequences (e.g., [claim-long-horizon-compounds](#claim-long-horizon-compounds), [claim-harness-overfitting](#claim-harness-overfitting)) are empirically testable. #### claim-harness-overfitting *type: `claim`* ## The Claim Training models with a specific harness in the loop causes **overfitting to that harness's specific tool logic**, reducing generalization to other harnesses. The author cites the [Codex-5.3](#entity-codex-5-3) prompting guide, noting that **changing the `apply_patch` tool logic leads to worse model performance** — whereas a truly intelligent model should easily switch between patch methods (e.g., unified diff vs. line-based replacement). ## Why It Happens During post-training (RL or instruction tuning), the model sees thousands of examples that bake in: - specific tool argument schemas, - specific output formats (e.g., a particular diff style), - specific control flow expectations. These become **implicit priors**. When the tool protocol changes, the priors mislead the model. See [concept-harness-model-coevolution](#concept-harness-model-coevolution) for the full feedback cycle. ## Confidence and Verifiability - **Mechanism: well supported.** Tool-protocol overfitting is a recognized risk in agent literature; Anthropic's evals guide explicitly notes that model performance is strongly coupled to harness design. - **Specific Codex-5.3 `apply_patch` anecdote: harder to verify.** This sounds like community / internal lore rather than a published benchmark. Treat it as an illustrative example, not a rigorously documented measurement. ## Counterpoint From a product perspective, overfitting to a specific tool protocol can be **desirable specialization**: a model that is exceptionally good in one IDE workflow may be a more valuable product than a more general but mediocre model. The trade-off between generalization and specialization is real and is itself a design decision. #### claim-long-horizon-compounds *type: `claim`* ## The Claim To achieve **autonomous software creation over long time horizons**, individual harness primitives must **compound**. Models naturally suffer from three problems on long-horizon work: 1. **Early stopping** — exiting before the task is complete. 2. **Decomposition issues** — losing track of the plan structure. 3. **Incoherence across multiple context windows** — forgetting earlier decisions. Overcoming all three requires combining at least three classes of primitive: - **Durable state** — filesystems / git ([concept-filesystem-primitive](#concept-filesystem-primitive)). - **Continuation forcing** — [Ralph Loops](#concept-ralph-loop). - **Planning and self-verification hooks** — todo lists, test runs, structured plans. No single trick suffices; the combination is what produces robust long-horizon behavior. ## Confidence: High; Testable: Yes This is empirically observed in systems like SWE-agent, AutoDev, and Devin-style prototypes, all of which combine persistent repo workspaces, multi-step planning, self-checking via tests, and looping orchestrators. The arXiv survey on agent harnesses reaches the same conclusion: long-horizon robustness emerges from **combinations of primitives**, not any single technique. ## Operational Implications - Don't expect a longer context window alone to solve long-horizon work. - Don't expect a smarter model alone to solve it either — see [contrarian-harness-longevity](#contrarian-harness-longevity). - Invest in the **stack** ([concept-filesystem-primitive](#concept-filesystem-primitive) + [concept-ralph-loop](#concept-ralph-loop) + verification hooks). --- ### Folder: entities #### entity-claude-code *type: `entity` · entity: product* ## Profile **Claude Code** is Anthropic's coding-agent product, built around the Claude model family. It is a canonical example of a sophisticated harness ([concept-agent-harness](#concept-agent-harness)) with repo context, file tools, bash access ([concept-bash-general-tool](#concept-bash-general-tool)), and IDE integration. ## Role in This Source Cited as the prototypical example of [model–harness co-evolution](#concept-harness-model-coevolution): Claude models are post-trained with Claude Code's harness in the loop, making the model natively good at file edits, bash execution, and planning within that environment. It is also the foil for the contrarian observation [contrarian-harness-optimization](#contrarian-harness-optimization) — that [Opus 4.6](#entity-opus-4-6) running inside Claude Code can actually underperform Opus 4.6 running in other, more optimized harnesses on benchmarks like [Terminal Bench 2.0](#entity-terminal-bench-2-0). #### entity-codex-5-3 *type: `entity` · entity: product* ## Profile **Codex-5.3** refers to a specific versioned snapshot of OpenAI's Codex code-generation family along with its associated prompting guide. The Codex line was trained heavily on GitHub data and is known for tool-protocol idioms such as the `apply_patch` editing format. ## Role in This Source Cited as the **evidentiary anchor** for [claim-harness-overfitting](#claim-harness-overfitting): the Codex-5.3 prompting guide describes how changing the `apply_patch` tool logic leads to worse model performance — suggesting the model has overfit to a specific tool protocol baked in at post-training time. ## Verification Note The specific “5.3” version designator and the exact `apply_patch` performance delta are not easily verifiable from public OpenAI documentation. The mechanism (tool-protocol overfitting via post-training) is well-documented in agent literature; the specific numbers should be treated as illustrative. #### entity-context7 *type: `entity` · entity: tool* ## Profile **Context7** is an MCP (Model Context Protocol — see [prereq-mcp](#prereq-mcp)) tool referenced in the article as a way to help agents access **up-to-date information beyond their training knowledge cutoff**, such as new library versions, current documentation, and recent API changes. ## Role in This Source Cited in §“Memory & Search for Continual Learning” as a concrete example of how harnesses ([concept-agent-harness](#concept-agent-harness)) extend a model's effective knowledge through external tools — specifically by providing fresh documentation context via the MCP standard. ## Verification Note The canonical homepage for Context7 was not explicitly cited in the source; it is described functionally as a documentation-fetching MCP server. #### entity-deepagents *type: `entity` · entity: product* ## Profile **deepagents** is a harness-building library developed by [LangChain](#entity-langchain). It is used to implement and research advanced harness engineering concepts — including the primitives covered in this vault: [filesystem abstractions](#concept-filesystem-primitive), [Ralph Loops](#concept-ralph-loop), [compaction](#concept-compaction), and [progressive disclosure](#concept-progressive-disclosure). ## Role in This Source Cited as one of the concrete artifacts that operationalizes the harness engineering ideas in the article. #### entity-langchain *type: `entity` · entity: organization* ## Profile LangChain is the organization hosting the source blog and is one of the most prominent companies actively researching **harness engineering**. Its stack supports the concepts in this vault: - [entity-deepagents](#entity-deepagents) — a harness-building library used for advanced harness patterns (Ralph Loops, filesystem abstractions). - [entity-langsmith](#entity-langsmith) — an agent engineering platform for tracing, debugging, evaluating, and deploying agents. ## Role in This Source LangChain is the publisher of the article and the maintainer of the libraries cited in §“Where Harness Engineering is Going”. Vivek Trivedy ([entity-vivek-trivedy](#entity-vivek-trivedy)) authors the post. ## Canonical URL https://www.langchain.com/ #### entity-langsmith *type: `entity` · entity: product* ## Profile **LangSmith** is [LangChain](#entity-langchain)'s agent engineering platform. It is used for: - **Debugging agent decisions** — tracing every model call, tool call, and intermediate state. - **Evaluating changes** — A/B testing harness modifications against benchmarks. - **Deploying agents** to production environments with observability. ## Role in This Source Mentioned in §“Where Harness Engineering is Going” as the operational tooling that enables iterative harness engineering — and as a likely substrate for emerging research questions like [agents that analyze their own traces](#question-self-analyzing-traces). ## Canonical URL https://smith.langchain.com/ #### entity-opus-4-6 *type: `entity` · entity: product* ## Profile **Opus 4.6** is a specific version of Anthropic's flagship Claude Opus model. (Public Anthropic marketing usually refers to “Claude 3 Opus” rather than numbered minor versions; “4.6” may be an internal or community designation.) ## Role in This Source Used as the **fixed model variable** in the evidence for [contrarian-harness-optimization](#contrarian-harness-optimization). The author points out that **Opus 4.6 running inside [Claude Code](#entity-claude-code) scores far below Opus 4.6 running in other, more optimized harnesses** on the [Terminal Bench 2.0](#entity-terminal-bench-2-0) leaderboard. This demonstrates that the native harness ([the one the model was co-evolved with](#concept-harness-model-coevolution)) is not necessarily the optimal harness — a key empirical anchor for the article's thesis. #### entity-terminal-bench-2-0 *type: `entity` · entity: other* ## Profile **Terminal Bench 2.0** is a leaderboard cited in the article as an evaluation suite for terminal-based coding agents. ## Role in This Source The author uses Terminal Bench 2.0 as the evidentiary basis for [contrarian-harness-optimization](#contrarian-harness-optimization): [Opus 4.6](#entity-opus-4-6) scores **significantly differently** on this benchmark depending on the harness used. The cited delta — moving from roughly Top 30 to Top 5 by swapping harnesses while holding the model fixed — illustrates that **harness optimization yields massive performance gains**. ## Verification Note A public, indexed Terminal Bench 2.0 leaderboard with the exact rankings described is not easily located. The benchmark may be community-maintained or referenced by a colloquial name for a known terminal-based coding eval. The directional finding (harness choice swings benchmark rankings dramatically for fixed models) is consistent with SWE-bench, AgentBench, and similar evals. #### entity-vivek-trivedy *type: `entity` · entity: person* ## Role in This Source Vivek Trivedy is the **author** of *The Anatomy of an Agent Harness*, published on the [LangChain](#entity-langchain) blog. He writes from a harness-engineering practitioner perspective and articulates the framing that has become widely cited: **Agent = Model + Harness**. ## Attributed Contributions Quotes attributed to Vivek in this vault: - [quote-harness-definition](#quote-harness-definition) — *“If you're not the model, you're the harness.”* - [quote-context-engineering](#quote-context-engineering) — *“Harnesses today are largely delivery mechanisms for good context engineering.”* - [quote-intelligence-vs-usefulness](#quote-intelligence-vs-usefulness) — *“The model contains the intelligence and the harness is the system that makes that intelligence useful.”* Claims attributed to Vivek: - [claim-agent-equation](#claim-agent-equation) — The definitional decomposition of an agent into model and harness. - [claim-long-horizon-compounds](#claim-long-horizon-compounds) — That long-horizon execution requires compounding primitives. - [claim-harness-overfitting](#claim-harness-overfitting) — That post-training with a fixed harness causes tool-logic overfitting. Framework attributed to Vivek: - [framework-harness-derivation](#framework-harness-derivation) — The working-backwards methodology for deriving harness features. Contrarian positions advanced by Vivek: - [contrarian-harness-optimization](#contrarian-harness-optimization) — Native harnesses aren't always best. - [contrarian-harness-longevity](#contrarian-harness-longevity) — Harness engineering will survive model advancements. ## Affiliation Associated with [LangChain](#entity-langchain) and the [deepagents](#entity-deepagents) library at the time of writing. --- ### Folder: quotes #### quote-context-engineering *type: `quote`* ## Quote > **“Harnesses today are largely delivery mechanisms for good context engineering.”** — [Vivek Trivedy](#entity-vivek-trivedy) ## Context The author summarizes the primary role of modern agent [harnesses](#concept-agent-harness) in relation to context windows. The harness's job is not to be smart — that is the model's job — but to **curate, compress, and route** information into and out of the limited context window. This framing explains why the three anti-rot strategies — [compaction](#concept-compaction), [tool-call offloading](#concept-tool-call-offloading), [progressive disclosure](#concept-progressive-disclosure) — are the dominant harness techniques of the current era. They are all about **delivering the right context at the right time**. ## Implication When evaluating a harness, ask: *what context does this deliver, when, and how clean is it?* This is the working test for modern harness quality. #### quote-harness-definition *type: `quote`* ## Quote > **“If you're not the model, you're the harness.”** — [Vivek Trivedy](#entity-vivek-trivedy) ## Context The author's succinct heuristic for dividing the architecture of an AI agent system. It collapses [the Agent = Model + Harness equation](#claim-agent-equation) into a single decision rule: when looking at any component of an agentic system, classify it as either *the model* (a forward pass over weights) or *the harness* (everything else). There is no third category. ## Why It Matters This framing has been adopted widely (Martin Fowler, Anthropic, Parallel AI, arXiv surveys). It clarifies design discussions: arguments about “agent improvements” can immediately be sorted into model-side improvements vs. [harness-side](#concept-agent-harness) improvements, which have entirely different teams, tools, and time horizons. #### quote-intelligence-vs-usefulness *type: `quote`* ## Quote > **“The model contains the intelligence and the harness is the system that makes that intelligence useful.”** — [Vivek Trivedy](#entity-vivek-trivedy) ## Context The concluding thesis statement of the article. It formalizes the relationship between the LLM and the surrounding system: **intelligence ≠ usefulness**. A model with infinite reasoning capacity but no [workspace](#concept-filesystem-primitive), no [execution environment](#concept-bash-general-tool), no [continuation logic](#concept-ralph-loop), and no [context management](#concept-compaction) still produces nothing useful in the world. The harness ([concept-agent-harness](#concept-agent-harness)) is the bridge from cognition to outcome. ## Why It Matters This quote is the strongest single-sentence support for [contrarian-harness-longevity](#contrarian-harness-longevity): even as model intelligence grows, the gap between intelligence and usefulness remains an engineering problem — and that gap is what harness engineering exists to close. --- ### Folder: action-items #### action-implement-filesystem *type: `action-item`* ## Action **Equip agents with filesystem abstractions and tools for durable storage and context offloading.** ## How Provide agents with a filesystem workspace and a set of fs-ops tools (`read_file`, `write_file`, `list_dir`, `grep`, `apply_patch`, etc.). This allows them to: - **Offload intermediate outputs** so they do not consume context tokens unnecessarily. - **Maintain state across sessions** — critical for [long-horizon work](#claim-long-horizon-compounds) and [Ralph Loops](#concept-ralph-loop). - **Collaborate via shared files** — Agent Teams (multiple agents and humans) coordinate naturally through the filesystem surface. Add Git on top for versioning, rollback, and experiment branching. ## Expected Outcome Agents can maintain state across sessions, avoid context window exhaustion, and recover from failures by re-reading durable state. ## Cross-Reference Grounding concept: [concept-filesystem-primitive](#concept-filesystem-primitive). #### action-provide-bash *type: `action-item`* ## Action **Ship harnesses with a bash tool to allow models to autonomously write and execute code.** ## How Instead of building a massive library of pre-configured tools for every edge case, give agents a general-purpose bash tool and code execution environment. This lets the model: - Write scripts on the fly to solve problems autonomously. - Inspect the environment (`ls`, `cat`, `grep`) without dedicated tools. - Compose existing UNIX utilities in arbitrary combinations. This is the operational expression of [concept-bash-general-tool](#concept-bash-general-tool) and the natural evolution of the [ReAct loop](#prereq-react-loop) toward general-purpose action spaces. ## Expected Outcome Reduces the need for exhaustive pre-built tools and dramatically increases agent autonomy on novel tasks. ## Caveat Sandbox the bash environment. Add audit logging. In high-risk deployments, weigh the autonomy gains against the governance costs of arbitrary shell access. #### action-tool-call-offloading *type: `action-item`* ## Action **Truncate large tool outputs in context to head/tail tokens, saving the full output to disk.** ## How Configure the harness middleware so that whenever a tool output exceeds a size threshold: 1. Only the **head and tail tokens** are kept in the active context window. 2. The **full output is written to the [filesystem](#concept-filesystem-primitive)** at a known path. 3. The model is shown the path / handle so it can re-read the full data on demand. This is the operational expression of [concept-tool-call-offloading](#concept-tool-call-offloading). ## Expected Outcome Prevents [context rot](#concept-context-rot) from noisy or oversized tool outputs while preserving full data access on demand. ## Tuning Notes - The size threshold and head/tail token counts should be tuned per domain (logs differ from code dumps differ from API JSON). - Pair with [concept-compaction](#concept-compaction) for older-context summarization and [concept-progressive-disclosure](#concept-progressive-disclosure) for tool-catalog management. #### action-use-ralph-loops *type: `action-item`* ## Action **Intercept model exit attempts and reinject prompts in clean context windows to force task continuation.** ## How For complex, long-running tasks, implement a [Ralph Loop](#concept-ralph-loop): 1. Install a hook that intercepts the model's exit attempt. 2. Clear the context window. 3. Reinject the original prompt. 4. Force the agent to read its prior state from the [filesystem](#concept-filesystem-primitive). 5. Continue working toward the goal. This works only if durable state is already in place — see [action-implement-filesystem](#action-implement-filesystem). ## Expected Outcome Overcomes **early stopping** and **incoherence across context windows** in long-horizon autonomous execution. Operationalizes [claim-long-horizon-compounds](#claim-long-horizon-compounds). ## Guardrails Add: - **Budget limits** (max iterations, wall-clock, tokens) to prevent runaway loops. - **Stop criteria** based on task-completion signals (tests pass, plan complete). - **Human approval gates** for high-stakes or production deployments. --- ### Folder: prerequisites #### prereq-mcp *type: `prereq`* ## What It Is The **Model Context Protocol (MCP)** is a standard introduced by Anthropic for connecting AI models to external data sources and tools. It defines how an LLM (or agent harness) discovers and invokes external capabilities — *“MCP servers”* expose tools, prompts, and resources; *“MCP clients”* (typically the harness) consume them. ## Why It Matters for This Source The article references "MCPs" and "MCP tools like [Context7](#entity-context7)" without defining the acronym. It assumes familiarity with MCP as the standard for tool integration. [concept-progressive-disclosure](#concept-progressive-disclosure) is specifically motivated by MCP: loading too many MCP servers' tool descriptions at agent start causes early [context rot](#concept-context-rot). Progressive disclosure manages this by injecting only the relevant subset on demand. ## Canonical URL https://www.anthropic.com/news/model-context-protocol #### prereq-react-loop *type: `prereq`* ## What It Is The **ReAct (Reasoning + Acting) loop** is the foundational pattern for tool-using LLM agents: 1. **Reason** — the model emits a thought / plan step. 2. **Act** — the model emits a tool call. 3. **Observe** — the harness executes the tool and returns the observation. 4. **Repeat** — the model sees the observation and decides the next step. This cycle continues in a while loop until the model emits a final answer or the harness terminates it. ## Why It Matters for This Source The article assumes the reader understands ReAct. Modern harnesses ([concept-agent-harness](#concept-agent-harness)) implement ReAct-style loops as their inner control flow. [Bash as a general-purpose tool](#concept-bash-general-tool) is best understood as an **evolution of ReAct**: instead of selecting from a small menu of typed tools at each Act step, the model emits arbitrary shell commands. The reasoning/observation cycle is identical; the action space is just much larger. [Ralph Loops](#concept-ralph-loop) also build on ReAct — they wrap the ReAct loop in an outer loop that re-initializes context. --- ### Folder: open-questions #### question-jit-tool-assembly *type: `open-question`* ## The Question How can harnesses **dynamically assemble the right tools and context just-in-time** for a given task, rather than relying on pre-configured environments? ## Resolution Path Likely directions: - **Advanced [progressive disclosure](#concept-progressive-disclosure)** that decomposes the user's request and fetches the matching skill set. - **Semantic routing over MCP catalogs** — embed task descriptions, retrieve relevant [MCP](#prereq-mcp) servers, mount them on the fly. - **Real-time task decomposition** — a planner subagent that emits the tool manifest needed for each subtask. ## Why It Matters The current state-of-the-art ships harnesses with fixed (or coarsely tiered) tool catalogs. A truly JIT harness would scale to **arbitrary domains** without context bloat — directly attacking the early-context-rot problem identified in [concept-context-rot](#concept-context-rot). #### question-orchestrating-hundreds *type: `open-question`* ## The Question How do you **orchestrate hundreds of agents working in parallel on a shared codebase** without them stepping on each other, duplicating work, or producing incoherent results? The author lists this as an open and interesting problem currently being explored in harness engineering research. ## Resolution Path Likely directions: - **Filesystem concurrency controls** — file-level locks, optimistic concurrency, conflict detection on the [filesystem primitive](#concept-filesystem-primitive). - **Git-based branching strategies for agents** — each agent on its own branch, with merge/rebase orchestration handled by the harness ([concept-agent-harness](#concept-agent-harness)). - **Hierarchical subagent routing protocols** — supervisor agents that decompose work and dispatch to worker subagents with non-overlapping scopes. ## Why It's Hard Classic distributed systems problems (consistency, partition tolerance, deadlock) re-emerge in agent-land — but with the added wrinkle that agents are **non-deterministic** and can hallucinate locks or invent invalid merges. Harness engineering must enforce determinism around the non-deterministic core. #### question-self-analyzing-traces *type: `open-question`* ## The Question How can agents be designed to **analyze their own execution traces** to identify and fix harness-level failure modes autonomously? ## Resolution Path Likely directions: - **Feedback loops on trace data** — execution logs (from platforms like [LangSmith](#entity-langsmith)) are fed back into a meta-reasoning prompt. - **Self-modifying harness configuration** — the agent proposes adjustments to tool selection, prompt structure, or compaction thresholds based on observed failures. - **Trace-conditioned RL** — using traces as a training signal for harness-aware behavior. ## Why It Matters This would close the loop on [model–harness co-evolution](#concept-harness-model-coevolution) at *runtime* rather than at training time — and would partially address some of the overfitting concerns raised by [claim-harness-overfitting](#claim-harness-overfitting). --- ### Folder: contrarian-insights #### contrarian-harness-longevity *type: `contrarian-insight`* ## The Conventional View As models natively absorb capabilities like planning and self-verification (the trajectory described in [concept-harness-model-coevolution](#concept-harness-model-coevolution)), one might assume **harness engineering will become obsolete** — that smarter models won't need scaffolding. ## The Contrarian Claim The author argues the opposite: just as **prompt engineering** remained valuable long after model improvements, **harness engineering** ([concept-agent-harness](#concept-agent-harness)) will continue to be critical. A well-configured environment with **durable state** ([concept-filesystem-primitive](#concept-filesystem-primitive)) and **verification loops** makes *any* model more efficient, regardless of its base intelligence. This is reinforced by [quote-intelligence-vs-usefulness](#quote-intelligence-vs-usefulness): the model contains the intelligence; the harness is what makes that intelligence *useful*. Even an infinitely smart model still needs a workspace, observability, safety constraints, and deterministic recovery paths. ## Nuance The *amount* and *location* of harness logic may shift inward (some planning may move into the model), but the need for **external orchestration, safety, observability, and state control** is unlikely to disappear. As models get more powerful and deployments more critical, harnesses become *more* valuable, not less. #### contrarian-harness-optimization *type: `contrarian-insight`* ## The Conventional View It is commonly assumed that the harness a model was post-trained with (e.g., [Claude Code](#entity-claude-code) for [Opus](#entity-opus-4-6)) is the **optimal environment** for that model. This view follows naturally from [co-evolution](#concept-harness-model-coevolution): if you trained the model in this harness, surely it performs best there. ## The Contrarian Claim The author challenges this directly. **Opus 4.6 running inside Claude Code scores far below Opus 4.6 running in other, more optimized harnesses** on the [Terminal Bench 2.0](#entity-terminal-bench-2-0) leaderboard. The cited delta is dramatic: moving from roughly Top 30 to Top 5 by swapping harnesses while keeping the model fixed. The lesson: **optimizing the harness for a specific task can yield massive performance gains, regardless of the model's native training environment.** ## Why It Matters This is the strongest empirical argument for treating harness engineering as an **independent vector** for performance gains — see also [contrarian-harness-longevity](#contrarian-harness-longevity) and [claim-agent-equation](#claim-agent-equation). Harness work is not subsumed by model improvements; it is a parallel discipline. ## Verification Caveat The specific “Top 30 → Top 5” numbers and the exact identity of the Terminal Bench 2.0 leaderboard are not easily verifiable from public sources. The *directional* claim (harness choice swings benchmark scores significantly) is well supported across SWE-bench, AgentBench, and related evals. ---