Harness: The Invisible Layer Behind the Best AI Agents

How Harness Design Shapes Token Economics, Cost, and Performance

When we think about AI agents, our attention usually goes straight to the language model (LLM).

Which LLM is the best? Is it worth switching models? Is the latest release really better than the previous one?

But as AI agents become more sophisticated, another layer is becoming increasingly important: orchestration.

Choosing a good model is only part of the equation. You also need to decide how that model will be used: what information it receives, when it should use tools, how it manages memory, when it summarizes context, how it delegates tasks, and how it executes complex workflows.

This layer is known as the harness.

In this article, we’ll explore what a harness is, why it plays a key role in AI agent architectures, and how good orchestration decisions can influence cost, performance, and scalability.

Follow our page on LinkedIn for more content like this! ❤

---

First Things First: What Is a Harness?

Imagine an AI agent that needs to:

• search documents;

• query a database;

• use external tools;

• delegate tasks to other agents;

• hold multi-turn conversations;

• summarize information;

• generate a report.

The LLM is only one part of that system.

Behind the scenes, there’s a software layer responsible for deciding:

• what context is sent to the model;

• which tools are available;

• when to perform retrieval (RAG);

• when to summarize the conversation history;

• when to spawn a sub-agent;

• when to reuse cached context;

• when to retry a request;

• when to end a task.

This layer is known as the harness (or orchestration layer).

A simple analogy:

• LLM = Engine

• Harness = Driver + Transmission + Brakes + GPS

You can put the world’s most powerful engine into a car, but you still need a system that knows how to drive it.

---

You might also like

• The biggest barrier to scaling AI in enterprises might not be what you think (in Portuguese)

• The era of chatbots is over. The era of AI agents has begun.

• The AI questions that land in my inbox every day

• 🚀 Ready to build? Get access to 30+ notebooks with runnable code (paid subscribers)

---

The Problem: Token Maxing

A study published by Writer in July 2026 examined this orchestration layer and introduced an interesting concept called token maxing.

As AI agents become more sophisticated, many applications end up solving problems simply by consuming more tokens.

For example:

• increasingly larger prompts;

• replaying the entire conversation history at every turn;

• exposing dozens of tools on every request;

• longer reasoning traces;

• multiple agents sharing large contexts.

The result is that costs grow rapidly while quality improvements become increasingly marginal.

The authors define token maxing as a development trajectory in which token consumption grows faster than the value delivered per task.

Token maxing describes a tendency for AI agents to consume more and more tokens while achieving progressively smaller quality gains.

---

Why the Orchestration Layer Matters

While much of the research in generative AI focuses on making language models themselves more efficient, the same study explored a different question:

To what extent can we reduce cost and improve an AI agent’s efficiency without changing the language model, simply by improving the harness?

To answer this question, the researchers conducted a controlled experiment using:

• the same 22 enterprise tasks;

• the same prompts;

• the same evaluation criteria;

• the same six language models.

The only variable that changed was the harness, allowing the researchers to isolate the impact of the orchestration layer.

The evaluated models included Claude Sonnet 4.6, Gemini 3.1, Gemini Flash 3.5, Qwen 3.6, GLM 5.1, and Palmyra X6.

The results?

Replacing only the orchestration layer led, on average, to:

• 38% fewer tokens per task

• 41% lower cost

• 44% lower execution time

• Comparable quality (0.78 → 0.81), a difference the authors consider statistically inconclusive given the study’s sample size.

Perhaps the most interesting finding was this:

Every one of the six models became cheaper to run.

Depending on the model, cost reductions ranged from approximately 33% to 61%.

---

How Can a Harness Reduce So Many Tokens?

1. Smart Prompt Caching

Many AI providers offer discounted pricing when the beginning of a prompt is identical to previous requests.

To take advantage of this, a harness structures the prompt into two parts:

• a stable section (system prompt, tool catalog, consolidated conversation history);

• a dynamic section (recent messages, current time, uploaded files, execution plan).

This allows almost the entire prompt prefix to remain unchanged across requests, dramatically increasing cache utilization.

---

2. Avoid Replaying the Entire Conversation

A naive implementation resends the entire conversation history with every interaction.

As conversations grow, token consumption increases roughly quadratically with the number of turns.

A harness can instead create summarized checkpoints and retain only the information that is still relevant, keeping token growth much closer to linear.

---

3. Keep Information Outside the Context Window

Not everything needs to live inside the prompt.

Large tool outputs, lengthy documents, and extensive search results can remain outside the model’s context window.

When needed, the agent retrieves that information again or receives only a concise summary.

---

4. Isolated Sub-Agents

Instead of having multiple agents share the same ever-growing context, the harness can spawn specialized sub-agents.

Each sub-agent works within its own context and returns only a summarized result to the main agent.

This prevents the primary context from growing indefinitely.

---

5. Waiting Without Spending Tokens

Another simple yet elegant idea.

If an agent needs to wait for:

• human approval;

• a user’s response;

• the completion of a background process;

it simply pauses.

Rather than continuously polling and consuming tokens, execution resumes only when the required event occurs.

---

An Interesting Concept: Harness Leverage

Not every model benefited from the new architecture in exactly the same way.

In the study, the authors observed that:

• stronger models improved in both efficiency and quality;

• smaller models primarily improved in efficiency.

They refer to this relationship as harness leverage; a model’s ability to translate better orchestration into better overall performance.

In other words, a well-designed harness benefits every model, but more capable models tend to extract greater performance gains from the same orchestration improvements.

---

What Does This Mean for AI Agent Developers?

One of the study’s key takeaways is that optimizing AI agents isn’t just about choosing a better language model.

It’s also about designing a better architecture.

That includes decisions such as:

• when to summarize context;

• how to structure prompts;

• how to leverage prompt caching;

• when to invoke tools;

• how to organize memory;

• how to decompose work across multiple agents.

These design choices are often invisible to end users, yet they can have a significant impact on the operational cost and scalability of LLM-based applications.

---

Conclusion

For a long time, building AI applications was largely about selecting the right language model.

Today, the conversation is evolving.

As AI agents become more capable, the way they manage context, memory, tools, and execution workflows is becoming just as important.

This is where the harness comes into play. It acts as the orchestration layer that connects these components and determines how an AI agent operates in practice.

Regardless of the framework or architecture you use, understanding this layer can help you build AI systems that are more efficient, scalable, and easier to maintain.

If you’re building AI agents, your next major optimization may not come from switching to a newer model—but from rethinking how your agent is orchestrated.

---

🚀 Ready to build?

Become a paid subscriber and get access to 30+ notebooks with runnable code, practical examples, and implementation details.

🔗 Join here.

---

This article was inspired primarily by the paper “The Harness Effect: How Orchestration Design Sets the Token Economics of Enterprise Agentic AI” (Writer, 2026), which investigates how different orchestration strategies affect cost, token consumption, latency, and quality in enterprise AI agents.