A practical guide to understanding what makes AI products behave differently
Two AI products can feel completely different even when they run the same underlying model. The same model can power a basic back-and-forth chatbot in one product, and in another product, placed inside a developer tool with access to files and a terminal, it can work for hours on a software task without stopping.
That difference is rarely the model. It comes from the software layer built around it, called the agentic harness.
This document explains the two parts separately. The first part is the AI model: the trained neural network that produces output text. The second part is the agentic harness: the tools the model can call, the way information is kept between steps, the loop that keeps the model running, and the rules about what the model is allowed to do.
By the end of this guide you will be able to tell which part of a system is responsible for a given behaviour, which matters when you are choosing a tool, debugging a failure, or building your own agent.
The guide is written for developers and technical readers. No prior experience with agent frameworks is assumed. Each technical term is defined the first time it appears.
1. Lesson 1: What an AI Model Actually Is
2. Lesson 2: What an Agentic Harness Is
3. Lesson 3: Tools
4. Lesson 4: Memory
5. Lesson 5: The Agentic Loop
6. Lesson 6: Why the Distinction Matters
7. Lesson 7: Applying This in Practice
8. Summary
9. Final Checklist
An AI model, in this context a large language model, is an artificial neural network with a fixed set of trained parameters. It receives a sequence of text, converts that text into numeric units called tokens, and predicts the next token. Repeating that prediction produces a full response. The process of running input through the trained network to get output is called inference.
The important property is that the parameters do not change during use. A model does not learn from your conversation. It produces output and nothing about it is modified.
On its own, a model has no way to affect anything outside its own output. Specifically:
- It cannot open, read, or write a file.
- It cannot execute code.
- It cannot make a network request or browse the web.
- It cannot remember anything after the request ends.
A model asked to read app/build.gradle.kts cannot do so. It can only produce text describing what such a file usually contains. The difference between describing a file and reading a file is the entire subject of this guide.
An evaluation, usually shortened to eval or benchmark, is a standardised set of tasks used to score models against each other. Benchmark results from the leading labs have converged to within a few points on most common evals.
That convergence has a practical consequence. When one AI product clearly outperforms another on real work, the model is usually not the explanation. The explanation is what has been built around it.
An agentic harness is the software layer that sits between the model and the outside system. It sends requests to the model, interprets what the model asks for, executes those requests against real resources, and returns the outcome to the model.
An agent is the combination of a model and a harness operating together toward a goal over multiple steps, rather than producing a single response.
Every agentic harness is built from three components. The rest of this document covers each one in detail.
| Component | Purpose | Failure when it is weak |
|---|---|---|
| Tools | Let the model act on real files, code, services, and interfaces | The model describes actions instead of performing them |
| Memory | Preserve relevant information within a session and across sessions | The model repeats questions and ignores project conventions |
| Agentic loop | Run plan, act, and observe cycles until the goal is reached | Work stops after one step, or continues without checking results |
A tool is a function the harness exposes to the model, described by a name, a purpose, and a set of parameters. The model does not execute the function. It emits a structured request naming the tool and its arguments. The harness validates that request, runs the function, and returns the result as new input to the model.
The sequence is:
1. The harness sends the model a list of available tools along with the conversation.
2. The model returns a tool call, for example a read request for a specific path.
3. The harness checks the call against its permission rules.
4. The harness executes the call and captures the output.
5. The harness appends that output to the conversation and calls the model again.
Every capability described below is an instance of this same mechanism.
File tools let the model read and write files on disk. A read returns actual file contents. A write modifies the file, either by replacing the whole file or by applying a targeted edit to specific lines.
Good harnesses provide edit tools that operate on a matched region of text rather than rewriting an entire file, because rewriting a 900-line file to change three lines wastes output tokens and risks losing unrelated content.
Execution tools run code and return the result, including exit status, standard output, and errors. A sandbox is an isolated execution environment with restricted access to the filesystem, the network, and system calls. Code that runs in a sandbox cannot modify the host machine outside the paths it is granted.
Sandboxing matters because generated code is not reviewed before it runs. Isolation limits the effect of an incorrect command.
Web tools fetch pages or run searches and return the text to the model. This matters because model parameters are frozen at training time. A model cannot know about a library version released after that point. A retrieval tool supplies the current information at request time.
Computer use is a tool category where the model receives a screenshot of a display and returns pointer and keyboard actions: move to a coordinate, click, type, scroll. The harness applies those actions to the real screen and returns an updated screenshot.
This allows the model to operate software that exposes no API, which is common with legacy internal applications. It is slower and less reliable than a direct API call, so it is normally the fallback rather than the first choice.
Rather than building a dedicated tool for every program, most harnesses expose a single shell tool. Any software already installed on the machine becomes available through it:
./gradlew :app:assembleDebug git log --oneline -20 adb shell am start -n com.example.app/.MainActivity npm run build
This is the same interface a developer uses, which is why it covers so much ground with one tool definition.
Services that do not live on the local machine, such as an internal database, a ticket tracker, or a third-party application, need their own integrations. Without a standard, every integration has to be rebuilt for every harness.
MCP, the Model Context Protocol, is a standard interface for exposing tools and data sources to models. A server written once against MCP works with any harness that supports the protocol, with no changes.
Configuration is typically a small declaration listing the servers to start:
{
"mcpServers": {
"issue-tracker": {
"command": "npx",
"args": ["-y", "@company/issue-tracker-mcp"],
"env": { "TRACKER_URL": "https://tracker.internal" }
}
}
}
| Category | Typical operations | Main constraint |
|---|---|---|
| File access | Read, write, targeted edit | Large files consume context quickly |
| Code execution | Run commands, tests, build scripts | Requires sandboxing or permission rules |
| Web retrieval | Fetch a URL, run a search | Retrieved content is untrusted input |
| Computer use | Screenshot, click, type, scroll | Slow and less reliable than an API |
| Command line | Any installed program | Broadest permission surface |
| MCP servers | External databases, apps, services | Each server must be configured and trusted |
Problem. A developer asks the model to fix a failing unit test in app/src/test/java/com/example/LoginViewModelTest.kt. Without tools, the model has neither the test file nor the failure output. It produces a plausible fix based on the test name alone, and the developer applies it, runs the suite, and finds the same failure.
Solution. Give the harness a file read tool and a shell tool. The model reads the test file and the class under test, runs the suite, and reads the actual assertion error:
./gradlew :app:testDebugUnitTest --tests "*LoginViewModelTest*"
LoginViewModelTest > emitsErrorOnEmptyPassword FAILED
expected: LoginState.Error(message=Password required)
but was: LoginState.Idle
The model now sees that the state is never updated because the validation branch returns before emitting, applies a targeted edit at line 47, and reruns the suite to confirm.
Benefit. One verified fix instead of a guess-apply-fail cycle. In practice this removes two to three review rounds per bug and eliminates fixes that address the wrong cause.
The context window is the maximum number of tokens a model can process in a single request. It holds the system instructions, the full conversation, every tool result returned so far, and the response being generated. It is the model's only working memory, and it is fixed in size.
Two consequences follow. First, the data sent with each request grows as the session continues, because every tool result stays in the conversation. Second, when the session ends, that working memory is gone. The next session starts with nothing.
The harness is what compensates for both.
An instruction file is a text file that sits in a project folder and is loaded into the model's context automatically at the start of every session. Common names are AGENTS.md and CLAUDE.md, placed in the repository root.
This is how a model knows a project's conventions without being told each time: which libraries are approved, how modules are laid out, which commands build and test the project, and which directories must not be modified.
# Project Conventions ## Architecture - MVVM with Clean Architecture: data, domain, presentation - Dependency injection with Hilt ## UI - Jetpack Compose only. Do not add XML layouts. ## Commands - Build: ./gradlew :app:assembleDebug - Test: ./gradlew :app:testDebugUnitTest - Lint: ./gradlew ktlintCheck ## Constraints - Do not edit files under /generated - Network calls go through Retrofit interfaces in data/remote
Keep instruction files short and specific. They are loaded on every session, so their cost is paid on every request.
Compaction is the process the harness runs when the context window approaches its limit. It replaces the earlier part of the conversation with a summary that preserves decisions, current state, and open work, while removing material that no longer carries information.
What is typically preserved:
- The original goal and any constraints given.
- Decisions made and the reason for each.
- Files modified and what changed in them.
- Unresolved errors and remaining steps.
What is typically pruned:
- Full contents of files that were read and then edited.
- Repeated command output, such as identical build logs.
- Search results already acted on.
- Superseded intermediate reasoning.
Without compaction, a long session ends when the window fills. With it, the session continues and the relevant state survives.
A production codebase is far larger than any context window. Loading it entirely is not possible, and loading most of it is wasteful even when it fits, because unrelated content dilutes the material that matters.
Harnesses instead give the model retrieval tools and let it pull in only what it needs:
1. Pattern search. Regular-expression matching across files, equivalent to grep -rn. Exact and fast when the identifier is known.
2. Semantic search. Matching on meaning rather than characters, using vector embeddings. Useful when the wording of the query does not match the wording in the code.
3. Code index. A structural map of definitions, references, and call relationships, which answers questions such as which functions call a given method.
Problem. A developer asks the model to change how authentication tokens are refreshed in a 4,000-file Android repository. A naive approach reads every file under app/src/main/java. The context window fills before the relevant class is reached, and the session ends without an edit.
Solution. The harness exposes a search tool. The model runs a targeted query first:
grep -rn "refreshToken" app/src/main/java --include=*.kt
data/remote/AuthInterceptor.kt:34: private suspend fun refreshToken(): String {
data/repository/AuthRepositoryImpl.kt:88: override suspend fun refreshToken() =
domain/usecase/RefreshTokenUseCase.kt:12: class RefreshTokenUseCase @Inject constructor(
Three files are read instead of four thousand, and the edit is applied to AuthInterceptor.kt at line 34.
Benefit. Context consumption drops from hundreds of thousands of tokens to a few thousand. The task completes in one session rather than failing partway, and the remaining context is available for the actual work.
The agentic loop is the repeating cycle in which the model and the harness work together toward a goal. Each iteration has three stages:
1. Plan. The model decides the next step and emits a tool call for it.
2. Act. The harness executes that call against the real environment.
3. Observe. The harness returns the result, and the model reads it and decides what follows.
The cycle repeats until the goal is met, the model stops, or a limit set by the harness is reached. A loop may run for a few seconds or for several hours across hundreds of iterations.
The critical property is the observe stage. A model that acts without reading the result of the previous action cannot correct itself and will keep building on a broken state.
Verification is the practice of checking work against reality inside the loop rather than after it. Common forms:
- Test execution. The model runs the suite after each change and reads the failures.
- Build and static checks. Compilation and lint catch errors that tests do not reach.
- Visual checks. For user interface work, the model captures a screenshot of the running application and inspects the result.
- Reviewer models. The harness starts a second model instance with the diff and a review instruction, and returns its findings into the loop.
Verification is the single strongest predictor of whether a long-running agent stays on track. Each check is an opportunity to detect a wrong step while it is still cheap to correct. A model that checks its own work can run far longer before its output degrades.
The harness also defines what the loop may not do. This is enforced in the harness, not in the model, because the model produces a request and the harness decides whether to run it.
Typical controls include an allowlist of commands that run without confirmation, a denylist of destructive operations, a prompt before writing outside the project directory, a cap on iterations, and a cap on total spend.
Problem. A harness is configured to make an edit and stop. A developer asks for a new field on a Room entity. The model edits data/local/entity/UserEntity.kt and reports completion. The build then fails, because adding a column requires a schema version bump and a migration. The developer discovers this later and reopens the task.
Solution. Configure the loop so that a build command runs after every file write and the output returns to the model. The model now sees the failure in the same session:
./gradlew :app:assembleDebug e: AppDatabase.kt:19 Room cannot verify the data integrity. Schema export is at version 4, entity hash does not match. Either bump version and provide a Migration, or set exportSchema to false.
It raises the version in AppDatabase.kt, adds a migration object, and rebuilds until the build passes.
Benefit. The failure is caught in the same iteration that caused it, while the relevant context is still loaded. This removes a full round trip through the developer, which is the most expensive step in the cycle.
Once the model and the harness are treated as separate things, recent progress in AI products becomes easier to read. Models have improved with every release. But a large share of the practical capability gain has come from harnesses getting better: broader and more reliable tools, better context handling, and loops that verify their own output instead of running blind.
This is also why a product can improve noticeably without any model change at all.
Questions such as "is AI good at writing code" or "is AI good for customer support" cannot be answered as stated, because they collapse two independent variables. The useful form specifies both:
1. Which model is being used.
2. Which harness is running it, meaning which tools, what memory handling, and what verification is in the loop.
The same model can be highly effective in one harness and get stuck in another. When an agent fails, identifying which of the two is responsible determines the fix.
| Observed failure | Usual source | Corrective action |
|---|---|---|
| References a file that does not exist | Harness: no file read tool | Expose read and search tools |
| Ignores project conventions | Harness: no instruction file | Add AGENTS.md to the repository root |
| Forgets earlier decisions in a long session | Harness: weak compaction | Improve summarisation, reduce tool output noise |
| Reports success on code that does not compile | Harness: no verification step | Run build and tests inside the loop |
| Produces valid but poor-quality logic | Model | Use a stronger model, or constrain the task |
The line between model and harness is not fixed. Capabilities that used to be implemented entirely in harness code, such as long-horizon planning and self-verification, are increasingly trained into the models themselves. In the other direction, behaviour that used to be considered a property of the model, such as consistency across a long task, is now shaped substantially by harness conventions and project instruction files.
The distinction remains useful for diagnosis and design even as the boundary shifts. It simply means that where a given capability lives should be checked rather than assumed.
When comparing AI development tools, compare harnesses, since the underlying models are often similar or identical. Useful questions:
1. Which tools are available, and can the model run shell commands and tests?
2. Does it load a project instruction file automatically, and which filename does it expect?
3. What happens when the context window fills: does the session end, or does it compact and continue?
4. How does it locate code: bulk loading, pattern search, semantic search, or an index?
5. Does the loop verify its own work before reporting completion?
6. What permission controls exist, and are destructive operations confirmed?
7. Can external services be connected, and does it support MCP?
If an agent performs poorly, work through the three components in order of cost before changing the model:
1. Instructions. Add or tighten the project instruction file. This is the cheapest change and often the highest impact.
2. Verification. Make the loop run the build and the test suite after edits, and feed the output back.
3. Retrieval. Replace bulk file loading with targeted search so context is spent on relevant material.
4. Tools. Add the specific capability that is missing, through a built-in tool or an MCP server.
5. Model. Change the model last, after the harness is no longer the limiting factor.
An AI model is a trained neural network that produces text. It cannot read a file, run a command, reach the network, or retain anything after a request ends.
An agentic harness is the software layer that supplies those abilities. It has three components. Tools let the model act on real systems through file access, code execution, web retrieval, computer use, the command line, and MCP servers. Memory manages the fixed context window through instruction files, compaction, and search-based retrieval. The agentic loop repeats plan, act, and observe until the goal is met, with verification steps that let the work stay correct over long runs.
Because model quality has converged across the leading labs, differences between AI products usually come from the harness. When judging whether AI is suitable for a task, specify both the model and the harness, since the same model performs very differently depending on what is built around it.
Use this to review an agent setup before relying on it for real work.
| Area | Check |
|---|---|
| Tools | File read, targeted edit, and shell execution are available |
| Tools | Code runs in a sandbox or under explicit permission rules |
| Tools | External services are connected through MCP rather than custom one-off code |
| Memory | An instruction file exists in the repository root and is loaded automatically |
| Memory | The instruction file lists build and test commands, and directories that must not change |
| Memory | Compaction is in place, so long sessions continue instead of ending |
| Memory | Code is located by search, not by loading whole directories |
| Loop | Tool results are returned to the model and read before the next step |
| Loop | The build and the test suite run inside the loop after edits |
| Loop | User interface changes are verified visually where relevant |
| Loop | Iteration and spend limits are set |
| Diagnosis | Failures are attributed to the model or the harness before any change is made |
| Diagnosis | The harness is improved before the model is replaced |