The Loop

Also known as: Agent

Problem

Imagine you're a soldier in the middle ages and your goal is to enter your tent for a cozy sleep, but there's a monster standing in front of it. If you don't defeat this monster, you will not achieve your goal. But here's the thing. You only have a single strike to defeat it. If you fail, the monster stays alive and probably ruins your tent (and your long dreamed sleep).

A raw LLM is a soldier (function) that has a single strike (text in > text out) to defeat the monster (hallucination/unreliability) and enter its tent (achieve what you asked it to). It's a stateless function. It does not store, modify, or remember any information or history between its executions. Each call is a brand-new call. Think of the model as a bare soldier: capable, trained, armed with every skill it learned during pre-training. A hell of a soldier depending on the pre-training. But it was trained to make exactly one single strike and stop. It doesn't know whether the strike landed. It doesn't adjust. It doesn't try again if the goal was not achieved.

Suppose you have a simple personal project made in Typescript. Your index.ts might be like this:

import { getUser } from './users';
              
function greet(userId: string) {
  const user = getUser(userId);
  return 'Hello, ' + user?.name;
}

console.log(greet('u_123'));

You wanna improve the greeting logic. So you send this to your model with a well constructed prompt:

You're a senior software engineer who's an expert in Typescript.
Look at the code snippet below. Your goal is to improve the greeting logic:
- If the user is a man, the greeting should end with 'What's up man?'
- If the user is a woman, it should end with 'What's up girl?'

--- CODE SNIPPET ---
${indexFile}

In my case, it returned:

import { getUser } from './users';

function greet(userId: string): string {
  const user = getUser(userId);
  
  if (!user) {
    return 'Hello, Guest!';
  }

  const greeting = `Hello, ${user.name}! `;
  
  switch (user.gender?.toLowerCase()) {
    case 'male':
    case 'm':
      return greeting + "What's up man?";
    case 'female':
    case 'f':
      return greeting + "What's up girl?";
    default:
      return greeting + "What's up?";
  }
}

The code is plausible and maybe even correct. But notice it guessed 'male'/'female'. You'll see later the real values inside users.ts are 'Man'/'Woman', so this code would hit the default case for every user. Plausible, but wrong. It won't be able to achieve the goal unless it has more context. It needs to know how the User interface is constructed.

Solution

We need a way to provide the model with the ability to ask and receive more context when needed, and remember its initial requests and iterations, before giving a final answer. And you might be wondering how that's possible, since a model is a stateless function. That's the 'magic' we're gonna dive into now.

The first thing we gotta do is tell the model it's okay to ask for more context if it's not confident to provide the final answer. We can even use the same prompt as before with a simple added paragraph:

You're a senior software engineer who's an expert in Typescript.
Look at the code snippet below. Your goal is to improve the greeting logic:
- If the user is a man, the greeting should end with 'What's up man?'
- If the user is a woman, it should end with 'What's up girl?'

--- CODE SNIPPET ---
${indexFile}

IMPORTANT: if you're not confident you can give a reliable answer
and feel you need more context, don't give the final answer.
Simply ask for more context and what specific context you want.

In my case, it returned:

I'd like to ask for some clarification before providing the final solution:

1. **User object structure**: What properties does the `User` object returned by `getUser()` have? Specifically, how is gender/sex information stored? Is it a property like `gender`, `sex`, `isMale`, etc.? What are the possible values?

2. **Null/undefined handling**: How should the function behave if:
   - `getUser()` returns `null` or `undefined`?
   - The gender property is missing or has an unexpected value?

3. **Type safety**: Should I add proper TypeScript types for the `User` object, or is there an existing type definition I should be aware of?

Once you clarify these points, I can provide a robust solution that handles edge cases properly!

We successfully did not hallucinate. But we still did not achieve our goal, which is to have an improved greeting logic. I could manually send the model a single code snippet from users.ts, but remember (again) the model is a stateless function. Each call is a brand-new call. If you simply send the model the users.ts content, it will try and do something with it with no recollection at all of the initial request you made.

If only there was a way to provide access to more context AND remember the initial intent... oh, there is! The loop!

A loop is as simple as a for/while loop that tells the model either to give the final answer if it's confident, or ask for more context. If the model gives the final answer, the loop ends. If it asks for more context, the loop gets this context and calls the model again with the initial iterations and the new context asked. For that to work, it relies on tools and arrays. Let's talk about tools first.

Tool is a concept that could use a whole different page only for it. We're gonna simplify things here, for learning purposes. That said, why do we need tools? Look at the last output, where the model asked for context. How will the loop use that as information to gather context and send it back? It's a simple string with semantic intention. And the intention may vary from reading files to asking if it can use browser functions to build the output. We need a way to provide it with tools that it can use to gather more context if needed. In this example, we could provide the model with a tool named read_files and tell it he's got a tool if it needs it, which is to read different files if needed. See, tools are specific, not open. They do a specific thing, and that's the point, you wanna provide the model with specific context. We also have to tell the model to return in a way the loop can understand if it must continue or stop. The prompt would be something like:

You're a senior software engineer who's an expert in Typescript.
Look at the code snippet below. Your goal is to improve the greeting logic:
- If the user is a man, the greeting should end with 'What's up man?'
- If the user is a woman, it should end with 'What's up girl?'

--- CODE SNIPPET ---
${indexFile}

IMPORTANT: if you're not confident you can give a reliable answer
and feel you need more context, don't give the final answer.
To gather more context, you may ask to read a specific project file.
ALWAYS return a JSON object in the exact shape below:
{
	finalAnswer: <true/false>,
	content: <your-response>
}

If you need more context, finalAnswer should de false and content should be 'read_file'.
If it's your final answer, finalAnswer should be true and content should be the code snippet.

In my case, it returned:

I need to gather more context before providing the final answer.

To implement gender-based greetings, I need to understand the structure of the user object returned by `getUser()`. Specifically, I need to know:

1. What properties does the user object contain?
2. How is the gender/sex information stored (e.g., is there a `gender`, `sex`, `isMale`, `isFemale` property)?

```json
{
  "finalAnswer": false,
  "content": "read_file"
}
```

Now the answer will be a json object which the loop can rely on to decide if it calls the read_files tool and re-call the model, or if it's the final answer and it stops. Note that we're also hardcoding the tool use into the prompt itself for learning purposes here. Foundation models such as Anthropic/OpenAi/Google models are pre-trained and have their own patterns for providing structured tools and returning structured objects for dealing with tool use. But here we're swimming in the layer underneath to understand how things actually work, so we're gonna keep hardcoding some stuff for now. Abstractions will come when convenient. We're almost ready to build our loop. But first we need to talk about the importance of the memory-array.

The model is stateless. I can't stress that enough. Each model call is brand-new. So the loop is responsible to hold the memory by replaying the whole conversation every call. The model will then "remember" what its goal was in the first place. The loop could even call different models on each pass. The loop doesn't care, nor does the model, since the memory of the conversation is stored in the memory-array. Now we're ready to build our loop:

const runAgent = async (client: Anthropic) => {
	let response: Response | null = null;
	
	let steps = 0;
	const MAX_STEPS = 3;
	while (steps < MAX_STEPS) {
		steps++;
		const message = await client.messages.create({
			max_tokens: 4096,
			model: 'claude-haiku-4-5',
			messages: messages,
		});
		
		messages.push({
			role: 'assistant',
			content: message.content,
		});
		
		response = extractResponse(message);
		
		if (response?.finalAnswer === true) {
			console.log(response.content);
			break;
		}
		
		if (response?.finalAnswer === false) {
			console.log(`Not the model's final answer. Model asked for: ${response.content}`);
			messages.push({
				role: 'user',
				content: `Here's the users file: ${usersFile}`,
			});
		}
	}
	
	if (!response) {
      console.error('Ended loop without a response (hit MAX_STEPS).');
      process.exit(1);
	}
};

Note how messages is an array initiated as [{ role: 'user', content: prompt }]. After every model return, we push its response as { role: 'assistant', content: message.content } into the messages array. If it's a tool call, we then push another object with user role and its content being what the model asked for. This happens in every pass of the loop. So each time the model is being called freshly, with no recollection whatsoever of previous calls, but each time it receives the whole conversation stored in the memory-array.

So first the model reasons it doesn't have enough context, so it asks for read_file. The loop then realizes that finalAnswer is false, so it pushes both the model's request and the user file into the messages array (it's hardcoded for learning purposes — in reality, the model would ask for a specific path and the loop would read that specific path). Here's users.ts:

export interface User {
	id: string;
	name: string;
	email: string;
	gender: 'Man' | 'Woman';
}

const users: Record<string, User> = {
	u_123: {
		id: 'u_123',
		name: 'Ada',
		email: 'ada@example.com',
		gender: 'Woman',
	},
	u_321: {
		id: 'u_321',
		name: 'John',
		email: 'john@example.com',
		gender: 'Man',
	},
};

export function getUser(id: string): User | undefined {
	return users[id];
}

On pass 2, the entire messages array (original prompt + model's question + the users file) is sent again. It's a brand-new model call, but with much more context. The model reasons it has enough context and returns a correct final answer:

1st pass:

Not the model's final answer. Model asked for: read_file

2nd pass:

import { getUser } from './users';

function greet(userId: string) {
  const user = getUser(userId);
  if (!user) {
    return 'Hello, stranger!';
  }
  const greeting = user.gender === 'Man' ? "What's up man?" : "What's up girl?";
  return 'Hello, ' + user.name + '! ' + greeting;
}

console.log(greet('u_123'));

Voilà. We got it to do what we asked. Now our soldier can ask what's the monster's weakness and strike much more precisely, defeating it and finally getting to his long dreamed sleep.

zzz

Congratulations, you built an AI agent. By now you're an expert and can click these things in your mind: the agent is NOT the LLM. Agency lives in the loop, not the model. The loop is what calls the model repeatedly, feeding it more information each time, turning a one-shot function into something that acts, observes (we're gonna talk about that in another moment), and acts again. That is the leap from "model" to "agent". Note that in our example we added a MAX_STEPS as a condition for the loop to cap. That is because depending on the request, the model might keep calling for tools and files indefinitely, spiraling the costs up indefinitely. Don't forget to add that cap to protect your wallet.

Structure

1 — The Model

The model (LLM/LMM) is the foundation. It has a single strike at achieving a specific goal. It needs enough context to give a final answer, or it'll ask for more.

2 — Tools

Tools are specific actions the model might ask the loop to do to give more context to the model. Note that the model itself does not execute tools. It requests them. Underneath, the loop executes and feeds back the results. Foundation models like Anthropic's don't require you to hardcode the tool use into the prompt. You pass tools as props to the message creator. Providing the model with tools is helping it achieve the desired outcome more precisely.

3 — The Loop/Agent

The loop is what wraps everything into a function that can call the model repeatedly, provide it with more info each time and in the end make the whole thing act, observe and act again until it's "satisfied". This is an agent. Not the model. This.

Applicability

Use loops when you know beforehand the model will need more context from a vast field and it needs to reason which part of the context it will need according to the input.

This is much better than adding the whole context inside the context window. Imagine a project with hundreds or even thousands of files that it could read for context. You DON'T want to send all the file contents to the model for MANY reasons. Instead, give the model a file list and let it choose which ones it will read increasing its context until it's "satisfied" and produces its output.

Extra Content

Read Claude API Docs to implement with Anthropic models using their built-in tools feature.

← All patterns