Ask most people what an AI agent is and they will describe a chatbot — you type a question, it types an answer. That is a language model, but it is not an agent. An agent is what you get when you wrap a language model in a loop, give it a goal instead of a single question, and hand it the ability to take actions in the world. The difference between answering and acting is the whole story. This article walks through what is actually happening inside an agent from the moment it receives a goal to the moment it decides it is done.
At the centre of every agent sits a large language model. It is useful to think of the model as the reasoning engine, not the agent itself. On its own the model does exactly one thing: given a sequence of text, it predicts the next chunk of text. That single capability turns out to be remarkably powerful, because if you frame a problem the right way, "predict the next token" becomes "decide what to do next." The job of everything surrounding the model — the scaffolding — is to turn that raw prediction into purposeful behaviour.
It begins with the goal and the context. When you give an agent a task like "find the three cheapest flights to Lisbon next month and summarise them," that instruction does not go to the model alone. The agent assembles a much larger prompt behind the scenes. This typically includes a system message describing who the agent is and how it should behave, a description of the tools it is allowed to use, any relevant memory from earlier in the conversation, and finally your actual request. This assembled context is the agent's entire world for that moment. Everything it knows, it knows because something put it into that context window.
Then the loop begins. This is the part that separates an agent from a one-shot model call. Instead of producing a final answer immediately, the agent runs a cycle that is often described as observe, think, act. The model observes the current state, reasons about what to do next, and emits an action. The action gets executed, the result is fed back into the context, and the cycle repeats. The agent keeps looping until it decides the goal has been achieved or it hits a limit you have set. A single user request might trigger two iterations or twenty.
The "think" step is where reasoning happens, and it is more deliberate than it looks. Modern agents are usually prompted to reason explicitly before acting — to lay out their plan in words, consider what information is missing, and choose a next step. This is not decoration. Forcing the model to articulate its intermediate reasoning measurably improves the quality of its decisions, because each token it generates becomes part of the context that informs the next one. The model is, in a sense, thinking out loud so that its later self can read what it just worked out.
Acting means calling tools, and tools are what give an agent reach beyond its own training. A tool is simply a function the agent is allowed to invoke — search the web, query a database, send an email, run a calculation, call an external API. The model does not execute these itself. Instead it produces a structured request that says, in effect, "call the flight_search tool with these parameters." The surrounding code intercepts that request, actually runs the function, and returns the result. This is the mechanism people mean when they talk about function calling or tool use, and it is the bridge between a model that only predicts text and a system that can change things in the real world.
Memory is what lets an agent stay coherent over a long task. There are two kinds worth distinguishing. Short-term memory is the running context of the current session — everything that has happened so far, kept in the prompt. But the context window is finite, so as a task grows the agent cannot simply remember everything verbatim. Long-term memory solves this by storing information outside the model, often in a vector database, and retrieving only the relevant pieces when they are needed. When the agent needs to recall a fact from three tasks ago, it searches that store, pulls back the most relevant snippets, and slots them into the current context. The model feels like it remembers; really, the scaffolding is feeding it the right memories at the right moment.
Tying all of this together is the orchestration layer, and this is where a lot of the real engineering lives. The orchestrator decides how the prompt is assembled, when to call which tool, how to handle a tool that fails or returns garbage, how many loop iterations are allowed before the agent gives up, and how to detect that the goal is genuinely complete rather than merely abandoned. Frameworks help here, but the hard problems — keeping the agent from looping forever, recovering gracefully from errors, deciding when a sub-task should be delegated to another agent — are design problems, not library calls.
It helps to trace a single request end to end. You ask for the cheapest Lisbon flights. The orchestrator builds the prompt with your goal, the agent's instructions, and the available tools. The model reasons that it first needs flight data and emits a call to the search tool with the relevant dates. The orchestrator runs that call and feeds the raw results back in. The model now observes real data, reasons that it has enough to compare prices, sorts them, and decides no further tools are needed. It composes a summary of the three cheapest options and signals that the task is complete. The loop ends, and you see a clean answer — but behind it were several model calls, a tool invocation, and a handful of decisions about what to do next.
The mental model to hold onto is this: the language model supplies the intelligence, but the agent is the system around it. The loop gives it persistence, tools give it reach, memory gives it continuity, and orchestration gives it judgement about when and how to use the rest. Building reliable agents is less about finding a smarter model and more about designing that surrounding system well — clear tool definitions, honest handling of failure, and prompts that make the model's reasoning explicit. Get the scaffolding right and an ordinary model starts to behave like something genuinely capable. Get it wrong and even the best model spins in circles. The intelligence is in the model; the behaviour is in the design.