AI Agents & Orchestration

Anatomy of an AI Agent

An AI agent is an LLM in a loop that can reason about tasks and use tools to accomplish them.

The ReAct Loop

while not done:
    thought = llm.think(task, history, tools)
    
    if thought.is_final_answer:
        return thought.answer
    
    action = thought.tool_call
    observation = execute_tool(action)
    history.append(thought, action, observation)

Each iteration:

1. Thought: The model reasons about the current state and what to do next
2. Action: It chooses a tool and provides arguments
3. Observation: The tool executes and returns results
4. Loop: The model incorporates the result and decides the next step

Tool Definition

Tools need clear, precise schemas:

{
  "name": "search_database",
  "description": "Search the product database by query. Returns top 5 matching products with name, price, and availability.",
  "parameters": {
    "type": "object",
    "properties": {
      "query": {
        "type": "string",
        "description": "Search query — product name, category, or description keywords"
      },
      "max_price": {
        "type": "number",
        "description": "Optional maximum price filter in USD"
      }
    },
    "required": ["query"]
  }
}

Critical tips:

• Descriptions matter more than names — the model reads them to decide when/how to use tools
• Include examples in descriptions for complex parameters
• Validate arguments before executing — the model can hallucinate invalid values

Memory Architecture

Agents need different types of memory:

Conversation Memory

The full message history. For long conversations, summarise older messages to stay within context limits.

Scratchpad (Working Memory)

Short-term notes for the current task:

Plan: 1) Search for user's order 2) Check return eligibility 3) Process refund
Step 1 result: Found order #4521, shipped March 3rd
Step 2 result: Within 30-day return window, eligible
Current: Processing refund...

Long-Term Memory

Persistent storage across sessions — user preferences, past interactions, learned facts. Typically stored in a database and retrieved as needed.

Multi-Agent Orchestration

Complex tasks benefit from multiple specialised agents:

Orchestrator Pattern

A “manager” agent delegates subtasks to specialist agents:

• Research agent → gathers information
• Analysis agent → evaluates findings
• Writer agent → produces the final output

Pipeline Pattern

Agents work in sequence, each transforming the output of the previous:

Input → Agent A (research) → Agent B (analysis) → Agent C (writing) → Output

Debate Pattern

Multiple agents with different perspectives discuss and converge on an answer — useful for reducing bias and improving reasoning.

Planning Patterns

Plan-then-Execute

The agent creates a full plan before acting:

1. Generate step-by-step plan
2. Execute each step
3. Revise plan if a step fails

Dynamic Planning

Re-plan after each step based on new information. More flexible but can be less efficient.

Safety Guardrails

Must-Have Guardrails

1. Action confirmation — Require human approval for irreversible actions (send email, delete data, make purchases)

2. Scope limits — Restrict what tools are available. A customer support agent shouldn’t have access to the database admin tool.

3. Rate limits — Cap the number of tool calls per task to prevent runaway loops

4. Output filtering — Check agent outputs for harmful content, PII leakage, or off-topic responses before delivering to users

5. Maximum iterations — Set a hard limit on the agent loop to prevent infinite loops

The Human-in-the-Loop Pattern

For high-stakes actions:

if action.risk_level == "high":
    approval = await request_human_approval(action)
    if not approval:
        agent.replan()

Common Failure Modes

• Tool call loops — Agent repeatedly calls the same tool expecting different results
• Hallucinated tool arguments — Model invents parameter values
• Lost context — Long agent runs exceed context window
• Error cascading — One failed step derails the entire task
• Scope creep — Agent takes actions outside its intended domain

Key takeaway: Agent engineering is about controlled autonomy — giving the AI enough freedom to be useful while maintaining guardrails that prevent costly mistakes.