AI Agents and Frameworks: Building Autonomous Decision-Making Systems

Travel and delivery apps now ship features that do more than answer a question: an IRCTC support bot that checks train availability, computes Tatkal charges, checks your wallet balance, and either books the ticket or explains why it can't, all without a human re-typing each intermediate step. A Swiggy support flow that looks up an order, checks the refund policy for that item category, and issues the refund on its own. These are not chatbots that produce one answer from one prompt. They are systems that decide, at each step, whether to call another system, what to call it with, and when to stop. That decision loop, not the language model inside it, is what this chapter builds, traces, and formalizes.

From Reflex Agents to a Decision-Making Loop

Russell and Norvig's classical definition still holds: an agent is anything that perceives its environment through sensors and acts on it through actuators, and a rational agent is one whose action, given its percept history, maximizes expected performance. The PEAS framing (Performance measure, Environment, Actuators, Sensors) that you used in Grade 11 to classify simple reflex and goal-based agents applies unchanged here; only the substrate has changed. In a large-language-model agent, the "sensors" are the tokens fed into the context window (the user's query plus everything the agent has observed so far), the "actuators" are structured function calls the surrounding system is willing to execute, and the "brain" is a stateless function: the model maps a sequence of tokens to a probability distribution over the next tokens, with no memory of any call that is not explicitly re-fed to it. Formally, an LLM agent's policy is a function π: (query, history) → action, where history is the entire prior transcript, re-supplied on every single call, because the model itself retains nothing between calls. That single fact, that the "memory" is external and must be re-transmitted every turn, is the source of almost every engineering tradeoff discussed later in this chapter.

The ReAct Loop: Thought, Action, Observation

The dominant pattern for structuring this loop was formalized by Shunyu Yao and colleagues as ReAct (Reasoning and Acting), presented at ICLR 2023. Instead of asking the model to jump straight to an action, ReAct prompts it to interleave three kinds of output on every turn: a Thought (free-text reasoning about what it knows and what it still needs), an Action (a structured call to a named tool with an argument), and, after that action is actually executed by the surrounding program, an Observation (the tool's real return value, appended back into the context for the next Thought). The loop repeats until the model emits a terminal action, conventionally called Finish, carrying the final answer. Yao et al.'s ablations showed that Act-only prompting (skipping the Thought) produces measurably worse tool-selection accuracy than the full Thought-Action-Observation cycle, because the reasoning text forces the model to commit to an explicit plan before its next tokens have to be a syntactically valid action, and gives it a place to notice, immediately after a bad Observation, that the plan needs revision before compounding one mistake into the next.

Worked Example: Tracing a ReAct Agent Step by Step

The clearest way to see where the "agent" actually lives is to run a minimal version of this loop by hand. Suppose the query is: "I'm driving a Swift Dzire (18 km/l) 350 km from Bengaluru to Chennai via NH44, diesel is ₹100/l, and I have ₹340 for tolls and fuel combined. Can I afford it?" This needs two tool calls: a toll lookup and an arithmetic calculation. The code below defines both tools and the executor loop that drives them.

import re

def calculator(expr: str) -> float:
    """Evaluates a pure arithmetic expression such as '350/18*100'.
    Production systems use a restricted parser (e.g. Python's ast
    module) instead of eval() for safety; eval() is used here only
    because the argument is a hardcoded literal, not user text."""
    return round(eval(expr, {"__builtins__": {}}, {}), 2)

def search_toll(route: str) -> str:
    """Mock of a live toll-lookup API. A real deployment would call
    an NHAI/FASTag endpoint; this returns a fixed value so the trace
    below is fully reproducible."""
    tolls = {"Bengaluru-Chennai-NH44": 390}
    return f"₹{tolls[route]}"

TOOLS = {"calculator": calculator, "search_toll": search_toll}

def run_agent(steps, max_iterations=5):
    """This loop, not the LLM, is the actual agent. It parses the
    Action string the model produced and executes a REAL function."""
    transcript = []
    for i, (thought, action) in enumerate(steps):
        if i >= max_iterations:
            transcript.append("STOP: max_iterations reached")
            break
        transcript.append(f"Thought {i+1}: {thought}")
        transcript.append(f"Action {i+1}: {action}")
        if action.startswith("Finish"):
            break
        name, arg = re.match(r"(\w+)\[(.*)\]", action).groups()
        result = TOOLS[name](arg)
        transcript.append(f"Observation {i+1}: {result}")
    return transcript

# What a competent ReAct-prompted model would emit, turn by turn,
# after reading each prior Observation:
steps = [
    ("I need the toll for this route before totaling trip cost.",
     "search_toll[Bengaluru-Chennai-NH44]"),
    ("Toll known. Now compute fuel cost: distance / mileage * price.",
     "calculator[350/18*100]"),
    ("Toll is ₹390, fuel is ₹1944.44, total ₹2334.44. Budget is "
     "₹340, so compare and finish.",
     "Finish[Total cost ₹2334.44 (₹390 toll + ₹1944.44 fuel) "
     "exceeds the ₹340 budget by ₹1994.44 -- trip is not "
     "affordable at this budget.]")
]

for line in run_agent(steps):
    print(line)

Tracing it by hand: iteration 0 appends Thought 1 and Action 1, sees the action does not start with Finish, matches the regex to get name="search_toll", arg="Bengaluru-Chennai-NH44", calls search_toll, gets back "₹390", and appends it as Observation 1. Iteration 1 does the same with calculator("350/18*100"): 350 ÷ 18 = 19.4444..., × 100 = 1944.44 after rounding, appended as Observation 2. Iteration 2 sees an action starting with Finish, appends Thought 3 and Action 3, and breaks immediately, so there is no Observation 3. The printed output is exactly:

Thought 1: I need the toll for this route before totaling trip cost.
Action 1: search_toll[Bengaluru-Chennai-NH44]
Observation 1: ₹390
Thought 2: Toll known. Now compute fuel cost: distance / mileage * price.
Action 2: calculator[350/18*100]
Observation 2: 1944.44
Thought 3: Toll is ₹390, fuel is ₹1944.44, total ₹2334.44. Budget is ₹340, so compare and finish.
Action 3: Finish[Total cost ₹2334.44 (₹390 toll + ₹1944.44 fuel) exceeds the ₹340 budget by ₹1994.44 -- trip is not affordable at this budget.]

Two things matter here beyond the arithmetic. First, the steps list was written by hand to represent what a well-behaved model would output at each turn; in a real deployment, each (thought, action) pair would come from a fresh model call that is shown the growing transcript so far and asked to produce only the next Thought and Action, one pair at a time, not all three up front. Second, run_agent is doing all of the real work: it is the only piece of code in this example that calls search_toll or calculator. The model never touches a toll database or a floating-point unit; it only ever produces text that looks like a request to.

Formalizing the Decision: Why the Agent Should Reach for the Tool

The choice the diamond in the loop makes on every turn (call a tool, or answer now) is a sequential decision problem, and it can be written down exactly as the Markov Decision Processes you met in the reinforcement-learning portion of this year's mathematics strand: a tuple (S, A, P, R, γ) of states, actions, transition probabilities, rewards, and a discount factor, with the Bellman optimality equation V*(s) = max_a [ R(s,a) + γ Σ_s' P(s'|s,a) V*(s') ] giving the value of behaving optimally from state s onward.

Model the trip-cost agent's own decision problem with three states: S0 (query received, no information gathered), S1 (toll and route confirmed but not yet reported to the user), and S2 (terminal, answer delivered). From S0 the agent can Search (cost of a wasted turn, reward −1, deterministically moves to S1) or AnswerDirectly without checking anything (reward −5, since a guessed cost is usually wrong and erodes user trust, moves to S2). From S1 it can Answer (reward +10 for a correct, grounded answer, moves to S2) or Search again pointlessly (reward −1, stays at S1). S2 is terminal with V(S2) = 0. Take γ = 0.9 and run value iteration from V₀ = 0 everywhere:

The optimal values converge to V*(S1) = 10 and V*(S0) = 8, both achieved by choosing Search at S0 and Answer at S1. Guessing pays a strictly worse expected return (−5) than paying the one-turn cost of retrieval and then answering (8), even though retrieval is the "slower" path. This is the formal version of a point ReAct makes empirically: grounding an answer in a real Observation is worth more, in expectation, than skipping straight to a plausible-sounding one, as long as the cost of one extra tool call is small relative to the reward gap between a right and a wrong answer.

From Prompt Templates to Stateful Graphs: Real Frameworks

The run_agent function above is, structurally, what the earliest production agent frameworks did: LangChain's original AgentExecutor is a while-loop around a prompt template that asks for text in exactly the Thought / Action / Observation shape, parsed with regular expressions much like the one used here. That approach is fragile in one specific way: it depends on the model reliably producing a parseable string, and any deviation (a stray character inside the brackets, a missing newline) breaks the regex and the whole run. OpenAI's function calling (mid-2023) and Anthropic's tool use (beta in November 2023, general availability in May 2024) both replaced free-text parsing with a JSON schema the model is constrained to fill in: instead of hoping the model writes search_toll[Bengaluru-Chennai-NH44] correctly, the API returns a structured object {"name": "search_toll", "input": {"route": "Bengaluru-Chennai-NH44"}} that is guaranteed to be syntactically valid, so the framework's job shrinks to validating the arguments and calling the function, not parsing prose.

Two extensions matter for anything beyond a single straight-line task. Reflexion, introduced by Noah Shinn and colleagues at NeurIPS 2023, adds a self-critique step: when a trajectory fails (the final answer is wrong, or a test the agent wrote fails), the agent generates a short verbal reflection on why, stores it in an episodic buffer, and includes that reflection in the context of its next attempt at a similar task, improving performance across attempts without touching any model weights. Toolformer, from Timo Schick and colleagues at Meta AI (NeurIPS 2023), tackled a different problem: rather than hand-writing a ReAct prompt, they showed a language model can be fine-tuned to decide for itself, from self-generated examples, when inserting a tool call would have reduced its own prediction loss, effectively teaching a base model when to reach for a calculator or search API without an explicit control loop in the prompt.

For tasks that do not reduce to one straight sequence of tool calls, LangGraph (built on top of LangChain) represents the agent explicitly as a graph: nodes are units of work (an LLM call, a tool call, a human-approval gate), and edges, including conditional edges chosen by the LLM's own output, connect them. Unlike a linear chain, this graph is allowed to contain cycles, so a node can route back to an earlier node, which is exactly what the ReAct loop needs and a strictly acyclic chain cannot express. It also supports checkpointing a run mid-graph, which is what lets a system pause an agent at a "confirm this refund" node, wait for a human to approve it out of band, and resume exactly where it left off. Multi-agent frameworks such as AutoGen and CrewAI take a different route to the same class of problem: instead of one model juggling every role, several model instances, each given a distinct system prompt (a "planner," a "researcher," a "critic"), pass messages to each other, trading the cost of inter-agent communication for the benefit of each instance carrying a shorter, more focused context.

Production Tradeoffs: Cost, Latency, and Failure Modes

Every property that makes the ReAct loop reliable also makes it expensive. Because the model has no memory of its own, the framework must resend the entire scratchpad on every single turn, so an n-turn run resends an accumulated transcript that grows on each turn, making total transmitted tokens grow roughly with the square of the number of turns, not linearly. Latency compounds the same way in the other direction: because each turn's prompt depends on the previous turn's Observation, the calls are inherently sequential, so an eight-turn agent pays for eight round trips to the model one after another, not in parallel, which is why agent responses routinely take tens of seconds where a single-shot answer takes two or three.

Because nothing in the loop guarantees the model will eventually emit Finish, every real executor needs the max_iterations guard that appears in run_agent above, or an equivalent timeout or cost cap; without it, a model stuck reasoning in circles (a known failure mode of early autonomous-loop projects such as AutoGPT in 2023) will keep calling tools and accumulating cost indefinitely. Structured function calling fixes syntactic tool-call failures but not semantic ones: a model can still emit a perfectly well-formed call to a tool that does not exist, or valid-looking arguments that are simply wrong, a failure mode distinct from ordinary text hallucination because the executor will happily run it if it matches a registered tool name. This is also why any tool with a real-world side effect (a refund, a payment, a message sent on someone's behalf) is typically wired behind an explicit confirmation node rather than let an agent execute it autonomously: the loop's job is to decide what should happen, not to be the last check before it does.

Misconception: A Prompt Does Not Make a Model Agentic

The most common error at this point is believing that a chatbot becomes an agent the moment its system prompt says "you have access to a calculator and a search tool." It does not. An LLM is a stateless function: given a sequence of tokens, it returns a probability distribution over the next token, and it has no way to execute anything, query a database, or move money, no matter what its prompt claims is available. What makes a system agentic is the surrounding control loop, the run_agent function in the worked example above, that parses a structured action out of the model's output, actually invokes the corresponding function against a real system, captures the genuine return value, and reinjects it as the next turn's input. Delete that loop and keep only the ReAct-style prompt, and the model will still produce text shaped exactly like Action: search_toll[...] immediately followed by a self-generated Observation: ..., because it has learned that pattern and next-token prediction completes patterns whether or not anything real backs them. There is no way to tell, from the transcript alone, whether an Observation came from a live API or from the model inventing a plausible-sounding number; the difference exists only in whether a real function actually ran, which is a property of the framework, not the prompt.

Diagram: The Agent Control Loop

The figure below traces the same five numbered steps used in the worked example above, from the query entering the scratchpad through the tool executor and back, to the terminal Finish branch.

Active Recall

Attempt each question before reading its answer.

1. In the MDP example, suppose the reward for AnswerDirectly from S0 is recalibrated from −5 to +12 (perhaps because for very simple routes, guessing is usually close enough). Holding every other parameter fixed, does the optimal policy at S0 change? What is the new V*(S0), and does V*(S1) change too?
2. In the worked ReAct trace, the Swift Dzire's mileage is revised from 18 km/l to 15 km/l (city driving instead of highway). Recompute the fuel cost, the total trip cost, and state whether the agent's final decision (affordable or not) changes.
3. Suppose an agent framework charges $3 per million input tokens and $15 per million output tokens (check current provider pricing before using these figures for real budgeting), each of its iterations sends roughly 1,200 input tokens and generates 300 output tokens, and it is capped at max_iterations = 5. Estimate the worst-case cost of one full run.
4. Why does ReAct ask the model to write an explicit Thought before every Action, instead of just asking it to emit the Action directly?
5. If you strip the run_agent executor loop out of the worked example and only keep the ReAct-formatted prompt, what specifically happens when the model reaches the point where it should call search_toll? Why can't you tell from the transcript alone whether a real tool ran?
6. Why do frameworks like LangGraph model an agent as a graph with cycles rather than as a linear chain, and what does that add beyond what the plain while-loop in run_agent already does?

Worked Answers

1. Recompute only the affected branch: at S0, Search still yields −1 + 0.9·V*(S1) = −1 + 0.9·10 = 8, but AnswerDirectly now yields 12 + 0.9·0 = 12. Since 12 > 8, the optimal action at S0 flips to AnswerDirectly, and V*(S0) = 12. V*(S1) stays at 10: nothing about S1's own transitions or rewards changed, and S0's parameters do not feed into S1's value calculation, only the other way around. The ripple is confined to S0's policy and value; it does not propagate downstream.

2. Fuel cost = 350 ÷ 15 × 100 = 23.333... × 100 = 2333.33 (rounded to 2 decimals). Total cost = 2333.33 + 390 (toll unchanged, since mileage does not affect the toll lookup) = 2723.33. Against the ₹340 budget, the shortfall is 2723.33 − 340 = 2383.33. The qualitative decision is unchanged (still not affordable), but the magnitude of the shortfall grows by about ₹389 compared to the original ₹1994.44 gap, purely because lower mileage burns more fuel per kilometre.

3. Per-iteration cost = 1,200 × ($3 / 1,000,000) + 300 × ($15 / 1,000,000) = $0.0036 + $0.0045 = $0.0081. Across 5 iterations: 5 × $0.0081 = $0.0405, roughly four cents per run in the worst case where the cap is reached. This is why loop caps matter for budgeting, not just correctness: an agent that reliably finishes in 2 turns instead of 5 cuts cost by more than half.

4. The Thought forces the model to commit, in its own generated tokens, to an explicit plan before it produces the action tokens that follow; since generation is autoregressive, the reasoning text becomes part of the context the action is conditioned on, which measurably improves how often the resulting action is well-formed and relevant to what is actually still needed (this is the effect Yao et al.'s ablations measured against Act-only prompting). It also gives the model a place, right after reading a new Observation, to notice a wrong assumption before compounding it into another action.

5. The model will still generate text shaped exactly like Action: search_toll[Bengaluru-Chennai-NH44] followed by a self-produced Observation: ₹..., because it has learned that surface pattern and next-token prediction completes it regardless of whether anything backs it. No real function runs, so any number in that Observation is invented, not fetched. You cannot tell from the transcript alone because a hallucinated Observation and a genuine one are syntactically identical text; the only distinguishing fact, whether search_toll actually executed against real data, lives outside the transcript, in whether an executor loop like run_agent was present to run it.

6. A plain while-loop has one implicit path: read the transcript, call the model, maybe call one tool, repeat. Production systems need conditional branching (route to a different tool depending on what the query needs), the ability to pause a run and resume it later at the exact node it stopped at (so a human can approve a risky action out of band), and composition (treat a whole sub-agent as one node inside a larger graph). Representing the agent explicitly as a graph of nodes and conditional edges, with cycles allowed so a node can route back to an earlier one, expresses all of that directly in the structure, whereas a single loop only expresses the one cycle it was written to run.

Think About It

Think about this: How would you explain ai agents and frameworks: building autonomous decision-making systems to a friend who has never seen a computer? What real-world analogy would you use? Imagine you had to build a system using these concepts — what would be your first step? Try this: before moving on, write down three things you learned and one question you still have.

Practice Exercises

Now it is time to practice! Complete these challenges to solidify your understanding:

• Exercise 1: Write a short program that demonstrates the core concept from this chapter. Test it with at least 3 different inputs.
• Exercise 2: Find a real-world example where ai agents and frameworks: building autonomous decision-making systems is used in an Indian company (like TCS, Infosys, Flipkart, or ISRO). Write a paragraph explaining the connection.
• Exercise 3: Create a mind-map connecting ai agents and frameworks: building autonomous decision-making systems to at least 3 other topics you have studied.

Key Takeaways — Summary and Recap

Let us recap what we covered: the core ideas behind ai agents and frameworks: building autonomous decision-making systems, how they connect to real-world applications, and why they matter for your journey in computer science. Remember these key points as you move forward. For competitive exam preparation (CBSE, JEE, BITSAT), focus on understanding the WHY behind each concept, not just the WHAT.