Agent Architectures: ReAct, Tree-of-Thought, Tool Use

Ask a large language model "What is the current status of PNR 4521867903?" and it will answer confidently — with a fabricated status, because IRCTC's live database is not part of its weights. The model has no way to know what it does not know; its training taught it to produce plausible-sounding continuations, not to check facts. This is not a knowledge gap that a bigger model fixes. No amount of parameters lets a frozen set of weights, trained months ago, read tomorrow's train schedule. The fix is architectural: give the model a way to call out to the world mid-reasoning, look at what comes back, and decide what to do next. That is what turns a language model into an agent — a policy that interleaves thinking with acting inside a loop, rather than a function that maps a prompt straight to an answer.

This chapter builds three specific architectural ideas that solve two different failure modes. The first failure mode is ungrounded reasoning — a model reasoning entirely from memory, unable to verify anything, prone to confident hallucination on any question requiring current or external facts. ReAct and tool use solve this. The second failure mode is irrecoverable commitment — a model reasoning once, linearly, unable to notice a dead end until it's too late to back up and try something else. Tree-of-Thought solves this. Both failure modes show up constantly in real systems, and the three ideas compose.

From chain-of-thought to ReAct

You already know chain-of-thought (CoT) prompting from your NLP work last year: ask the model to "think step by step" before answering, and it produces a sequence of intermediate reasoning tokens that improve accuracy on multi-step problems. CoT is powerful but closed — every step is generated from the model's parameters alone. If a CoT chain needs a fact the model never saw, or saw incorrectly, or that has changed since training, the chain reasons flawlessly from a wrong premise and lands on a wrong, fluently-justified answer. This is the core failure mode: a model that is good at reasoning is not thereby good at knowing.

Shunyu Yao and coauthors addressed this directly in "ReAct: Synergizing Reasoning and Acting in Language Models" (arXiv:2210.03629, published at ICLR 2023). Their proposal is structurally simple: instead of generating one uninterrupted reasoning chain, the model generates a Thought, then an Action (a call to an external tool — search, a calculator, a database query, a code interpreter), the harness executes that action and returns an Observation, and the model conditions its next Thought on that Observation. The loop repeats until the model emits a designated terminal action (commonly written Finish[answer]) that signals the trajectory is complete. Reasoning-only (CoT) has no way to update its beliefs against reality; acting-only (no Thought steps) has no way to plan which action to take next or explain why an observation matters. ReAct's contribution is showing that interleaving both, in the same autoregressive stream, lets each one correct the other — the Thought decides what to look up, and the Observation keeps the Thought honest. The paper reports that this interleaving reduces the hallucinated and error-propagating trajectories seen in CoT-only baselines and outperforms act-only baselines on multi-hop QA (HotpotQA), fact verification (FEVER), and interactive decision-making environments (ALFWorld, WebShop) — precisely because grounding and reasoning correct each other's blind spots.

Tool use: what actually crosses the model boundary

"The model calls a tool" is a convenient shorthand that hides an important boundary. The model has no side-effect capability whatsoever — it cannot open a socket, query a database, or execute code. All it can do is emit tokens. What makes tool use work is a contract external to the model: the harness (your application code, or the inference provider's serving layer) declares a set of tools as structured schemas, and the model is trained or prompted to emit a structured request matching one of those schemas instead of free text when it wants to act.

A tool schema is typically a name, a natural-language description (which the model reads to decide when the tool is relevant), and a JSON Schema describing its arguments:

{
  "name": "get_weather",
  "description": "Get tomorrow's rain forecast for an Indian city.",
  "parameters": {
    "type": "object",
    "properties": {
      "city": {"type": "string"},
      "date": {"type": "string", "format": "date"}
    },
    "required": ["city", "date"]
  }
}

When the model decides to act, it does not run this function — it emits a structured object matching the schema, for example {"name": "get_weather", "arguments": {"city": "Chennai", "date": "2026-08-28"}}. Modern serving stacks enforce this structure either through constrained decoding (the sampler is restricted to only tokens that keep the output valid JSON matching the schema) or through fine-tuning the model to reliably emit a recognizable call format that the harness parses with a regular parser. Either way, the harness — not the model — is the thing that actually calls get_weather(city="Chennai", date="2026-08-28"), catches exceptions, retries on malformed arguments, and enforces which tools are even reachable (an agent should never be handed a schema for "delete_database" unless deleting the database is genuinely an intended action). The result of that real function call is serialized back to text and appended to the conversation as an Observation, and only then does control return to the model. This round trip — propose structured call, execute outside the model, re-inject result as text — is the entire mechanism of "tool use." Every agent architecture in this chapter is built on top of it.

Two production consequences follow directly from this design. First, the conversation — and therefore the token count the model must process — grows by roughly the size of every Thought, Action, and Observation on every loop iteration, so latency and API cost scale with trajectory length, not just answer length. Second, a wrong Observation (a search tool that returns a stale or incorrect page) poisons every downstream Thought exactly as a wrong training fact would, because the model has no independent way to verify what the tool told it — grounding removes hallucination from the model's own memory, but it does not remove the possibility of being confidently wrong about the world if the tool itself is wrong.

Worked example: a ReAct trajectory end to end

Consider the question: "Which state is ISRO's Sriharikota launch site in, and how far is that state's capital from Bengaluru?" This needs two chained facts a model is unlikely to have pinned down precisely, plus an arithmetic-free lookup — a good minimal case for tracing the full loop. Below, a toy search tool is backed by a small fixed dictionary standing in for a real retrieval API, so the trace is fully reproducible without a live model call:

KB = {
    "sriharikota launch site state": "Andhra Pradesh",
    "andhra pradesh capital": "Amaravati",
    "bengaluru to amaravati distance": "740 km (approx., by road)",
}

def search(query):
    """Toy retrieval tool: looks up a fixed knowledge base.
    A real agent would call a live search API instead."""
    key = query.strip().lower()
    return KB.get(key, "No result found.")

# Scripted for reproducibility: in a live agent every Thought/Action
# string below is generated token-by-token by the model. Here they are
# fixed strings so the trace can be checked line by line.
trace = [
    ("Thought", "I need the state where ISRO's Sriharikota launch site sits."),
    ("Action",  'search("sriharikota launch site state")'),
    ("Observation", search("sriharikota launch site state")),
    ("Thought", "Now I need that state's capital."),
    ("Action",  'search("andhra pradesh capital")'),
    ("Observation", search("andhra pradesh capital")),
    ("Thought", "Now the road distance from Bengaluru to that capital."),
    ("Action",  'search("bengaluru to amaravati distance")'),
    ("Observation", search("bengaluru to amaravati distance")),
    ("Thought", "I have state, capital, and distance — ready to answer."),
    ("Action",  'finish("Sriharikota is in Andhra Pradesh; its capital Amaravati is about 740 km by road from Bengaluru.")'),
]

for step, content in trace:
    print(f"{step}: {content}")

Tracing this line by line: each search(...) call's argument, after stripping the surrounding quotes, is exactly a key present in KB — "sriharikota launch site state", "andhra pradesh capital", and "bengaluru to amaravati distance" — so every lookup hits and none falls through to the "No result found." branch. The printed output is therefore exactly eleven lines: four Thought/Action pairs — the fourth Action being the terminal finish(...) call — interleaved with the three Observations returned by search. Notice what the Observations actually contribute: without them, a model asked this question in one shot would need to recall Sriharikota's state, that state's capital, and a road distance, all from memory — three independent chances to be subtly wrong, with no way for the model itself to detect the error. With ReAct, each fact is fetched and checked against the previous Thought before the next one is formed.

The scripted trace above only prints a trajectory — it doesn't show the control logic that actually drives one. A real orchestrator dispatches on the model's Action text and stops when it sees finish:

def run_agent(question, llm_step_fn, max_steps=6):
    """Generic ReAct control loop.
    llm_step_fn(history) -> (kind, content) is the model call itself
    (assumed helper, not shown — in deployment this hits the model API)."""
    history = [f"Question: {question}"]
    for _ in range(max_steps):
        kind, content = llm_step_fn(history)
        history.append(f"{kind}: {content}")
        if kind == "Action":
            if content.startswith("finish("):
                return content[len("finish("):-1].strip('"')
            elif content.startswith("search("):
                q = content[len("search("):-1].strip('"')
                obs = search(q)
                history.append(f"Observation: {obs}")
    return None  # ran out of steps without finishing

Two design choices here matter beyond this toy example. max_steps is a hard budget — without it, a model that never emits finish(...) loops forever, silently consuming cost. And parsing is done on plain string prefixes only for illustration; a production system uses the structured tool-calling contract from the previous section instead of prefix-matching free text, precisely because free text is far easier for a model to get subtly wrong than a schema-validated JSON object.

When one trajectory isn't enough: Tree-of-Thought

ReAct still commits to a single linear trajectory — one Thought leads to one Action leads to one next Thought, with no mechanism to notice mid-stream that an earlier choice was bad and try a different one. That is fine for tasks where each step either succeeds or fails cleanly and grounding prevents most errors. It breaks down on tasks that are genuinely combinatorial — where a step that looks locally reasonable can still lead to a dead end several steps later, and the only way to find a working solution is to explore several candidate continuations and abandon the ones that stop looking promising.

Yao and coauthors (a mostly overlapping author list to the ReAct paper) formalized this in "Tree of Thoughts: Deliberate Problem Solving with Large Language Models" (arXiv:2305.10601, NeurIPS 2023). Tree-of-Thought (ToT) treats each intermediate reasoning state as a node in a search tree rather than a line in a single transcript. At each node, the model is prompted to propose several candidate next steps (branching factor b); a separate prompt asks the model to evaluate each resulting state's promise of reaching the goal (a self-assessment, often phrased as "sure / likely / impossible" or a numeric score); and a search algorithm — breadth-first or depth-first — uses those evaluations to decide which states to keep expanding and which to prune, with the option to backtrack if every child of a kept state turns out to be a dead end. The load-bearing shift from ReAct is that the model's own output is now used twice: once to generate candidates, and once to judge them, and only the judged-best few survive to the next depth.

Worked example: Game of 24 as a search tree

The ToT paper uses the "Game of 24" as one of its controlled benchmarks precisely because it is small enough to trace by hand: given four numbers, combine all of them using +, −, ×, ÷ (each used exactly once, in any order) to reach exactly 24. Take the numbers {3, 3, 8, 8}. A single linear guess is unreliable here — most combinations don't reach 24, and a model committing to one path with no way to reconsider will often fail outright. Framed as a tree, reaching 24 from four numbers takes exactly three binary combine-steps (four numbers → three → two → one), so the search tree has depth 3.

Set the search parameters to branching factor b = 3 (the model proposes three candidate operations from each open state) and keep width k = 2 (only the two most promising states survive each round of evaluation):

The exhaustive column is derived independently, as a sanity bound, and its Total row deliberately repeats the depth-3 figure rather than summing the column (36 + 648 + 3,888 = 4,572 states would exist across all three depths combined) — it is the depth-3 leaf count alone that the speedup comparisons below use throughout, since that is the full space of finished four-number combinations, not partial states at earlier depths. At depth 1 there are C(4,2) = 6 unordered pairs among the four numbers, and each pair admits up to 6 distinct operation-results (a+b, a×b, a−b, b−a, a÷b, b÷a), giving 6 × 6 = 36 possible next states if nothing is pruned. After one combine, three numbers remain, so depth 2 has C(3,2) = 3 pairs × 6 = 18 possible next states per surviving depth-1 node, and with all 36 depth-1 states kept that's 36 × 18 = 648. Depth 3 has one remaining pair × 6 = 6 final combines per depth-2 node, giving 648 × 6 = 3,888 leaf states total. ToT's sampled proposals (3 per expansion, not all 36 possible operations) combined with pruning down to the best 2 states per depth touches only 15 states across the same three depths — 3,888 ÷ 15 ≈ 259 times fewer states examined, at the cost of roughly 5 proposal calls (one per expanded node: 1 + 2 + 2) plus roughly 15 evaluation calls (one per generated state), around 20 model calls total, against the single call a one-shot chain-of-thought guess would cost.

One path the search actually finds: from {3, 3, 8, 8}, combine 8 ÷ 3 = 8/3 (state becomes {3, 8, 8/3}); combine 3 − 8/3 = 1/3 (state becomes {8, 1/3}); combine 8 ÷ (1/3) = 24 exactly (goal reached). This is exactly 8 ÷ (3 − 8 ÷ 3) = 24, using each of the two 3s and two 8s once. A pure CoT attempt that committed to a different first move — say 3 + 3 = 6, leaving {6, 8, 8} — cannot reach 24 from there at all (6+8+8=22, 6×8÷8=6, 8×8÷6≈10.7, none work), and a linear chain has no built-in mechanism to notice this and go back; it either fails silently or needs an entirely separate retry from scratch. ToT's evaluator is what catches a state like {6, 8, 8} as low-promise before three more expensive steps are spent expanding it.

Common misconception: Tree-of-Thought is not "self-consistency in disguise"

A very natural mistake at this point is to conflate ToT with self-consistency (Wang et al., "Self-Consistency Improves Chain of Thought Reasoning in Language Models," arXiv:2203.11171, ICLR 2023): sample several independent full chain-of-thought answers to the same question and take a majority vote over the final answers. Both methods spend extra compute generating multiple reasoning paths, which is exactly why they get confused for each other. But the mechanisms are structurally different. Self-consistency's paths are generated independently from the root and never interact — there is no intermediate evaluation, no pruning, and no way for one path to learn anything from another; the only aggregation happens at the very end, by counting which final answer appeared most often. ToT's paths share a tree: intermediate states are explicitly scored by an evaluator prompt, low-scoring branches are cut before they consume any further compute, and the search can backtrack. Self-consistency answers "which of these fully-formed guesses do most of my samples agree on?" ToT answers "which partial attempt is most worth continuing, checked at every step along the way?" A depth-1 ToT run is still not the same as self-consistency at that depth, either — ToT selects its single kept state using the evaluator's judgment of quality, while self-consistency selects using a vote count with no notion of quality at all.

Combining the ideas, and when each one is worth its cost

ReAct and ToT are not competitors — they answer different questions and compose. Zhou et al. (ICML 2024), "Language Agent Tree Search Unifies Reasoning, Acting, and Planning in Language Models," build an agent that runs ToT-style branching search where each node's evaluation is grounded by actually taking ReAct-style tool actions and observing results, rather than trusting the model's unaided self-assessment of a branch's promise — and adds reflection, where a failed branch's trajectory is summarized back into the prompt so the next attempt doesn't repeat the same mistake. This is expensive: multiplying tree search's already-large call count by ReAct's per-step tool-call overhead compounds fast, so it is reserved for tasks where being wrong is costly and a single grounded guess demonstrably isn't reliable enough — coding agents verifying against real test execution, or multi-step planning tasks with real external dependencies.

The practical choice is a cost-accuracy tradeoff, and the numbers above make it concrete. A single deterministic lookup — "what's the weather," "what's this PNR's status" — needs plain tool use: one Action, one Observation, done, no search required because there's only one correct answer to fetch. A multi-hop but linear question — the Sriharikota example — needs ReAct: several grounded facts chained together, each one a single correct lookup, no branching required because no step has multiple plausible candidates worth comparing. A combinatorial or planning task where a locally-reasonable first move can still be a dead end three steps later — Game of 24, or comparing several candidate multi-leg itineraries under a changing constraint — needs ToT's branch-evaluate-prune search, at roughly twenty times the model calls of a single guess, because a single guess is measurably unreliable there. Reaching for ToT on a task that ReAct already solves reliably is wasted compute; reaching for a single ReAct trajectory on a task that requires comparing competing candidates is a coin flip dressed up as reasoning.

Active recall

Attempt each question before reading its answer.

Q1. In the Game of 24 search, keep width stays k = 2 but branching factor increases from b = 3 to b = 4. Recompute the total states generated across all three depths, the total model calls (proposals + evaluations), and the speedup factor over the 3,888-state exhaustive bound.

Q2. Now instead keep b = 3 fixed and raise keep width from k = 2 to k = 3. Recompute the same three quantities, and note which quantity changes for a different reason than in Q1.

Q3. In the ReAct trace, suppose the search tool had a stale index and returned "Telangana" instead of "Andhra Pradesh" for the first query. Trace what happens to the rest of the trajectory. Does ReAct's architecture detect this error anywhere?

Q4. Why does the ReAct loop need a designated terminal action like finish(...) rather than simply stopping whenever the model produces a plausible-looking answer?

Q5. True or false, with justification: "Self-consistency is just Tree-of-Thought with tree depth 1."

Q6. Using the get_weather(city, date) schema from the tool-use section, write the Thought and Action lines a ReAct agent should produce for "Should I carry an umbrella in Chennai tomorrow?" (today is 2026-08-27), followed by a plausible Observation and the concluding Thought and finish Action.

A1. Depth 1: 1 parent × 4 = 4 states. Depth 2: 2 kept parents × 4 = 8 states. Depth 3: 2 kept parents × 4 = 8 states. Total states = 4 + 8 + 8 = 20 (up from 15). Proposal calls are unchanged at 5 (1 + 2 + 2), because that count depends on how many nodes get expanded, not on b; evaluation calls rise to 20 (one per generated state), so total model calls = 5 + 20 = 25 (up from 20). Speedup over 3,888 exhaustive states = 3,888 ÷ 20 ≈ 194×, smaller than the original ≈259× — wider proposals examine more of the space per step, still an enormous reduction versus exhaustive search, but a less efficient one.

A2. Depth 1: only 3 states exist (b = 3), and k = 3 keeps all of them (no pruning at depth 1). Depth 2: 3 kept parents × 3 = 9 states. Depth 3: 3 kept parents × 3 = 9 states. Total states = 3 + 9 + 9 = 21. The non-obvious ripple: proposal calls rise too, from 5 to 1 + 3 + 3 = 7, because raising k means more parent nodes survive to be expanded at the next depth — a parameter that only appeared to control "how many states you keep" also controls "how many expansion calls you pay for" at every subsequent depth. Total model calls = 7 + 21 = 28. Speedup over 3,888 = 3,888 ÷ 21 ≈ 185×.

A3. The second query becomes search("telangana capital"), which is not a key in KB, so search returns the literal string "No result found." — the trajectory doesn't silently continue with a wrong-but-plausible capital, it stalls on a visibly failed lookup, which a well-designed agent can treat as a retry or backtrack signal. But if the first Observation itself had returned a wrong-but-real answer (say a different, actually-existing state), nothing in ReAct's architecture would catch that — grounding only guarantees the model is reasoning over what the tool said, not that what the tool said is true. ReAct removes reasoning-time hallucination; it does not remove upstream data error, and this is exactly the tool-reliability caveat raised in the tool-use section.

A4. Without a fixed terminal signal, the harness has no principled way to know the trajectory is finished — the model's free-text output doesn't carry a machine-checkable "done" flag on its own. The loop would either run to an arbitrary step cap regardless of whether an answer was already found (wasting cost and latency), or the harness would need a separate, unreliable heuristic to guess when the model "sounds finished," which fails silently on any answer that happens not to match the heuristic's pattern.

A5. False. Even at tree depth 1, ToT generates b candidates and selects among them using an evaluator prompt that judges each candidate's quality before selection. Self-consistency generates k full independent chains and selects by majority vote over final answers, with no evaluator ever judging intermediate or final quality — it only counts agreement. A depth-1 ToT tree can discard every candidate that shares a wrong intermediate step if the evaluator flags it; self-consistency would still let a majority of subtly-wrong-but-mutually-agreeing chains outvote a lone correct one.

A6.

Thought: To advise on an umbrella, I need tomorrow's rain forecast for Chennai.
Action: get_weather("Chennai", "2026-08-28")
Observation: {"temp_c": 29, "precip_probability": 0.72}
Thought: A 72% chance of rain is high enough to recommend carrying one.
Action: finish("Yes — about 72% chance of rain in Chennai tomorrow, carry an umbrella.")

Think About It

Think about this: How would you explain agent architectures: react, tree-of-thought, tool use to a friend who has never seen a computer? What real-world analogy would you use? Imagine you had to build a system using these concepts — what would be your first step? Try this: before moving on, write down three things you learned and one question you still have.

Practice Exercises

Now it is time to practice! Complete these challenges to solidify your understanding:

• Exercise 1: Write a short program that demonstrates the core concept from this chapter. Test it with at least 3 different inputs.
• Exercise 2: Find a real-world example where agent architectures: react, tree-of-thought, tool use is used in an Indian company (like TCS, Infosys, Flipkart, or ISRO). Write a paragraph explaining the connection.
• Exercise 3: Create a mind-map connecting agent architectures: react, tree-of-thought, tool use to at least 3 other topics you have studied.

Key Takeaways — Summary and Recap

Let us recap what we covered: the core ideas behind agent architectures: react, tree-of-thought, tool use, how they connect to real-world applications, and why they matter for your journey in computer science. Remember these key points as you move forward. For competitive exam preparation (CBSE, JEE, BITSAT), focus on understanding the WHY behind each concept, not just the WHAT.