Production AI Agents with LangGraph and Langfuse: Part 2, Tools, Human-in-the-Loop, and Traceable Loops

A returns agent went live on a Friday. By Monday it had refunded one customer twice because a tool call timed out, retried, and the agent had no idea the first refund went through. Another conversation looped eleven times trying to "confirm the order status" before hitting a token limit. The on-call engineer's only evidence was a 4,000-line JSON log with no structure. Nobody could answer the one question that mattered: what did the agent actually do, and why?

Agents fail differently from RAG. RAG is a mostly-fixed pipeline; an agent decides its own next step, calls tools that have real side effects, and loops until it thinks it is done. That freedom is the point. It is also the danger. Production safety for agents needs three things RAG can defer longer: checkpointing so state survives, human-in-the-loop so risky actions get approved, and per-step tracing so "what did it do and why" is a question you answer by reading a trace.

Part 1 built production RAG as a LangGraph graph with Langfuse on every node. This post does the same for tool-using agents. The walkthrough follows one e-commerce order and returns agent: it looks up orders, checks refund policy, issues refunds (with human approval above a threshold), and escalates when it cannot help.

The agent mental model in LangGraph#

Strip an agent to its core and it is a loop: the model looks at the conversation and decides either to call a tool or to answer. If it calls a tool, you run the tool, append the result to the conversation, and let the model look again. It repeats until the model answers without a tool call.

That loop (model → tools → model) is the ReAct pattern (Yao et al., arXiv:2210.03629). LangGraph expresses it as a tiny graph: one reasoning node, one tool node, and a conditional edge that loops back while there are tool calls to run.

LangGraph gives you two ways to build this. create_react_agent is a prebuilt one-liner that wires the standard loop. Use it to start. The moment you need custom routing, approval gates, or multiple agents, move to a hand-built StateGraph where you control every edge. We build the returns agent the explicit way because that is what production looks like.

What LangGraph gives you (and where each piece fits)#

Before the walkthrough, here is the agent toolbox in one place. Each row is a LangGraph primitive, the phase where it earns its place, and what it does in the returns-agent graph. Treat this table as the roadmap; every phase section below maps back to a row.

The rest of Part 2 walks these phases in order. Read it top to bottom the first time; after that, jump to the row you need.

Phase 0: State and scope#

Every node in the graph reads and writes shared state. For an agent, that state is mostly the conversation. LangGraph ships MessagesState, a state type whose messages field uses the add_messages reducer. Each node appends to the history rather than overwriting it. That append semantics is what the agent loop needs: the model's tool-call request and the tool's result both get added, and the model sees the full thread on its next turn.

We extend MessagesState with fields the returns agent needs: the resolved customer id and a flag for whether a refund is pending human approval.

from langgraph.graph import MessagesState

class AgentState(MessagesState):   # inherits the `messages` field + add_messages reducer
    customer_id: str
    pending_refund: dict | None

Scope is the other half of Phase 0, and it is a safety decision, not a code one. This agent handles order status, returns, and refunds. Nothing else. Anything outside that scope escalates to a human. Define that boundary up front so the agent does not confidently improvise in situations it was never meant to handle.

Phase 1: Perception and routing#

Not every incoming message needs the full reasoning loop. A returns desk would route "where is my order" differently from "I want a refund for a damaged item" or "I want to speak to a manager." The first node in the graph does that triage: cheap routing up front saves model calls and keeps high-risk paths explicit.

In LangGraph a node can return a Command that both updates state and names the next node. That combines "what did I learn" and "where do I go" in one object.

from langgraph.types import Command
from typing import Literal

def triage(state: AgentState) -> Command[Literal["agent", "human_escalation"]]:
    last = state["messages"][-1].content.lower()
    if "manager" in last or "lawsuit" in last:
        return Command(goto="human_escalation")
    return Command(goto="agent")

Command is documented under LangGraph's control flow. In the Langfuse trace, this triage decision is the first span. When an interaction goes wrong, you see immediately which path it took.

Phase 2: Reasoning and planning#

Once triage sends a message to the agent node, the model decides what happens next. Given the conversation and the schemas of the tools it can call, it either answers directly or requests tool calls. Bind tools with bind_tools. We use Claude Opus 4.8 (claude-opus-4-8) here because tool selection and policy reasoning are where a stronger model earns its cost.

from langchain_anthropic import ChatAnthropic

llm = ChatAnthropic(model="claude-opus-4-8", temperature=0)
llm_with_tools = llm.bind_tools([lookup_order, check_refund_policy, issue_refund])

def agent(state: AgentState) -> dict:
    system = (
        "You are an order and returns assistant. Use tools to look up orders and "
        "check policy before any refund. Never invent order details. "
        "Refunds over $100 require human approval."
    )
    response = llm_with_tools.invoke(
        [{"role": "system", "content": system}, *state["messages"]]
    )
    return {"messages": [response]}

For a returns flow, step-by-step ReAct is the right default. Each tool result changes what to do next. In Langfuse this node is a generation span showing the tool calls the model chose and the arguments it filled in. That is the first place to look when an agent misbehaves.

Phase 3: Tool execution#

When the model requests a tool call, something outside the LLM has to run. Tools are where the agent touches the real world, and where real money moves. LangGraph's ToolNode takes the tool calls the model produced, runs the matching functions, and appends their results back to the conversation as tool messages. Define tools as plain functions with the @tool decorator; the docstring becomes the schema the model reads.

from langchain_core.tools import tool
from langgraph.prebuilt import ToolNode

@tool
def lookup_order(order_id: str) -> dict:
    """Look up an order's status and contents by its ID."""
    return orders_db.get(order_id)

@tool
def issue_refund(order_id: str, amount: float) -> dict:
    """Issue a refund for an order. Amounts over $100 require approval."""
    return payments.refund(order_id, amount)

tool_node = ToolNode([lookup_order, check_refund_policy, issue_refund])

Two production concerns live here.

Tool errors: an API timeout should not crash the agent. ToolNode can return the error to the model so it can react ("the lookup failed, let me retry or escalate") instead of killing the graph.

Idempotency: tools get retried. A side-effecting tool like issue_refund must be safe to call twice. Pass an idempotency key so a retry does not double-refund. That is the bug from the intro, and it is a tool-design fix, not an agent-design one.

In the trace, every tool call is its own span with arguments, return value, and latency.

Phase 4: The agent loop and termination#

Phases 2 and 3 cover one turn: the model reasons, then tools run. Phase 4 wires them into the ReAct loop from the mental model section. After the agent node reasons, a conditional edge checks whether the model requested tool calls. If yes, go to the tool node, then back to the agent to interpret the results. If no, the agent answered and you end. LangGraph ships this router as tools_condition.

from langgraph.graph import StateGraph, START, END
from langgraph.prebuilt import tools_condition

builder = StateGraph(AgentState)
builder.add_node("triage", triage)
builder.add_node("agent", agent)
builder.add_node("tools", tool_node)

builder.add_edge(START, "triage")
builder.add_conditional_edges("agent", tools_condition)   # -> "tools" or END
builder.add_edge("tools", "agent")                        # loop back

The danger of any loop is that it never terminates. That is the eleven-times-around bug from the intro. LangGraph guards this with recursion_limit: a hard cap on steps per run. Set it based on how many tool calls a legitimate interaction needs.

graph.invoke(input, {"recursion_limit": 12, **config})

In Langfuse the loop renders as a nested step tree (agent → tools → agent → tools). A pathological loop is visually obvious: the same tool called with the same arguments, over and over.

Phase 5: Human-in-the-loop#

Tool calls can move money or cancel orders. Some actions are too consequential to let an agent take alone. LangGraph's interrupt() pauses the graph mid-execution, surfaces a payload to your application for a human to review, and waits. When the human responds, you resume from the exact pause point with Command(resume=...). This only works because the checkpointer persisted the full state at the pause. Interrupts and checkpointing are inseparable.

from langgraph.types import interrupt, Command

def approve_refund(state: AgentState) -> dict:
    refund = state["pending_refund"]
    if refund["amount"] > 100:
        decision = interrupt({          # pause; show this to a human
            "action": "refund",
            "order_id": refund["order_id"],
            "amount": refund["amount"],
        })
        if decision != "approved":
            return {"messages": [("assistant", "Refund needs manager review; escalating.")]}
    result = issue_refund.invoke(refund)
    return {"messages": [("assistant", f"Refund processed: {result}")]}

Your application detects the interrupt, shows the approval UI, and resumes:

state = graph.invoke(input, config)
if "__interrupt__" in state:                    # agent is waiting for a human
    human_choice = show_approval_ui(state["__interrupt__"])
    graph.invoke(Command(resume=human_choice), config)   # resume exactly where it paused

This is documented under human-in-the-loop. Because state was checkpointed, the human can take five minutes or five hours. The agent waits and resumes cleanly. In Langfuse, the interrupt and resumption are both visible: who approved which refund, and when.

Phase 6: Memory#

Once the graph runs across multiple turns or sessions, you need memory beyond the current message list. LangGraph separates two kinds.

Short-term memory is the conversation, handled by the checkpointer keyed on thread_id. Same mechanism as Part 1. Pass the same thread id and the agent remembers the current conversation, including any pending approval.

Long-term memory is what persists across conversations: past returns, customer tier, a note that they had a bad experience last month. LangGraph's store is a cross-session key-value layer for this. The agent reads from the store at the start to personalize and writes at the end to remember.

from langgraph.store.memory import InMemoryStore

store = InMemoryStore()
graph = builder.compile(checkpointer=checkpointer, store=store)

# inside a node, namespaced per customer:
def agent(state, store):
    history = store.get(("customer", state["customer_id"]), "returns_history")
    ...

Short-term memory is automatic and per-thread. Long-term memory is explicit: you decide what to write. The store is documented under persistence and memory. In production, back it with Postgres, same as the checkpointer.

Phase 7: Multi-agent and subgraphs#

A single agent stays manageable while its tool set and instructions stay narrow. One agent with ten tools becomes hard to reason about and easy to confuse. When that happens, split into specialists (returns, billing) coordinated by a supervisor that routes each request. LangGraph supports this because a compiled graph can be a node inside another graph (a subgraph). The supervisor is a graph whose nodes are other agents.

supervisor = StateGraph(AgentState)
supervisor.add_node("returns_agent", returns_graph)    # a compiled subgraph
supervisor.add_node("billing_agent", billing_graph)
supervisor.add_node("route", route_to_specialist)

Do not reach for this early. A single well-scoped agent is simpler and easier to trace. Graduate to multi-agent only when one agent's tool set and instructions have genuinely grown too broad. Patterns are in multi-agent systems. For evaluation, see Evaluating Small Agent Workflows. In Langfuse, subgraph spans nest under the parent trace: supervisor routing decision, then the chosen specialist's full loop underneath.

Phase 8: Streaming#

Agents feel slow because they think in steps, not because each step is inherently slow. Streaming makes the steps visible. Instead of a spinner, the user sees "Looking up your order… Checking the return policy… Processing your refund." LangGraph's stream_mode="updates" emits each node's output as it completes, which maps to user-facing status. stream_mode="messages" streams the model's tokens for the final answer.

for chunk in graph.stream(input, config, stream_mode="updates"):
    node, update = next(iter(chunk.items()))
    show_status(node)   # "tools", "agent", ...

The steps you stream are the same node boundaries Langfuse records. One mechanism, two audiences: the user sees status, you see a trace.

Phase 9: Observability and evaluation with Langfuse#

Part 1 wired Langfuse at the callback layer so every RAG node became a span. Agents use the same hook, but the trace tree matters more here because the failure mode is a sequence of decisions, not a single bad retrieval.

Pass the Langfuse callback handler in config and the entire agent run becomes a trace:

from langfuse.langchain import CallbackHandler

langfuse_handler = CallbackHandler()
config = {
    "configurable": {"thread_id": "returns-session-3310"},
    "callbacks": [langfuse_handler],
    "recursion_limit": 12,
    "metadata": {"langfuse_session_id": "returns-session-3310", "langfuse_user_id": "cust_4471"},
}

A single agent trace shows: triage → agent (with chosen tool calls) → tools (arguments, results, latency) → agent → approval interrupt → resume → final answer. Full setup: Langfuse LangChain docs.

Tracing tells you what happened. Evaluation tells you whether it was correct. Agents need both, and RAG pipelines often skip the second:

• Cost and latency per session. Agents make many model calls. Langfuse sums them so you catch the conversation that cost forty calls instead of four.
• Tool-call correctness. Score whether the agent chose the right tool with the right arguments. This is the most common failure mode.
• Task success. Did the conversation resolve the issue? Score it (LLM judge or human label) and trend it.
• Datasets and replays. Build representative conversations and replay on every release so a prompt change that breaks the refund path is caught before deploy.

Phase 10: Productionizing and safety#

Agents take actions in the real world, so their production checklist is heavier than RAG's. The phases above introduce the primitives; this section is the minimum bar before you ship.

Idempotency for side effects. Every tool that changes the world needs an idempotency key. The second call with the same key should be a no-op. That is the direct fix for the double-refund bug.

Recursion and cost limits. Always set recursion_limit. Consider a per-session cost ceiling. A confused agent without limits is an unbounded bill.

Durability. With a Postgres checkpointer, an agent paused for human approval survives a deploy. A crash mid-loop resumes from the last completed step instead of re-running tool calls. See durable execution.

Approval gates on anything irreversible. Start with humans in the loop for every consequential action. Remove gates only when trace data earns your trust. Broader context: Observability for AI Systems.

Common mistakes and tradeoffs#

The core tradeoff is autonomy versus control. More tools and fewer gates make the agent capable and fast. More gates and tighter scope make it safe and predictable. LangGraph does not resolve that tension. It gives you the dials (interrupt, recursion_limit, scope) to set deliberately, and Langfuse gives you the evidence to know when you can safely loosen one.

Key Takeaways#

• An agent is a loop (model → tools → model). LangGraph expresses it with ToolNode and tools_condition. Start with create_react_agent; move to a custom StateGraph when you need control.
• interrupt() is the key production primitive. It pauses for approval; Command(resume=...) continues. It only works because the checkpointer made the paused state durable.
• Side-effecting tools must be idempotent. Consequential ones belong behind approval gates. That prevents the double-refund class of bug.
• Always set recursion_limit so a confused agent cannot loop forever.
• Langfuse renders the agent as a step tree with arguments, results, latency, and cost per step.
• Evaluate agents on tool-call correctness and task success, not just the final answer. Replay datasets before every release.

FAQ#

When should I use an agent instead of RAG or a fixed workflow?

Use a fixed workflow when the steps are known in advance. Use RAG when the task is "answer from documents." Reach for an agent only when the system needs to decide its own next action and call tools with side effects. Agents are more powerful and more dangerous. Do not use one where a workflow suffices.

Should I use create_react_agent or build a custom graph?

Start with create_react_agent for prototypes. Move to a custom StateGraph when you need custom routing, approval gates, multiple agents, or anything the prebuilt loop does not express. The returns agent in this guide needed approval gates, so it is custom.

How do I stop an agent from looping forever?

Set recursion_limit on every invocation. Watch the Langfuse trace for the same tool called repeatedly with the same arguments. That is the signature of a stuck loop. Tighten the prompt or add a termination condition.

Is human-in-the-loop required?

Not technically. For any irreversible or costly action it should be your default until trace data proves the agent is reliable. interrupt() is cheap to add and easy to remove later. An unsupervised mistake on a real customer almost always costs more than an approval click.

What is the difference between the checkpointer and the store?

The checkpointer is short-term, per-thread memory: the current conversation, persisted so interrupts and resumes work. The store is long-term, cross-session memory: what you want to remember about a customer between conversations. Checkpointing is automatic; writing to the store is explicit.

How do I evaluate an agent, not just an answer?

Score the steps, not only the output: tool-call correctness and task success. Build a Langfuse dataset of representative conversations and replay on every release. See Evaluating Small Agent Workflows for the methodology.