How I Built a Customer Support AI Agent with LangGraph

Customer Support AI Agent with LangGraph

A mistake I often see from people new to Generative AI is mixing up a chatbot with an agent. A chatbot just talks to you, while an agent can actually take action. If you’ve ever used a support bot that keeps apologizing but never checks your account, you know what I mean. Real industry solutions need AI that can think, get real-time data, and know when to let a human step in. I recently built a fully functional customer support AI agent using LangGraph.

In this article, I’ll show you how I built it. We’ll go over the architecture, state management, tool use, and how to hand things off to a human.

Why LangGraph?

When I started building LLM applications, simple step-by-step chains worked fine. You’d take an input, send it to a prompt, run it through the LLM, and get some text back. But customer support isn’t usually that straightforward. A user might ask about their order, then decide to cancel, or get upset and want to talk to a manager.

Standard chains can’t handle this kind of back-and-forth. That’s where LangGraph made a big difference for me. By setting up the AI workflow as a stateful graph with loops and memory, you let the LLM reason, take action, see what happens, and try again if needed.

Let’s get started with the implementation.

Customer Support AI Agent with LangGraph

Before getting started, make sure you have Ollama installed on your system. Once installed, pull the model:

ollama pull qwen3:8b

Next, install/update the dependencies:

pip install -U langgraph langchain-core langchain-ollama

Now, let’s go through the steps one by one.

Step 1: Defining the Agent’s Brain and State

The heart of any LangGraph app is its State. You can think of the state as the agent’s memory for each conversation. Every node in the graph reads from this state and writes back to it:

from typing import Annotated
from typing_extensions import TypedDict
from langgraph.graph.message import add_messages
from langchain_core.tools import tool
from langchain_core.messages import HumanMessage
from langchain_ollama import ChatOllama
from langgraph.graph import StateGraph, START, END
from langgraph.prebuilt import ToolNode, tools_condition
from langgraph.checkpoint.memory import MemorySaver

# 1. DEFINE THE SHARED STATE
class GraphState(TypedDict):
    messages: Annotated[list, add_messages]

Here, I define a simple state that keeps a list of messages. The add_messages function is important because it tells LangGraph to add new messages to the list instead of replacing them, so the conversation history stays intact.

Step 2: Equipping the Agent with Real Tools

An LLM by itself can’t do much. To turn it into a customer support agent, we need to give it tools so it can interact with the outside world. In my experience, making good tools with clear docstrings is key, since the LLM depends on those docstrings to know when and how to use each tool:

# 2. DEFINE CUSTOMER SUPPORT TOOLS

@tool
def check_order_status(order_id: str) -> str:
    """Check the shipping status of a customer's order."""
    # In a real application, this would query
    # a SQL database or an order-management API.
    if order_id == "12345":
        return "Order 12345 is out for delivery."
    return "Order not found. Please verify the order ID."

@tool
def escalate_to_human(reason: str) -> str:
    """Escalate the conversation to a human support agent
    when the customer is frustrated or the issue requires
    manual review.
    """
    return (
        f"ESCALATION TRIGGERED: {reason}. "
        "A human support agent will take over."
    )

tools = [
    check_order_status,
    escalate_to_human,
]

I added two practical tools here. check_order_status acts like it’s checking a company database. escalate_to_human is probably the most important feature in any enterprise AI. You don’t want an AI trying to handle an emotional dispute. By giving the LLM a clear way to escalate, it learns to step aside when needed.

Step 3: Initializing the Local Open-Source LLM

For this project, I wanted to avoid expensive API calls. One of the best ways to prototype now is with local models. I used Ollama with the Qwen3:8b model. It’s lightweight, great at calling tools, and completely free:

# 3. INITIALIZE THE LOCAL LLM

llm = ChatOllama(
    model="qwen3:8b",
    temperature=0,
)

# Bind the tools to the model
llm_with_tools = llm.bind_tools(tools)

Setting the temperature to 0 makes the model as factual and predictable as possible. Customer support isn’t the place for creative guesses.

Step 4: Building the Nodes and Graph Logic

Now let’s look at the architecture. In LangGraph, you define “nodes” that do the work and “edges” that control how things flow:

# 4. DEFINE THE AGENT NODE
def chatbot_node(state: GraphState):
    """
    The agent reads the conversation history and decides
    whether to answer the customer directly or call a tool.
    """
    response = llm_with_tools.invoke(
        state["messages"]
    )
    return {
        "messages": [response]
    }

# 5. CREATE THE TOOL NODE
tool_node = ToolNode(tools)

# 6. CREATE THE LANGGRAPH WORKFLOW
builder = StateGraph(GraphState)

# Add the agent node
builder.add_node("agent", chatbot_node)

# Add the tool node
builder.add_node("tools", tool_node)

The chatbot_node is the decision-maker. It checks the user’s message and either answers directly or asks to use a tool. If it needs a tool, the flow goes to the tool_node, which runs our Python functions.

Now, we wire them together with conditional routing:

# 7. DEFINE THE GRAPH ROUTING

# Start → Agent
builder.add_edge(START, "agent")

# Agent → Tools OR END
# tools_condition checks whether the LLM response
# contains a tool call.
builder.add_conditional_edges("agent", tools_condition)

# Tool → Agent
# After the tool executes, its result is sent back
# to the agent so the model can generate the final answer.
builder.add_edge("tools", "agent")

This routing logic is how I build real-world systems. The tools_condition checks what the LLM wants to do. If it needs a tool, it routes to “tools.” After the tool gets the data, the flow returns to the “agent” node so the LLM can read the result and create a helpful response for the customer.

Step 5: Persistent Memory and Execution

An agent isn’t helpful if it forgets what you said a few minutes ago. By adding a simple MemorySaver, LangGraph saves the state after every interaction:

# 8. ADD MEMORY
memory = MemorySaver()

# Compile the graph
support_agent = builder.compile(checkpointer=memory)

# 9. CREATE A CONVERSATION THREAD
config = {
    "configurable": {
        "thread_id": "customer_support_thread_1"
    }
}

# 10. GET USER INPUT
user_input = HumanMessage(
    content="Where is my order? The ID is 12345."
)

# 11. RUN THE AGENT
print("\nCustomer:")
print(user_input.content)

print("\nAgent:\n")

for event in support_agent.stream(
    {
        "messages": [user_input]
    },
    config
):
    for value in event.values():
        message = value["messages"][-1]
        # Print messages that contain text
        if message.content:
            print(message.content)
Customer:
Where is my order? The ID is 12345.

Agent:

Order 12345 is out for delivery.
Your order #12345 is currently out for delivery! 🚚 Please ensure someone is available to receive it. If you need further assistance tracking the delivery, let me know!

When you run this code, the LLM figures out the intent, pulls out “12345,” uses the check_order_status tool, reads the result, and then replies: “Your order 12345 is out for delivery.” All of this happens smoothly behind the scenes.

Want to Build More AI Agents?

If you liked building this customer support agent, my book, Hands-on GenAI, LLMs and AI Agents, is a great next step. It uses a practical, project-based approach to Generative AI, LLMs, RAG, and AI agents, helping you go from learning the basics to building real AI applications.

If you want a more structured learning experience, you can also look at Building AI Agents and Agentic Workflows by IBM. It covers how agentic AI systems are built using frameworks and workflows like the ones in this tutorial, so it’s a natural next step after learning the basics with LangGraph.

The Takeaway

Building this agent reminded me of an important lesson I share with everyone I mentor: Don’t stress too much about the models; focus on mastering how everything works together.

The industry is changing fast. Today it’s Qwen3; tomorrow it will be something new. Models will keep getting smarter, cheaper, and faster. But the real value is in the engineering: handling state, failing gracefully, and safely connecting AI with older systems.

I hope you enjoyed this article on building a fully functional customer support AI agent with LangGraph. For more AI and machine learning tips, you can follow me on Instagram.

Aman Kharwal
Aman Kharwal

AI/ML Engineer | Published Author. My aim is to decode data science for the real world in the most simple words.

Articles: 2223

Recommended For You

3 Comments

Leave a Reply

Discover more from AmanXai by Aman Kharwal

Subscribe now to keep reading and get access to the full archive.

Continue reading