From what I’ve seen, the main challenge teams face when building a multi-agent system isn’t the language models. It’s getting agents to communicate with their tools, access local data, and share context. If you’re using MCP (Model Context Protocol), you’re already ahead of the curve.
When I deployed RAG-based apps and AI agents to production, I used to spend a lot of time writing custom API wrappers just so an LLM could read a local file or query a database. Each new tool meant more integration code, which quickly became a maintenance headache. Things got much easier when the Model Context Protocol became an open standard.
Today, I’ll show you how to build a practical multi-agent system with MCP using only free, open-source tools. We won’t use any paid APIs, just Python, the official MCP SDK, and local models with Ollama.
Why MCP?
Before we get to the code, let’s clear up what MCP does. The Model Context Protocol is an open-source standard that makes it easier and safer for AI models to access external data and tools.
You can think of MCP as a universal USB-C port for AI agents. Rather than hardcoding connections between your AI and your local files, you set up an MCP server. Your agents connect to this server using a standard protocol to request data or run tools.
This approach is a game-changer for multi-agent systems. For example, both an Analyzer Agent and a Writer Agent can use the same MCP tools, so they get consistent data access without repeating backend logic.
Multi-Agent System Using MCP
Let’s walk through a real-world example: building a research pipeline.
We’ll set up a simple MCP server that shares a local data tool, then create two agents using a local open-source model, such as Mistral or Llama 3 with Ollama.
→ Agent 1 (The Researcher): This agent queries the MCP server to pull raw data.
→ Agent 2 (The Editor): This agent takes the raw data from the Researcher and turns it into a clear summary.
Learn to build production-ready AI systems with Hands-On GenAI, LLMs & AI Agents.
Step 1: Setting up the Environment
First, make sure you have Ollama installed on your computer and that you’ve downloaded a local model:
ollama run mistral
Next, install the necessary Python libraries:
pip install mcp ollama
Step 2: Implementation
Here’s the full code to set up our MCP server and two-agent workflow. We’ll use FastMCP, which makes it easy to build MCP servers in Python:
import asyncio
from mcp.server.fastmcp import FastMCP
import ollama
# 1. Initialize the MCP Server
# We define a server that will act as the data layer for our agents.
mcp = FastMCP("ResearchServer")
# Expose a tool to the agents via MCP
@mcp.tool()
def read_company_report(section: str) -> str:
"""Simulates reading a specific section of a local financial report."""
data_store = {
"financials": "Q3 Revenue grew by 15% due to AI infrastructure investments.",
"risks": "Supply chain delays in GPU procurement are causing project bottlenecks.",
"strategy": "Shifting focus to edge AI computing and multi-agent workflows."
}
return data_store.get(section.lower(), "Section not found.")
# 2. Define the Multi-Agent Workflow
async def run_multi_agent_system():
print("Starting Multi-Agent System via MCP...")
# In a fully distributed system, the MCP server would run on a separate port.
# For this tutorial, we are calling the tool directly to simulate the client-server interaction
# without needing complex asynchronous subprocess management in a single script.
# Agent 1: The Researcher (Data Extraction)
target_section = "financials"
print(f"\n[Researcher Agent] Requesting data for: {target_section}")
# The agent uses the standardized MCP tool to get context
raw_data = read_company_report(target_section)
print(f"[Researcher Agent] Retrieved Data: {raw_data}")
# Agent 2: The Editor (Formatting and Synthesis)
print("\n[Editor Agent] Processing raw data into a professional summary...")
prompt = (
f"You are an expert financial editor. Summarize the following raw data "
f"into a single, professional sentence suitable for an executive brief.\n"
f"Raw Data: {raw_data}"
)
# Using Ollama for a 100% free, local inference
response = ollama.chat(model='mistral', messages=[
{'role': 'user', 'content': prompt}
])
final_output = response['message']['content']
print(f"\n[Final Output]\n{final_output}")
if __name__ == "__main__":
asyncio.run(run_multi_agent_system())Let’s break down how the implementation works:
- The FastMCP Server: We set up FastMCP and use the @mcp.tool() decorator. This is the key step. In production, the server runs on its own, and any AI agent on your network can connect to it and use read_company_report without needing to know how the database works.
- The Researcher Agent: In this script, the first agent finds the target data and uses the MCP tool to extract it. A common mistake is giving agents instructions that are too broad. Here, we focus on the “financials” section to keep the context window clear.
- The Editor Agent: We send the extracted, verified context to our open-source LLM through Ollama. By keeping the extraction logic (MCP) separate from the reasoning logic (Ollama), we avoid hallucination risks with the data.
Here’s what the final output will look like:
(airflow_env) (base) amankharwal@Amans-MacBook-Pro multi agent mcp % python main.py
Starting Multi-Agent System via MCP...
[Researcher Agent] Requesting data for: financials
[Researcher Agent] Retrieved Data: Q3 Revenue grew by 15% due to AI infrastructure investments.
[Editor Agent] Processing raw data into a professional summary...
[07/29/26 11:04:24] INFO HTTP Request: POST _client.py:1025
http://127.0.0.1:11434/api/chat "HTTP/1.1
200 OK"
[Final Output]
Q3 Revenue experienced a 15% increase primarily as a result of strategic AI infrastructure investments.
Recommended Courses to Learn More
If you want to learn more about building production-ready AI applications before moving on to complex multi-agent systems, I recommend these two programs:
- Generative AI with Large Language Models: This course is great for learning the basics of open-source models and LLMs. It helps you move from local experiments to building strong, scalable AI systems.
- AI Engineering Professional Certificate: This program matches the software engineering mindset I always recommend. It covers practical, end-to-end AI deployment, which is the skill set you need for working with standards like MCP.
Both of these resources will help you build stronger backend engineering skills, so you can handle advanced integrations more easily.
Final Thoughts
When you’re coding AI applications, it’s easy to get distracted by the latest model weights or benchmark scores. But in my experience, reliable AI isn’t about having the smartest model; it’s about having the strongest architecture.
Building a multi-agent system with MCP helps you think like a software engineer, not just a prompt engineer. Standardizing how your agents access data will save you hours of debugging and rewriting code when you need to change a backend tool or scale up.
I hope you found this article on building a multi-agent system with MCP helpful.
For more AI and machine learning tips, follow me on Instagram. My book, Hands-On GenAI, LLMs & AI Agents, can also help you advance your AI career.
