Creating an end-to-end local AI project is one of the most useful and freeing things you can do as an AI/ML developer today. In this article, we’ll build a working AI application, a Local AI Email Assistant, using only free, open-source tools that run on your own computer.
We’ll go through the process step by step so you can understand both the code and the thinking behind it.
The Tech Stack
To build a Local AI Project, you need two main parts: an inference engine to run the model and a framework to create the user interface.
- Ollama: This tool lets you run large language models on your own machine. It handles the complicated setup that local ML inference usually needs.
- Streamlit: If you don’t have time to learn React just to make a prototype, Streamlit is a Python library that lets you turn data scripts into web apps in minutes.
Before we get to the code, set up your environment:
- Download and install Ollama.
- Open your terminal and pull the Llama 3 model by running:
ollama pull llama3. - Install the required Python libraries:
pip install streamlit ollama.
Want to build more real-world AI projects? Check out my book Hands-On GenAI, LLMs & AI Agents.
End-to-End Local AI Project: Email Assistant
The goal of this app is simple and useful: it takes rough notes from a user and turns them into a polished, professional email in the tone you choose.
Here’s the full code for our app. Make a file called app.py and write this code inside it:
import streamlit as st
import ollama
# 1. Page Configuration
st.set_page_config(
page_title="Local AI Email Assistant",
page_icon="✉️",
layout="centered",
)
st.title("✉️ Local AI Email Responder")
st.write("Turn your rough notes into a professional email, running 100% locally.")
# 2. User Inputs
tone = st.selectbox(
"Select Email Tone",
["Professional", "Friendly", "Apologetic", "Direct"],
)
user_input = st.text_area("Enter your rough draft or notes here:", height=150)
# 3. The Core Logic
if st.button("Generate Professional Email"):
if not user_input.strip():
st.warning("Please enter some text to get started.")
else:
with st.spinner("Drafting your email..."):
# 4. Prompt Engineering
system_prompt = f"""
You are an expert corporate communicator. Your task is to rewrite the user's rough notes
into a well-structured, grammatically correct email.
The tone of the email must be: {tone}.
Do not include any explanations or pleasantries in your output, just the email itself.
"""
# 5. Local Model Invocation
try:
response = ollama.chat(
model="llama3",
messages=[
{"role": "system", "content": system_prompt},
{"role": "user", "content": user_input},
],
)
# 6. Displaying the Output
st.subheader("Your Polished Email:")
st.info(response["message"]["content"])
except Exception as e:
st.error(
f"An error occurred: {e}. Is Ollama running in the background?"
)Let’s break down what’s happening so you can use these ideas in your own projects:
- Page Configuration: Start by setting up the UI details. Using st.set_page_config is a good habit because it controls how your app looks in the browser tab and helps your prototype feel more complete from the beginning.
- User Inputs: I recommend letting users control the model’s output settings. Using a selectbox for tone limits choices to a set list, which makes prompt engineering more predictable.
- The Core Logic: Notice the validation check (if not user_input.strip():). You should never send empty strings to an LLM. It wastes resources and can lead to strange or confusing results.
- Prompt Engineering: This is the most important part. We keep the instructions (system_prompt) separate from the data (user_input). This is important for both security and structure in AI projects. By telling the model to “not include any explanations,” we stop it from adding phrases like “Sure! Here is the email you requested:” before the actual output.
- Local Model Invocation: We use the ollama.chat function and specify llama3. The messages array uses the standard conversational format that’s common in the industry.
- Error Handling: Notice the try-except block. Many beginners assume the environment will always work. But if the user forgets to start Ollama, the app should show a clear error message. This is what makes your app reliable instead of just a script.
You can run this file by executing the command below:
streamlit run app.py
Here’s what the final output will look like:
Closing Thoughts
When I mentor new ML engineers, the most common problem I see is getting stuck in ‘tutorial hell’; reading lots of papers and watching videos, but never building or deploying anything.
Building an end-to-end Local AI Project is the best way to break out of that cycle. It connects what you know in theory to real engineering work. It also makes you consider the user interface, edge cases, error handling, and prompt design together.
I hope you enjoyed this article about building an end-to-end local AI project.
For more AI and machine learning tips, follow me on Instagram. My book, Hands-On GenAI, LLMs & AI Agents, can also help you grow your AI career.
