In July 2025, an AI coding agent on Replit deleted a live production database during a code freeze, despite repeated instructions from the founder not to make changes. The agent proceeded regardless and then misrepresented its actions. This incident did not involve a malicious model, but rather an agent with excessive access, insufficient safeguards, and a lack of oversight before executing a destructive step. Most agent tutorials overlook this critical gap. Let’s examine why AI agents make mistakes and which engineering skills are essential for building reliable systems.
Why AI Agents Make Mistakes
When a chatbot makes a mistake, it produces an incorrect answer. When an agent errs, it takes an incorrect action. This distinction is significant.
Several common factors can cause agents to fail:
- Probabilistic decisions: The model selects the next step based on likelihood rather than certainty, so identical prompts may yield different tool calls across runs.
- Ambiguous instructions: Phrases like “clean up the old records” may be clear to you, but the model must infer the meaning of terms such as “old” and “clean up.”
- Compounding errors: In multi-step tasks, an early mistake can undermine all subsequent steps.
- Prompt injection: Text from sources such as web pages or emails may include instructions that the model interprets as commands. For example, in 2023, a Chevrolet dealership’s chatbot was manipulated into “agreeing” to sell a car for one dollar. While that bot lacked real purchasing authority, an agent with actual tools could face more serious consequences.
- Confident errors: Models seldom indicate uncertainty and often proceed regardless.
In my experience, most agent failures result from system design issues, not model errors. Often, there are no clear restrictions on what the agent is permitted to do.
The Engineering Skills That Make Agents Reliable
1. Tool permissions: least privilege
Grant the agent only the minimum capabilities required. For example, if it only needs to read customer data, assign a read-only database role rather than admin credentials. Separate tools by risk; actions like “search orders” and “issue refund” should have different access levels.
Enforce restrictions through code rather than prompts. A prompt stating “never delete data” is only a suggestion, while a database user without delete permissions enforces a rule.
2. Sandboxing
All agent executions, including code, shell commands, or browser actions, should occur in an isolated environment. Use containers or restricted runtimes that lack access to production secrets, have limited network connectivity, and enforce resource limits. This ensures any destructive actions remain contained within the sandbox.
Separate environments are essential. Agents should operate on staging or copies by default, with production access granted only as a controlled exception.
3. Human-in-the-loop controls
Not all actions require approval, but irreversible or high-impact actions do, such as sending customer emails, transferring funds, deleting records, or deploying code. A best practice is to allow the agent to plan and prepare, then require human approval before final execution.
The key skill is determining where to place approval checkpoints. Too many can lead to superficial approvals, while too few increase the risk of critical errors.
Preparing for AI/ML Interviews?
If you are building AI projects to prepare for engineering roles, be sure to practice articulating your technical decisions and problem-solving strategies. I recommend Cracking Your First AI/ML Interview as a helpful resource.
4. Evaluation
Improvement requires measurement. Develop an evaluation set with realistic tasks, including edge cases and adversarial scenarios, and test your agent against it whenever you modify prompts, models, or tools.
Assess more than just final outputs. Review the agent’s process: did it use the correct tools, in the proper sequence, with appropriate arguments? Teams that neglect this often find that minor prompt changes disrupt previously functional workflows. LLM-based scoring, rule-based checks, and regression tests in continuous integration are valuable tools.
5. Monitoring and observability
Once an agent is deployed, monitor its activity closely. Log every prompt, tool call, result, and decision. Trace multi-step executions end to end to enable failure analysis.
Track operational metrics such as cost per task, number of steps, retry rates, and tool error rates. An unexpected increase in steps per task may indicate looping behavior. Set strict limits on iterations and spending to prevent unexpected costs.
6. Error recovery
Tools can fail, APIs may time out, and models sometimes return malformed output. Reliable agents are designed to anticipate these issues. Learn to:
- Validate tool inputs and outputs against schemas before acting on them.
- Retry transient failures with backoff, but cap the retries.
- Design actions to be idempotent where possible, ensuring repeated execution does not result in duplicate charges or unintended effects.
- Implement rollback or undo mechanisms for any operation that alters system state.
- Provide the agent with a clear process to halt execution and escalate issues to a human when necessary.
Build These Skills by Building Real Projects
Reading about permissions and evaluation is only the first step. These skills are best developed through implementation, such as building a sandboxed code agent, an approval workflow, an evaluation harness, or a tracing dashboard.
For more advanced project ideas, consider my 40 AI Projects That Show Real AI Engineering Skills. This collection offers hands-on projects to help you practice and demonstrate these skills in your portfolio, which is highly valued by hiring managers.
The Takeaway
The most challenging aspect of building AI agents is not enabling action, but determining when the agent should refrain from acting and how to handle errors.
Assume your agent will make mistakes. Restrict its permissions, isolate its execution, involve humans at critical decision points, test continuously, monitor in production, and prepare for failure.
Developers who master these skills will build trustworthy agents, which is the true opportunity in AI engineering.
Thank you for reading this article on why AI agents make mistakes and the engineering skills required for reliability. For more insights on AI and machine learning, follow me on Instagram.
