Running an AI agent on our own agency's helpdesk before offering it to anyone else taught us things a client project would not have. The decisions we would repeat, and the ones we would make earlier.
Agentic coding tools change how software is built, but only when the project instructions, standards, review gates and budgets are right. The setup we use ourselves, and recommend to clients.
"Human in the loop" is a slogan until someone decides where the loop closes. The real design question is the threshold: how sure must the system be before it acts alone, and who decides that, on what evidence.
An agent that cannot be measured is not ready for production. What an evaluation set is, how to build one with your own experts in a few hours, what to measure, and why it has to run on every change.
The engineering story behind our own helpdesk agent: why a generic chatbot could not do the job, how code and documentation are indexed so retrieval is grounded in the actual build, and the decisions that made it safe to put in front of customers.
Retrieval-augmented generation answers from passages that look like the question. GraphRAG adds the relationships between things. Which one a knowledge assistant needs depends on the questions people actually ask.
The gap between a working pilot and a running system is where most AI spend disappears. Five reasons it happens and the scoping decisions that prevent it.
The mechanism that makes an AI agent safe to put in front of customers: how confidence is scored, where the threshold goes, what gets handed to a person, and how the evaluation set keeps it honest.
Off-the-shelf AI products force your business into their shape. A framework is a proven architecture with the hard parts solved, tailored to your systems in weeks. Here is the difference, and when each is right.