We are building an internal AI agent right now. Production system, not a demo.
It pulls from a CRM, an operations database and a handful of other sources. It runs on an orchestration platform we wrote ourselves, partly for security and partly because off-the-shelf options do not pass our requirements. Two months in, here is the honest version.
The AI is the easy part.
The hard part is the work that has to happen before the model ever runs. Defining what “right” looks like for each output. Writing the evaluation suite. Setting the line between what the agent decides and what a human still has to. Pressure-testing the prompts against every edge case in the data. Building the rails. Reviewing outputs every day until the failure modes stop being surprises. That work does not get the LinkedIn carousel treatment. It is also where most of the cost lives.
Here is the uncomfortable part. The volume of operator discipline required to make the agent worth using is so substantial that it almost erases the productivity advantage you signed up for. For the first six months, you would have been faster with a REST API and a sharp analyst.
The industry data points the same way. Analyses of agent deployments that are still losing money a year in keep landing on the same causes: unclear success criteria, insufficient tool or data access, and evaluation that drifts out of date. Those are not model-quality problems. They are product ownership problems wearing a model costume.
The tech will keep getting better. The orchestration layer matures. Operator overhead drops. When that day comes, the systems we are building now will be quietly beautiful, and the firms that committed early will have a real moat.
Until then, here is the part the mid-market needs to hear. If you do not have a product owner who can hold the ground between the engineers and the business, the project will burn capital and your team will end up asking, six months in, whether a REST API would have been simpler.
A lot of the time, it would have been.
The bottleneck is not the model. It never was.