Where AI Actually Belongs in a Business Process (and Where It Will Cost You)
Language models should read, extract, classify and draft. They should not decide, approve, or write to your system of record unsupervised. That single line between interpreting information and acting on it is what separates AI automation that runs reliably for years from the kind that quietly produces wrong answers for six weeks before anyone notices.
What are language models genuinely good at?
Turning unstructured human input into structured data. This is the capability that actually changed what is possible, and it is worth being precise about it.
An enquiry email written by a person in a hurry contains a name, an intent, a property or role reference, an urgency level and sometimes a deadline, none of it labelled. Before, extracting that needed a human to read it. Now a model does it in under a second at a cost measured in fractions of a penny. Same for classifying support tickets, summarising a call transcript into a structured note, matching a free-text job description to a taxonomy, or drafting a first-pass reply for a person to approve.
These tasks share one property: a wrong answer is visible and cheap. If the model misclassifies an enquiry, a human notices at the next step and corrects it. Nothing irreversible happened.
What should never be handed to a model?
Anything where a wrong answer is expensive, invisible, or irreversible. Concretely:
- Financial actions: issuing a refund, approving an invoice, releasing a deposit, changing a payment amount.
- Compliance determinations: whether a candidate's right-to-work documentation is valid, whether a property meets a regulatory requirement.
- Writes to the system of record without validation: creating, merging or overwriting a customer, tenancy or candidate record.
- Anything a customer sees unreviewed, where a confidently wrong statement becomes a commitment you have to honour.
The problem is not that models are frequently wrong. It is that they are wrong without signalling it. A model that is uncertain produces output in exactly the same confident register as a model that is correct. Traditional software fails loudly, it throws an error. A language model fails fluently. That is a fundamentally different failure mode, and it needs a different design.
What does a safe architecture actually look like?
Deliberately boring. Three layers, with the model confined to the first.
Layer 1: Interpretation (the model)
Reads the input and produces structured output: fields extracted, a classification, a confidence signal, a draft. It does nothing else. It calls nothing, writes nothing and triggers nothing.
Layer 2: Validation and decision (deterministic code)
Checks the structured output against rules you can read: is the reference format valid, does the amount fall within an expected range, does this record already exist, is the confidence above threshold. Then it decides what happens. This layer is ordinary code, testable, auditable, and identical every time it runs.
Layer 3: Action, with proper failure handling
Writes to the system of record with an idempotency key so a retry cannot create a duplicate. Retries transient failures with backoff. Sends anything that fails repeatedly to a dead-letter queue rather than dropping it silently. Alerts a human when the queue grows.
And running alongside all three: a human review queue. Anything the model flagged as low confidence, anything that failed validation, anything ambiguous. Typically 3–8% of records. The point is not to remove humans; it is to make sure the only things reaching a human are the things that genuinely need judgement.
This is not novel thinking. It is standard distributed-systems practice applied to a new component. What is novel is how often it gets skipped, because the demo works without it.
Why do AI automation projects fail?
Almost never because the model underperformed. Three causes account for most of it.
The first is no line between reading and deciding, the model was handed authority it cannot exercise responsibly, and it exercised it confidently anyway. The second is no exception path: the build handles the case where everything works and has nothing for the 4% of inputs that are malformed, duplicated or unusual, so those records vanish or corrupt something. The third is no observability: nobody can answer "what did it do last Tuesday and why", so when a discrepancy surfaces months later there is no way to trace it, and trust in the whole system collapses at once.
All three are design decisions made at the start, not model limitations discovered later.
What should I ask a vendor selling me an AI agent?
Five questions. The answers tell you more than any demo.
- Which decisions does the model make, and which are made by deterministic code? A vendor who cannot separate these has not thought about it.
- "What happens when the model is uncertain: does it guess, or does it escalate?" "It escalates" should come with a description of the queue and who monitors it.
- What happens if the target system is down mid-write? You want to hear the words retry, idempotency, and dead-letter queue, or plain-English equivalents.
- How do I see what it did and why, three months from now? An audit log of inputs, model output and actions taken.
- Who owns and hosts this if we part ways? Open tooling on infrastructure you control means you keep the system.
Does this make AI automation more expensive?
The model calls are the cheapest part of any of this, usually a rounding error next to the engineering. The validation, exception handling and observability are most of the build cost, and they are exactly what separates a system running unattended in year three from one abandoned in month four.
It is also why the sequencing matters. Most operations do not need a model at the centre. If the underlying problem is that two systems do not talk to each other, the fix is an integration, and adding a language model to it adds cost and a failure mode for no benefit. The model earns its place specifically where input is unstructured and human. Everywhere else, deterministic code is cheaper, faster and more reliable.
Frequently asked questions
Which model should we use?
Far less important than the architecture around it. For extraction and classification in a well-designed pipeline, mid-tier models are usually sufficient and materially cheaper than frontier ones. Structure your system so the model is a swappable component and this stops being a strategic decision.
Is our data safe if we use an LLM?
It depends entirely on the deployment. Self-hosted or UK/EU-hosted inference with a no-training agreement is achievable and is what I default to for anything touching personal data. Ask where inference runs, what is retained, and for how long — and get it in writing.
Can AI replace our admin person?
It can remove most of the transcription and triage from their week. It cannot take responsibility for a decision, and someone still has to work the exception queue. In practice, the same person handles two to three times the volume rather than being replaced.
Where to start
Take one process you were considering handing to AI and split it on paper into reading, deciding and acting. If the model ends up in the deciding column, that is the piece to redesign before anything gets built.
If you already have an AI workflow running and cannot answer the five vendor questions above about your own system, an Advanced AI Agent Systems review is worth doing before it produces a discrepancy you have to explain.