A company with around 200,000 products wanted what everyone wants now: ask a question in plain language, get the answer from the catalogue. The data was clean and sat behind well-built APIs. It looked like a weekend’s work.
Then we let a model query the data directly, and watched it do what models do. It hallucinated fields that did not exist. It missed ones that did. Worst of all, it began building quiet shadow copies of the database in its own context, answering from a version of the truth that drifted a little further from reality with every question.
The rule: the model never talks to your data
The fix was not a better prompt. It was a rule we now apply everywhere: the model never talks to your data directly.
Instead, we put a layer in between. A question in natural language becomes a schema-validated query, checked against what the data actually is, which then runs against the real source and returns a real result. The model helps phrase the question. It never invents the answer.
AI proposes. The system decides what is real.
Why the extra layer pays for itself
The difference is accountability. A model can be wrong about anything. A validated query can only return something that is genuinely there, and every answer it produces is traceable back to a source page. If a number looks off, you can follow it home.
This is human-in-the-loop moved down a level. AI proposes; the system decides what is real. The model is allowed to be creative about language, and nothing else. It is slightly more work to build, and it is the difference between a demo that wins a meeting and a system people still trust on the two-hundredth question.
An answer you cannot trace is not an answer. It is a guess with good grammar.