The Sabivox support agent cut average response time from 8 minutes to 30 seconds and forecasts ticket volume at 94% accuracy. It works because we used LLMs for what they are actually good at.
LLMs excel at: understanding ambiguous natural language at scale, context-aware response generation, summarising documents, and routing complex queries.
LLMs will ruin your system if you trust them for: exact numerical precision, hard constraints (dates, amounts, IDs), or anything where a hallucination has a real-world consequence.
The engineering question is not "should we use an LLM?" It is "what specific, bounded task is this LLM responsible for, and what validates its output before it touches anything that matters?"
— Dick Bassey | DevDick | 2023