First architecture, then "magic": our path from scripted voice bots to smart assistants

When scenarios begin to "burst at the seams"

But this approach has its limits. Over time, the number of phrase variations becomes too large, exceptions grow faster than the team can describe them, and the scenarios start to fall apart. Meanwhile, the business needs not a formal response, but a normal conversation: to check a payment, explain a status, guide the customer through several steps, and not lose context. This is precisely where the need for an LLM arises.

It is important not to confuse its role. The model is not a source of facts or the "brain of the system." Its task is to understand speech, maintain context, make decisions, and formulate responses. Facts must come from systems—via APIs and services. When this separation is maintained, everything works predictably. When it is not, the system starts to sound confident but makes mistakes. This is usually where architecture begins.

It doesn't look like a replacement of one block with another, but like a gradual layering. Below is a convenient diagram of this transition in a simplified form, which we used to synchronize the team's work.

Risks of "quick" implementation

Therefore, connecting LLM directly to telephony or CRM is a bad idea. At demos, it looks impressive: responses sound livelier, the voice is more pleasant, and the dialogue seems more natural. But in a real system, this is not enough. If there is no normal architecture between the model and data sources - routing, API, knowledge layer, and constraints - the result is not a smart agent, but a very convincing, yet unreliable interface. It sounds confidently, but can make mistakes, and in a regulated environment, this is already a risk, not just a drawback.

There's another point that is often remembered too late - personal data. The law requires a clear understanding of what data is being processed and why, not collecting unnecessary data, and monitoring its relevance.

In search of balance

At the same time, the transition to LLM indeed brings value. The conversation becomes more natural, the team spends less time supporting scenarios, and the bot handles non-standard situations better. But this works only when the model is responsible for formulating the response, not for facts or interaction rules. When this separation is maintained, the system remains manageable and becomes noticeably more convenient for the user.

Checklist for the team: how not to confuse a demo with a working system

• First, draw the routes, and only then argue about the models.

• Separate facts from formulations: facts should come from internal systems, and the LLM should only explain and gather the response.

• Build knowledge management as a process with owners, versions, and metrics, not as a folder of PDFs.

• Legal aspects and human translation rules are not an appendix to the Terms of Reference, but a mandatory part of solution development.

• Evaluate not an abstract "model quality," but the share of actually completed scenarios, fact accuracy, and handoff quality.

• Choose infrastructure not by trend, but by where data requirements and SLAs are best met.

• Consult with information security specialists so that the project implementation does not carry additional or hidden risks.

Conclusion

In the end, the transition to LLM is not a story about "it was simple, now it's smart." It's more about the complication of the system. Another layer appears that works well with language: it understands what a person is saying, what a person is saying, and correctly formulates the response.

But everything else does not disappear. Routes, data, rules, and responsibility still remain at the architecture level. And if they are not thought through, the model alone will not save the situation.