Before the rise of Large Language Models (LLMs), conversational chatbots were designed with rigid flowcharts that mapped out every conversational step, predicted user responses, and scripted specific outcomes.
Teams spent millions creating these flows, hiring conversation designers to generate thousands of utterances, and training natural language processing [NLP] classifiers to guide users along the correct routes.
However, this converted only a minimal number of user queries into successful support, and more often than not, customers resorted to human agents.
One might think that today, the LLMs have largely solved the problem, given their ability to conduct conversations significantly better than traditional systems, while requiring less effort. However, similar problems persist even as organisations have integrated LLMs at scale.
Bolting LLMs to Existing Frameworks Won’t Work
Most organisations have simply bolted LLMs onto existing conversation structures. While responses sound more natural, users still hit dead ends when they deviate from predetermined flows. Additionally, LLMs introduce failure modes that traditional systems have never encountered.
A flow-based system fails predictably—it either recognises an intent or it doesn’t. LLMs, on the other hand, can fail spectacularly thanks to hallucinations and can generate responses that seem plausible but violate business policies or provide incorrect information.
For high-stakes interactions, even a 1 in 1000 failure rate is unacceptable. Without proper grounding, these agents act like inexperienced customer representatives, lacking competence and leaving users feeling unsupported.
Organisations map common paths, but real-world conversations diverge. Users provide details out of order, in batches, or skip steps entirely. Covering these variations requires exponential work, yet most cases remain unhandled.
Thus, when companies invest heavily in using LLMs for conversations, an outdated and misdirected approach yields minimal ROI.
For instance, a 2024 Gartner survey, with over 5,000 participants, revealed that only 14% of customer support and service issues were resolved through a company’s self-service channel.
Flow-based methods, whether LLM-enhanced or not, demand extensive modelling of every conversational turn. The solution isn’t better integration of LLMs into existing frameworks, but reimagining conversation architecture entirely.
Emcie, founded by engineers from leading software firms such as Microsoft, Check Point, and Dynamic Yield, alongside NLP researchers, has built Parlant. This open-source platform offers fundamentally different approaches to conversational AI, moving beyond the constraints of flow-based systems.
“It’s like going to a supermarket — but instead of going on your bicycle, you’ve bought a Ferrari to do so. You’re paying for the car, all the extra gas, and you’re risking the car getting scratched, stolen, or damaged. There are so many risks involved, but what you’re still getting is the ROI of just groceries,” said Yam Marcovitz, CEO and co-founder of Parlant, in an interaction with AIM. He pointed out that companies can spend millions on LLMs, yet achieve minimal gains, accompanied by a long list of risks.
Why Parlant
Emcie’s Parlant introduced structured guidelines and clear rules that addressed specific scenarios or user intents. These guide the AI chatbot on how to respond to specific situations. This modular method eliminates the need for creating a lengthy, comprehensive prompt.
“Think of an LLM as a highly knowledgeable stranger who’s just walked into your business. They may have years of general experience, but they don’t know your specific context, preferences, or approach to things. Yet, this stranger is eager to help and will always try to, even when uncertain,” said Parlant, underscoring the need to build guidelines.
Guidelines are added when business experts request behavioural changes in an agent. And before each response, Parlant loads only the guidelines relevant to the conversation’s current state. “This dynamic management keeps the LLM’s ‘cognitive load’ minimal, maximising its attention and, consequently, the alignment of each response with expected behaviour,” said Parlant, in its documentation.
“Once the action is accomplished in a session, Parlant will deactivate the guideline—unless it has reason to believe the action should re-apply.”
Moreover, the company allows human experts to modify guidelines, glossaries, context variables, and API usage instructions to ensure consistent behaviour across all interactions.
For instance, an AI customer service agent is instructed to improve a customer’s mood when they display disappointment, but this instruction is rather vague. Does it mean making them smile by cracking a joke, or perhaps offering a substantial discount? Situations like these highlight the significant influence of guidelines.
If the ‘condition’ states that the customer is unhappy, deploying an ‘action’ statement that says “acknowledge their frustration specifically, express sincere empathy, and ask for details about their experience so we can address it properly” can be considered.
Here’s another example that the company provides to implement guidelines. “Suppose an AI Travel Agent’s vanilla response to a customer asking to book a room may be, Sure, I can help you book a room. When will you be staying?’
“If we wanted the agent to first ask how many guests would be staying, we can add a guideline to that effect. If we wanted the agent’s response to be more enthusiastic, we could also add a guideline for that. Parlant will ensure our guidelines are consistently applied at the right situations,” the company said.
A well-defined guideline that is bounded, specific, and contextual is more likely to lead to the desired behaviour.
By employing these techniques, Parlant guarantees a reliable experience for both business teams and, more importantly, customers. More importantly, it helps organisations earn the ROI that generative AI essentially promises today, in the customer support sector.
The company maintains comprehensive technical documentation on their website covering all of the platform’s offerings, while the source code is available under the Apache 2.0 license in its public GitHub repository.
The post Why LLMs Alone Can’t Fix Broken Customer Support Chatbots appeared first on Analytics India Magazine.
