Depositphotos
CX leaders are under pressure to increase self-service and automate more customer interactions. But the knowledge systems built to help trained employees find answers were not necessarily designed to let AI deliver those answers independently.
In agent assist, an employee can recognize an outdated policy, question a search result, or escalate a case. When AI answers customers directly, those safeguards must be designed into the system.
For this article, Searchplex spoke with Rohan Rath, Director, Product Management at Laivly. He has built RAG systems for agentic customer experiences, agent assist, customer-facing chats, voice, and operational insights, giving him a practical view of what changes when retrieval moves in front of the customer.
New CX platforms can support the shift from human-led service to more automated, agentic resolutions. To control cost, protect customer trust, and understand revenue impact, teams need to get information retrieval right before they scale agentic CX.
Information retrieval is the discipline behind search and retrieval-augmented generation, or RAG. It selects the knowledge that grounds an AI response. Agent assist, case containment, chatbots, and voice agents can all depend on a shared retrieval foundation. The following failure modes show what happens when that foundation was built for internal use rather than direct customer service.
When the Human Leaves the Loop
In agent assist, a human remains in the loop. An agent can discard a weak answer before the customer sees it, ask a team lead, or rely on an informal workaround. Those interventions can hide problems in the knowledge base and retrieval layer.
Customer-facing automation removes that review step. The same retrieval layer now answers customers directly and in real time, increasing the cost of every retrieval error.
We always vet the information before it’s used for agentic solutions. When you’re going direct to the customer, there’s no human in the loop to catch a mistake, so the liability is much higher than when an agent reviews the answer first.
A well-designed retrieval architecture can support several CX applications, including chatbots, voice agents, agent assist, tool configuration, reporting, and operational insights. Improving retrieval once can therefore improve several customer and employee experiences.
These retrieval problems also appear in tax research, legal search, and commerce. CX teams can apply established practices from those fields, including version controls, metadata, ranking, and evaluation.
Customer-Facing AI Raises the Stakes
Trust starts before the model generates a response.
Air Canada is a prominent example of what happens when an AI system gets the answer wrong in front of a customer. Its website chatbot gave a customer incorrect information about the airline’s bereavement fare policy, the customer relied on that information when making a purchase, and a British Columbia tribunal held Air Canada responsible for the resulting loss. The chatbot may have delivered the answer, but the company carried the legal and reputational cost. At Chevrolet of Watsonville, a user prompted the dealership’s chatbot to agree to any request, and the bot agreed in chat to a $1 price for a 2024 Tahoe. No sale occurred, but the incident showed the reputational risk of letting a customer-facing system make statements beyond its role.
Knowledge base quality is still the biggest challenge we see. A RAG system is only as good as the content it retrieves from, and most knowledge bases weren’t written with that in mind.
Keeping the right document in the knowledge base does not guarantee the right answer. When a vacation-rental AI concierge on WhatsApp began offering refunds without being asked, Searchplex first examined the retrieval layer. The team tightened source selection and grounding so the model received the relevant policy and context. A grounded product depends on retrieval architecture, answer evaluation, and careful integration with the existing technology stack.
Why CX Needs a Retrieval Layer Between Data and the LLM
Selecting relevant content before generation reduces context, tokens, and delay.
Several of the failure modes above can arise when teams do not treat information retrieval as its own discipline. A common AI-first design stores documents in a vector database, retrieves the closest matches, and sends them to an LLM. That may work for a prototype. In production, teams also need an indexing strategy for chunking, metadata, versioning, ranking, and refresh logic. Sending stale or irrelevant content into every prompt increases input tokens, latency, and model cost.
A database or knowledge repository is typically the system of record for policies and other content accumulated over time. A search index is a retrieval-optimized representation of that knowledge. It helps the system select the most useful content for a specific query before the LLM generates an answer. The same pattern can support internal knowledge search, public website search, chatbots, and interactive voice assistants.
Moving from a knowledge base in Guru, Confluence, or an internal wiki to a production index takes deliberate work. Teams must split documents into useful chunks, enrich them with metadata, manage versions, and define what relevant means for each content type.
Versioned content makes this distinction especially important. A retriever may score an old policy and its replacement as equally relevant because embedding similarity measures likeness to the query, not the policy’s effective date. Without version metadata and filtering, the vector index may continue to surface stale content.
Most organizations have knowledge bases built up over many years, and older versions rarely get retired. New versions keep getting added alongside the old ones, because archiving or deleting content feels risky.
Version stacking—having multiple versions of the same document influence retrieval quality—isn’t unique to CX either. Nextens, part of LexisNexis Risk Solutions, runs tax research products used daily by accountants and advisors; Searchplex audited and redesigned their production search, improving relevance, handling of versioned content, and retrieval control. Tax law and refund policy can exhibit the same failure mode: without effective-date metadata and retrieval rules, older versions can remain eligible and rank.
Knowledge base conflict detection gets a lot of interest, because everyone wants a cleaner knowledge base. But the tool is only the first step. It can show you the problems, and someone still has to go in and fix them.
Policy Search and Product Search Need Different Retrieval
Not every wrong answer has the same business impact. A wrong policy answer can create repeat contacts, escalations, refunds, or reputational risk. A weak product recommendation can reduce conversion and leave revenue on the table.
Policy search and product search often require different relevance rules. A policy question such as “What is the return window?” may depend on semantic matching plus the policy’s date, market, and customer context. A product request may require exact filters for SKU, size, color, availability, or compatibility. A result that is semantically similar but the wrong color does not help a customer looking for a green dress.
Voice Makes Retrieval Speed Essential
You can’t deliver a good voice experience without removing latency from every step of the interaction. That’s what convinced us to invest in our search and retrieval to improve it from every angle.
Voice exposes every slow step in the retrieval pipeline.
Chat interfaces can mask delay with a typing indicator. Voice has much less room to hide delay. A few seconds of silence on a phone call feels broken, and filler language can create an awkward interruption when the answer arrives. Passing a long policy document into a large context window also adds processing time. A selective, low-latency retrieval layer narrows the context to the most useful content so the voice agent can respond within a natural conversational pause.
In 2023, a popular view was that you didn’t need to invest much in RAG, because context windows would keep growing, models would keep getting cheaper, and you could put everything in the prompt. Context windows did grow, but costs didn’t fall the way people expected, and latency rarely came up. In many of our use cases, latency is the main reason you still need RAG.
Questions for Your Next CX Review
Use these questions in your next quarterly review to surface retrieval quality, risk, cost, and performance:
- Ask the same question in separate sessions. Does the bot return a consistent answer, and can the team explain any variation?
- Ask the voice agent one policy question and one product-selection question. Does each answer use the right retrieval method and constraints?
- Measure the full response time for a voice query. Does the agent answer within a natural conversational pause?
- For each top contact driver, how many active and obsolete versions of the supporting articles exist?
- How would input-token volume, latency, and model cost change if retrieval reduced the average context by 50% or 75%?
- What is the p95 retrieval latency, and how much of the end-to-end response time does retrieval consume?
- How does the team evaluate retrieval quality before launch and monitor it after launch?
- When the bot delivers a wrong answer, how does the team determine whether to fix the source content, metadata, index, ranking, prompt, or model?
- Can the team trace each chatbot or voice answer to the exact source document and version used?
How reliable is the retrieval layer behind your customer-facing AI?
Searchplex provides AI-native search and retrieval engineering for enterprises where search affects revenue, productivity, and customer experience. A Search Stack Audit can help teams answer these questions and identify the highest-impact improvements.