Written version
Search problems often hide behind symptoms that look like product, frontend, infrastructure, or AI problems. Users see irrelevant results, empty result pages, poor suggestions, stale information, or answers that are not grounded in the right source material. The business sees abandoned searches, lower engagement, weak conversion, or customer churn.
This episode of the Vespa Voice podcast examines how those problems surface, why conventional team boundaries make them difficult to diagnose, and what it takes to improve search as an ongoing product capability.
What a hidden search problem looks like
Many companies begin with a business problem rather than a search problem. An e-commerce company wants to sell products. A tax platform needs to help people find the correct guidance. A chatbot is expected to answer questions. Search is one of several technologies assembled to deliver that experience, and the people using the product are the ones who discover how well the retrieval layer actually works.
The resulting problems are not always reported as search failures. A user may:
- receive zero results for something that should exist;
- reformulate the same query several times;
- select a suggested product that is no longer in stock;
- receive an apparently relevant document from the wrong year;
- get a useful result only after it has been buried several pages deep;
- receive a fluent chatbot answer that is incomplete or factually wrong.
From the outside, the interface may appear to be working: the user submits a query and the system returns something. The real question is whether it returned the right information for that user and situation.
Diagnose the content, match, and rank layers
A useful starting point is to separate three layers of the search problem.
First, does the system contain the right content? The required information may be absent, incomplete, out of date, or transformed incorrectly before indexing. Data from several sources may need cleaning, joining, and normalization before it is suitable for search.
Second, if the content exists, why did it not match? The cause may lie in text processing, stemming, language-specific analysis, query formulation, or the way the data was modeled and indexed.
Third, if the content matched, why was it not among the first results? That is a ranking and relevance question. A technically matching document is not useful when the correct answer is buried below less relevant material.
The episode gives an example from a Dutch tax product. Users can file taxes for several previous years, so the system maintained similar tax information for multiple years. Searches returned documents, but sometimes from an older tax year. Search appeared operational while still producing the wrong answer because freshness and context were not handled correctly.
Chatbots expose the same distinction. A generated answer may sound like an AI-model problem, but the model cannot produce a reliable grounded answer if the retrieval layer supplies the wrong context. The quality of the answer depends on which information was found before generation began.
Search is a multidisciplinary engineering domain
Search sits across several disciplines, which is one reason ownership can become unclear.
The user experience covers the query box, suggestions, filters, result presentation, and—when generative answers are involved—citations and the source passages supporting the response. Engine-specific engineering covers the operational behavior of systems such as Elasticsearch, Solr, or Vespa under different query, indexing, update, and scale characteristics.
Data engineering is needed to combine and prepare source data. Data modeling determines how that information should be represented for the queries the product must support. Relevance work then defines how results are scored, how search quality is evaluated, and which technical measures correspond to business outcomes.
These responsibilities do not fit neatly inside a single conventional role. The episode describes a company whose business depended on a continuously updated international search product, but search was assigned to one backend engineer for only half of that person's time. The staffing did not reflect the importance or breadth of the capability.
Product and search expertise need shared ownership
At minimum, search improvement needs close ownership from two perspectives.
A product owner or product manager connects the work to customer needs, business objectives, and the outcomes that matter. A search expert understands the retrieval stack and can determine how content, matching, ranking, evaluation, and operational behavior contribute to the observed problem.
Those two roles can bring in frontend, data, machine-learning, platform, or domain specialists when needed. The important point is that search should not become an unowned gap between teams. Technical improvements need product direction, while product priorities need someone capable of tracing them through the search system.
AI applications still depend on information retrieval
Machine learning is not new to mature search teams. It has long been used to improve data quality, derive ranking signals, train ranking or reranking models, and evaluate experiments.
What has changed is how quickly teams can now assemble an AI proof of concept. Embedding APIs, vector databases, and hosted language models make it possible to produce an initial system without the specialist team that earlier implementations might have required.
The cracks often appear when that proof of concept moves toward production. Before a model can answer from context, the application must retrieve and assemble that context. Retrieval may involve keyword search, vector search, or a hybrid of both. The team still has to decide how to evaluate those approaches and how to select the most relevant evidence efficiently.
AI engineering is therefore rediscovering information retrieval. A model may generate the final response, but the sophistication and differentiation of the product increasingly depend on the quality of the retrieval beneath it.
UX, performance, and relevance solve different problems
A strong search experience requires three complementary forms of work.
UX makes it easier to express a need and understand the response. It covers query entry, suggestions, filters, result presentation, generated answers, citations, and links to supporting passages.
Performance determines whether the complete pipeline responds within an acceptable time. Users may tolerate a short wait for a generated answer, but the retrieval and generation stages still need to work together predictably.
Relevance determines whether the system found and ordered the right information. It requires evaluation datasets, representative queries, regression tests, and baseline quality measures such as NDCG. Improvements should be tested against those baselines rather than judged only through isolated examples.
None of these is a one-time task. A polished interface does not repair weak retrieval, faster infrastructure does not make an irrelevant result useful, and a ranking improvement cannot compensate for missing content. The disciplines have to improve together.
Start with a traceable problem
Once a team suspects a search issue, it should begin with a concrete problem or query class and trace it through the system.
Was the required content present and prepared correctly? Did the query match it? If it matched, how was it scored and why did it appear at that position? What happened between the submitted query and the result shown to the user?
Good observability makes those questions answerable. Logging and tracing should reveal what happened at each layer, allowing the team to distinguish a data problem from text processing, matching, ranking, latency, or presentation.
This approach also creates a more productive conversation with leadership. Rather than arguing abstractly that the company needs search expertise, the team can show where a user-visible failure originates and why work confined to the frontend or infrastructure will not resolve it.
Search quality is a continuing product capability
There is no single tool or framework that fixes search for every organization. Teams can develop the necessary understanding through practitioner material, vendor training, or a focused project in which internal engineers work alongside experienced search specialists.
The lasting value is not a one-off adjustment. It is the ability to observe search behavior, evaluate quality, identify the failing layer, and improve the system repeatedly. As search becomes the retrieval foundation for more AI applications, that capability becomes part of the product itself rather than a feature that can be set once and forgotten.