Hybrid Search: BM25, Semantic Search or the Best of Both Worlds

Everybody knows what search is. You type a few words into a box, you hit enter, you get what you wanted. Google trained the entire human race to expect that.

What Google did not train anyone to expect is how badly that experience falls apart the moment you leave google.com. Google is amazing at searching the web, but it is also famously mediocre at searching your site — anyone who ever had the pleasure of trying the good old Google Custom Search Engine and watched it return a three-year-old PDF above the product page knows exactly what I mean.

I too, used to think site search was one of those things you just bolt on. Google made it look trivial, so it must be trivial. But the longer I worked on real systems the more I understood that the split-second between "click search" and "here are your results" is where an enormous amount of engineering hides.

This post is a tour of what's in there. Not a deep dive into search internals — more of a map of why keyword search works, where it breaks, why vectors showed up, and why in 2026 practically every serious system runs both at once.

The naive way to search

If you're building your first real application, you probably have a SQL database or a document store. You put your users, configurations and products there. And then at some point somebody asks you to find things — users in your admin panel, products in your catalog — so you add a search bar.

The first thing you write might look like this:

SELECT * FROM products WHERE name = 'red snoopy shirt';

That works for exactly zero real queries, because nobody types the product title verbatim. So you reach for wildcards:

SELECT * FROM products WHERE LOWER(name) LIKE '%snoopy%';

Then someone searches for a brand, so you add a field. Then a description. Then you want any of the words to match, not all of them:

SELECT * FROM products
WHERE LOWER(name)        LIKE '%red%'
   OR LOWER(name)        LIKE '%snoopy%'
   OR LOWER(name)        LIKE '%shirt%'
   OR LOWER(description) LIKE '%red%'
   OR LOWER(description) LIKE '%snoopy%'
   OR LOWER(brand)       LIKE '%snoopy%'
ORDER BY ???;

And there it is — the ORDER BY ???. That's the moment the whole approach quietly falls over.

It works for a while. Then it doesn't, for four separate reasons:

• It doesn't scale. A leading % wildcard can't use a B-tree index, so every query is a full scan. Fine at 10k rows, a disaster at 10M.
• It has no notion of relevance. Every row that matched is equally matched. A product literally named "Red Snoopy Shirt" ranks the same as a product whose 900-word description happens to contain the word "red".
• It's brittle about spelling. snopy returns nothing. Not fewer results — nothing.
• It treats every word as equally important. Matching "the" counts exactly as much as matching "snoopy", which is backwards in the most damaging possible way.

Search databases and BM25

This is where most people search (pun intended) for something better and land on Elasticsearch, OpenSearch, or an equivalent. Full disclosure: I worked at Elastic for five years, so my instincts here are shaped by that.

The pitch is genuinely close to magic. You dump in a large volume of unstructured text, you run a query, and without tuning anything you get results that are ranked well in single-digit milliseconds. Coming from a relational database, where you had to invent your own ORDER BY, that's a quite a magic upgrade.

Behind the magic

Alas, there's no magic. There are a few simple ideas stacked on top of each other.

1. Zipf's law. Word frequency in natural language is wildly non-uniform. The most common word appears roughly twice as often as the second most common, three times as often as the third, and so on. A handful of words ("the", "of", "and") make up a huge share of all text, and the vast majority of words are rare.

The practical consequence: rarity is signal. If a word is rare across your whole corpus, a document containing it is much more likely to be about that thing.

2. Term frequency and inverse document frequency (TF-IDF). This is the idea turned into arithmetic. A term matters more in a document if it appears often in that document (term frequency), and it matters more overall if it appears in few documents across the corpus (inverse document frequency).

$$\text{tf-idf}(t, d) = \text{tf}(t, d) \cdot \text{idf}(t) \qquad\qquad \text{idf}(t) = \log \frac{N}{\text{df}(t)}$$

Where $N$ is the number of documents in the index and $\text{df}(t)$ is how many of them contain the term.

The intuition falls out immediately. Take the query "the red snoopy shirt" against a catalog of 1,000,000 products:

term	in how many products	IDF	what it means
the	~900,000	0.11	almost worthless as a signal
red	~120,000	2.12	narrows things down a bit
shirt	~40,000	3.22	a real category signal
snoopy	~300	8.11	this is what the user is actually looking for

Nobody wrote a rule saying "ignore stopwords, prioritize brand names." It comes for free from counting. A document that matches "snoopy" earns roughly 77× the score of one that matches "the", which is exactly the ranking a human would want.

3. BM25. TF-IDF is right in spirit but naive in two specific ways, and BM25 (Best Match 25, from the 1990s Okapi project) patches both with one extra knob each:

$$\text{score}(D, Q) = \sum_{q_i \in Q} \text{IDF}(q_i) \cdot \frac{f(q_i, D) \cdot (k_1 + 1)} {f(q_i, D) + k_1 \cdot \left(1 - b + b \cdot \dfrac{|D|}{\text{avgdl}}\right)}$$

It looks intimidating, but it's essentially the same TF-IDF idea from a paragraph ago — rare word, appearing often, in this document — with two corrections bolted on, one per knob.

$k_1$ (default 1.2) saturates term frequency — instead of growing linearly, repeated matches flatten toward a ceiling of $k_1 + 1$. The second mention of a word is informative; the two hundredth is not. $b$ (default 0.75) normalizes for length, penalizing a document by how far it runs past the average ($\text{avgdl}$), because long text accidentally contains more of everything.

Both, on the running example, with a 40-word average document:

document	"shirt" count	length	raw TF	BM25
Red Snoopy T-Shirt	1	4 words	1.0	1.58
2,000-word catalog page	1	2,000 words	1.0	0.05
keyword-stuffed listing	20	40 words	20.0	2.07

Raw TF hands the win to the stuffed listing by 20×. BM25 hands it to the title, and prices the stuffing at 2.07 — it never gets past 2.2 no matter how many times you say "shirt."

Put those three ideas together with an inverted index (a map from every term to the list of documents containing it, so you never scan rows) and you get the thing that felt like magic: relevance-ranked full-text search over millions of documents in milliseconds.

In practice you rarely touch any of this. You write:

from elasticsearch import Elasticsearch

es = Elasticsearch("http://localhost:9200")

res = es.search(
    index="products",
    query={
        "multi_match": {
            "query": "the red snoopy shirt",
            "fields": ["title^3", "brand^2", "description"],
            "fuzziness": "AUTO",
        }
    },
    size=10,
)

for hit in res["hits"]["hits"]:
    print(round(hit["_score"], 2), hit["_source"]["title"])

title^3 says a match in the title is worth three times a match in the description. fuzziness: AUTO allows a typo or two depending on word length. BM25 does the rest, and you didn't configure it.

The limitations of keyword search

BM25 is thirty years old, extremely fast, needs no GPU, and is still a genuinely strong baseline. It also has hard ceilings:

• Typo tolerance only goes so far. Fuzzy matching works on edit distance, so snopy → snoopy is fine. b4n4n4 is not a searchable term, and never will be under an edit-distance model.
• Synonyms are a manual list you maintain forever. "sneakers" / "trainers" / "running shoes" are three unrelated tokens as far as the index is concerned. Somebody has to write that mapping down, keep it current, and maintain it per language. Ask anyone who has run a search team which task they hate most.
• Word order barely exists. To a bag-of-words model, "dog bites man" and "man bites dog" are the same document. Phrase queries and shingles help, at a cost, if you remember to configure them.
• It has no idea what words mean. "red" and "crimson" are as unrelated as "red" and "carburetor". "Snoopy" and "cartoon dog" share nothing. If the user's words aren't your words, you return nothing — and returning nothing is the worst possible failure mode, because it looks like you have no inventory.

Semantic meaning and search

Those limits pushed search people, NLP researchers, and linguists toward the same question long before anyone said "GenAI": can we represent text by what it means, not just which tokens it contains?

Or, concretely: how does a machine work out that "red snoopy t-shirt" and "crimson cartoon dog garment" are roughly the same thing, when they share exactly zero words?

The lineage is longer than most people assume. Latent Semantic Analysis was doing this with matrix factorization in 1990. Word2Vec (2013) made it practical and famous — the king − man + woman ≈ queen party trick — followed by GloVe and fastText. Those gave you a vector per word. BERT (2018) made the vectors context-dependent, and Sentence-BERT (2019) made whole-sentence vectors that were actually comparable to each other, which is the part search needed. Everything since — OpenAI's embedding models, Cohere's embed-v4, the open-weight zoo on MTEB — is a refinement of that same shape.

But if you don't feel like diving into the research papers - let me simply show you how it works.

How it actually works

A model is trained so that texts humans consider similar end up close together in a high-dimensional space. (How it's trained is its own article — contrastive objectives, hard negatives, a lot of data.) "Close together" has a specific definition: cosine similarity, the cosine of the angle between two vectors.

$$\cos(A, B) = \frac{A \cdot B}{\lVert A \rVert \, \lVert B \rVert} = \frac{\sum_{i=1}^{n} A_i B_i} {\sqrt{\sum_{i=1}^{n} A_i^{2}} \; \sqrt{\sum_{i=1}^{n} B_i^{2}}}$$

In 2D this is easy to picture: the smaller the angle between two arrows, the more similar the things they point at.

In 1,536 dimensions it's impossible to picture — but the math doesn't care how many dimensions you have, so if you trust linear algebra you can extrapolate. Two identical directions give 1, unrelated directions give ~0.

In practice: you embed each text (convert it into a long numeric vector), and comparing two texts is one dot product.

import cohere
import numpy as np

co = cohere.ClientV2(api_key="...")

res = co.embed(
    texts=["red snoopy t-shirt", "crimson cartoon dog garment"],
    model="embed-v4.0",
    input_type="search_document",
    embedding_types=["float"],
)

a, b = (np.array(v) for v in res.embeddings.float_)
print(a @ b / (np.linalg.norm(a) * np.linalg.norm(b)))
# ~0.7 — clearly related.
# BM25 scores this pair exactly 0.0. Not one shared token.

There are plenty of tricks and sharp edges here — asymmetric query vs. document embeddings, chunking, dimensionality, normalization, domain drift — and they get their own post. The general idea holds: meaning becomes geometry, and similarity becomes arithmetic.

If you want the intuition without the math, I did a talk on exactly this.

Semantic search in the age of AI

This technique's full potential got unlocked in the past few years, for a reason that took a while to become obvious: a lot of what we call "AI problems" are search problems wearing a costume.

When a ReAct shopping agent is asked to find the perfect shirt (the Snoopy one, obviously), its core challenge is not reasoning. It's exploring a catalog and finding the right item — the same problem as before, just with a different caller.

But AI interfaces invite queries that look nothing like keywords. Two decades of Google trained people to self-compress into keyword-ese: we type snoopy shirt kids 4t because we learned that's what machines eat. Chat interfaces take query understanding for granted and invite top-of-mind, sloppy, conversational queries: "something with that dog from peanuts for my kid, he's like four, nothing itchy." There is no keyword index on earth that handles that sentence.

That's where searching by meaning stopped being a nice-to-have. It's also what lets ordinary code map free text onto structured objects, labels, or tool names without paying for a slower and more expensive LLM call to do it.

Hence the explosion: dedicated vector databases like Pinecone and Qdrant, and vector search bolted onto everything else — Elasticsearch, Postgres via pgvector, MongoDB, Redis. Same idea, wildly different packaging.

# Embed the query, then find the nearest product vectors.
qvec = embed("something with that dog from peanuts, for a 4 year old")

res = es.search(
    index="products",
    knn={
        "field": "title_embedding",
        "query_vector": qvec,
        "k": 10,
        "num_candidates": 100,
    },
)

No shared tokens required. The catalog entry "Peanuts Snoopy Toddler Tee, 4T" comes back anyway.

Hybrid search

The best of both worlds

Once you run this in production for a few weeks you discover semantic search has its own failure modes, and they're roughly the mirror image of BM25's.

Vectors are fuzzy by design, which is exactly wrong when the user is being precise. Someone searching for SKU AX-42B, or an error code, or a person's surname, wants that exact string — and an embedding model will cheerfully hand back things that are vibe-adjacent to AX-42B. Rare, out-of-domain, and brand-new tokens are the worst case: the model never learned them, so it approximates, and approximating an exact identifier is just being wrong with confidence.

And then there's everything that has nothing to do with meaning at all. Filter to one country. Restrict to the user's organization. Under $40, in stock, within 20km. No amount of cosine similarity gets you that.

So, like most things in engineering, the answer wasn't picking a winner. It was running both and merging the results.

The merge is the interesting part, because the two scores aren't comparable. A BM25 score of 14.7 means nothing on its own — it depends on your corpus. Cosine similarity is always between 0 and 1. Adding them together is meaningless.

So the standard trick, Reciprocal Rank Fusion, ignores the scores entirely and looks only at position. Each list votes for its top results, a document's votes get summed, and whatever ends up with the most votes wins:

$$\text{RRF}(d) = \sum_{r \in R} \frac{1}{k + \text{rank}_r(d)}$$

Being 1st in a list is worth more than being 10th, which is worth more than being 50th — and that's all the arithmetic is doing. Documents that both retrievers liked collect votes from both and rise to the top. Documents only one retriever found still get a fair shot.

It's not as pretty as the single-line queries above, but this is still a reasonably compact query — and it delivers both kinds of search at once:

res = es.search(
    index="products",
    retriever={
        "rrf": {
            "retrievers": [
                {   # lexical: exact terms, SKUs, brand names
                    "standard": {
                        "query": {
                            "bool": {
                                "must": {
                                    "multi_match": {
                                        "query": user_query,
                                        "fields": ["title^3", "brand^2", "description"],
                                    }
                                },
                                "filter": CATALOG_FILTERS,
                            }
                        }
                    }
                },
                {   # semantic: intent, synonyms, sloppy phrasing
                    "knn": {
                        "field": "title_embedding",
                        "query_vector": embed(user_query),
                        "k": 50,
                        "num_candidates": 200,
                        "filter": CATALOG_FILTERS,
                    }
                },
            ],
            "rank_window_size": 50,
            "rank_constant": 20,
        }
    },
    size=10,
)

Where CATALOG_FILTERS is the boring, non-negotiable business logic:

CATALOG_FILTERS = [
    {"term":  {"country": "IL"}},
    {"term":  {"in_stock": True}},
    {"range": {"price": {"lte": 40}}},
]

The payoff is that each retriever covers the other's blind spot. AX-42B hits dead-on in the lexical branch. "that dog from peanuts" hits in the vector branch. "red snoopy shirt under $40, in stock" hits all three mechanisms — lexical, semantic, and filters — and the fused list is better than anything either side produced alone.

Conclusion

In 2026, all of this is likely to be hidden inside a single search_products tool that your agent calls. That's a good thing — it's necessary complexity, packed behind a name so plain it sounds like it took an afternoon.

From the user's side, the agent just finds the thing. They click add to cart. Or they don't even click — they say "yeah, get it" while watching something on the other monitor.

From our side — the AI engineers, the search people — every one of those calls is a box full of heuristics, linguistics, geometry and tuning that took decades to get right. Worth knowing what's in the box, especially on the day it returns nothing and somebody has to work out why.

---

Further reading: Elastic on semantic search · Dr. Patrick Lewis (Cohere) on RAG · WTF are Embeddings