Yashotantra Academy logo
← RAG & Vector Search theorytheory 0/12 · 0%
level 1
Curious · 0 XP
streak
0d
perfect
0
badges
0

100 XP to Apprentice

Retrieval · easy

5. Hybrid search: BM25 plus vectors

Keywords catch what embeddings miss, and vice versa.

1 min read3 questionsup to 50 XPtopic 5/12

Read in another language

AI translates the explanation and quiz text while preserving code and answer scoring.

Dense vectors are great at paraphrase but weak at exact identifiers: error codes, SKUs, function names, contract addresses. BM25 lexical search nails those. Hybrid retrieval runs both and fuses the results.

Reciprocal Rank Fusion (RRF) is the simple, robust fuser:

python
def rrf(rankings, k=60):
    scores = {}
    for ranking in rankings:                 # e.g. [dense_ids, bm25_ids]
        for rank, doc_id in enumerate(ranking, start=1):
            scores[doc_id] = scores.get(doc_id, 0) + 1 / (k + rank)
    return sorted(scores, key=scores.get, reverse=True)

Also useful: - Metadata filters before search (tenant, language, date range, product version). - Query rewriting — expand pronouns from chat history into a standalone query before retrieving. - Multi-query — generate 3 paraphrases, retrieve for each, fuse.

Gotcha: apply the tenant/ACL filter inside the search call, not after ranking, or you will retrieve k documents and show the user zero while paying full latency — or worse, leak them.

How to study this topic

  1. 1. Read the explanation once end to end, skipping the code.
  2. 2. Re-read the code snippets and say out loud what each line does.
  3. 3. Answer the quiz without scrolling back — that is the real check.
  4. 4. Read every explanation, even for the answers you got right.

Scoring

  • · Up to 30 XP from quiz accuracy.
  • · +15 XP for a perfect first attempt.
  • · +5 XP while a 3-day streak is running.
  • · 75% or higher marks the topic complete.

Check your understanding

  1. 1. Why add BM25 to a vector search system?

  2. 2. What does Reciprocal Rank Fusion combine?

  3. 3. Where must access-control filters be applied?

Badges

First light
Pass your first topic
Sharp shooter
3 perfect first-try quizzes
On a roll
Study 3 days in a row
Week warrior
Study 7 days in a row
Grinder
Earn 500 XP
Deep diver
Earn 1500 XP