When an AI agent asks “what did we decide about vector indexing last month?”, the answer is rarely a clean keyword match. Sometimes the query is strictly lexical (“vector indexing”), sometimes it’s conceptual (“storing high-dimensional embeddings”), and frequently it’s a mix of both.

Building search that catches both intents — without unfairly biasing one ranking system over the other — is essential for a dependable AI memory layer.

The Two Complementary Shapes of Search

Full-text search and semantic vector search solve fundamentally different problems:

Lexical Search (BM25 / Keyword) is precise about exact terminology, codes, identifiers, and boolean queries. It is fast, deterministic, and ideal when the query matches specific names.
Semantic Search (Dense Vectors) understands intent, concepts, and synonyms. It discovers relevant knowledge even when phrasing differs completely from the original note.

Neither engine is sufficient on its own. Pure keyword matching misses natural paraphrases, while pure vector retrieval can lose precision on exact error codes or technical constants. Combining both in a unified retrieval pipeline yields the strongest recall.

Why Simple Score Blending Fails

A common initial approach is score combination (alpha * keyword_score + beta * vector_score). In practice, this creates subtle ranking instability:

Incompatible Scales: BM25 scores are unbounded positive numbers, whereas cosine similarities typically sit between -1 and 1.
Score Drift: Tuning fixed weights creates bias across different corpora sizes.

To solve this, we rely on Reciprocal Rank Fusion (RRF):

score(document) = Σ [ 1 / (k + rank_in_system) ]

With k = 60, each retrieval system contributes its relative ranking order rather than an arbitrary raw score. A document that ranks #2 in keyword search and #3 in vector search accumulates reciprocal weight naturally.

High-Level Architecture

The retrieval system evaluates parallel streams and fuses results through an isolated ranking stage:

function HybridSearch(query, topK):
    # Step 1: Query both retrieval engines in parallel
    keywordRanks = KeywordEngine.search(query, limit=50)
    vectorRanks  = SemanticEngine.search(query, limit=50)
    
    # Step 2: Combine rankings using Reciprocal Rank Fusion
    fusedScores = Dictionary()
    
    for engineResults in [keywordRanks, vectorRanks]:
        for rank, documentId in engineResults.enumerate():
            fusedScores[documentId] += 1.0 / (60 + rank)
            
    # Step 3: Sort by fused score and return top results
    return fusedScores.sortByScoreDescending().take(topK)

System Resilience & Graceful Fallback

Two key architectural principles ensure stability:

Dual-Engine Fault Isolation: If the vector service encounters network latency or an outage, the pipeline gracefully degrades to lexical search without service interruption.
Scale-Agnostic Ranking: Because RRF evaluates relative ranks rather than absolute magnitudes, the ranking quality remains consistent whether a workspace contains 50 notes or 50,000 documents.