Elastic Hybrid Search Agent

Hybrid Search, Human Touch

Comment

# Project Story: Conversational Hybrid Search Agent

About the Project

Our project is a Conversational Hybrid Search Agent that combines ElasticSearch and Vertex AI to retrieve, reason, and answer user queries in a chat-like interface. The system leverages both keyword-based search and semantic embeddings to provide accurate, context-aware responses.

Inspiration

The inspiration came from the observation that traditional search engines often return results that are technically relevant but lack contextual understanding. We wanted to build a system where users could ask questions naturally and get answers that understand the meaning of their query, not just match keywords.

We were also motivated by the Elastic x Google Cloud Challenge, which encouraged innovative use of ElasticSearch and Vertex AI.

What We Learned

During this project, we gained hands-on experience with:

  • Hybrid Search: Combining keyword-based and semantic vector search to improve retrieval quality.
  • Vertex AI: Using embeddings (Gecko) and generative models (Gemini) to create intelligent, conversational responses.
  • System Integration: Connecting ElasticSearch with Vertex AI in a seamless workflow.

We also learned about the trade-offs between search precision and response generation, and how context selection influences answer quality.

How We Built It

  1. Data Preparation: We uploaded FAQs, documentation, and knowledge base content into ElasticSearch.
  2. Embedding Generation: For semantic understanding, queries are converted to vector embeddings using Vertex AI’s Gecko model.
  3. Hybrid Retrieval: ElasticSearch combines traditional keyword search with vector-based similarity to fetch relevant documents.
  4. Conversational Response: Retrieved documents are passed to Vertex AI’s Gemini generative model to produce a natural, explanatory response.

Mathematically, the relevance score of a document (d) for query (q) is computed as a combination of keyword match score and vector similarity:

[ \text{score}(q, d) = \alpha \cdot \text{keyword_score}(q, d) + (1-\alpha) \cdot \text{cosine_similarity}(embedding_q, embedding_d) ]

where (0 \leq \alpha \leq 1) balances the two search methods.

Challenges Faced

  • Hybrid Search Tuning: Finding the right balance between keyword relevance and semantic similarity was challenging.
  • Context Management: Ensuring that the generative model received only the most relevant documents without overwhelming it with irrelevant data.
  • API Integration: Smoothly integrating ElasticSearch queries with Vertex AI embeddings and generative responses required careful orchestration.

Despite these challenges, the final system provides accurate, conversational answers, making it a strong demonstration of hybrid search in action.

Conclusion

This project taught us how to bridge traditional search and modern AI to create a more intelligent, conversational retrieval system. It was an exciting journey combining ElasticSearch’s speed and reliability with Vertex AI’s semantic and generative power, demonstrating the future of AI-powered search.

Built With

  • ai
  • elasticsearch
  • generative
  • google
  • languages:-python-platforms-/-cloud-services:-google-ai-studio-(vertex-ai)-frameworks-/-libraries:-elasticsearch-py-databases-/-storage:-elasticsearch-(knowledge-base-storage)-apis:-vertex-ai-embeddings-(gecko)
  • model
  • vertex
Share this project:

Updates