Requirements
Before starting, ensure you have the following packages installed:Setup
Start by setting up the environment:The Dataset
In this cookbook, we’ll work with a product catalog dataset containing fashion items with structured metadata. The dataset includes:- Basic product information: titles, descriptions, brands, and prices
- Categorization: categories, subcategories, and product types
- Attributes: structured characteristics like sleeve length, neckline, and fit
- Materials and patterns: fabric types and design patterns
Prepare DataFrame for LanceDB
We’ll use a Pandas DataFrame as the ingest interface.description
as the “text” field, although you could concatenate title/description/etc.
Generate Embeddings (OpenAI)
Now, let’s create embeddings for our product descriptions. We’ll use OpenAI’s text-embedding-3-large model:Combine all text fields into a single searchable text field
We’ll create a single text field that combines the product name, description, and category. This will allow us to perform a single search over all relevant text content:Ingest Data into LanceDB
We’ll use LanceDB to store our product data and embeddings. LanceDB makes it easy to experiment, as it provides both vector and hybrid search capabilities within one single API.Generating Synthetic Data
When you don’t have production data to start with, you can generate synthetic data to simulate a real-world scenario. We already have the ‘output’, which is the clothing item we just embedded. We now want to generate synthetic queries that would be relevant to the clothing item. In this case, we’ll use GPT-5 to generate realistic user queries that would naturally lead to each product in our catalog. This gives us query-product pairs where we know the ground truth relevance.Hybrid Search in LanceDB
LanceDB makes it easy to combine vector search with full-text search in a single query. Let’s see how this works with a practical example:title | brand | description | category | subcategory | product_type | price | _relevance_score |
---|---|---|---|---|---|---|---|
Elegant Wedding Guest Dress | Zara | A stunning formal dress perfect for wedding ceremonies and receptions | Women | Dresses | Formal Dresses | 189.99 | 0.87 |
Floral Maxi Dress for Special Occasions | H&M | Beautiful floral pattern dress ideal for weddings and formal events | Women | Dresses | Maxi Dresses | 149.50 | 0.82 |
Satin Wedding Guest Jumpsuit | ASOS | Sophisticated alternative to dresses for wedding guests | Women | Jumpsuits | Formal Jumpsuits | 165.75 | 0.79 |
Men’s Formal Wedding Suit | Hugo Boss | Classic tailored suit perfect for wedding guests | Men | Suits | Formal Suits | 399.99 | 0.71 |
Beaded Evening Gown | Nordstrom | Elegant floor-length gown with beaded details for formal occasions | Women | Dresses | Evening Gowns | 275.00 | 0.68 |
Implementing Different Search Methods
To properly compare different search approaches, we’ll implement three search functions:- Semantic search: Uses only vector embeddings to find similar products
- Lexical search: Uses only BM25 text matching (similar to what traditional search engines use)
- Hybrid search: Combines both approaches for potentially better results
Evaluation Metrics
To objectively compare our search methods, we’ll use two standard information retrieval metrics:- Recall: The proportion of relevant items successfully retrieved
- Mean Reciprocal Rank (MRR): How high relevant items appear in our results
Prepare Evaluation Data
Assuming your synthetic queries are a list of dicts with"query"
and "id"
.
Run the Experiment
Now we can run the experiments. The code does the following:- Tests each search method with different numbers of results (k=3, 5, and 10)
- Aggregates the metrics by calculating the mean recall and MRR for each method
- Organizes the results in a DataFrame for easy comparison
k | method | recall | mrr |
---|---|---|---|
3 | semantic | 0.906 | 0.816 |
3 | lexical | 0.937 | 0.815 |
3 | hybrid | 0.916 | 0.848 |
5 | semantic | 0.937 | 0.823 |
5 | lexical | 0.969 | 0.822 |
5 | hybrid | 0.948 | 0.860 |
10 | semantic | 0.974 | 0.828 |
10 | lexical | 0.984 | 0.824 |
10 | hybrid | 0.990 | 0.868 |
Conclusion
Our evaluation demonstrates that hybrid search consistently outperforms both pure vector search and lexical search across all tested k values. Key findings:- Hybrid search achieves the highest MRR scores, showing that combining semantic understanding with keyword matching places relevant results higher in the result list.
- Lexical search performs surprisingly well on recall, reminding us that traditional keyword approaches remain valuable for explicit queries.
- Vector search provides a solid baseline but benefits significantly from the precision that text matching adds.
- Test multiple retrieval strategies on your specific data
- Measure performance with appropriate metrics
- Consider the trade-offs between implementation complexity and performance gains