Self-Learning Agent-Based Retail Search

Introduction Published

Why ranking in retail is a multi-objective problem before it is a relevance problem.

1. Self-Learning Agent-Based Retail Search, Part 1: Why Retail Search Is Harder Than It Looks

   Retail search is not just about returning relevant products. The hard part is deciding which relevant product should come first when customer needs, business goals, inventory, promotions, trust, and mobile behavior all compete.

Phase 1 — Cranfield Foundation Complete

Prove the method on 1,400 academic abstracts that cannot lie: a measured BM25 baseline, failure-driven experiments, published rejections, and evidence-gated promotion. Ends at a keyword ceiling, then breaks it with embeddings.

1. Self-Learning Agent-Based Retail Search, Part 2: The Baseline Before the Agents

   Before search agents tune anything, the system needs a measurable baseline. This article shows Phase 1 of the project: an out-of-box OpenSearch BM25 baseline on Cranfield, the live relevance numbers, and the failures agents will need to improve.

2. SLAB-RS, Part 3: The Agent Climbs Keyword Search to Its Ceiling

   An AI agent ran five ranking experiments on a live OpenSearch baseline in six days — work that would have taken a core search team months. This article shows the process, the failures, and how a rarely-shipped old technique won the keyword round.

3. SLAB-RS, Part 4: The Agent Breaks Its Ceiling with Embeddings

   Embeddings from a chat model scored five times worse than random vectors. A purpose-built retrieval model beat the keyword ceiling by 8.4%. The agent’s three embedding attempts, a machine-learned ranking detour, and a live OpenSearch vector index.

Phase 2 — Cross-Domain Validation with BEIR Complete

Every Phase 1 technique, re-run across fifteen public BEIR datasets spanning science, finance, argument retrieval, fact-checking, and web search. The verdict: the keyword rerankers were domain-conditional, and hybrid search was the one Universal win. Result: ARCH-0.5, the first architecture chosen from cross-domain evidence, deployed live.

1. SLAB-RS, Interlude: Why the Agent Is Not in a Store Yet

   Four articles into a series called retail search and there is not a single product — only aeronautics abstracts. Here is the map: what BEIR is, why the same BM25 scores 0.158 on one dataset and 0.789 on another, why I was wrong to call Phase 2 a filter, and what has to happen before the agent reaches real products.

2. SLAB-RS, Part 5: How the Agents Discovered Hybrid Search

   The agents took their Phase 1 wins to fifteen new domains — and discovered hybrid search: the one method that improved every single one, by up to 64%. The keyword tricks that looked brilliant on aeronautics turned out to be aeronautics-shaped, and even the cleverest ranker got demoted by Occam’s razor.

Phase 3 — Retail Relevance with Amazon ESCI Complete

Documents become products: 1.2 million real Amazon items with graded Exact/Substitute/Complement/Irrelevant labels. The academic BGE hybrid transfers at parity, on one free-tier box inside 700 ms; the full classical toolkit proves the ceiling but not worth its cost; and the one shipped win — a negation rule — comes from reading the failures, not a bigger model.

1. SLAB-RS, Part 6: The Agent Meets a Million Real Products

   Five parts of a retail search series, and the documents were never products. Phase 3 changes that: 1.2 million real Amazon products, graded Exact/Substitute/Complement/Irrelevant labels, and one question — does the academic stack survive contact with real retail? It does. The BGE hybrid lands at parity with the published single-model baseline, at production latency, and int8 quantization makes a live million-vector index fit on a free-tier box, losslessly.

2. SLAB-RS, Part 7: The Textbook Wins the Benchmark, and Loses the Trade

   Retail is the first dataset in the project with real graded labels — so it is the first place the classical retail-search toolkit gets a fair audition. Field weighting stayed dormant. Learning-to-rank finally woke up, and then relearned the hybrid it was supposed to beat. The full textbook cascade won the benchmark — by +0.13% over the lean stack that was already shipped. Measure everything, ship almost nothing.

3. SLAB-RS, Part 8: The Word the Embeddings Couldn't Read

   Phase 3 measured the whole classical toolkit and shipped almost none of it. One thing did ship — and it didn’t come from a bigger model. It came from reading a thousand failures and noticing that a million-vector embedding model cannot read the word ‘without’. The fix was a single linguistic rule. +4.8 nDCG on negation queries, live.

Phase 4 — Behavioral Retail Search Planned

Add the one thing academic data cannot provide: customers. Clicks, add-to-cart, and purchases as ranking signals — and a decision on whether they justify a learned ranking layer.

Not written yet — this phase is planned.