AI & ML
Generative Search and Recommendation at Shopify
arXiv cs.IR0 views
arXiv:2611.08804v1 Announce Type: new Abstract: Search and recommendation at Shopify require retrieving relevant products from a changing catalog of billions of products under strict latency constraints. We present SID-LLM, which casts both tasks as generating semantic IDs: short sequences of content-derived tokens representing products. We jointly pre-train SID-LLM on catalog and behavioral data, giving search and recommendation a shared representation and training objective. We show that reinforcement learning further improves judged search relevance. Furthermore, distillation from task-specific experts improves recommendation quality over joint supervised fine-tuning without a statistically detected loss in search quality. Finally, for efficient serving, we pair multi-token prediction with a lightweight ranker, improving retrieval quality over our autoregressive baseline while reducing latency. We evaluate SID-LLM both as a standalone retriever and as a component of Shopify's production search and recommendation systems. As a standalone search system, SID-LLM achieves quality comparable to Shopify's existing multi-model search stack while meeting production latency requirements. Integrating SID-LLM into the search and recommendation stacks provides significant gains in both offline evaluations and online A/B tests. Together, these results demonstrate the practical value of semantic-ID generation as a unified approach to product discovery at scale.
Source:https://arxiv.org/abs/2611.08804PreviousSteerGR: Steering Generative Recommendation via Sparse Autoencoders for Controllable Game DiscoveryNextSkinMind: Equipping Agentic RAG with Human-Inspired Graph Probing for Reliable Dermatology AI
Related posts
A small business running 8 projects on Claude Code for 30 days: rules ignored, projects crossing into each other, and what finally helped
0dev.to AI
L'IA trouve les peptides, la VR les révèle : pourquoi Antigravity change la donne pour la découverte de médicaments
0dev.to AI
Why Tesseract Fails on Cursive Handwriting (And How We Built a Two-Tier Vision Pipeline in Node.js)
0dev.to AI
OFX looks like XML. It isn't, and QuickBooks will reject it
0dev.to WebDev
Before You Buy GEO Monitoring Software, Build a 50-Line Baseline
0dev.to Python
Building Self-Improving AI Models Without Blinding Yourself to Failure
0dev.to Python