Teaching Sidekick to say no: automated data curation with LLM judge consensus

Shopify Engineering2026-06-16 03:300 views

Production training data only captures successful queries; it can't teach a model when to say no. We built an automated curation pipeline using LLM judge consensus to close that gap.

Source：https://shopify.engineering/sidekick-curation

My favorites

PreviousClustering billions of products for agentic commerce with Catalog API NextQuick: An internal hosting platform for the AI era

Choosing QuePaxa, an unusual consensus algorithm

Cloudflare recently announced Meerkat , an experimental distributed consensus service designed to manage our internal gl

0Cloudflare Blog

China’s Nvidia Honeypot Isn’t Surprising. Jensen Huang Fell for It Anyway

China’s use of sexpionage is a well-established intelligence tactic with recent, high-profile examples. So how did the w

0Wired

Google DeepMind just mathematically verified key parts of Signal

Google DeepMind researchers used agentic AI to mathematically prove, or “formally verify,” a number of the Signal protoc

0Ars Technica

Adobe’s AI experiments let you walk around in photos and interview your characters

Project Screen Test lets you talk with AI-generated characters to shape their personality. | Image: Adobe Some of the mo

0The Verge

Google’s AlphaGenome 2 zooms in on the cellular roots of disease

A graphic representing Google’s AlphaGenome Atlas. | Image: Google Researchers at Google DeepMind have unveiled a new AI

0The Verge

Plurai wants to help AI agents work better using the same approach that helped get self-driving cars to roads

Like the end-to-end models trained to drive cars, AI agents are probabilistic systems operating in unpredictable environ

0TechCrunch