Teaching Sidekick to say no: automated data curation with LLM judge consensus

Shopify Engineering2026-06-16 03:300 views

Production training data only captures successful queries; it can't teach a model when to say no. We built an automated curation pipeline using LLM judge consensus to close that gap.

Source：https://shopify.engineering/sidekick-curation

My favorites

PreviousClustering billions of products for agentic commerce with Catalog API NextQuick: An internal hosting platform for the AI era

Log Receipts to Notion Expense Tracker with Make and OpenAI (No Code)

If you want a notion expense tracker that logs every receipt automatically, one no-code Make scenario does the whole job

0dev.to AI

The Cognitive Debt of AI Shortcuts: Why Mastering Hard Skills Still Matters

In software engineering, automating repetitive work is standard practice, from relying on compilers and linters to deleg

0dev.to AI

OpenAI Q3 Revenue Tops $10B as ChatGPT Paying Users Hit 100 Million

OpenAI's Q3 revenue reached about $10B, up ~50% quarter-over-quarter, with ~100M paying ChatGPT users and a ~$50B annual

0dev.to AI

Alibaba Launches Qwen4-Max Preview, Opening Its Next-Gen Model Family

On Nov 11 Alibaba opened Qwen4-Max (qwen4-max-preview) on its Token Plan - the first callable model of the Qwen4 generat

0dev.to AI

Torus Raises $500M to Build Modular AI Foundry Compute

Torus raised $500M led by Magnetar to build 'Foundry' - prefab AI compute packing power, storage, cooling, networking an

0dev.to AI

What goes into a corporate agent harness

We build internal AI systems for companies. Most of our work is the layer around the model, which we call the harness. H

0dev.to AI

Teaching Sidekick to say no: automated data curation with LLM judge consensus

Related posts

Log Receipts to Notion Expense Tracker with Make and OpenAI (No Code)

The Cognitive Debt of AI Shortcuts: Why Mastering Hard Skills Still Matters

OpenAI Q3 Revenue Tops $10B as ChatGPT Paying Users Hit 100 Million

Alibaba Launches Qwen4-Max Preview, Opening Its Next-Gen Model Family

Torus Raises $500M to Build Modular AI Foundry Compute

What goes into a corporate agent harness