During the Flip the Switch Hackathon, I worked on a problem that can happen pretty easily in data engineering but can cause serious issues later.
Sometimes, a developer changes a field in a dataset, and everything seems fine. But another model that depends on that field might stop working correctly. The worst part is that these problems can go unnoticed and affect business reports.
That's what made me build SchemaCourt.
What is SchemaCourt?
SchemaCourt is a project that helps data teams find problems caused by schema changes and fix them before they affect reporting.
For example, imagine a checkout service changes a field from discount_cents to promo_struct.discount_cents.
The service itself still works, but a revenue model that uses the old field might start returning NULL values. This can affect revenue calculations without anyone immediately noticing.
In my simulation, the affected revenue metric represents $17.2M across 100,000 rows of data.
How does it work?
When SchemaCourt detects a schema change, it checks which models are affected and measures how much the change could impact the data.
It then suggests a SQL fix that works with both the old and new data structures:
COALESCE(discount_cents, promo_struct.discount_cents, 0) Instead of applying the fix automatically, SchemaCourt shares the findings and proposed changes in a Switch Room.
The data owner can review everything and approve the change before it is applied.
After approval, SchemaCourt tests the fix against both the old and new data to make sure the results remain correct.
Using Switch
I used Switch to make this process easier to follow and manage.
The project uses a Telegram Switch Room where SchemaCourtBot shares the detected change, affected model, impact, and proposed SQL fix.
The data owner can approve the change through a Wispr Flow voice interaction. Once approved, the workflow applies the fix and checks both versions of the data.
I liked this part because it makes the process more than just an AI tool giving suggestions. The team can actually review what's happening and take responsibility for the final decision.
Testing the fix
For testing, I used:
- 100,000 rows of legacy data
- 100,000 rows with the changed schema
After applying the fix, the measured difference was $0.00.
This means the repaired model produced the expected revenue result for both versions of the data in my simulation.
What I learned
Building SchemaCourt helped me understand how small changes in data can create bigger problems downstream.
It also gave me a chance to explore how AI can be useful in data engineering without giving it complete control over important changes.
I wanted to build something where AI helps find the problem and suggest a solution, but a human still gets to make the final decision.
That was the main idea behind SchemaCourt, and it was a really interesting project to work on during the hackathon.
GitHub: https://github.com/miheergathibandhe/SchemaCourt
