This is a submission for the MLH x DEV Writing Challenge

What I Built

My team, Order of the Golden Eagles, built Signal2Speech, an accessibility prototype for people who have difficulty speaking or typing. People with conditions like ALS, stroke, or cerebral palsy can lose the ability to speak or type, even though they still know exactly what they want to say. Most assistive tools either make them spell out every letter, which is slow, or guess for them, which takes the decision out of their hands. Our main rule came from that: the app suggests what to say, but the wearer decides what they actually want to say.

A conversation partner types a question, and Claude suggests two short replies based only on facts the app has verified. The wearer picks one with a P300 response, which is a brain signal that shows up about 300 milliseconds after the card you're watching flashes. A preview then shows the exact words, and nothing is spoken until the wearer deliberately confirms with a second, different signal. The app also has a NASA example where the wearer can ask which Michigan county's air quality got worse when wildfire smoke arrived in June 2023, and it shows NASA satellite images next to the EPA's daily county Air Quality Index (AQI).

My part was mostly the reliability and testing side. I built the simulated brain input so the team could rehearse without a headset, the first eyes-closed confirm switch, the smoke check and recovery tests, the hour-long soak test, the Model Context Protocol (MCP) server that lets an AI assistant read summaries from our signal pipeline, and the science note for the NASA data. I also started the team's devlog.

The biggest thing I learned is that separating "choose" from "confirm" matters more than how accurate any single step is. A P300 pick can be wrong, but if the wearer has to confirm it with a different signal, a wrong pick costs a few seconds instead of the wrong words being said. I also learned that AI has to be grounded before it speaks for someone. Before we added grounding, every model we tried answered our demo question wrong. It said Wayne County's AQI rose more, when Washtenaw actually rose 186 points and Wayne rose 168. Finally, I learned that the hardware's design decides what signals you can actually measure, and simulated data can't tell you that because it only does exactly what you tell it to.

Demo

The repo is public: https://github.com/Order-of-the-Golden-Eagles/Signal2Speech

You can try it without a headset by using the keyboard:

git clone https://github.com/Order-of-the-Golden-Eagles/Signal2Speech.git cd Signal2Speech pip install -r requirements-lock.txt python -m signal2speech 

That opens a small launcher with a button for each tool, and each button says which input it uses. A row of labels at the bottom of the app always shows whether the input is Keyboard, Simulated, Recorded, or Live EEG, so simulated input is never mistaken for a real headset.

Partner Technologies

g.tec (Mind and Machine track). g.tec's tools are the core of the project. We first planned to build in Unity with g.tec's starter. About an hour and a half in, we switched the whole project to Python and g.Pype, g.tec's Python toolkit for EEG, because g.Pype lets you actually see the signal pipeline. EEG goes through g.Pype's filters and OSCAR artifact removal (cleaning out noise from blinks and muscle movement). We then cut out the EEG around each flash, filter it to 1 to 30 Hz, average five flashes per card, and classify them with Linear Discriminant Analysis (LDA). Our pipeline diagrams are built by the same code the app runs and read back from g.Pype's own record of nodes and connections, so a diagram can't show a pipeline we don't actually run.

Working with g.Pype went well overall, but there were two rough spots. With no headset connected, g.Pype's start() waits forever instead of failing, so we had to put a time limit on every step of the device check. We also found that OSCAR delays the signal by about a third of a second, but g.Pype doesn't shift the flash markers to match. Because of this, the classifier ends up looking at the wrong part of the brain response after each flash, which can cost up to 0.14 accuracy if you don't account for it. On the good side, OSCAR made the biggest blink and movement swings 14 to 65% smaller on g.tec's recorded data while barely changing the rest of the signal.

Our g.tec Unicorn Hybrid Black headset was lost in transit, so we didn't get access to one until the last minute, so for most of the event we used g.tec's own recorded P300 dataset of five real people. The app plays each recorded brain response through the same pipeline and classifier the live path uses. Across the five people, it picked the right card out of two 94% of the time from one flash and 100% from five. When we scrambled the labels, it dropped to chance (42 to 46%), which tells us it wasn't fooling itself. Near the end of the event, we were able to run the app on a real Unicorn headset through Unicorn Suite's Unicorn Recorder over Lab Streaming Layer (LSL, a standard way to send live sensor data between programs). On the real headset, we switched the default confirm to a double blink, calibrated on rest and three blink pairs, and labeled it on screen as an eye signal instead of a brain signal.

PiEEG. Overnight, we borrowed PiEEG Octopus 16 boards from the venue library and tested them on two people, with consent. PiEEG's simulator streamed into our g.Pype pipeline over LSL without problems, and the real board connected and recorded well. However, we could not find a usable brain signal. Eyes-closed alpha actually went slightly down, and a flickering square gave no response. The likely reason is that the board's reference pin sits within 2 cm of its signal pins, which cancels a brain rhythm before it gets measured. We did not prove this, but it fits two people, two kinds of brain signal, and good contact. We also wrote a keepalive tool that reconnects when the board's Bluetooth link drops, since PiEEG's own script quits quietly when that happens.

Claude (Anthropic). AI Answering uses Claude Sonnet 5.5 through the Anthropic API. Each request only includes the typed question and a "Verified facts" block built from our NASA and EPA data, never EEG, and Claude is told to say "I don't know" for anything outside those facts. For map answers, the app speaks our own fixed line, and the numbers come straight from the dataset. We also used Claude as a coding agent throughout the event, with a shared CLAUDE.md so every teammate's agent started from the same rules, like keeping API keys and raw EEG out of git.

Hackathon Experience

We attended Hack the Nest 2026 as a team of four: Vasanth Anbukumar, Long Le, In Young Song, and me. We pulled an all-nighter, and the commit history shows it, with over 200 pull requests merged from Saturday afternoon to Sunday. Four people merging that fast meant we needed CI on every pull request and a shared selection contract (one set of rules every input has to follow), so the keyboard, simulator, recorded, and headset inputs could be built in parallel and still behave the same.

What I will remember most is how much the plan changed. We switched from Unity to Python early on, our headset got lost in transit, the borrowed PiEEG boards gave no usable signal in the middle of the night, and we finally got a real headset working after the deadline. Through all of that, we tried to keep the project honest about what we had actually tested. Every recorded or simulated input is labeled that way, and we never called anything live until it actually was. For an app that is meant to speak for someone, I think that honesty matters as much as the code.