I trained an 825k-parameter model to generate drawing programs that execute exactly on an RP2040 [P]

I’ve been working on a small research project about whether sub-million-parameter models can learn to generate executable drawing programs for constrained hardware. The current system is an 825k-parameter autoregressive transformer which generates ~100 bytes of drawing bytecode rather than pixels. The bytecode is then transferred to a Raspberry Pi Pico, where a small fixed-point virtual machine executes it and streams the resulting geometry back over UART.

The model runs on the host. The Pico only stores and executes the generated program, so this is not a claim that the transformer itself runs on the microcontroller. The execution side is currently the most solid part of the project: 12,670/12,670 generated traces matched the Python reference VM exactly, 1,862 bytes of flash for the interpreter, 0 bytes of static RAM and 492 bytes of peak stack, 7,334 cycles per drawing at 12 MHz, or about 0.61 ms for the measured QuickDraw programs and no floating-point hardware or tensor runtime is needed on the Pico.

I have been comparing token, byte, bit, typed-token, and delta-coordinate representations while keeping the underlying drawing information equivalent. so far, perhaps trivially, the conclusion is that the answer depends strongly on the corpus: on a synthetic program corpus, a bit-level representation was essentially equivalent to bytes at the converged budget, while on real QuickDraw sketches it incurred an approximately 11.6-bit penalty per drawing.

I’ve also tested whether a model can discover repeated structure such as loops from flat bytecode, and whether hierarchical stroke planning helps at this scale. The planner did not improve likelihood, although it substantially improved termination and generated-length behavior. Another experiment found that the model can show a strong preference for compatible relational context under teacher forcing, while still struggling to produce the exact compatible continuation when sampling freely.

The project is still very much a work in progress. The current direction is to add an explicit source-span / affine-relation / copy-or-emit action while keeping the final output as ordinary flat drawing bytecode. The goal is to test whether making relations explicit helps with exact generation on unseen combinations.

The repository, demo instructions, figures, captured RP2040 traces, and experiment details are here:

https://github.com/roodriigoooo/drawing-machine

I would especially appreciate feedback on 1) how to evaluate novelty and memorization more convincingly 2) better ways to measure exact program generation rather than only teacher-forced likelihood and 3) experiments that would make the microcontroller result more meaningful