“Perfection is achieved, not when there is nothing more to add, but when there is nothing left to take away.” — Antoine de Saint-Exupéry
Can you train a vision model from scratch on raw medical pixels to detect Tuberculosis, quantize it down to 8 bits, and run inference on a $2.50 microcontroller with less memory than a vintage digital watch?
Most of the industry today is focused on scaling up: massive GPU clusters, hundreds of billions of parameters, and megawatts of power. But there is a complementary frontier at the extreme low end: how much useful intelligence can you squeeze into tiny, dirt-cheap microcontrollers?
The target here is an ESP32 ($2.50, running on a single lithium cell) that can sit on a desk in a rural field clinic without internet access, cloud connectivity, or an on-site radiologist.
This is a walkthrough of building TinyTB-Net from scratch: the architecture, why naive accuracy fails in clinical triage, what broke on my first ResNet attempt, how Squeeze-and-Excitation attention fixed it, how INT8 post-training quantization works under the hood, and how to serve a self-contained diagnostic web interface directly from microcontroller flash without exhausting the heap.
In order to fully understand this article, knowledge of neural network design is preferable, yet not mandatory.
1. Motivation: Edge AI for Tuberculosis
Tuberculosis (Mycobacterium tuberculosis) still kills roughly 1.3 million people each year, making it one of the deadliest infectious diseases globally.
The frustrating part is that effective antibiotics (isoniazid, rifampicin) have been dirt cheap for over seventy years. The primary bottleneck is timely diagnosis. A chest radiograph (CXR) is inexpensive and fast to acquire, but reading it requires a trained radiologist. In low-resource regions, the radiologist-to-patient ratio can drop below 1 to 500,000.
An offline screening model running on an inexpensive microcontroller turns any battery-powered digital X-ray station or flatbed digitizer into an automated triage station.
(Quick disclaimer: this is an educational engineering build, not a commercial medical device. Clinical systems in hospitals undergo strict regulatory validation. But exploring how far a 300k-parameter model can go on edge silicon is deeply satisfying.)
2. Why Accuracy Fails in Clinical Triage
In standard classification benchmarks like CIFAR-10 or ImageNet, tracking top-1 accuracy (correct / total) is standard practice. In medical triage, accuracy is almost uninformative because error costs are heavily asymmetric.
Consider a test cohort of 120 patients: 59 confirmed active tuberculosis cases and 61 healthy controls.
In this confusion matrix, errors fall into two buckets with very different consequences:
False Negatives (top-right, 24 patients): An algorithm tells someone with an active bacterial infection that they are healthy. They return home untreated and continue transmitting bacilli.
False Positives (bottom-left, 9 patients): A healthy patient is flagged for follow-up testing (like a GeneXpert sputum swab). The confirmatory lab comes back negative two days later. The patient dealt with temporary anxiety and an extra swab, but no one was harmed.
If your loss function weights false negatives and false positives equally, the optimizer will happily trade missed infections for fewer false alarms. In clinical screening, missing an infection is far worse than ordering an extra swab.
Diagnostic Metrics
Instead of overall accuracy, clinical triage relies on three core measures:
Sensitivity (Recall / True Positive Rate):
The fraction of genuinely sick patients detected by the model. In screening, this needs to stay high, typically ≥ 85–90%.
Specificity (True Negative Rate):
The fraction of healthy patients correctly cleared. High specificity prevents confirmatory testing labs from being swamped with false alarms.
The Trade-off: Sensitivity and specificity pull in opposite directions. A model predicting “sick” for everyone achieves 100% sensitivity and 0% specificity. Predicting “healthy” for everyone does the reverse.
The Pair Bet: ROC-AUC
Because sensitivity and specificity shift depending on where you set your classification threshold (for example, ≥0.50 versus ≥0.35), we need a threshold-independent measure of ranking quality.
I track ROC-AUC (Area Under the Receiver Operating Characteristic Curve).
A straightforward way to think about ROC-AUC: imagine drawing one confirmed TB patient and one healthy control at random from the test set. ROC-AUC is the probability that the network assigns a higher suspicion score to the sick patient than to the healthy patient.
AUC=0.50: Equivalent to a random coin flip.
AUC=1.00: Perfect separation.
AUC≥0.85: A clinically viable screening candidate.
The Loss Function: Weighted Cross-Entropy
For training, we use standard binary cross-entropy, but with one key tweak: we weight positive examples more heavily.
In standard training, a false alarm and a missed infection carry the exact same penalty. By scaling up the positive term with pos_weight > 1.0, we tell the optimizer that letting an active case slip through hurts far more than ordering an extra confirmatory swab.
Under the hood, we use PyTorch’s nn.BCEWithLogitsLoss(). It operates directly on raw logits rather than probabilities, using the numerically stable log-sum-exp formulation to avoid floating-point underflow:
3. Microcontroller Constraints: Hardware Arithmetic on the ESP32
Why the ESP32? An ESP32 or ESP32-S3 module costs roughly $2.50, runs a dual-core 32-bit Xtensa CPU at 240 MHz, includes built-in Wi-Fi, and draws around 100 to 240 mA at 3.3V. A single 18650 cell can power it through a full day of clinic triage.
Working with microcontrollers requires careful memory budgeting before writing any architecture code.
Weights vs. Intermediate Activations
People often assume model weight size is the primary constraint. On an ESP32 with 4 MB or 8 MB of external SPI flash, storing 500 KB to 1 MB of weights in read-only .rodata is straightforward.
The real constraint is SRAM.
The ESP32 has roughly 320 KB to 512 KB of internal static RAM. During inference, the runtime needs scratch memory for intermediate activation maps (each layer's output tensors).
Consider an input image of 224×224 pixels. A convolution with 16 output channels and stride 2 yields an activation map of 112 × 112 × 16. In standard 32-bit floats (float32, 4 bytes per number):
That single buffer exceeds the entire internal SRAM of the microcontroller. Allocating it on the heap triggers an immediate heap panic and crash.
Quantized to 8-bit integers (int8, 1 byte per value), that buffer drops to ~196 KB:
Residual architectures also require ping-pong memory arenas to hold skip-connection tensors while evaluating the residual branch, pushing peak activation memory toward ~392 KB. Channel counts, strides, and pooling stages all had to be planned around this limit.
4. Dataset Assembly: The Initial 800 Images
Training on chest radiographs is quite different from training on ImageNet. I started by combining two public, de-identified clinical datasets:
NIH NLM Montgomery County Dataset (Maryland, USA): 138 frontal radiographs (58 confirmed active TB cases, 80 healthy controls), captured at 12-bit grayscale depth.
Shenzhen No.3 Hospital Dataset (Guangdong, China): 662 frontal radiographs (336 confirmed TB cases, 326 healthy controls).
Together, these provided 800 radiographs (394 TB, 406 healthy).
Eight hundred images is a very small dataset for training a convolutional network from scratch, yet I was determined to try it out.
Furthermore, radiographs from different hospital systems exhibit noticeable distribution shifts. The Shenzhen scans were acquired on a Philips digital unit with high contrast and sharp bone edges, while the Montgomery scans have a softer dynamic range. Without strong regularization, a small network easily memorizes scanner artifacts (collimator margins, edge contrast) instead of lung tissue.
5. Architecture Phase 1: A Baseline Residual Network
Before adding attention mechanisms or custom kernels, I needed a clean baseline.
A standard dense network was out of the question. Flattening a single 224×224 grayscale radiograph yields 50,176 input values. Feeding that into a modest 512-neuron hidden layer produces:
In 32-bit floats, that single layer weighs over 102 MB. That is twenty-five times larger than our entire 4 MB SPI flash chip before writing a single convolution or activation buffer.
Convolutions avoid this parameter explosion through spatial weight sharing: a 3×3 kernel has only 9 learnable numbers, regardless of the input resolution.
But stacking convolutions into a deep network runs directly into vanishing gradients during backpropagation:
When multiplying dozens of layer Jacobians, singular values below 1.0 cause the error signal to decay toward zero before it reaches the early filters. When training from scratch on only 800 medical scans without ImageNet pretraining, an unassisted deep stack refuses to learn.
Residual connections solve this by letting layers fit a residual mapping, bypassing the transformation entirely:
Differentiating the block reveals why training stays stable:
The +1 term creates an unobstructed highway for backpropagation. Even if the learned weights produce a Jacobian near zero, gradients flow backward through the identity shortcut unchanged.
The Initial Candidate: A 4-Stage ResNet
Using residual units, I assembled a compact 4-stage network for 224 × 224 grayscale scans:
Stem: Conv 7 × 7 (stride 2, padding 3) → BatchNorm → ReLU → MaxPool 3 × 3 (downsampling from 224 × 224 to 56 × 56).
Stage 1: 2 residual blocks with 16 channels.
Stage 2: 2 residual blocks with 32 channels (stride 2 downsampling).
Stage 3: 2 residual blocks with 64 channels (stride 2 downsampling).
Stage 4: 2 residual blocks with 128 channels (stride 2 downsampling).
Head: Global Average Pooling (7 × 7 × 128 → 1 × 1 × 128) → Linear(128 → 1).
The whole network has roughly 280,000 parameters. In 32-bit floats, that is about 1.12 MB. Quantized to INT8, it drops to 280 KB, easily fitting into ESP32 flash with plenty of room left for firmware and static assets.
6. Training & Results: Phase 1 Evaluation
I trained the baseline model for 40 epochs using a standard set of regularized hyperparameters:
Phase 1 Training Configuration: - Optimizer: AdamW (lr = 1e-3, weight_decay = 1e-4) - Schedule: Cosine Annealing (40 epochs) - Batch Size: 32 - Augmentations: Horizontal flips, ±8° rotations, mild affine scaling Evaluating the model on a held-out test cohort of 120 patients gave:
Phase 1 Test Evaluation (120 Patients: 59 TB, 61 Healthy): - ROC-AUC: 0.762 - Accuracy: 72.5% - Specificity: 85.2% (52/61 healthy cleared) - Sensitivity: 59.3% (Only 35/59 TB cases caught!) A sensitivity of 59.3% means missing roughly 4 out of every 10 active cases. For clinical triage, this is unacceptable.
Two main factors explained the poor recall:
Channel Indifference: Standard convolutions apply spatial filters across all channels equally. In chest radiographs, most variation comes from normal anatomy (ribs, spine, heart border). A 280k-parameter model easily wastes capacity modeling bone structures rather than subtle infiltrates.
Sample Size: Eight hundred images across two scanner types did not provide enough diversity for training a convnet from scratch.
7. Phase 2: Squeeze-and-Excitation Attention
In commercial settings, the usual workaround is to pull down an ImageNet-pretrained ResNet-50 and fine-tune it. But a ResNet-50 has 25.6M parameters (~100 MB), which cannot fit in ESP32 flash. I wanted to see if a compact model trained solely from scratch on medical pixels could achieve solid diagnostic numbers.
That meant addressing both data volume and architectural feature selection.
Expanding the Dataset: Adding Belarus Scans
To increase training diversity, I incorporated the NIAID TB Portals Belarus Dataset (Minsk, Belarus), adding 308 culture-confirmed tuberculosis radiographs.
This expanded the dataset to 1,108 chest radiographs:
702 Confirmed Tuberculosis Scans (Shenzhen + Montgomery + Minsk)
406 Healthy Controls
It is not a lot, but it’ll help.
Squeeze-and-Excitation (SENet) Blocks
In a standard convolution, every channel carries equal weight regardless of image content. In a chest radiograph, most channels inevitably spend their capacity tracking dense, high-contrast anatomy like ribs and the spine. What we want instead is a dynamic volume knob that can boost channels tracking faint soft-tissue opacities while dampening normal bone contours.
To do this without ballooning parameter count or memory, I added Squeeze-and-Excitation (SE) blocks from Hu et al. (2018). The idea is straightforward: average each feature map down to a single summary number (squeeze), pass those summaries through a tiny two-layer bottleneck network to learn channel-wise importance scores (excite), and multiply each channel by its score (scale). It acts like a content-aware gain control for feature maps.
Because the bottleneck operates on a single channel vector rather than full 2D spatial maps, SE blocks added fewer than 12,000 parameters (~48 KB in float32) across the entire network and zero additional bytes to peak SRAM activation memory. Total model parameters rose modestly from ~280k to 313,153.
Phase 2 Validation Results
Training TinyTB-Net on the expanded dataset for 40 epochs yielded a substantial improvement:
Sensitivity improved from 59.3% to 82.86% at the default threshold of 0.50, and reached 89.5% when calibrated to the clinical screening threshold of 0.35.
ROC-AUC rose to 0.8932, a solid result for a 313k-parameter network trained entirely from scratch.
8. Quantization: Mapping Floats to 8-Bit Integers
The model reached 0.8932 ROC-AUC, but weights and activations were still in 32-bit floats:
While 1.25 MB fits into flash, executing 194 million floating-point operations per inference on a 240 MHz microcontroller without a vector FPU takes several seconds per image.
Post-Training Quantization (PTQ)
Quantization maps continuous 32-bit floating-point numbers to 8-bit signed integers (int8, -128 to +127). For each layer, we determine the dynamic range, divide it into 256 steps, and snap each float to the nearest integer index:
This provides two key benefits for microcontrollers:
Memory: An 8-bit integer uses 1 byte instead of 4, shrinking weight storage from 1.25 MB down to 305.8 KB (a 4x reduction).
Execution Speed: Microcontroller ALUs execute 8-bit integer multiplications and additions much faster than software-emulated or multi-cycle floating-point math.
Impact on Diagnostic Accuracy
Compressing 32-bit floats into 8-bit integers discards precision. To check whether this affected clinical output, I evaluated both models on the same 166-patient test cohort:
The evaluation showed:
Weight storage decreased from 1,252 KB to 305.8 KB (a 4x reduction).
Test ROC-AUC shifted by +0.0007 (0.8932 to 0.8939).
Average probability drift across the cohort was only 1.15%, changing the classification of exactly one patient out of 105 positive cases.
The model now comfortably fits within 7.5% of the ESP32’s flash capacity.
9. The Web Interface: Serving from Flash
The web interface is not the focus of this build, but a diagnostic model locked inside a serial terminal is not much use in a field clinic.
To keep the setup completely self-contained, the ESP32 hosts its own local Wi-Fi Access Point (TinyTB-Diagnostic-AP) and serves a minimal diagnostic web interface directly out of flash memory. You connect with a phone or laptop browser, drop in a radiograph, and run inference locally without internet access or external servers.
Making room for an uploaded image buffer in ~320 KB of shared RAM without triggering heap panics was not exactly a piece of cake, and I will spare you the blow-by-blow of that debugging session with Claude :) The gist of the solution: we downsample the radiograph client-side on an HTML5 canvas before POSTing a compact raw buffer, and keep all web assets gzipped in read-only program memory.
10. Takeaways & Next Steps
A few practical takeaways from this build:
Small datasets require strong inductive bias: With only ~1,100 training radiographs, generic convolutions spent too much capacity on dominant rib contours. Squeeze-and-Excitation blocks provided the biggest diagnostic gain (+0.13 ROC-AUC) for almost no parameter overhead (under 12k parameters).
Metric alignment matters more than raw accuracy: Optimizing for naive accuracy produces models that miss active infections. Threshold calibration and weighted cross-entropy are necessary whenever error costs are asymmetric.
INT8 quantization has minimal clinical penalty: Quantizing weights to 8-bit integers achieved a 4x size reduction with an average probability drift of 1.15% and virtually identical ROC-AUC.
Much of the recent focus in machine learning centers on expanding cluster scale and parameter count. But building small, focused vision pipelines that run reliably on $2.50 silicon is a reminder of how much utility can be extracted from compact models.
