48-Hour Field Notes: I Ran a Tiny AI Code-Review Job on Free Model Access and a Free Server

Two days ago I gave myself a boring constraint: one small code-review job, free model access, a free server, and a hard stop at 48 hours. No new subscriptions, no GPU box, no "just spin up a bigger instance" escape hatch. I wanted to know whether a free tier could survive contact with a real workflow, and I wanted the answer to be something I could re-run instead of a vibe. Isn't that the whole problem with cheap-tier experiments — you finish them and realize you measured nothing?

So the notes below are ordered by what broke first, not by what sounds impressive. The code is a template you can run as-is; the timings are from one small sample, not a benchmark, and your numbers will differ. Keep the method, throw away my constants.

What I actually set up in the first six hours

The job itself is unglamorous: take a unified diff, ask a model for a short review, and log whether a human kept any of the suggestions. I deliberately picked something with a cheap ground truth, because "did this feel helpful" is not a metric I can defend in a post. Here is the shape I settled on:

1. One fixture per review type — a null-check diff, an error-handling diff, and a config diff. Three files, no more.
2. Five repetitions per fixture, same prompt, same temperature, so a change in output has a chance of being a model change rather than a prompt change.
3. A JSONL ledger with one row per request, written before I look at the answer.
4. A request cap and an input-token cap, both enforced in code, so a bad afternoon cannot quietly become a bill.

On the infrastructure side I used MonkeyCode's free server option rather than my laptop, mainly because I wanted the process to keep running while I slept. The project is open source, and the free tier as described at the time of writing includes a token allowance for model access — check the project's current docs for the exact number and terms, because free-tier quantities move and this post should not be your source of truth on that.

Disclosure: This article was prepared as part of MonkeyCode's product outreach.

The ledger came first, and it immediately embarrassed my assumptions

My first instinct was to budget output tokens, since that is what "long answer" feels like. Wrong axis. The diff, the system prompt, and the project conventions I pasted in dominated the request, and output was the small part. So the ledger records input and output separately, plus wall-clock time, plus a human acceptance flag:

# ledger.py - append one JSON line per request, never rewrite history import json, time, pathlib LEDGER = pathlib.Path("ledger.jsonl") def log_request(*, label, started, ended, usage, accepted, note=""): row = { "label": label, "wall_s": round(ended - started, 2), "input_tokens": usage.get("input_tokens"), # None if the API omits it  "output_tokens": usage.get("output_tokens"), "accepted": bool(accepted), # did a human keep the diff?  "note": note, "ts": time.strftime("%Y-%m-%dT%H:%M:%SZ", time.gmtime(ended)), } with LEDGER.open("a") as f: f.write(json.dumps(row) + "\n") 

If your endpoint does not return a usage block, write None and estimate with a tokenizer in a separate column. Labeling an estimate as an estimate is the difference between a ledger and a story.

Guard the budget where the request is built

A cap that lives in your head is not a cap. This is the smallest guard that actually stopped me:

class Budget: def __init__(self, max_input_tokens, max_requests): self.max_input = max_input_tokens self.max_requests = max_requests self.input_used = 0 self.requests = 0 def charge(self, input_tokens): if self.requests + 1 > self.max_requests: raise RuntimeError(f"request cap hit at {self.requests}") if self.input_used + input_tokens > self.max_input: raise RuntimeError(f"would exceed input cap: {self.input_used}+{input_tokens}") self.requests += 1 self.input_used += input_tokens 

Notice that the guard charges before the call, not after. If you charge afterwards you have already paid for the mistake.

Failure one: cold start made my timeout a decoration

The free server was idle overnight, and my first morning batch was not warm in any sense of the word. Requests that took a few seconds the night before took far longer on the first call, which meant my per-request timeout fired on work that was still fine. The fix was boring and effective: measure with a monotonic clock, and set the timeout generously for the first request of a batch only.

import time def timed(call, first_of_batch, base=90.0, cold=180.0): budget = cold if first_of_batch else base start = time.monotonic() result = call(timeout=budget) return result, round(time.monotonic() - start, 2) 

One more thing I checked rather than assumed: that the timeout actually stopped the server-side work. A client-side give-up that leaves a request running is how you get phantom spend, and I have been bitten by exactly that before.

Failure two: retries spent budget twice and taught me nothing

My retry wrapper was cheerful and wrong. On a malformed response it retried, which doubled the input tokens for that fixture, which then made the fixture look expensive in the ledger. Worse, I could not tell a retry from a fresh attempt when reading results later. Two rules fixed it:

• Every attempt gets its own ledger row, with the attempt index in label (for example nullcheck-r3-a2).
• A retry is only allowed for transport errors, never for a response I merely disliked; disliked output is data, not a failure.

After that, "this fixture costs more" turned into "this fixture produces invalid output often", which is a much more useful sentence.

Failure three: I could not separate model drift from prompt drift

By hour thirty I had edited the prompt four times and lost the ability to attribute anything. Is the output better, or did I just move the goalposts? The matrix runner exists so that question stays answerable:

#!/usr/bin/env bash # run_matrix.sh - 3 fixtures x 5 reps, one label per cell set -euo pipefail for fixture in fixtures/*.diff; do for rep in 1 2 3 4 5; do python review_once.py \ --fixture "$fixture" \ --rep "$rep" \ --label "$(basename "$fixture" .diff)-r$rep" \ --timeout 90 done done 

And the summary I actually read, which is two queries, not a dashboard:

# acceptance rate over the whole sample jq -s 'map(select(.accepted)) | length' ledger.jsonl # per-fixture view: attempts, accepted, and input tokens spent jq -s 'group_by(.label[0:9]) | map({ fixture: .[0].label[0:9], attempts: length, accepted: (map(select(.accepted)) | length), input_tokens: (map(.input_tokens // 0) | add) })' ledger.jsonl 

If a prompt change moves acceptance and input tokens together, I keep it. If it only moves my mood, I revert it. That single rule saved me from two prompt rewrites that felt like progress.

The decision table I would hand to someone starting today

What I would repeat, and what I would skip

Repeat: the ledger, the pre-charge budget guard, the three-fixture matrix, and the acceptance flag. All four are reusable with any provider and any model, which is the point of writing them down.

Skip: my first two hours of prompt polishing. Without a ledger it produced opinions, not evidence. Also skip per-request dashboards — jq over JSONL answered every question I had, and the dashboard would have been a day of work for one chart.

Limitations, and who should not do this

This is a single 48-hour sample on a tiny workload, and it says nothing about throughput, stability under load, or anything at scale. Free-tier terms, model lists, and allowances change, so treat my setup as a method and the current docs as the facts. The free server is fine for a batch process that can tolerate a cold start; it is a bad fit for anything user-facing, anything with an on-call rotation, and anything where a stalled request costs more than the compute you saved. If your team already has a paid endpoint with usage reporting, port the ledger to it and skip the infrastructure experiment entirely — the measurement is the valuable part, not the free part.

My one open question after two days: how much of a review's value survives when the reviewer is cheap? I still do not know, but I now have a JSONL file that will tell me eventually.

If you want to run this matrix without spending anything first, MonkeyCode's free model access and free server option are a reasonable place to start — build the ledger before you build the prompt, and you will learn more in an afternoon than in a week of trying prompts by feel.