Most teams customize their coding agent heavily: a tuned system prompt, a chosen set of tools, skills for recurring workflows, a turn budget that fits their repo. But the model underneath usually gets no such treatment. It was post-trained on someone else's generic agent setup, then dropped into yours — and the mismatch shows up as wrong tool calls, ignored skills, wasted steps, or agents that burn through their turn budget before finishing.

A November 3 post from OpenHands and Snowflake AI Research proposes a different split of labor: instead of rebuilding your harness inside a reinforcement-learning framework, post-train the model inside the harness you already run.

Why training usually means rebuilding

Reinforcement learning for agents works by letting the model attempt tasks, scoring the outcome, and updating the weights toward what worked. The catch is that most RL frameworks bring their own agent loop, tool formats, and task structure. To train, you re-implement your production harness inside the training framework — extra engineering, plus a quiet risk that the thing you trained is not quite the thing you deploy.

The OpenHands approach removes that second implementation. With the OpenHands Agent SDK, your agent's configuration — system prompt, tools, skills, turn budget, context length — is passed as configuration, not recreated. Every training rollout is a real execution of your agent.

How a rollout becomes training data

The mechanism is deliberately boring, which is the point. The agent does not know it is being trained: it sends ordinary chat-completion requests, and a proxy between the agent and the model records the exact tokens the model produces on each turn. Those tokens become the training data. Prompts and tool results are excluded from the loss, so the model learns from its own outputs, not from memorizing your instructions.

Tasks use the open Harbor format, which many teams already use for agent evaluation. A Harbor task is a directory with an instruction.md, an environment/ sandbox definition, and a tests/test.sh script — and that test script's score becomes the reward signal. Each rollout runs in its own Snowflake Cortex Sandbox, with the agent's terminal and the grader inside the same sandbox, so the reward reflects what the agent actually did. Training compute comes from Snowflake Cortex Training.

Configuration maps directly onto familiar knobs: the system prompt via OPENHANDS_SYSTEM_PROMPT_FILE, tools via OPENHANDS_TOOLS, skills via HARBOR_SKILL paths, turn budget and context length via MAX_TURNS and MAX_CONTEXT_LENGTH, and the base model via MODEL — the example in the post uses Qwen/Qwen3.5-4B.

The part most developers will actually use

Running RL still involves operational work: allocating GPUs, setting batch sizes, launching jobs, watching for failed rollouts, releasing compute. OpenHands ships a cortex-training-rl skill (in OpenHands/extensions) that lets you describe the run in plain language and have the agent handle setup and execution:

Use /cortex-training-rl to post-train Qwen/Qwen3.5-4B on my Harbor tasks in ~/my-harbor-tasks/train and ~/my-harbor-tasks/eval with my agent's skill at ~/my-skill/SKILL.md, a 15-turn budget, for 50 steps.

The agent configures the launcher, checks GPU availability on Cortex Training, starts the run in the background, reports reward metrics and failed rollouts at every step, plots the reward curve, and releases GPU resources when done. The skill also works from Claude Code and Cursor. Your local machine only runs the agent rollouts on CPU; model training and sampling run on Snowflake GPUs, four H200s per run by default.

Does it actually learn? One experiment says yes

To test the idea end to end, the teams built a custom code-localization harness: given a GitHub issue from a real Python repository in the SWE-smith dataset, the agent must identify the files, modules, and functions that need to change. The harness used a localization-specific system prompt, a terminal tool, a custom localization_finish tool for submitting answers, and a 10-turn budget. The reward was the sum of F1 scores at the file, module, and function levels, for a maximum of 3.0.

They trained Qwen3.5-4B in this harness for 25 steps at 64 rollouts per step. Mean reward climbed across training, and — the result that matters — the outcome was comparable to a purpose-built RL implementation. The full run, from Harbor tasks to a trained model, finished in a couple of hours.

That comparison is the honest benchmark for this approach. If training inside your existing harness only matched a bespoke setup, that is already a win, because the bespoke setup is the thing most teams will never build.

The caveats, stated plainly

This is not a drop-in for every agent. A few details from the post are worth knowing before you try it:

  • Sandboxed by default, but constrained. Each rollout gets its own Cortex Sandbox, and training-machine credentials are not exposed to it. That isolation is good; it also means the agent's terminal runs each command in a fresh shell. The working directory persists, but exported environment variables and background processes do not.
  • No file-editor tool during training. Agents must edit files through the terminal, since the OpenHands file_editor tool is not currently available in training runs.
  • GPU defaults are opinionated. The launcher targets Hopper GPUs by default with FlashAttention 3; other configurations are available, but you are starting from Snowflake's managed stack, not your own cluster.
  • You still need the tasks. Harbor tasks are the reward signal. If you do not have a set of tasks with real grading scripts, there is nothing to train against — prompt tuning remains the cheaper first step.

When this is the right move

Post-training inside your harness makes sense when three things are true: you have a stable harness you plan to keep, you have graded tasks that represent real work, and your failures are behavioral — the model reaching for the wrong tool or ignoring a skill — rather than a lack of raw capability. In that situation, the model is not failing because it cannot code; it is failing because it has never practiced inside your setup. Training it there is the direct fix.

If those conditions do not hold yet, the cheaper sequence is still: fix the prompt, tighten the tools, build the task set, then train. What OpenHands and Snowflake changed is what happens after that sequence — post-training stops being a separate research project with its own framework and GPU cluster, and becomes one more step in the harness development loop.

Sources: OpenHands, "Post-Train a Model for Your Own Agent Harness with OpenHands and Snowflake" (November 3, 2026); Snowflake, Cortex Training announcement at Expedition 2026.