DevOps
FAQ: Five Myths That Confuse the Model With the Box
Jordan Huang Dev.to (EN Zone)
2 views
Did the model run that, or did a shell?
I keep getting that same question in code reviews.
A reply can look like a real terminal session.
That still does not make it a shell.
A free model only predicts the next tokens.
A free server actually runs those processes for you.
Mix those two, and your "fix" never lands.
Disclosure: This article was prepared as part of MonkeyCode's product outreach.
I use MonkeyCode for free models and a free server.
Those are two knobs, not one magical laptop.
I still refuse to trust the chat transcript.
Why this mix-up keeps winning
The UI shows one thread, so brains merge hosts.
Your brain wants a single shared computer underneath.
The model says cd /app with total confidence.
Do you actually print pwd after that claim?
Until a process prints a fact, you have a story.
Chat stories do not create directories on disk.
How to read this FAQ
Each myth has a claim, a check, and a correction.
Run the checks yourself on a real shell.
Do not take my word for any result.
I label every snippet as a runnable checklist.
These snippets are not benchmarks or product scores.
I will not invent quotas, GPUs, or uptime.
Myth 1: Free model access means you got a box
Claim: The model is free, so a machine is included.
Evidence: A chat window that types shell commands fluently.
Corrected model: Inference and compute remain two different systems today.
A model can emit gcc without a compiler present.
A server can compile without a smart model attached.
Did you confirm both sides of that pair?
Ask a blunt question before you debug the build.
Where did the process run, if it ran at all?
Checklist: prove a host exists
# checklist — run on the server, not in chat fiction
date -u
hostname
id -un
pwd
uname -s
If those five lines never appear, no box spoke.
A free model can still draft the next script.
A free server is what executes the script.
Please do not collapse those two roles together.
You can draft offline and still fail on PATH.
Myth 2: export in the chat mutates the server
Claim: I set DATABASE_URL, so the next command has it.
Evidence: A fenced code block that contains export.
Corrected model: The environment only lives inside a process tree.
An export inside markdown changes zero running processes.
An export in a dead shell dies with that shell.
Did the same PID keep your variables alive?
Probably not, unless you stayed in one shell.
Checklist: prove the variable
# checklist — same shell session as the real work
export SMOKE_MARK=faq-2026-09-09
printenv SMOKE_MARK
python3 - <<'PY'
import os
print("py:", os.environ.get("SMOKE_MARK", "MISSING"))
PY
If Python prints MISSING, the chat story leaked.
Or you started a new process without the var.
Both of those failures happen more than people admit.
Myth 3: The model's cwd is the server cwd
Claim: The model says we are in the repo root.
Evidence: The model named a project folder with confidence.
Corrected model: The current directory is a kernel fact.
os.getcwd() does not care about the story.
Relative paths fail hard when cwd is wrong.
That is chdir, not a model-quality mystery.
Checklist: pin the directory
# checklist — fail fast if the tree is wrong
set -euo pipefail
pwd
ls -la
test -f pyproject.toml || test -f package.json || test -f go.mod
Still unsure about two views of the same tree?
# checklist — unexecuted until you actually run it
from pathlib import Path
import os
print("cwd:", os.getcwd())
print("script:", Path(__file__).resolve() if "__file__" in globals() else "repl")
If those paths disagree, stop editing files now.
Fix the directory first, then touch code.
Myth 4: Install prose means a binary on disk
Claim: I installed pytest, so we can test now.
Evidence: A sentence that says the install finished.
Corrected model: The command -v check is the only judge.
The model can recite pip output from training.
The box may lack the interpreter entirely today.
The box may lack network for a real install.
The box may have a different PATH order.
Checklist: prove the tool exists
# checklist — no poetry, just presence facts
command -v python3 || echo "python3: missing"
command -v pip || echo "pip: missing"
command -v pytest || echo "pytest: missing"
python3 -m pytest --version || echo "pytest module: missing"
Need a decision, not another vibe check?
Chat claim
Box check
If they disagree
python is ready
command -v python3
Install it, or fix PATH
deps installed
python3 -m pip show PKG
Reinstall inside this env
tests passed
the pytest exit code
Rerun; ignore the prose
we are in /app
pwd
cd correctly, or abort
FOO is set
printenv FOO
Export in this same shell
a host exists
hostname and id
You only had inference
If the table cell fails, the chat does not count.
Myth 5: Green on a free server equals CI green
Claim: Zero on the box means zero in production.
Evidence: One local command returned an exit code of zero.
Corrected model: One host is a sample, not a pipeline.
CI pins images, caches, runners, and secrets too.
A free coding server is a scratch space.
It is useful, but it is not your merge gate.
Do not ship releases from a mystery PATH.
Do not paste production tokens into that box.
Would you merge from a laptop you never inspected?
Then do not merge from a chat summary either.
Checklist: what this box is not
# checklist — inventory, not a green badge
echo "user=$(id -un)"
echo "home=$HOME"
echo "pwd=$(pwd)"
git rev-parse --is-inside-work-tree 2>/dev/null || echo "not a git work tree"
git status -sb 2>/dev/null || true
Treat this output as reconnaissance only, nothing more.
Then run the real job in your real CI.
What a tool trace is not
A tool trace is not a hostname line.
A tool trace is not a pwd result.
If the trace never calls a shell, stop now.
Ask for a host fact before the next patch.
Save this as test_host_facts.sh
I want one file I can run after every "done".
Copy this script, then run it on the box.
#!/usr/bin/env bash
# test_host_facts.sh — checklist, not a score
set -euo pipefail
fail() { echo "FAIL: $*"; exit 1; }
[[ -n "$(hostname)" ]] || fail "no hostname"
[[ -n "$(pwd)" ]] || fail "empty pwd"
command -v python3 >/dev/null || fail "python3 missing"
python3 - <<'PY'
import os
import sys
import socket
print("host=", socket.gethostname())
print("cwd=", os.getcwd())
print("exe=", sys.executable)
print("path_ok=", bool(os.environ.get("PATH")))
PY
echo "PASS: host facts printed"
Run it like this after any claimed success.
chmod +x test_host_facts.sh
./test_host_facts.sh
echo "exit:$?"
If the script dies, the chat did not finish the job.
If the script prints, you finally have a host.
A tiny workflow I actually follow
I draft commands with a free model first.
I execute those commands on a free server.
I keep only lines that printed host facts.
Here is the loop I refuse to skip.
Ask the model for a command, not a novel.
Run that exact command on the box.
Capture pwd, command -v, and the exit code.
Only then ask for the next edit.
# checklist — wrap any claimed success
run() {
echo "++ $*"
"$@"
echo "exit:$?"
}
run pwd
run command -v python3
run python3 -c "import sys; print(sys.executable)"
run ./test_host_facts.sh
That wrapper is the whole method, honestly speaking.
It is boring on purpose and debuggable on purpose.
Limitations
This FAQ does not measure raw model quality.
This FAQ does not prove a product SLA.
Free model access can still hallucinate file paths.
A free server can still be empty or reset.
I am not giving quotas, GPUs, or uptime numbers.
Those numbers would be fiction in this post.
Secrets do not belong in this scratch loop.
Regulated data does not belong in this loop.
Long production builds may not belong here either.
If you need pinned runners, use real CI.
If you need a laptop, use your laptop.
Who should skip this
Skip it if you already have a locked pipeline.
Skip it if you cannot run a shell at all.
Skip it if the work is production-only traffic.
Skip it if hosted models are off-limits for you.
This FAQ is for people who mix chat with PATH.
It is not a deployment guide for anything.
What I want you to keep
The model talks, and the box runs processes.
A pwd line beats a confident paragraph every time.
A command -v check beats the word installed.
Exit codes beat any "looks good" summary.
If you try this loop, paste pwd and command -v.
Keep the novel out of the review thread.
Read original: https://dev.to/gitlab_3188/faq-five-myths-that-confuse-the-model-with-the-box-3b0m
← Previous
From "twenty minutes per caption" to a real launch: the CAPTD story so far
Next →
I Hid a Rule in CLAUDE.md. Only One Reviewer Could Prove It Read It.
Related
GitHub Workflows Strain Under AI Agent Development
DevOps
0
DevOps.com
Flaw in DeepSeek Harness AI Coding Tool Let Agents Disable Their Sandbox
DevOps
0
DevOps.com
GitHub Puts Guardrails on Copilot’s Sandbox Inside JetBrains IDEs
DevOps
0
DevOps.com
Rust’s First Debugging Survey Shows Most Developers Skip the Debugger Entirely
DevOps
2
DevOps.com
Comments0
No comments yet — be the first