The Coding Agent Was in the Wrong Directory
A 25-minute local coding failure turned into a verified patch after fixing OpenCode's working directory. Here is what our benchmark actually measured.
Approximately 8 min read
I spent an afternoon trying to make a local 27-billion-parameter model repair some JavaScript. One coding agent read files for 25 minutes and never produced a patch.
I was ready to blame the model.
That seemed reasonable at the time. The machine had a 24 GB RTX 4090. The model had 80K tokens of context. I had increased the output allowance, stopped splitting inference across two GPUs, and given the agent much longer to work. OpenCode still exited with the code untouched.
Then I tried a task so small that there was almost nothing to reason about: a two-file repository with an addition function that subtracted instead.
OpenCode failed that task too.
This time I looked at where its shell was actually running. It was not in the repository. It was in the launcher’s directory.
We had been blaming a local model for a problem partly caused by the program launching the agent.
One stale environment variable
The benchmark launched agents from Python. It created an isolated Git checkout, then passed the checkout as the child process’s working directory:
env = os.environ.copy()
subprocess.Popen(
command,
cwd=target_repo,
env=env,
stdin=subprocess.DEVNULL,
)
That looks normal. The catch is that setting a child process’s working directory does not automatically rewrite the inherited shell variable `PWD`.
In our environment, `PWD` still referred to the launcher’s own directory.
OpenCode 1.19.5 used that value when choosing the workspace for a headless `opencode run` session. As a result, a shell tool reported the wrong directory. Relative file searches missed the project. An attempt to read the actual repository through an absolute path was rejected by the tool permission system.
The agent had not run out of coding ability. It had been given the wrong workspace and permission boundary.
Worse, this particular failure could end with exit status `0`. An automated supervisor looking only at the process exit code could count it as a clean completion.
This is a real upstream issue, not a novel claim about the internals of OpenCode. The same `PWD`-over-process-cwd behavior was reported in OpenCode #47080 for v1 and again in #56386 for v2. Our independent reproduction used v1.19.5. We did not rerun the latest v2 release ourselves.
The small test that changed our conclusion
I made a clean repository containing only `math.mjs` and `math.test.mjs`. The function returned `a-b` when it should have returned `a+b`. A single Node test checked `add(4, 3) === 7`.
No large context window was needed. No lengthy planning. The agent only had to read the file, make a one-character change, and run the test.
I kept the task, local model endpoint, and test the same, and changed the launcher settings.
| Launcher configuration | Result |
|---|---|
| Child `cwd` correct; inherited `PWD` stale; no `–dir` | No edit, test failed |
| Child `cwd` correct; `PWD` corrected | Test passed, 12.9 s |
| Child `cwd` correct; `–dir` specified | Test passed, 12.8 s |
| Child `cwd`, `PWD`, and `–dir` all aligned | Test passed, 16.4 s |
The successful runs executed file-reading, editing, and shell-testing tools. No tool errors appeared in those runs.
A minimal reproduction matters here because it removes most of the excuses. We are no longer arguing about whether a 27B model can understand a complicated game state machine. We are looking at an agent that could not find a trivial file until its launch directory was corrected.
For automation on OpenCode v1, I now treat the process cwd, `PWD`, and the agent’s effective workspace as separate values that must agree. Explicit `–dir` worked in the version we tested. The v2 CLI has changed, so the portable workaround is to set `PWD` correctly in the child environment.
Back to the 25-minute failure
The original job was not trivial. It involved two linked bugs in a multiplayer encounter.
Defeated enemies could remain in an active list and interfere with an escort objective. At the same time, players far above or below the encounter could qualify as participants or receive rewards. The repair had to preserve kill idempotency, attack suppression, multiplayer behavior, and persistent state.
The regression suite contained 56 tests. On the untouched baseline, 49 passed and seven failed. A separate evaluator-only check saved state as JSON, reloaded it, and verified that duplicate kill reports were still rejected.
For the rerun, I kept the same local inference setup: Qwen3.8-27B Q4_K_M served by llama.cpp, a single RTX 4090, 81,920 context tokens, a 24,576-token completion limit, and a 1,500-second wall-clock allowance.
The important change was the OpenCode launcher. Both `PWD` and `–dir` now pointed at the intended checkout.
| OpenCode run | Time | Regression suite | Persistence check | Files modified |
|---|---|---|---|---|
| Earlier run, stale workspace environment | 1,500.1 s | 49/56 | Fail | None |
| Rerun, workspace explicitly corrected | 877.3 s | 56/56 | Pass | Two target files |
The corrected run made 35 successful tool calls, including six edits, and reported no tool errors. It exited normally after 14 minutes and 37 seconds. The resulting patch passed an independent rerun of the regression suite, the JSON persistence holdout, and `git diff –check`.
The patch remained in an isolated checkout. Nothing was deployed.
I would not claim that the directory fix alone caused the improvement in this complex task. These were sequential runs of a generative model, not repeated trials with identical tool trajectories and controlled sampling. The small test isolates the directory problem much better.
Still, the practical conclusion changed. We had previously treated OpenCode’s 25-minute no-edit run as evidence against the local model’s ability. That result was compromised by a launcher configuration error. With the workspace corrected, the same harness/model combination completed the repair.
What llama.cpp did and did not break
The working directory was not our only configuration mistake.
An early OpenCode run had a completion cap of 4,096 tokens. Its trace ended with `finish_reason=length` at exactly that limit, before it produced a patch. Giving the process more wall-clock time would not fix an individual response being cut off.
Crush had a similarly restrictive initial output setting. Raising its completion allowance to 24,576 tokens preceded a successful long run: 56/56 tests and the persistence check passed in 1,016.7 seconds.
That does not prove larger output caps always improve results. It does show why a benchmark should record why an agent stopped. A response ending at a token ceiling is not the same as a model deciding its work is finished.
I also suspected that llama.cpp’s OpenAI-compatible interface was failing to pass tool calls correctly. We tested that directly against the running server rather than inferring it from agent behavior.
In a basic tool-calling probe, llama.cpp returned a structured function call, valid arguments, and `finish_reason=tool_calls`. It accepted a tool result and produced a follow-up answer. A separate streaming probe preserved the function-call fragments and ending signal.
Those probes do not prove compatibility with every coding harness or every reasoning format. They do rule out the simplest theory that the local endpoint could not carry OpenAI-style tool calls at all.
Goose offered another clue. In a small repair task it completed successfully in 13.2 seconds, but first attempted to use an image-reading tool on source code and received a format error. The tool protocol was working; the tool choice was bad. Those are different failure categories and deserve different fixes.
The other false positive: all green, still broken
Not every problem was caused by a launcher.
Aider completed the original visible regression suite in 387.9 seconds, scoring 56/56. That sounded excellent until the JSON persistence check ran.
Its duplicate-kill tracking used a JavaScript `Set`. That works while the process stays alive. It does not survive a normal `JSON.stringify` / `JSON.parse` round trip as a Set, so the restored state could no longer use the ledger as intended.
The visible tests passed. The stored-state contract failed.
This is why I still want the upstream task to be explicit about acceptance conditions while leaving the downstream harness free to choose the implementation. A model’s declaration of success is not the deliverable. The patch and the tests are.
The benchmark gate I should have built first
Before running another 25-minute coding benchmark, I would make the harness prove four boring things:
- Its shell sees the intended working directory.
- Its file tools can read and edit a scratch file inside that directory, without crossing the workspace boundary.
- Its model endpoint correctly completes a tool-call and tool-result round trip.
- It can run a small test, observe failure, make a patch, and then observe success.
Only after that would I give it a large codebase and count a timeout as evidence about task-solving ability.
The upstream system can submit work; the downstream harness can decide how to execute it. But a failed harness invocation is not a failed model inference experiment. If the harness is looking at the wrong repository, the model’s size, quantization, and context window are almost beside the point.
There is still an economic question. Fourteen minutes for one successful repair is slow for interactive development, and one success is not a reliability estimate. A cloud coding model may still be the better everyday choice. We have not demonstrated a cost advantage for this local setup.
What we have demonstrated is narrower, and more important for anyone testing local agents: the original OpenCode failure was not a valid reason to dismiss the model. Fixing the launch environment turned the same workload into a passing patch.
That is a better conclusion than the one I published after the first round.
Technical note: The sanitized working-directory evidence was added to OpenCode issue #56386. Reproduction details intentionally omit machine identifiers and private project paths. Related RAMGPT research: We Tried to Break a 27B Q4 Model. The Benchmark Broke First..