I spent the better part of an afternoon convinced I had a bad model.
Goose wasn't applying edits. No error. No timeout. No complaint. It would read the file, think
about it, produce what looked like a plan — and then nothing. The file sat unchanged. I swapped
prompts. I tried a different task. I restarted Ollama. Same result. The model —
qwen2.5-coder:32b, 32 billion parameters of carefully coder-tuned weights — was
apparently just vibing.
It wasn't the model.
The thing that's actually happening
When an agentic coding tool calls a function — "edit this file," "read this path," "apply this
patch" — it sends a request to the model that includes a structured tools block. The
model is supposed to respond with a structured tool_calls array: here is the tool I
want to invoke, here are the arguments.
On raw Ollama endpoints, most qwen-family models don't do that.
They put the tool call in content — the regular text field — and return an empty
tool_calls array. The format varies by model. qwen2.5-coder emits JSON:
content: '{"name": "edit_file", "arguments": {"path": "app.py", "content": "..."}}'
tool_calls: []
qwen3-coder does something different — XML:
content: '<function=edit_file><parameter=path>app.py</parameter>...</function>'
tool_calls: []
The agentic tool checks tool_calls. Finds nothing. Reports no edits. Exits
cleanly.
That's not a failure. That's a success with an empty result. And there is no log line that tells
you why.
How we found it
We built a systematic bake-off at QCoda — part of our labs infrastructure for testing and
graduating capabilities before we ship them. The goal was to understand which agentic coding engines
work with which local models, so we could route automatically rather than guess. We ran aider,
goose, pi, claude-code, and opencode against seven local models, ten trials each, same task, same
context.
The task: add four specific routes to an existing Flask application. Specific, measurable,
unambiguous. Either the routes were in the file or they weren't.
Some of the results looked like hardware problems. Goose went 0/10 on qwen2.5-coder:32b.
Claude-code did the same. These aren't obscure tools — they're well-maintained, widely used. And
qwen2.5-coder:32b is not a bad model. It was the fastest model in the entire matrix when
paired with a compatible engine — finishing in 21 seconds median, nothing else close.
Zero edits applied on one engine. Twenty-one seconds to completion on another. Same model. Same
hardware. Same task.
The only difference was whether the engine depended on structured tool_calls or
not.
Aider doesn't use the tool-call protocol. It asks the model to emit edits in a SEARCH/REPLACE
format, reads the response as text, and parses it directly. No tool_calls dependency.
Zero compatibility surface for this failure mode. That's why aider's results were clean across every
model — not because aider is better engineered in some abstract sense, but because it sidesteps the
broken layer entirely.
The vLLM variant is worse
We also probed qwen2.5-coder-7b running behind a vLLM server. Raw Ollama is at least
consistently broken — it always emits text. vLLM without the right startup flag is
non-deterministic.
Three consecutive identical requests, temperature zero:
- Response 1:
<response>-fenced JSON
- Response 2:
<function-call> XML
- Response 3: structured
tool_calls ✓
Same model. Same prompt. Same temperature. Three different formats.
At temp=0, you expect determinism. You don't get it. The fix is a vLLM startup flag:
--enable-auto-tool-choice --tool-call-parser hermes. It has to be set when the server
starts. If you didn't set it, your model is serving non-deterministic tool-call output and every
agent running against it is seeing a different interface on each request. Nothing in the agent's
logs will tell you this is happening.
The fix is at the serving layer
Here's the thing about per-client shims: there are five of them by the time you've installed a
few coding agents. Each one has its own way of handling — or not handling — this failure. Some fail
silently. Some error cryptically. Some, like aider, just happen to use a different protocol that
avoids the problem.
The right fix is one layer down: normalize tool-call output at the serving layer, before any
client sees it. Parse the text-form call out of content, build the structured
tool_calls array, return that. Every client downstream gets working tool calls without
knowing a normalization step happened.
That's what we built into LLMesh — QCoda's inference node layer. When you point
your local coding agent at a LLMesh endpoint instead of raw Ollama, the normalization happens
transparently. qwen2.5-coder emits JSON-in-content; LLMesh parses it, returns structured
tool_calls. qwen3-coder emits XML; same treatment. The agent never knows.
The practical consequence: qwen2.5-coder:32b behind LLMesh becomes a viable target for goose,
pi, and claude-code — tools that were 0/10 against the raw endpoint. Same model, same hardware.
Different endpoint URL.
Worth noting: LiteLLM proxy does similar normalization, and if you're already running it, you're
getting a lot of this for free. The point isn't that we invented the solution — it's that the
fix needs to live somewhere in the serving layer, and most teams running raw Ollama or vLLM
don't have anything there. The default is broken. Silently.
Go check what your agents are pointed at
If it's a raw Ollama endpoint — and most local setups start there — run one test. Pick a
qwen-family model. Pick a tool-dependent agent. Give it a one-line edit task. Watch whether the
file changes. Then look at the agent's output and see if it mentions tool calls at all.
If the file didn't change and there was no error, you found it.
The fix isn't complicated. But you can't fix what you don't know is broken. And this failure is
specifically designed, by accident, to look like a model problem when it's a protocol problem. Teams
swap models. They conclude local OSS isn't ready. They move back to cloud.
That's not an accuracy problem. That's a serving-layer misconfiguration masquerading as a
capability gap.
This is part of an ongoing series from QCoda Labs — where we run structured experiments to
understand what actually works in local AI-assisted software development, before building on top of
it. Episode 2: which agentic coding engines work for which kind of work, and why the results
surprised us.
