Making Local AI Tool Calls More Reliable
fix: improve tool call reliability and add automated tests for repairs
#20
Pull Request
Summary
I fixed an intermittent error where the assistant sometimes returned a text response instead of calling the required local-data tool. I added a safe one-time retry with a required tool choice, protected write operations from duplicate execution, and verified the solution with 67 automated tests and a live local Gemma model.
Verification
List the tests or checks you ran.
Checklist
- [x] I updated my section in the matching weekly report.
- [x] I reviewed the final changes and ran
scripts/verify-contribution.ps1. - [x] I confirmed that this branch uses the correct base branch and that the pull request targets the intended branch.
While testing our local-first AI assistant, I found an intermittent problem: the model sometimes answered with plain text instead of calling the tool needed to read or update local data.
The original system used tool_choice="auto". This normally worked, but it allowed the model to skip a required tool call. A prompt can guide a model, but it cannot guarantee that the model will always follow the tool protocol.
I fixed this by adding a safe recovery step. The first request still uses automatic tool selection, so normal conversation works as before. If an explicit local-data request returns no tool call, the assistant retries once with tool_choice="required". The retry happens only before any tool has run, which prevents duplicate database writes.
The assistant also checks whether the recovered tool belongs to the correct read or write group. During streaming, it buffers the first response so an incorrect, ungrounded answer is not shown before recovery completes.
I tested the change with automated regression tests and the running local Gemma model. The live recovery flow was:
auto -> required -> auto
Enter fullscreen mode Exit fullscreen mode
The main lesson was simple: prompts describe expected behavior, but reliable agent systems also need program-level checks around model decisions.