General-purpose LLMs are bad at browser automation. I spent the last few months working with various computer-use agents, and the performance gap between specialized models and general ones is way bigger than most people expect.
Heres a concrete example. On WebRetriever Protocol I, a benchmark that tests real browser navigation and form-filling, Mano-CUA 1.1 scored 41.7 NavEval. Gemini 2.5 Pro Computer Use got 40.9. Claude 4.5 Computer Use landed at 31.3. The full project is at github.com/Mininglamp-AI/Mano-P if you want to look at the model and benchmark setup.
That 10-point gap between Mano-CUA and Claude isnt noise. It shows up consistently across tasks that require multi-step form interaction, dropdown selection, and navigating through paginated results.
Why general models struggle with browser tasks
GPT-4o and Claude are incredibly good at reasoning over text. But browser automation is a different kind of problem. You need to understand spatial layout, click targets that change after each interaction, and UI elements that look different across every website.
Most general models handle this by parsing the DOM or relying on accessibility APIs. That works for simple pages. It breaks down fast on complex web apps with dynamic rendering, custom components, or heavy JavaScript.
Mano-P takes a different approach. Its pure vision-driven. The model looks at screenshots the same way a human user would, identifies clickable elements from pixels, and plans the next action based on visual context alone. No DOM parsing, no API hooks, no accessibility tree dependency.
This matters because real-world web pages are messy. A DOM parser sees a div with an onclick handler. A vision model sees a blue button that says Submit. When the page structure changes but the visual layout stays the same, the vision approach keeps working while the DOM parser breaks.
The form-filling problem
Form-filling is where most general models completely fall apart. A typical form has text inputs, dropdowns, radio buttons, date pickers, and sometimes nested sections that appear conditionally. Each interaction changes the page state.
General models tend to lose track of where they are in the form after 4 or 5 steps. They click the wrong field, skip required inputs, or get stuck in loops where they keep trying the same failed action.
On WebRetriever Protocol I, the tasks include real web forms from actual websites. Not simplified test pages. Real sites with captchas, multi-page flows, and validation errors that redirect you back to fix something.
Mano-CUA handles these because the think-act-verify loop in its architecture forces it to check whether each action actually worked before moving to the next step. If it clicks a dropdown and nothing opens, it tries again. If a form field rejects the input, it reads the error message from the screenshot and adjusts.
Running it locally
One thing that sets Mano-P apart from cloud-based solutions is that it runs entirely on your machine. The 4B quantized model runs on Apple M5 Pro hardware at about 80 tokens per second decode speed. All your data stays local. No screenshots get sent to any server.
The setup is straightforward if you have a Mac with Apple silicon and 32GB RAM:
brew tap Mininglamp-AI/tap && brew install mano-cua
mano-cua check
mano-cua install-sdk
mano-cua install-model
mano-cua run "fill out the application form on example.com" --local
Enter fullscreen mode Exit fullscreen mode
The --local flag ensures everything stays on-device. Without it, inference goes through their cloud endpoint, which is faster but sends screenshots to their server.
The specialization argument
Theres a broader trend here. General models keep getting better at everything, but specialized models are pulling ahead on specific verticals. Browser automation is one of them. Code generation is another. Medical imaging is a third.
The reason is training data composition. A general model trains on everything and learns a bit about GUIs from the fraction of its data that includes UI screenshots. A specialized model like Mano-CUA trains specifically on GUI interaction sequences, with reinforcement learning that optimizes for task completion, not just next-token prediction.
Mano-P uses a three-stage training pipeline. First supervised fine-tuning on GUI interaction data. Then offline reinforcement learning with logged trajectories. Then online RL where the model actually interacts with live interfaces and learns from success and failure.
That last stage is the key differentiator. The model doesnt just learn what a correct click looks like from labeled data. It learns what happens when it clicks wrong and how to recover.
Benchmark context
Some context on the numbers. On OSWorld, which tests desktop-level tasks across multiple operating systems, Mano-CUAs 72B evaluation model hit 58.2% success rate. Thats the top score among specialized models, 13.2 points above the second place. Important note: the 72B model is an evaluation model, not what you actually run locally. The local 4B thinking model scored 56% on a 100-task macOS GUI test, still well above Qwen3-VL-Plus at 39% on the same test set.
WebRetriever Protocol I is more focused. It tests specifically web navigation and retrieval tasks. The 41.7 score there puts Mano-CUA ahead of both Gemini 2.5 Pro and Claude 4.5 on their computer use modes.
What this means for developers
If youre building browser automation workflows, the practical takeaway is simple. General models give you flexibility but mediocre results on complex web tasks. Specialized models give you significantly better completion rates on the specific things they were trained for.
For anything involving form submission, data extraction from web UIs, or multi-step navigation through complex sites, a vision-based specialized agent will save you a lot of failed runs compared to throwing GPT or Claude at the same task.
The project is Apache 2.0 licensed. The code, the CLI tool, and the 4B model weights are all open source. Check the repo at github.com/Mininglamp-AI/Mano-P for benchmarks, model downloads, and the full technical report.