Claude in Chrome keeps missing clicks. I measured why — there are exactly two causes.

작성자

카테고리:

← 피드로
DEV Community · Mxhlix · 2026-07-11 개발(SW)

Mxhlix

If you use Claude in Chrome (Chrome MCP) for browser automation, you have probably hit some of these:

  • Clicks land next to the target, not on it
  • The pointer drifts right of the element, and retrying never fixes it
  • You take a screenshot, and moments later the page dimensions have changed
  • Double-clicks don’t enter edit mode — you get a row selection instead

I run 600+ web tools built and QA’d through AI-driven browser automation, so I hit these constantly. I burned hours suspecting the page elements. They were fine.

The real problem: the assumptions behind coordinate-based clicking were breaking. It comes down to exactly two causes.

Cause 1: the window width settles late, so coordinates drift right

Right after a page loads, innerWidth can report a smaller value than the real window. On my machine it started at 1664. Even after document.readyState hits complete, it keeps widening for 2–3 seconds until it reaches the true width (1920). During that window, outerWidth / outerHeight sometimes return 0.

Now take a screenshot during that settling period and click using its coordinates. The page has widened since the capture, so your click lands to the right of the target. 1664→1920 is a 1.15× stretch — the further right the element, the bigger the miss.

That’s the whole mystery: “the page changed size after my screenshot” and “the pointer keeps drifting” are the same bug.

This is not limited to exotic canvas apps. Ordinary HTML pages with lazy loading or dynamic layout do it too. Some of my “this input just won’t click” sessions were nothing more than coordinates swinging at empty air.

How to diagnose it

Read the viewport with javascript_tool and check whether it has stabilized:

({ url: location.href, readyState: document.readyState,
   dpr: window.devicePixelRatio,
   innerW: window.innerWidth, innerH: window.innerHeight,
   outerW: window.outerWidth, outerH: window.outerHeight })

Enter fullscreen mode Exit fullscreen mode

Four rules. If outerW is 0, the window size hasn’t settled — do not interact yet. Read again after a moment; if innerW changed, it’s still settling — keep waiting. Two consecutive identical innerW readings with a non-zero outerW and the size is clear. The fourth rule guards against a different failure entirely: dpr has to match whatever it was when you took your screenshot. Drag the window onto a monitor with different DPI scaling and every coordinate rescales — same math as the 1664→1920 stretch — except nothing ever looks unsettled, because the new values are correct and stable from the very first read. An innerW-only check waves it straight through. (Pointed out by @skillselion in the comments — see the update note below.)

Cause 2: double_click fires too slowly and becomes two single clicks

The computer tool’s double_click action can leave a gap between the two clicks that exceeds the app’s double-click threshold. The app interprets it as two single clicks — so instead of entering text-edit mode, you get a row selection or nothing.

The nasty part: the tool believes it sent a double-click. The logs look correct.

You cannot see this failure in any log.

Fixes

Plan A (preferred): stop using coordinates, use element references

On a normal HTML page this is the reliable path. Drop coordinate clicks entirely. Use find (natural-language element search) or read_page (accessibility tree) to get element references, then click and fill through those references (form_input for values). Element references survive layout shifts.

They’re not a free pass, though. A ref is a pointer to one specific node and nothing more — it knows nothing about where that node sits on the page or what it currently means to the user. Replace the node in a rerender and the ref dies with an explicit error. Move the node, or let it take on a different role, and the ref keeps right on working: it reports success and clicks whatever that node has become, wherever it now lives. That second case is a silent miss with a clean log. Both reproduce in plain vanilla JS, so there’s no framework magic involved — a React rerender just does the replacing for you. The fix is the same either way: treat refs as disposable. Grab one right before you use it, use it once, and never carry it across a navigation or a form submit (credit to @skillselion in the comments for the single-use rule). One more caveat I’d rather admit than bury: I’ve watched a perfectly fresh ref click no-op silently while the window was still settling, which tells me ref clicks get resolved to coordinates somewhere under the hood. Plan A shrinks the coordinate problem. It does not make it go away.

The exception: canvas/Flutter apps (the Rive editor, for example). Their UI doesn’t exist as HTML elements, so find and read_page return nothing. Then Plan B is your only option.

Plan B: if you must use coordinates, pair “wait for settle” with “rapid double-fire”

  1. Wait for the size to settle before doing anything — use the diagnostic snippet above until innerW stabilizes, then take your screenshot
  2. Don’t let time pass between screenshot and click. Capture → click on those fresh coordinates, as one unit. If anything intervened, re-capture
  3. Never use double_click. Build your own with two rapid left_clicks at the same coordinates via browser_batch — the gap gets tight enough to pass the threshold:
browser_batch([
  { name: "computer", input: { action: "left_click", coordinate: [x, y], tabId } },
  { name: "computer", input: { action: "left_click", coordinate: [x, y], tabId } },
])

Enter fullscreen mode Exit fullscreen mode

  1. After every click, zoom into the target and verify the expected state change (selection, edit mode, highlight) before moving on. One unnoticed miss desyncs everything after it

Cheat sheet

Symptom First suspect Clicks miss Size not settled (Cause 1) — wait Double-click doesn’t work Switch to rapid double-fire (Cause 2) Stuck on a normal HTML page Switch to Plan A (element refs) Ref click errors out or hits the wrong element Stale ref — re-find immediately before the interaction (refs are single-use) Canvas/Flutter app Plan B only — there are no elements

Limits and caveats

The numbers here (1664→1920, 2–3 seconds) are from my setup (Windows, 1920×1080). Display config and DPI scaling will change them. Take the principle, not the numbers: wait for settle before you capture.

Claude in Chrome ships updates frequently. If the tool ever starts waiting for size-settle itself, this whole workaround becomes unnecessary. And coordinate-based automation is inherently fragile — whenever element references are available, they win.

Update (2026-07-11)

@skillselion left a comment below that made this article better, and I’ve revised Plan A and the settle rules accordingly. I didn’t take either point on faith — both got re-verified in plain vanilla JS before I touched the text. Refs really do track node identity and nothing else: kill the node and the ref dies loud, move it and the ref follows it silently into whatever it has become. And the innerW-only settle check really is blind to DPI changes — by design, since a rescaled-but-stable viewport is indistinguishable from a settled one if all you compare is innerW. I couldn’t physically test the cross-monitor move (one display here), but my own setup argued the point for me: the same physical 1920px screen that settled at innerWidth 1920 in June now reports 1745 at a fractional dpr of 1.1. The full back-and-forth is in the comments.

Verified: June 2026; Plan A and settle conditions re-verified July 2026 (dpr 1.1 / 110% scaling environment). Environment: Windows 11 / Chrome + Claude in Chrome extension / 1920×1080, including automation of the Rive editor (Flutter/canvas).

원문에서 계속 ↗

코멘트

답글 남기기

이메일 주소는 공개되지 않습니다. 필수 필드는 *로 표시됩니다