005 / research
ax-snapshot
What an accessibility tree shows a browser agent, and what it quietly drops.
What
Comparing an accessibility snapshot of a page with a screenshot of the same page, as input to a tool-using agent. No product around it. Just the two observations, side by side.
Why
The agent project uses the accessibility tree as its eye. I want to know which pages that eye cannot see, before I pretend the choice is universal.
Architecture
One page, two captures. The tree is text: roles, names, a rough structure. The screenshot is pixels. The question is which one lets a later tool call point at the thing a person meant.
Discoveries
The tree is excellent on documents, forms, and ordinary links. It goes thin on canvas, maps, and custom drawing. Screenshots keep that information and lose the structure that makes a click target obvious. The default eye should be the tree. The screenshot belongs as a tool I turn on, not as the air the agent breathes.