shot-scraper video: Your AI Agent Can Now Record Its Own Screen — and That Changes Everything

Simon Willison released shot-scraper 1.10 today, and the headline feature is a new shot-scraper video command. It accepts a storyboard.yml file that defines a routine — open URLs, click buttons, fill forms, wait for elements — and uses Playwright to record a full video demo of that routine executing.

The demo in the launch post? Generated entirely by GPT-5.5 xhigh running in Codex Desktop. The agent read the --help output, built a storyboard, and filmed itself working.

This is one of those releases that looks like a small utility update but actually unlocks something fundamentally new: the ability for AI agents to produce visual proof of their own work without any human behind the camera.

How the Storyboard Works

The format is refreshingly simple. A storyboard.yml file has two levels:

Global setup — the web server command, viewport size, cursor visibility, global JavaScript overrides, and initial URL:

output: output/demo.webm
server: uv run datasette -p 6419 --root --secret "1" /tmp/demo.db
url: http://127.0.0.1:6419/demo/tasks
viewport:
  width: 1280
  height: 720
cursor: true

Scenes — each scene has a name and a sequence of do actions:

scenes:
  - name: Bulk insert existing table rows
    do:
      - pause: 0.8
      - click: 'button[data-table-action="insert-row"]'
      - wait_for: "#row-edit-dialog[open]"
      - fill:
          into: ".row-edit-bulk-textarea"
          text: |
            title,owner,status,priority
            Prepare release video,Ana,doing,1
            Check pasted CSV import,Ben,review,3
      - click: ".row-edit-save"
      - wait_for: "text=3 rows inserted."

The action set covers the fundamentals: pause (wait N seconds), click, wait_for (CSS/text), wait_for_url (glob patterns), fill (type text into an element), and open (navigate mid-scene). It also supports authentication via a --auth <cookies.json> flag.

That's it. The simplicity is the point — the format is minimal enough that an AI agent can generate it from a single --help read, but expressive enough to produce genuinely useful demos.

The Two-Year Payoff

The feature request for shot-scraper video dates back to February 2024. Willison says he "almost certainly wouldn't have taken on this feature without coding agent support" — and that's the kind of honest admission about AI collaboration that more maintainers should make.

The blocker was Playwright itself. Older versions of Playwright's video output included browser chrome, which makes for terrible demos. Playwright 1.59 introduced a screencast API with fine-grained control, but the resulting videos were locked at 800px wide — too narrow for anything involving modern web UIs. A fix PR landed and shipped in playwright-python 1.61.0, which unblocked the release immediately.

The timeline tells a story:

• Feb 2024: Feature request filed
• Late 2025: Playwright screencast API lands (but 800px wide)
• Early 2026: Width fix merges
• June 2026: playwright-python 1.61.0 ships → shot-scraper 1.10 ships two days later

That two-year gap isn't laziness. It's a feature that needed the right combination of upstream capability and AI-assisted implementation velocity to become viable.

The Agent That Wrote Its Own Demo Script

The demo storyboard wasn't hand-crafted. Willison gave GPT-5.5 xhigh a three-step prompt:

1. Review the branch changes
2. Run shot-scraper video --help
3. Use the command to record a demo against a local dev server

The agent generated the full storyboard autonomously. Willison's framing is worth quoting:

"I really like this pattern where the --help output for a command provides enough detail that a coding agent can use it — it works kind of like bundling a SKILL.md file directly inside the tool."

This is the pattern I expect to see every CLI tool adopt within a year. If your --help text can't serve as an agent prompt, your CLI is stuck in the human-only era.

Even more interesting: the documentation for shot-scraper video was also written by the AI. Willison found that reviewing AI-generated docs was the best way to spot design inconsistencies and iterate on the feature. The YAML schema was defined using Pydantic, which made the AI's design decisions easier for a human to audit.

This is a workflow worth stealing: let the agent write the code AND the docs, then review the docs first. If the agent can't explain its own feature coherently in prose, there's probably a design problem.

Where This Fits in the Agent Observability Stack

Shot-scraper video lands in a rapidly growing ecosystem of tools for observing and recording AI agent behavior. The landscape breaks down roughly like this:

Most teams building agent systems have the trace and log layers covered. Very few have the visual recording layer — and that's a problem. When an agent makes a wrong decision in a browser, the trace tells you what it did (clicked button X, sent form Y), but the video tells you why (the page looked broken, the button was obscured, a loading spinner never finished).

Video is the most human-friendly debugging artifact an agent can produce. It's also the artifact that builds trust with non-technical stakeholders. "Here's the video of my agent filing the report" lands much harder than "here's the trace ID."

What This Unlocks

I see three immediate use cases beyond what the launch post covers:

1. CI-pipeline visual regression testing. Imagine your CI runs a storyboard.yml against staging after every deploy. If the video looks wrong (button not found, page didn't load, data didn't appear), you catch it before it ships. The video becomes a visual assertion system.

2. Agent portfolio artifacts. If you're building agent workflows for clients, each agent can produce a video demo of its own execution as part of the output. The deliverable isn't just "the job ran successfully" — it's "here's the recording of it running, with visual proof."

3. Bug report reproducers. An agent that encounters an error mid-task can record the session up to that point and attach the video to the bug report. No more "steps to reproduce" written in prose — just a 15-second video showing exactly what happened.

The Catch

The current storyboard format is read-only for interactive targets. There's no assert action — you can wait for an element to appear, but you can't verify a value or trigger a failure condition. That limits its use for testing. You also can't record sessions interactively (record-what-I-just-did) and export them as storyboard.yml, which would be the obvious next step for human-generated demos.

But Willison built this with AI agent generation as the primary workflow, and for that use case it's already complete. Agents read specs and produce YAML; they don't need a record-and-export flow. The assertion gap matters more, and I'd expect it to close in a 1.11 release if the community asks for it.

Bottom Line

Shot-scraper video is the kind of release that seems modest on paper — a new subcommand in a niche CLI tool — but reveals a much bigger shift in how we build and verify AI agent work. The combination of AI-generated storyboards and AI-generated documentation, validated by human review of the docs-first workflow, is a genuine productivity pattern that more tool builders should adopt.

The fact that the entire feature — code, storyboard, and docs — was produced by an AI agent collaborating with a human maintainer makes this a meta-demo as much as a tool release. The agent that built shot-scraper video can now turn around and film itself building the next feature.

That's the kind of recursive productivity gain that's hard to un-see once you've watched it work.

Sources: Simon Willison: Have your agent record video demos of its work, shot-scraper 1.10 release notes, GitHub: shot-scraper, Playwright screencast API