feat(skills): add computer control with Cua Driver
Merged
F-
3mo ago+284+284+012345678901234567890123456789-0-0-0123456789
Adds a superset:computer skill alongside superset:browser so agents can operate native desktop apps and system browser windows through Cua Driver.
The skill:
- routes Superset in-app panes to
superset:browserand host desktop UI tosuperset:computer; - preflights the installed Cua Driver, daemon, and OS permissions without silently installing or widening access;
- creates a task-scoped Cua Driver session and cleans it up without stopping a shared daemon;
- requires a fresh snapshot → targeted action → fresh verification loop;
- prefers semantic and accessibility operations before pixel, foreground, or desktop escalation;
- preserves confirmation boundaries for consequential actions and authentication.
Also adds a responsive /computer docs page with current macOS, Linux, and Windows installation commands, invocation examples, browser-versus-computer routing, and safety guidance.
Refs #6643
How I tested it
- Skill Creator
quick_validate.py(pass) - Real Cua Driver smoke:
start_session→get_accessibility_tree→end_session(pass) bun run lint(pass)bun run typecheck(38/38 tasks pass)bun run --cwd apps/docs build(150 static pages pass)- Visual QA in standalone Chrome at 1200×917 and 500×800; restored the original window geometry and removed only the task-created tab afterward
bun run testreached unrelated@superset/agent-setupshell-wrapper tests; 2/121 failed because the real hostclaudebinary was selected as the system fallback and rejected a no-input invocation. CI runs the suite in an isolated environment and passes.- External-readiness scan for placeholders, private paths, credentials, and drafting residue (pass)
Checklist
- PR title follows conventional commits
-
bun run lintandbun run typecheckpass - Allow edits from maintainers is enabled on the fork
Summary by CodeRabbit
- New Features
- Added the
computerskill for accessibility-first control of desktop applications and system browsers. - Added support for Cua Driver and macOS Peekaboo, with driver selection and installation guidance.
- Added Windows driver autostart configuration and cross-platform setup guidance.
- Added the
- Documentation
- Added the skill to documentation navigation and the skills reference.
- Included setup, permissions, telemetry opt-out, invocation, session management, verification, browser interaction, and safety guidance.
- Added examples for opening desktop apps and modifying settings.