Computer Use

Computer use is a capability class in which a multimodal model perceives a graphical user interface via screenshots and emits low-level pointer, keyboard, and scroll actions to operate arbitrary software.

Why It Matters

Computer use removes the integration tax on enterprise AI: the long tail of legacy software with no API becomes accessible the moment the model can see and click.

Timeline

1. 2024-10

   Anthropic introduces Computer Use beta in Claude 3.5 Sonnet.

2. 2025-01

   OpenAI Operator ships consumer browser-use product.

3. 2025-Q2

   Major RPA vendors (UiPath, Automation Anywhere) reposition around vision-driven agents.

• 12 monthsmedium confidence

  Computer-use agents pass 70% on a major end-to-end office-task benchmark.

Related Trends

• Agentic Workflows

  Rising

  LLMs that plan, call tools, and complete multi-step tasks autonomously.

• Tool-Use Protocols

  Emerging

  Standardized ways for models to discover and call external tools (MCP, A2A).

• Enterprise AI Adoption

  Mainstream

  AI moves from pilots into core P&L line items.