C/ua: Empowering AI Agents with Full Operating System Control

C/ua (pronounced "koo-ah") is a groundbreaking open-source project designed to provide AI agents with full operating system control within virtual containers. Billed as "Docker for Computer-Use AI Agents," c/ua allows developers and AI enthusiasts to run sophisticated AI agents locally or deploy them to the cloud, enabling them to interact with and automate tasks across entire operating systems.

What is C/ua?

At its core, c/ua is a robust framework that facilitates the integration of AI agents with virtualized environments. It leverages Lume CLI for high-performance macOS/Linux VM management and Lumier for a Docker-like interface to these VMs. This powerful combination means AI agents can perform real-world tasks like navigating a desktop, interacting with applications, and executing commands just like a human user. With 8.5K stars on GitHub and an active community, c/ua is rapidly gaining traction as a vital tool in the evolution of AI.

Key Capabilities and Features:

• Virtual Containerization: Provides secure and isolated virtual environments for AI agents to operate within.
• Operating System Control: Allows AI agents to control macOS and Linux systems, with potential for Windows (via WSL).
• Automated Desktop Tasks: Automate complex workflows, from data entry to software interactions.
• Multiple Agent Loops: Supports various agent paradigms including:
  • UI-TARS-1.5: Local execution on Apple Silicon with MLX.
  • OpenAI CUA: Utilizes OpenAI's Computer-Use Preview model.
  • Anthropic CUA: Integrates with Anthropic's Computer-Use capabilities.
  • OmniParser-v2.0: Enables UI control with Set-of-Marks prompting using any vision model.
• Developer-Friendly: Offers a Python SDK for easy integration into custom applications.
• Comprehensive Modules: Includes Lume (VM management), Lumier (Docker interface for VMs), Computer (VM control interface), Agent (AI agent framework), and MCP Server (integration with Claude Desktop).

Demos in Action

[Image: vibe-photoshop.mp4] * Photoshop Integration: See an AI agent manipulating Photoshop, demonstrating fine-grained control over graphical applications.

[Image: mcp-claude-tableau.mp4] * MCP Server with Claude Desktop and Tableau: Watch an agent seamlessly interact with business intelligence tools.

[Image: ai-gradio-clone.mp4] * Multi-app Workflow (Browser, VS Code, Terminal): Witness an AI agent navigating between different applications to accomplish a task.

[Image: notebook-github-cursor.mp4] * Fix GitHub Issue in Cursor: An agent intelligently identifies and resolves a GitHub issue within a code editor.

Getting Started with C/ua

C/ua offers flexible installation options to suit different user preferences:

Option 1: Fully-Managed Install (Recommended)

For a guided and hassle-free setup on macOS/Linux/Windows (via WSL), simply run the following command (requires Python 3.11+):

/bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/trycua/cua/main/scripts/playground.sh)"

This script handles dependencies, VM image downloads, Python package installation, and launches the Computer-Use Agent UI.

Option 2: Key Manual Steps

For those who prefer a more hands-on approach:

For C/ua Agent UI (any system, cloud VMs only):

pip install -U "cua-computer[all]" "cua-agent[all]"
python -m agent.ui.gradio.app

For Local macOS/Linux VMs (Apple Silicon only):

# 1. Install Lume CLI
/bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/trycua/cua/main/libs/lume/scripts/install.sh)"

# 2. Pull macOS image
lume pull macos-sequoia-cua:latest

# 3. Start VM
lume run macos-sequoia-cua:latest

# 4. Install packages and launch UI
pip install -U "cua-computer[all]" "cua-agent[all]"
python -m agent.ui.gradio.app

Developer Guide

Integrating c/ua into your own projects is straightforward with its Python SDK:

from computer import Computer
from agent import ComputerAgent, LLM
import asyncio

async def main():
    # Start a local macOS VM
    computer = Computer(os_type="macos")
    await computer.run()

    # Or with C/ua Cloud Container
    # computer = Computer(
    #    os_type="linux",
    #    api_key="your_cua_api_key_here",
    #    name="your_container_name_here"
    # )

    # Example: Direct control of a macOS VM with Computer
    await computer.interface.left_click(100, 200)
    await computer.interface.type_text("Hello, world!")
    screenshot_bytes = await computer.interface.screenshot()

    # Example: Create and run an agent locally using mlx-community/UI-TARS-1.5-7B-6bit
    agent = ComputerAgent(
        computer=computer,
        loop="uitars",
        model=LLM(provider="mlxvlm", name="mlx-community/UI-TARS-1.5-7B-6bit")
    )
    async for result in agent.run("Find the trycua/cua repository on GitHub and follow the quick start guide"):
        print(result)

if __name__ == "__main__":
    asyncio.run(main())

Community and Contributions

C/ua is an MIT-licensed open-source project that welcomes contributions. You can join their Discord community to discuss ideas, get assistance, or share your demos. The project also provides clear contributing guidelines for those looking to get involved.

By providing a robust platform for AI agents to control operating systems, c/ua is pushing the boundaries of what AI can achieve, making complex automation and interaction simpler and more accessible than ever before.