Turn Old Android Phones Into AI Agents | DroidClaw Tutorial

IN INTRODUCTION DroidClaw is an open‑source AI agent that turns an Android phone into a hands‑free assistant. By using a large language model to interpret screen contents and generate taps, pulls, and keystrokes via ADB, it can perform tasks like searching YouTube, sending WhatsApp messages, or running full multi‑app workflows—without any dedicated APIs.

WHAT IT IS The core of DroidClaw is a perception → reasoning → action loop: 1. It reads the screen through an accessibility tree (or a screenshot if the tree is missing). 2. The screen state, the user’s plain‑English goal, and prior actions are fed to an LLM. 3. The model returns a plan of high‑level actions (e.g., tap, type, swipe). 4. DroidClaw executes those via ADB, observes the result, and feeds it back into the next step.

Key Features • Natural‑language goal setting • Automatic handling of repeated actions and stuck loops • Vision fallback for inaccessible UI trees • Multi‑turn conversational memory • Local Ollama support for fully offline use • Remote control via Tailscale

SYSTEM REQUIREMENTS Hardware: Android phone (USB debugging enabled) or Wi‑Fi‑debuggable phone Software: ADB (Android Platform Tools), Bun runtime, Node/npm not required LLM: Any provider supported by DroidClaw (Groq, OpenAI, Ollama, OpenRouter, Bedrock)

INSTALLATION STEPS 1. Install Bun curl -fsSL https://bun.sh/install | bash 2. Install ADB • Mac: brew install android-platform-tools • Linux: sudo apt install android-tools-adb • Windows: download from https://developer.android.com/tools/releases/platform-tools 3. Clone the repo git clone https://github.com/unitedbyai/droidclaw.git cd droidclaw 4. Install project dependencies bun install 5. Copy the example env file cp .env.example .env 6. Configure your LLM provider in .env Example for Groq (free tier): LLM_PROVIDER=groq GROQ_API_KEY=YOUR_GROQ_KEY Or for local Ollama: LLM_PROVIDER=ollama OLLAMA_MODEL=llama3.2-vision

CONNECT YOUR PHONE • Enable Developer Options → USB debugging • Plug the phone via USB; accept the prompt • Verify: adb devices Should list your device.

RUNNING THE AGENT Interactive mode (plain English goal): bun run src/kernel.ts # Type your goal and press Enter

Workflow mode (AI‑powered multi‑app sequence): bun run src/kernel.ts --workflow examples/workflows/telegram-send-message.json

Deterministic flow (no AI): bun run src/kernel.ts --flow examples/flows/send-whatsapp.yaml

EXAMPLE USAGES - Open YouTube and search for "lofi hip hop": $ bun run src/kernel.ts enter your goal: open youtube and search for "lofi hip hop" - Send a WhatsApp message via a preset workflow. - Run a nightly morning briefing that reads calendar events, weather, and unread Slack messages.

REMOTE CONTROL WITH TAILSCALE Install Tailscale on both phone and host. Enable wireless debugging in Phone Settings → Developer Options. Connect over network: adb connect : Now DroidClaw can be launched from any VPS, laptop, or cron job, letting a phone stay plugged in at home while your scripts run it from anywhere.

ADVANCED FEATURES • Vision Fallback : When the accessibility tree is empty, the agent sends a screenshot to the LLM. • Stuck‑Loop Detection: If the screen doesn’t change after a few steps, the agent injects recovery hints. • Multi‑turn Memory: Keeps the last N actions in conversation context.

TRUBLESHOOTING • "adb: command not found" – make sure it’s installed and in $PATH or set ADB_PATH in .env. • "no devices found" – double‑check USB debugging enabled and confirm the cable supports data transfer. • Repeated actions – increase MAX_STEPS or switch to a larger model. • Empty accessibility tree – set VISION_MODE=always to force screenshot sends.

CONCLUSION DroidClaw demonstrates how modern LLMs can orchestrate UI automation without needing dedicated APIs. By combining open‑source tooling (Bun, ADB, Ollama) with a simple perception–reasoning–action loop, you can instantly repurpose an old phone into a powerful, always‑on personal assistant. Whether you want to automate social media posting, run nightly research workflows, or simply experiment with AI‑driven phone control, the installation process is straightforward and the flexibility is immense. Happy automating!