Turn Screen Recordings into Reusable AI Skills with Microsoft's Skill Recorder
Microsoft's Skill Recorder captures your screen activity and uses GitHub Copilot CLI to turn it into reusable Skills or Automations for AI agents. Learn how it works and how to install it.
Microsoft has released an open-source desktop app called Skill Recorder that records your on-screen work session and uses the GitHub Copilot CLI to reconstruct it as an intent plus ordered steps, then builds a reusable Skill or Automation for Microsoft Scout, Microsoft Copilot Cowork, or Copilot Studio. This is a game-changer for developers and power users who want to teach AI agents to perform repetitive tasks without writing a single line of code.
Why Skill Recorder Matters
AI agents are only as useful as the skills they possess. Traditionally, creating a skill for an agent like Microsoft Copilot required manually writing a SKILL.md file with detailed instructions, which is time-consuming and error-prone. Skill Recorder flips this process: you simply perform the task once on your screen, and the app captures everything—clicks, window switches, URLs, and even your voice narration—then uses Copilot to analyze the recording and generate a structured procedure. This means you can teach an agent to do anything you can do on your computer, from filling out forms to navigating complex workflows.
The key insight is that the generated skill prefers the agent's native tools (like the gh CLI or web_fetch) over replaying UI clicks. This makes the skill more robust and generalizable. For example, recording yourself submitting one form can teach the agent to submit all forms of that type, because the skill is based on the intent and steps, not on pixel-perfect click coordinates.
How It Works
The workflow is elegantly simple:
Record: Hit the record button (or use the global shortcut
⌘⇧Ron macOS /Ctrl+Shift+Ron Windows) and perform your task. Skill Recorder captures your screen and activity locally, in the background. A small always-on-top bar shows capture and microphone state, allowing you to mute/unmute or switch mics on the fly.Analyze: Click Analyze and GitHub Copilot reconstructs one overall intent and an ordered list of steps from the recording. You can review and edit the analysis until it reads correctly.
Create: From the approved analysis, generate a reusable Skill (a
SKILL.mdprocedure) and/or a scheduled Automation (the same procedure on a trigger).
This process is designed to be iterative—you can record, analyze, tweak, and regenerate until the skill perfectly matches your intent.
What Gets Captured
Skill Recorder is privacy-conscious. Recording, storage, frame extraction, and optional narration transcription all happen on your device; nothing leaves your computer while you record. Only when you choose Analyze does the app send the event timeline (window/document titles, URLs, clipboard previews), extracted screen images, and narration text to GitHub's cloud for Copilot to process.
Specifically, the app tracks:
- Window tracking: active-app and window switches.
- Browser URLs: the page you're on (macOS).
- Screen video: recorded by Chromium; low-rate snapshots are kept only when the screen changes or a heartbeat is due.
- Clipboard: short previews of copied text that tie steps together.
- Narration (optional): spoken commentary, transcribed on-device in any of Whisper's 99 supported languages (a one-time ~252 MB model download on first use).
⚠️ Important: Keep secrets out of your recordings. Don't record, type, paste, or narrate passwords, tokens, API keys, or other confidential info. The app reminds you before every recording.
Installation
Skill Recorder is published as a source release. One command downloads a pinned Node.js runtime, builds the exact release commit on your machine, and adds a Skill Recorder (Source) app you can relaunch anytime. Nothing is installed globally. You'll need a GitHub account with Copilot access; the Copilot CLI ships with the app.
macOS is the primary target, but Windows 11 (x64 and ARM64) is also supported.
macOS / Ubuntu
Open the latest release and copy the command for your platform. Each release pins an exact commit, so the real command looks like this (replace <40-character-release-commit> with the actual commit hash):
commit="<40-character-release-commit>"
curl -fsSL "https://raw.githubusercontent.com/microsoft/skill-recorder/$commit/install.sh" | SKILL_RECORDER_COMMIT="$commit" bash
To keep the app running after the terminal closes, add SKILL_RECORDER_DETACHED=1:
commit="<40-character-release-commit>"
curl -fsSL "https://raw.githubusercontent.com/microsoft/skill-recorder/$commit/install.sh" | SKILL_RECORDER_COMMIT="$commit" SKILL_RECORDER_DETACHED=1 bash
On macOS this adds a Skill Recorder (Source) app to ~/Applications (relaunch from Spotlight, Launchpad, or the Dock). On Ubuntu it adds a matching application entry.
Windows (PowerShell)
$commit="<40-character-release-commit>"
$env:SKILL_RECORDER_COMMIT=$commit
irm "https://raw.githubusercontent.com/microsoft/skill-recorder/$commit/install.ps1" | iex
This adds Skill Recorder (Source) shortcuts to your desktop and Start Menu.
First Steps
After installation, launch the app. On first launch, macOS will ask for Screen Recording permission; grant it and you're ready to record.
Then follow the three-step workflow: Record, Analyze, Create. The first time you click Analyze, Skill Recorder will offer to sign in to Copilot if you aren't signed in yet.
For detailed instructions on inspecting the install script, setting install options, updating, or uninstalling, see the INSTALL.md file in the repository.
Developing from Source
If you want to hack on the code, you'll need Node.js 24. After checking out a release revision:
npm ci
npm run compliance:licenses
npm run dev
npm run dev starts Vite and launches the Electron app with hot-reload. The global shortcut ⌘⇧R (macOS) / Ctrl+Shift+R (Windows) toggles recording from anywhere.
There's also an eval suite for the Copilot describer and builders:
npm run eval # score the describer against synthetic recordings
npm run eval:builder # score the skill/automation generalization
Conclusion
Skill Recorder is a powerful tool that bridges the gap between human demonstration and AI automation. By turning your screen recordings into structured, reusable skills, it makes AI agents far more practical for everyday tasks. Whether you're a developer building custom automations or a power user looking to streamline repetitive work, this tool is worth exploring.
Check out the GitHub repository for the latest releases, documentation, and to contribute. With 2.4k stars and active development, it's a promising addition to the AI agent ecosystem.