playbooks

home / skills / openclaw / skills / android-adb

android-adb skill

/skills/staticai/android-adb

This skill helps you control Android devices via ADB for UI automation, screenshots, and complex command sequences.

npx playbooks add skill openclaw/skills --skill android-adb

Review the files below or copy the command above to add this skill to your agents.

Files (2)
SKILL.md
2.1 KB
---
name: android-automation
description: Control Android devices via ADB with support for UI layout analysis (uiautomator) and visual feedback (screencap). Use when you need to interact with Android apps, perform UI automation, take screenshots, or run complex ADB command sequences.
---

# Android Automation

Control and automate Android devices using ADB, uiautomator, and screencap.

## Connecting Devices

### USB Connection
1. Enable **Developer Options** and **USB Debugging** on the device.
2. Connect via USB and verify with `adb devices`.

### Wireless Connection (Android 11+)
1. Enable **Wireless Debugging** in Developer Options.
2. **Pairing**: Find the IP, port, and pairing code in the "Pair device with pairing code" popup.
   `adb pair <ip>:<pairing_port> <pairing_code>`
3. **Connecting**: Use the IP and port shown on the main Wireless Debugging screen.
   `adb connect <ip>:<connection_port>`
4. Verify with `adb devices`.

## Common Workflows

### Launching an App
Use the monkey tool to launch apps by package name:
`adb shell monkey -p <package_name> -c android.intent.category.LAUNCHER 1`

### Analyzing the UI
Dump and pull the UI hierarchy to find coordinates:
`adb shell uiautomator dump /sdcard/view.xml && adb pull /sdcard/view.xml ./view.xml`

Then grep for text or resource IDs to find `bounds="[x1,y1][x2,y2]"`.

### Interacting with Elements
- **Tap**: `adb shell input tap <x> <y>`
- **Text**: `adb shell input text "<text>"` (Note: Use `%\s` for spaces in some environments or handle quoting carefully)
- **Keyevent**: `adb shell input keyevent <keycode>` (Home: 3, Back: 4, Power: 26, Search: 84, Enter: 66)
- **Swipe**: `adb shell input swipe <x1> <y1> <x2> <y2> <duration_ms>`

### Visual Verification
Take a screenshot to verify the state:
`adb shell screencap -p /sdcard/screen.png && adb pull /sdcard/screen.png ./screen.png`

## Tips
- **Search**: Use `input keyevent 84` to trigger search in many apps.
- **Wait**: Use `sleep <seconds>` between commands to allow the UI to update.
- **Coordinates**: Calculate the center of `[x1,y1][x2,y2]` for reliable taps.

Overview

This skill automates Android devices using ADB with support for UI layout analysis (uiautomator) and visual feedback (screencap). It enables launching apps, tapping or swiping by coordinates, entering text, and capturing screenshots for verification. Use it to build repeatable UI flows, run complex ADB command sequences, or integrate device control into scripts.

How this skill works

The skill uses ADB to communicate with a connected Android device over USB or wireless debugging. It can dump the UI hierarchy with uiautomator to extract element bounds, compute tap coordinates, and send input commands (tap, swipe, text, keyevent). Screencap is used to capture the device display and pull images to the host for visual checks.

When to use it

  • Automate repetitive interactions in Android apps or test flows.
  • Perform headless or remote device control from scripts or CI systems.
  • Capture and verify app screens during automated tests or debugging.
  • Extract UI element positions when no accessibility API is available.
  • Run complex, ordered ADB command sequences across multiple devices.

Best practices

  • Enable Developer Options and USB Debugging before connecting devices.
  • Prefer wireless debugging for remote or multiple-device setups; pair securely with the provided code and ports.
  • Dump the UI hierarchy and compute element centers from bounds to make taps reliable.
  • Insert sleep/wait steps between commands to allow UI transitions to settle.
  • Quote and escape input text carefully; spaces or special characters may require percent-encoding or different quoting strategies.

Example use cases

  • Launch an app by package name and navigate through menus using coordinate taps.
  • Dump the on-screen UI to view.xml, grep for resource IDs, compute center coordinates, and send tap events to specific elements.
  • Take a screencap after each test step and pull screenshots for visual diffs or reporting.
  • Automate form entry by sending input text and keyevents (e.g., Enter/back) in sequence.
  • Connect to a device over wireless debugging, run a scripted sequence, then disconnect and gather logs/screenshots.

FAQ

How do I connect wirelessly to a device?

Enable Wireless Debugging in Developer Options, use the pairing popup to get IP:pairing_port and code, run adb pair <ip>:<pairing_port> <pairing_code>, then adb connect <ip>:<connection_port> and verify with adb devices.

How can I reliably tap a UI element found in the uiautomator dump?

Parse the bounds attribute, calculate the center as ((x1+x2)/2,(y1+y2)/2), and use adb shell input tap <x> <y>. Add a short sleep after the tap to allow the UI to update.