How to Automate Android Devices with Natural Language: A Practical Guide to Google's ARTEMIS

If you've done Android development or testing, you've probably been through this scenario: you've built an APK and want to quickly verify a flow—open the app, log in, tap a few buttons, take screenshots, and compare. Traditional approaches mean writing Appium scripts from scratch or recording UI interactions manually, and just setting up the environment can eat up half a day.

What if you could simply tell your computer, "Open Settings, check the battery level, and take a screenshot," and it just does it?

That's exactly what we're going to put into practice today with ARTEMIS—an open-source project from Google. It converts natural language instructions into reliable Android device actions, achieving a 99%+ task completion rate on Google's own AndroidWorld benchmark. In this tutorial, we'll walk through setting up the environment from scratch, running your first natural language command, and writing a Python script to integrate this capability into your testing workflow. No theory dumps—just hands-on practice.

Prerequisites

Before we begin, make sure you have the following:

Why USB debugging? Because ARTEMIS communicates with the device through ADB under the hood. Without USB debugging enabled, it simply cannot control the screen. On a real device, enable it in "Settings → Developer Options." Emulators have it enabled by default.

Installation & Quick Start

ARTEMIS provides a one-click startup script that automatically installs ADB, scrcpy, FFmpeg, and other system tools, saving you a lot of hassle. Let's get started:

Step 1: Clone the Repository

git clone https://github.com/google/artemis.git && cd artemis

Step 2: Start the Project

macOS / Linux:

./start.sh

Windows PowerShell:

git clone https://github.com/google/artemis.git
cd artemis
.\start.bat

A Windows gotcha: PowerShell doesn't search for executable scripts in the current directory by default, so you must write .\start.bat (with the backslash prefix). Running start.bat directly may throw a "command not found" error. In CMD, you can omit the prefix.

After running the script, it will automatically detect and install dependencies. uv is the Python package manager ARTEMIS uses (much faster than pip), and if it's not already on your system, it will be installed automatically.

Step 3: Verify the Connection

Once started successfully, it will open http://localhost:8000, where you'll see a web console with a live screen mirror and an input box for commands. At this point, run a command in the terminal to test if the connection is working:

uv run artemis run "Open Settings, find the Battery option, and tell me the current battery level" --profile flash

The --profile flash flag uses "Fast Mode," where each operation step takes about 3–5 seconds. It's ideal for clear, deterministic operations. ARTEMIS also has a pro mode (multi-agent planning + validation, suited for complex long-running workflows), which we'll cover later.

Why use uv run instead of plain python? Because uv automatically manages the project's virtual environment and dependency versions, avoiding global environment pollution. This is becoming the best practice trend in the Python toolchain.

Hands-On: Drive a Complete Workflow with Natural Language

Let's do something interesting—have ARTEMIS automatically complete a cross-app task: set up a navigation route in Google Maps, then go to YouTube and play a song.

Option 1: Run Directly from CLI

uv run artemis run "Set up a driving route in Google Maps and calculate the total duration, then open YouTube and play a song by Coldplay" --profile flash

ARTEMIS will automatically recognize the current screen, locate elements, and execute taps and text input. You can see every step of its operation in real-time on the web console—what it perceives, the coordinates, and what action it performs. Very intuitive.

Option 2: Integrate with Python SDK into Your Scripts

As a backend developer, you'll likely want to embed this capability into automated testing scripts. ARTEMIS provides a Python SDK:

import asyncio
from artemis_client import ArtemisClient


async def main():
    client = ArtemisClient(
        "http://artemis-host:8000",         # ARTEMIS service address
        device_serial="emulator-5554",     # Optional: specify device serial number
        default_profile="flash",           # flash=fast response / pro=deep planning
    )

    # Issue a natural language instruction
    result = await client.run(
        "Open System Settings, navigate to the Battery page, confirm the battery percentage is displayed, and check if there are any crash dialogs.",
    )

    # Assert the execution result
    assert result.succeeded, f"Test failed: {result.error or result.status}"
    print(f"✅ Test passed! Device: {result.device_serial} | Trace ID: {result.trace_id}")


if __name__ == "__main__":
    asyncio.run(main())

First, install the SDK:

uv add "artemis-client @ git+https://github.com/google/artemis.git#subdirectory=packages/artemis-client"

Why this architecture? ARTEMIS's server-side runs on the device side (handling ADB, screenshots, element recognition), while the SDK is a lightweight client with zero runtime dependencies—it only sends commands and receives results. This design means your test scripts can run anywhere, as long as they can reach the ARTEMIS service address.

Advanced: Control Your Phone Directly from AI IDEs

If you use AI coding assistants like Claude Code, Cursor, or Antigravity, ARTEMIS natively supports MCP (Model Context Protocol), allowing AI to directly drive your real phone.

Install MCP for all supported AI IDEs with a single command:

## Install MCP configuration for all supported AI IDEs
uv run artemis mcp --install all

Once installed, simply say in the IDE's chat box:

"Build the latest code into an APK, install it on the connected device, open the login page with a test account, check if there are any error dialogs after login, and return a screenshot of the final page."

The AI will hand this instruction off to ARTEMIS for execution, then bring back the results to you. For mobile development teams, this dramatically shortens the feedback loop for bug reproduction and UI verification.

Troubleshooting & Common Pitfalls

1. ADB Cannot Connect to Device

This is the most common issue. Checklist:

• Real device: Ensure "USB Debugging" is enabled, and select "File Transfer" or "MTP" mode
• Emulator: Confirm the AVD is running and appears in adb devices
• Run adb devices to verify the device is enumerated

2. Flash Mode vs Pro Mode: Which Should I Choose?

• Flash: Each operation step takes 3–5 seconds. Single-model reactive loop with no task planning or checkpoint validation. Ideal for daily, well-defined workflows.
• Pro: Each operation step takes 15–40 seconds. Multi-agent collaboration (Planner plans → Operator executes → Checker validates), with pre-execution safety nets, auto-recovery on failure, and final report generation. Suited for 100+ step long workflows and stability stress testing.

Simply put: run Flash for daily automation, run Pro for formal test cases.

3. What is the "Artemis Accessibility Helper" on My Device?

On the first run, ARTEMIS installs an accessibility service on the phone to read screen layout information. It runs locally on the device only and does not upload any data. If you prefer not to auto-install it, you can set ARTEMIS_HELPER_AUTO_INSTALL=false in your .env file.

4. Want the artemis Command to Be Globally Available?

Run this once in the project root:

uv tool install -e .

After that, you won't need to prefix every command with uv run.

Next Steps

Today, we've covered setting up ARTEMIS, executing your first natural language instruction, integrating the Python SDK, and learned how to let AI IDEs directly drive your phone via MCP. Here are some directions you can continue exploring:

1. Integrate ARTEMIS into your team's CI/CD pipeline to automatically run a round of UI smoke tests with every release
2. Write end-to-end test cases with validation points and final reports using Pro mode
3. Combine with the pytest framework, encapsulating client.run() into reusable test fixtures

Google's official AndroidWorld benchmark covers 20+ apps and 100+ multi-step tasks. The 99%+ completion rate shows that its ability to handle complex scenarios is already quite reliable. Once you start using it, you'll find that phone automation is much simpler than you might think.

Project: github.com/google/artemis (currently 12,000+ Stars). Feel free to Star and follow for updates. Questions? Join their Discord community to discuss.