Concepts

This page explains how Droidline thinks about a phone. You do not need all of it to get started, but each section answers a question people run into in their first week. The examples are in Python and Node.js; the command reference shows every command in every interface.

The screen as a tree

Android describes what is on the screen as a tree of elements, the same tree screen readers use for blind users. A screen is a big box; inside it are smaller boxes such as a toolbar and a list; inside those are buttons, texts and images.

dump() returns that tree. Each element carries these fields:

{
  "text": "Log in",
  "id": "com.example:id/login",
  "desc": "",
  "class": "android.widget.Button",
  "bounds": [60, 1000, 1020, 1140],
  "clickable": true,
  "enabled": true,
  "checked": false,
  "children": []
}

d.dump("screen.json")          # saves the tree to a file on your PC
tree = d.dump()                # or returns it as a dict

await d.dump("screen.json");   // saves the tree to a file on your PC
const tree = await d.dump();   // or returns it as an object

Finding elements walks through reading a dump and picking the right element.

Selectors: a field and a value

Commands that act on one element take two arguments first: which field to compare (by) and the value to look for. Together they are a selector.

When several elements match, nth picks one, counting from 0 in the order they appear on screen. touchById, touchByText and touchByDesc are shorthands for the three most common fields:

d.touch("id", "login")
d.touchById("login")                                   # the same
d.touch("class", "android.widget.CheckBox", nth=1)     # the second checkbox

await d.touch("id", "login");
await d.touchById("login");                                      // the same
await d.touch("class", "android.widget.CheckBox", { nth: 1 });   // the second checkbox

Which field to choose: id survives language changes and most app updates, so prefer it when it exists. text is easy to read but changes with the phone's language. desc is what icon buttons usually have.

Waiting is built in

Phones are slow in unpredictable ways: an app takes a moment to start, a list loads from the network. Droidline handles this for you. touch, long_touch, input, clear and wait wait for their element to appear, 10 seconds by default. Do not add your own sleep before them.

d.touch("text", "Done")                 # waits up to 10 seconds
d.touch("text", "Done", timeout=30)     # waits up to 30 seconds
d.wait("text", "Upload complete", 120)  # just wait, without tapping
d.wait_gone("text", "Loading")          # wait for something to disappear

await d.touch("text", "Done");                    // waits up to 10 seconds
await d.touch("text", "Done", { timeout: 30 });   // waits up to 30 seconds
await d.wait("text", "Upload complete", 120);     // just wait, without tapping
await d.waitGone("text", "Loading");              // wait for something to disappear

The waiting happens on the phone, so it costs nothing on the network while it waits.

How a tap is delivered

When touch finds its element, it tries three ways to tap it, in order, and reports which one worked in the via field of the reply:

1. node: the element's own click action, the cleanest way.
2. parent: the nearest clickable parent, for a label inside a clickable row.
3. gesture: a real tap at the center of the element's bounds.

input works the same way: it sets the text through accessibility (via: "set_text") and falls back to typing with the Droidline keyboard (via: "ime") for fields that ignore the first method.

Conditions answer at once

Some commands are questions: exists, get_text, checked, enabled, selected, count, in_app, keyboard_shown, last_toast, color, installed and the device checks such as battery. They never wait and never fail. A missing element gives false, "", 0 or -1 instead of an error, so they fit straight into an if:

if d.exists("text", "Close ad"):
    d.touch("text", "Close ad")

if d.battery()["level"] < 20:
    print("charge me")

if (await d.exists("text", "Close ad")) {
  await d.touch("text", "Close ad");
}

if ((await d.battery()).level < 20) {
  console.log("charge me");
}

Waiting for one of several screens

which is the one condition that waits. Give it a list of selectors and it returns the position of the first one to appear, or -1 if none appears before the timeout. It is the cleanest way to handle a step that can end in different places:

state = d.which([("text", "Log in"), ("id", "main_tab")], timeout=15)
if state == 0:      # the login screen came first
    d.input("id", "email", "me@example.com")
elif state == -1:   # neither appeared
    d.screenshot("unknown.png")

const state = await d.which([["text", "Log in"], ["id", "main_tab"]], { timeout: 15 });
if (state === 0) {          // the login screen came first
  await d.input("id", "email", "me@example.com");
} else if (state === -1) {  // neither appeared
  await d.screenshot("unknown.png");
}

Errors

A command that cannot do its job fails with an error. Every error has a code, a readable message, and retryable, which says whether trying again might help.

NOT_FOUND: Could not find text 'Log in' within 10s. Current screen: com.example / .MainActivity

from droidline import NotFoundError, DroidlineError

try:
    d.touch("text", "Log in", timeout=5)
except NotFoundError:
    d.back()
except DroidlineError as e:
    print(e.code, e.retryable, e)

import { DroidlineError } from "droidline";

try {
  await d.touch("text", "Log in", { timeout: 5 });
} catch (e) {
  if (!(e instanceof DroidlineError)) throw e;
  if (e.code === "NOT_FOUND") await d.back();
  else console.log(e.code, e.retryable, e.message);
}

Messages come in the server's language: English, Korean or Chinese, from the lang setting in config.toml or your system language. The codes are always the same. Error codes lists all of them.

One phone or many

With exactly one phone online, you never have to name it. With more than one, name the phone you mean, or the call fails with DEVICE_AMBIGUOUS:

from droidline import Droidline

dl = Droidline()
for info in dl.devices():
    print(info["name"], info["online"], info.get("route"))

a = dl.device("shelf-01")
b = dl.device("shelf-02")
a.launch("com.android.chrome")

import { Droidline } from "droidline";

const dl = new Droidline();
for (const info of await dl.devices()) {
  console.log(info.name, info.online, info.route);
}

const a = dl.device("shelf-01");
const b = dl.device("shelf-02");
await a.launch("com.android.chrome");

droidline --device shelf-01 launch com.android.chrome

Every interface also reads the DROIDLINE_DEVICE environment variable as the default phone. Give phones short names with droidline rename <id> <name>.

Commands to the same phone run one at a time, in the order you sent them. Commands to different phones run at the same time. Many phones shows how to drive a shelf of phones in parallel.

If a phone is offline, a command waits up to 30 seconds for it to come back (offline_wait in config.toml) before it fails with DEVICE_OFFLINE.

Commands that cut the phone's own connection

data(False), wifi(False) and airplane(True) take away the network the phone uses to talk to your PC. Droidline handles this in two ways.

By default, these commands answer {"accepted": true} before they act, and the final result arrives after the phone reconnects. Pass wait=True (CLI: --wait) to get the final result as the normal return value instead:

d.airplane(True, wait=True)

await d.airplane(true, { wait: true });

droidline airplane true --wait

Anything that has to happen while the phone is offline goes into one batch. The phone runs the whole list by itself and reports back when it is online again:

d.batch([("airplane", True), ("sleep", 3000), ("airplane", False)], wait=True)

await d.batch([["airplane", true], ["sleep", 3000], ["airplane", false]], { wait: true });

droidline batch '[["airplane",true],["sleep",3000],["airplane",false]]' --wait

That is the usual way to get a new mobile IP address. sleep exists only inside a batch.

Nothing runs twice

On mobile data the link between the PC and a phone can drop at any moment, including halfway through a command. Droidline makes sure a command still runs exactly once:

1. Every command carries an ID.
2. The phone keeps each reply until the PC confirms it received it.
3. If the link drops, the PC resends the command with the same ID when the phone is back.
4. The phone recognizes the ID and sends the stored reply instead of running the command again.

If the Droidline app on the phone was restarted in the meantime, its memory of IDs is gone. The command then fails with AGENT_RESTARTED, so your code can decide whether running it again is safe.