Skip to main content
Ouro UI

No thumbs needed: the robot drives all three screens with ls and do.

A little white robot leans over a phone, a tablet and a laptop on a garden bench, each showing the same green card with a switch, while Ouro smiles beside them.

Ouro Reins

Hand an agent the reins.

Agents used to drive apps the way a tourist drives abroad: squinting at screenshots, tapping where the label probably is. With Ouro Reins an agent reads the screen as state and actions and drives it with ls and do. Headless, from fixtures, with the network blocked, in milliseconds.

The idea is Shopify's Reflex, which showed view models driven this way. We took the contract, left the macro, and brought it to SwiftUI. Thank you, Shopify.

Take the reinsSee what an agent sees

Nineteen milliseconds against seventeen seconds.

  1. Reins, headless

    on the line and again, 0 times so far

  2. XCUITest, on an iPhone simulator

    on the line

Spoonjoy's journey from the Kitchen to a recipe, into cook mode, a step forward and back, and home took a median 18.7 ms headless through Reins, over 50 runs, and 16.7 s through XCUITest on a simulator, over 5. The simulator's lane runs at real speed, so bring a snack, or skip to the end.

Your turn

Drive a kitchen from here.

This is a pretend kitchen in your browser, shaped like the tree Spoonjoy's real app hands Reins: the same screens and actions, refusing in Reins' own words. Open the pasta, start cooking, turn a step, then ask for something that isn't on offer and watch it say no nicely.

spoonjoy-reins a pretend kitchen, running in your browser

  1. ls

    focus kitchen spoonjoy › tabs › stack › kitchen

    {
  "counts": {"recipes":2,"shoppingItems":3},
  "recipes": [
    {
      "id": "recipe_lemon_pantry_pasta",
      "title": "Lemon Pantry Pasta",
      "steps": 3
    },
    {"id":"recipe_tomato_toast","title":"Tomato Toast","steps":1}
  ],
  "lead": {"id":"recipe_lemon_pantry_pasta","title":"Lemon Pantry Pasta"}
}

    actions openRecipe from kitchenstartCooking from kitchenselectTab from tabs

Or press one. These are exactly what ls offers right now:

The model

What an agent sees.

  • Nodes

    A screen is a little tree. A node has a name, its state as JSON, and the actions it offers, with an input schema for those that take input.

  • The focus

    The deepest node is the screen in front of you. ls shows it, its path, and the actions you may run right now.

  • Lending

    A navigation stack lends goBack once you're deeper than its root; a tab bar lends selectTab. If two could lend the same action, the nearest one does.

  • Refusal

    Ask for something that isn't on offer and Reins says no before any app code runs, and lists what is.

Make it drivable

One object, built from what you have.

You don't rewrite your view models. You add one object that builds the tree from them, and move any glue that lives in a view into a model both can call.

Shell.swift
import OuroReins

@MainActor
final class Shell: ReinsDrivable {
    let store: AppStore
    var path: [Route] = []
    var page: ReinsNode { path.last.map(\.node) ?? .kitchen(store) }
    func select(_ tab: String) { path = []; store.tab = tab }
    var reinsNode: ReinsNode {
        .tabs(selected: store.tab, tabs: ["kitchen", "shopping"],
              select: { self.select($0) }, child:
            .stack(depth: path.count,
                   goBack: { self.path.removeLast() }, child: page))
    }
}
A journey, in a test
try await ReinsNetworkGuard.withNetworkBlocked {
    let journey = ReinsJourney(ReinsSession(Shell(store: .fixtureKitchen())))
    try await journey.do("openRecipe", ["id": "recipe_pasta"],
                         until: { $0.focus == "recipe" })
    try await journey.do("goBack", until: { $0.focus == "kitchen" })
}

Offline, on purpose

The network is blocked, and checked.

While the guard is up, every request through URLSession.shared fails, and the test fails afterwards even when the app recovered. Spoonjoy's journeys made no attempts, and a control run that went live was caught at it. A journey that needs the internet isn't a test, it's a weather report.

In a running debug build the same lines work over a local socket, so an agent can drive the real app on a simulator. Release builds leave it out entirely, and CI keeps it that way. Ouro Reins is on ouro-ui-apple's main branch now, and comes in a later release. Read the guide or hand your agent its skill. One honest limit, shared with Reflex: a screen shows one column at a time, so a split view shows only the column in focus.

Beyond our apps

Any app on a Mac, same reins.

Reins is a protocol as well as a package: the screen, the requests and replies, every refusal with its exit code, journeys, and the batches and diffs a remote driver uses, written down on their own. A second driver speaks it for whole Macs: Finder, System Settings or anyone's app, as state and actions, with no model in the loop and no clicking by guesswork.

  • The tree

    Any app's tree of controls, pruned to what a person can act on. An element has a role, a label, a value and a frame, and an address like toolbar > button[Back] that a recorded journey replays.

  • The pixels

    No tree to read (a login window, a canvas, a stuck screen)? A screenshot's text boxes become the same elements, the state says "layer": "pixels", and an IP-KVM's keyboard and mouse do the clicking.

  • Refusals first

    person-active while someone is at the Mac. app-protected for an app that refuses automation, a password prompt or a security setting. permission-missing names the step a person takes. Nothing is touched.

  • One round trip

    Only lines cross the network. batch runs a whole form on the Mac and checks its untils there, and after the first screen only a diff against the rev you hold comes back.

What an agent sees in Calculator
$ ls Calculator
{"focus":"Calculator","path":["mac","studio","Calculator"],
 "state":{"layer":"ax","elements":[
   {"i":3,  "ref":"statictext[Edit field]", "value":"0"},
   {"i":8,  "ref":"button[7]",              "can":["click"]},
   {"i":11, "ref":"button[Multiply]",       "can":["click"]},
   {"i":23, "ref":"button[Equals]",         "can":["click"]} …]},
 "context":{"person":{"active":false,"idle_s":312}, …},
 "actions":[{"name":"click","title":"press it","from":"Calculator",
             "input":{"properties":{"element":{"enum":["button[7]", …]}, …}}}, …],
 "rev":"6c64…"}
Clicks and a check, one request
$ batch '[{"action":"click","input":{"element":"button[7]"}},
         {"action":"click","input":{"element":"button[Multiply]"}},
         {"action":"click","input":{"element":"button[6]"}},
         {"action":"click","input":{"element":"button[Equals]"},
          "until":{"@statictext[Edit field].value":"42"}}]'
{"ok":true,
 "steps":[{"n":0,"action":"click","ok":true,"ms":25}, …
          {"n":3,"action":"click","ok":true,"ms":36}],
 "screen":{"rev":"b81c…","since":"6c64…",
           "put":[{"ref":"statictext[Edit field]","role":"statictext",
                   "value":"42"}]}}

Write your own driver against the protocol, or read how the any-Mac driver reads trees and pixels and when it says no. Its code isn't public yet.

Ways of knowing

Three layers: claims, behaviour, and how it feels.

  1. Claims held

    AX X-ray

    Does it say what it does?

    Reads the built app's App Intents from outside and checks that its claims hold.

  2. Behaviour

    Reins (this page)

    Does it do it?

    Drives the screens as state and actions, headless, with the network blocked.

  3. How it feels

    Feel

    Does it feel right while it does?

    Walks the real app in a simulator and measures its jank, genre by genre.

NextGet started