Skip to main content
Ouro UI

Ouro AX

ax-flows

An adopter's agent flows as data, run in CI without a model: a flow is the calls a good agent makes for one task, and the runner checks the calls' exit codes, JSON shapes and fields, error text, time, size and voice. The findings are in the flow-review/0 format (findings.jsonl, one check a line, and report.json), with an X-ray report attached when given. The X-ray checks what every command must do; flows check what a task needs, and each finds what the other can't.

node lib/ouro/ax/flows.ts run <suite module> [--out report] [--flows a,b] [--slack n] [--xray report.json]
npx shadcn@latest add ourostack/ouro-ui/ax-flows#v1.3.10

It installs lib/ouro/ax/flows.ts. Its tests are ax-flows-tests. Every item runs on Node 22.18 or later as it is copied: the files import each other as ./<file>.ts. The Ouro AX guide shows how they fit together.

Exports

at

The values at a path, and whether the path fanned out over an array ([] or a filter).

function at(doc: unknown, path: string): { values: unknown[]; fan: boolean; }

checkCall

The findings for one call: one per check its expect declares, then the voice checks every CLI call gets: stderr stays plain (no "I", exclamation or emoji outside quotes), JSON alone under --json, and no brand name the suite forbids. A path through why, by, agent, audience or who is an honesty check.

function checkCall(flow: Pick<Flow, "id" | "surface">, step: Pick<RunStep, "id" | "expect" | "fix" | "known">, ran: Ran, options: { tool: string; slack?: number | undefined; argv?: readonly string[] | undefined; brandNames?: readonly string[] | undefined; }): Finding[]

defineFlow

function defineFlow<T extends Flow>(flow: T) => T

defineSuite

function defineSuite<T extends FlowSuite>(suite: T) => T

Driver

Calls the CLI with the words after its name and the environment for the call.

type Driver = (argv: readonly string[], env: Readonly<Record<string, string>>) => Promise<Ran>;

Expect

interface Expect {
  exit: number | readonly number[];
  jsonShape?: readonly string[];
  fields?: Readonly<Record<string, Match>>;
  stderrRules?: readonly StderrRule[];
  maxMs?: number;
  maxBytes?: number;
  text?: { has?: readonly string[]; not?: readonly string[] };
}

Finding

One check's result: one line of findings.jsonl (flow-review/0).

interface Finding {
  flow: string;
  step: string;
  surface: Flow["surface"];
  cell: null;
  check: string;
  pass: boolean;
  measured?: unknown;
  threshold?: unknown;
  selector?: string;
  evidence?: string;
  fix?: string;
  ms: number;
  rubric?: Rubric;
  known?: string;
}

Flow

interface Flow {
  id: string;
  surface: "agent-cli" | "agent-http";
  title: string;
  steps: readonly Step[];
  rubric: readonly Rubric[];
  source?: string;
  readOnly?: boolean;
  env?: Readonly<Record<string, string>>;
  known?: Readonly<Record<string, string>>;
}

FlowReport

interface FlowReport {
  schema: "flow-review-report/0";
  summary: {
    checks: number;
    pass: number;
    fail: number;
    unknownFail: number;
    knownPassing: { flow: string; step: string; known: string }[];
    flows: { flow: string; pass: boolean; checks: number; failed: number; known: number }[];
  };
  findings: Finding[];
  attachments: { kind: "ouro-ax-xray"; path: string; claimSetHash: string | null; checkerVersion: string | null; conformant: boolean | null }[];
}

FlowSuite

An adopter's flows and how to run them.

interface FlowSuite {
  tool: string;
  flows: readonly Flow[];
  cli: Driver;
  start(dir: string): Promise<Mock>;
  guard(mock: Mock): void | Promise<void>;
  state?: readonly string[];
  brandNames?: readonly string[];
}

Harness

A request to the mock's control port, such as a reply arriving; it is set-up, not checked.

interface Harness {
  path: string;
  body?: unknown;
}

HarnessStep

interface HarnessStep {
  id: string;
  harness: Harness;
}

inProcess

A driver that runs the tool's own entry in this process (fast): main(argv, { stdout, stderr, env }) returns the exit code. stderr's write calls its callback, so a tool that waits for an announcement to flush before acting goes on.

function inProcess(main: (argv: string[], io: { stdout: { write(s: string): boolean; }; stderr: { write(s: string, done?: ((error?: Error | null | undefined) => void) | undefined): boolean; }; env: Record<string, string>; }) => number | Promise<number>): Driver

listedCommands

The command words the overview lists: from help --json (the commands not hidden, with their aliases), else from --help text.

function listedCommands(help: Ran, overview: Ran): Set<string>

main

The command: run a suite module, print one line per flow and per failure, write the files. Exits 1 on a failure no known names, and on a known defect that now passes.

function main(argv: readonly string[], io?: { stdout: { write(s: string): unknown; }; stderr: { write(s: string): unknown; }; }): Promise<number>

Match

A value a JSON field must have: the value itself, a pattern (ignoring case), one of several values, or a type.

type Match = string | number | boolean | null | { re: string } | { oneOf: readonly (string | number | boolean | null)[] } | { type: "string" | "number" | "boolean" | "array" | "object" | "null" };

matches

function matches(value: unknown, m: Match): boolean

Mock

A started mock: where its API and control port are, what the CLI's calls need in their environment, and how to stop it.

interface Mock {
  url?: string;
  control?: string;
  env?: Readonly<Record<string, string>>;
  vars?: Readonly<Record<string, string>>;
  stop(): void | Promise<void>;
}

Ran

What a call gave: its exit code (or HTTP status), stdout, stderr and how long it took.

interface Ran {
  code: number;
  out: string;
  err: string;
  ms: number;
}

report

The flow-review/0 report: a summary per flow, the findings, and the X-ray's report as an attachment. A step marked known that passes is listed in knownPassing, so the mark is removed once the defect is fixed.

function report(findings: readonly Finding[], xray?: { path: string; doc: unknown; } | undefined): FlowReport

RequestStep

A request to the mock's HTTP API; exit is the status.

interface RequestStep extends StepBase {
  request: { method: "GET" | "POST" | "PUT" | "PATCH" | "DELETE"; path: string; body?: unknown };
}

Rubric

An adopter's agent flows as data, run in CI without a model: a flow is the calls a good agent makes for one task, and the runner checks the calls' exit codes, JSON shapes and fields, error text, time, size and voice. The findings are in the flow-review/0 format (findings.jsonl, one check a line, and report.json), with an X-ray report attached when given. The X-ray checks what every command must do; flows check what a task needs, and each finds what the other can't.

node lib/ouro/ax/flows.ts run <suite module> [--out report] [--flows a,b] [--slack n] [--xray report.json]
type Rubric = "discoverable" | "predictable" | "honest" | "forgiving" | "fast" | "beautiful";

runFlows

Runs the flows in order against one started mock and returns the findings. The suite's guard runs first, before any call; a flow's harness steps set up what happens next, and its saved values fill later steps' {name}.

function runFlows(suite: FlowSuite, mock: Mock, options: { dir: string; slack?: number | undefined; flows?: readonly Flow[] | undefined; }): Promise<Finding[]>

RunStep

A call to the CLI: the words after the tool's name; {name} is a saved value.

interface RunStep extends StepBase {
  run: readonly string[];
}

runSuite

Starts the suite's mock in <out>/work, runs the flows, writes findings.jsonl and report.json to out, and stops the mock.

function runSuite(suite: FlowSuite, options: { out: string; only?: readonly string[] | undefined; slack?: number | undefined; xray?: string | undefined; }): Promise<FlowReport>

spawned

A driver that runs a built or installed CLI as its own process: the command's words, then the step's.

function spawned(command: readonly string[], timeoutMs?: number): Driver

StderrRule

Nothing on stderr ("empty"), or one line ("oneLine") that names what to do instead ("namesFix"), without a stack trace ("noStack").

type StderrRule = "empty" | "oneLine" | "namesFix" | "noStack";

Step

type Step = RunStep | RequestStep | HarnessStep;

unlisted

The command words a flow runs successfully that the overview does not list: an agent reading --help alone won't find them.

function unlisted(flow: Flow, listed: ReadonlySet<string>): string[]
Source on GitHubThis page as Markdown