tensteps.yml

One file at the root of your repo describes what to test. Tensteps also accepts tensteps.yaml (not both). Check it any time with npx tensteps@beta validate.

Add this first line for autocomplete and checking in your editor:

# yaml-language-server: $schema=https://tensteps.ai/schema/v1.json

A complete example

version: 1

defaults:
  max_steps: 30

studies:
  - name: checkout
    objective: "Buy any pair of shoes under $50 and get it shipped to your home address"
    success:
      url_matches: "/order/confirmation"
    sessions: 8
    guardrails:
      payment: test_card
      forbid_actions: [delete]
    matrix:
      device: [desktop_chrome, android_low_end]
      network: [wifi, slow_3g]

  - name: signup
    objective: "Create a new account"
    start_path: /signup
    success:
      text_visible: "Welcome aboard"
    population: low_tech_comfort

follow_up:
  - key: frustration
    type: rating
    scale: 5
    prompt: "How frustrating was that? (1 = not at all, 5 = very)"

triggers:
  pull_request: { sessions: 6, branches: [main] }

Top level

| Key | Required | What it is | | --- | --- | --- | | version | yes | Always 1. | | studies | yes | 1 to 25 studies. Each is one goal tested by its own personas. | | defaults | no | Values every study uses unless it sets its own. Any study field except name, objective and success. | | follow_up | no | Questions every persona answers after their session. | | triggers | no | How many personas a pull request gets, and which pull requests run. |

Studies

| Key | Default | What it is | | --- | --- | --- | | name | required | Short id: lowercase letters, numbers and underscores. | | objective | required | The goal in plain words, as you'd say it to a person. | | success | required | How Tensteps knows a persona made it. See below. | | start_path | / | Where personas start, relative to the site's URL. | | sessions | 50 | Personas per run, 1 to 1,000. A trigger's sessions overrides it. | | max_steps | 40 | Steps a persona may take before the session ends, up to 200. | | timeout | 5m | Real-person time a persona will spend, like 90s or 5m (up to 30 minutes). | | population | general_us | Who the personas are. See below. | | auth | none | How personas log in. See below. | | guardrails | none | What personas must not do. See below. | | matrix | desktop Chrome, Wi-Fi, en-US | Devices, networks and browser languages to spread personas across. | | fail_if | success_rate_drop: "10%" | Thresholds for comparing against a baseline. Saved now; used once baselines ship. |

success

Exactly one of:

| Rule | Passes when | | --- | --- | | url_matches: "/order/confirmation" | The page URL matches this regular expression. | | text_visible: "Welcome aboard" | This text is visible on the page. | | element_visible: "#receipt" | An element matching this CSS selector is visible. | | all: [ ... ] | Every rule in the list passes. | | any: [ ... ] | At least one rule in the list passes. |

population

| Preset | Who | | --- | --- | | general_us | US adults 18+, approximate census mix of age, gender and region. | | low_tech_comfort | Adults who avoid unfamiliar interfaces; skews older. | | low_patience | Adults who abandon quickly at friction, errors or slow pages. | | screen_reader | Blind or low-vision adults navigating with a screen reader. | | non_native_english | Adults reading English as a second language. |

Preview any population with npx tensteps@beta personas -p screen_reader. The same seed always gives the same persona, so runs are repeatable.

auth

• none: personas arrive logged out.
• { email: "test@example.com", password: "secret:TEST_PASSWORD" }: personas log in with this account when the site asks.
• { cookie: "secret:SESSION_COOKIE" }: personas arrive already logged in with this cookie.

secret:NAME values are read from the environment variable NAME. Never put a password in the file.

guardrails

| Key | What it does | | --- | --- | | payment: test_card | Personas pay with the test card 4242 4242 4242 4242, so use payments in test mode. | | forbid_paths: ["/admin/**"] | Pages personas can't open. * matches within a path segment, ** across segments. | | forbid_actions: [delete, purchase, send] | Buttons and links personas can't use, by what they say ("Delete account", "Place order", "Send"). |

matrix

Personas are spread evenly across every combination.

| Key | Values | | --- | --- | | device | desktop_chrome, desktop_safari, iphone_15, android_mid, android_low_end, tablet | | network | wifi, 4g, slow_3g, offline_flaky | | locale | Browser languages, like en-US or es-MX |

follow_up

Each question has a key (snake_case), a prompt, and a type:

| Type | Needs | | --- | --- | | rating | scale: 2 to 10 | | single_select | options: at least 2, each { key, label } | | text | nothing more |

triggers

pull_request controls runs on pull requests:

| Key | What it does | | --- | --- | | sessions | Personas per study on a pull request. | | branches, branches_ignore | Only PRs into (or not into) these branches. | | head_branches, head_branches_ignore | Only PRs from (or not from) these branches. | | labels | Only PRs with at least one of these labels. | | paths, paths_ignore | Only PRs that change (or don't only change) matching files. |

Without a triggers block, a pull request gets 30 personas per study. If the block exists but has no pull_request, pull requests aren't tested. push_main and nightly are accepted and saved for scheduled runs, which aren't automated yet.