Wyrd docs v0.4.1 Use with your coding agent Reference GitHub

Build and verify / Recipes

Recipes

Pick the check you need, bind its Verifier under verified_by to what it checks (a Service, one of its Agents, Models, or Data, or a standalone Agent), and call it from your code. Each recipe’s Python declares support-reply.yaml, a Service that binds the recipe’s Verifier, as in Run your first verification.

To checkUseKind
Fields of an answerAssertionsassert
That your code took the right stepsTracetrace
Logic an operator cannot expressPython checkcode
Meaning, tone, or policyModel judgeagent
An answer against a documentMulti-stepextract and code steps, then assert
An answer or decision against a policy you upload onceUploaded filesagent with a workspace
A whole conversation, not one replyGrouped sessionassert over a group
Inputs or outputs shifting over timeDriftdrift

Steps and checks explains each kind. Start with the cheapest check that catches the failure. A model judge costs a model call per requirement; an assertion costs nothing.

Assertions

Compare fields of a JSON value with an operator.

reply-rules.yaml yaml
apiVersion: wyrd/v1
kind: Verifier
metadata: {name: reply-rules, version: 1.0.0}  # the name your code calls
spec:
  description: A support reply never promises a refund and stays within the refund limit.
  inputs:
    answer: {type: json}  # one named slot for the evidence; json takes any JSON value
  requirements:  # each requirement gets its own verdict
    - id: no-guarantee
      statement: The reply never promises a guaranteed refund  # shown in results
      # path selects a value from the evidence; operator compares it with expected
      check: {kind: assert, path: inputs.answer.text, operator: not_contains, expected: guaranteed refund}
    - id: within-limit
      statement: The refund offered is at most 100 dollars
      # operators that take parameters are written as {kind: ..., ...}
      check: {kind: assert, path: inputs.answer.refund, operator: {kind: in_range, min: 0, max: 100, inclusive: true}}

Comparison operators lists every operator.

Trace

Check the spans your code recorded: that a step ran, in order, with the right attributes.

looked-up-policy.yaml yaml
apiVersion: wyrd/v1
kind: Verifier
metadata: {name: looked-up-policy, version: 1.0.0}
spec:
  description: The reply was drafted after looking up the refunds policy.
  inputs:
    trace: {type: trace}  # a trace ID; Wyrd reads the trace's stored spans
  requirements:
    - id: looked-up-first
      statement: The first span is the policy lookup
      # spans are ordered by start time, so spans[0] is the first one
      check: {kind: trace, on: inputs.trace, span_selector: "$.spans[0].name", operator: equals, expected: lookup-policy}
    - id: read-refunds
      statement: The lookup read the refunds policy
      # attributes are the ones your code set on the span
      check: {kind: trace, on: inputs.trace, span_selector: "$.spans[0].attributes.policy_id", operator: equals, expected: refunds}

Python check

Run your own function when no operator fits. It returns a verdict with a reason and a citation.

names-the-policy.yaml yaml
apiVersion: wyrd/v1
kind: Verifier
metadata: {name: names-the-policy, version: 1.0.0}
spec:
  description: A support reply points the customer to the refund policy.
  inputs:
    answer: {type: json}
  requirements:
    - id: names-the-policy
      statement: The reply names the refund policy
      check:
        kind: code
        entrypoint: check  # the function to call
        inputs: {answer: inputs.answer}  # passed to it as keyword arguments
        source:
          inline: |
            def check(answer):
                named = "policy" in answer["text"].lower()
                # Return a verdict: outcome, reason, and the evidence it cites.
                return {
                    "outcome": "passed" if named else "failed",
                    "reason": "names the policy" if named else "does not name the policy",
                    # slot names an input; json_pointer points inside it
                    "citations": [{"slot": "answer", "locator": {"json_pointer": "/text"}}],
                }

The function runs sandboxed, with no files, network, or environment.

Model judge

Ask a model whether a claim holds. A pass must cite the evidence it relied on, or it counts as inconclusive.

follows-policy.yaml yaml
apiVersion: wyrd/v1
kind: Verifier
metadata: {name: follows-policy, version: 1.0.0}
spec:
  description: A model judges whether a support reply follows the refund policy.
  inputs:
    answer: {type: json}
  requirements:
    - id: follows-policy
      statement: The reply follows the refund policy and never promises more than it allows
      check:
        kind: agent
        agent: ./policy-judge.yaml  # the judge, below
        inputs:  # fill the judge's prompt placeholders
          claim: requirement.statement  # this requirement's statement
          answer: inputs.answer.text
policy-judge.yaml yaml
apiVersion: wyrd/v1
kind: Agent
metadata: {name: policy-judge, version: 1.0.0}
spec:
  execution_route: {kind: wyrd_gateway, deployment: openai}  # a deployment in your LLM gateway
  run_config: {max_iterations: 1}  # one model call, no tools
  prompt:
    provider: openai
    model: gpt-4o
    messages:
      - role: user
        content: |
          Refunds of up to 100 dollars are approved within 30 days; larger refunds need a manager.
          Claim: {{claim}}  # filled from the check's inputs
          Reply: {{answer}}
          Decide whether the claim holds for the reply, and cite the reply.
    # The judge answers with a Verdict: an outcome, a reason, and citations.
    response_format: verdict

The judge’s model calls go through the LLM gateway with your provider key. Verdicts and citations explains the locators a citation can use.

Return scores and labels with the verdict

Declare a schema under data and the judge returns that object beside its verdict. The server checks it against the schema; output that does not match is inconclusive and keeps no data.

policy-judge.yaml (excerpt) yaml
    response_format:
      kind: verdict
      data:
        type: object
        properties:
          score: {type: number, minimum: 0, maximum: 1}
          category: {type: string}
        required: [score, category]
        additionalProperties: false

A code check declares the same response_format beside entrypoint, and its function returns data with its verdict. Each requirement carries its own data, which stays absent when its check declares none. Read it as JSON, or ask for your own type:

python
from pydantic import BaseModel

class JudgeData(BaseModel):
    score: float
    category: str

result = run.observe.verify_typed("follows-policy", JudgeData, answer={"text": "Refunds up to 100 dollars follow our policy."})
data = result.requirements[0].get("data")  # a JudgeData, or None when the check returned none

Data that does not fit your type raises WYRD_SDK_422_VERIFICATION_DATA_INVALID naming the requirement; the stored result is unchanged and nothing runs again. Its path is always "", the root of that requirement’s data, because a deeper path could name a key the check returned.

Multi-step

Prepare evidence before checking it. Here a step reads the policy PDF, a second step compares the refund with the limit it states, and a requirement checks the result.

within-policy.yaml yaml
apiVersion: wyrd/v1
kind: Verifier
metadata: {name: within-policy, version: 1.0.0}
spec:
  description: The refund offered is within the limit the policy PDF states.
  inputs:
    answer: {type: json}
    policy: {type: media, formats: [pdf]}  # a file, uploaded by your code
  steps:  # run before the requirements; later steps read earlier outputs
    - id: policy-text
      action: {kind: extract, from: inputs.policy}  # PDF to text, page by page
    - id: compare
      action:
        kind: code
        entrypoint: compare
        # steps.<id>.output reads a step's result; this step waits for policy-text
        inputs: {pages: steps.policy-text.output.pages, answer: inputs.answer}
        source:
          inline: |
            def compare(pages, answer):
                text = " ".join(page["text"] for page in pages)
                # "Refunds of up to 100 dollars are approved ..."
                limit = int(text.split("up to ")[1].split(" ")[0])
                return {"within_limit": answer["refund"] <= limit}
  requirements:
    - id: within-limit
      statement: The refund offered is within the policy limit
      check: {kind: assert, path: steps.compare.output.within_limit, operator: equals, expected: true}  # reads the step's output

Steps can also be agent actions, which read files with built-in tools. Actions lists each kind.

Uploaded files

Upload a file or directory once as an Artifact Card, then mount the exact files a judge needs into its workspace.

bash
wyrd artifact upload ./handbook --name staff-handbook --version 1.0.0 --extract

The command prints the exact Active reference, UID included. Each file keeps its path relative to ./handbook, byte for byte. Nothing is parsed unless you pass --extract; then each supported document P is also published as extracted/P.json before the Card is declared, and spec.metadata.extraction records the report.

The version follows the ordinary rules: uploading the same bytes to a pinned version returns the same Card, changed bytes under it are refused with WYRD_REGISTRY_409_SPEC_DRIFT, and --bump patch without --version publishes the next version. List what a version holds from the server:

bash
wyrd artifact files --kind Artifact --name staff-handbook --version 1.0.0

The SDKs run the same upload:

LanguageUploadInventory
PythonCards().upload_artifact(source, {"name": ..., "version": ...}, version_bump=None, spec=None, extract=False)Cards().list_artifacts(card_ref)
RustCards::upload_artifact(source, metadata, spec, extract)Cards::list_artifacts(selector)
TypeScriptcards.uploadArtifact(source, { metadata, spec, extract })cards.listArtifacts(ref)

metadata takes labels and annotations too, and spec defaults to {artifact_kind: file}.

A refused upload declares no Card. A WYRD_SPEC_400_VALIDATION refusal names its rule in details:

You seeFix
Rule artifact_source_regularThe source, or an entry under it, is a symbolic link or another non-regular file. Replace or remove the entry details.source names
Rule artifact_source_emptyThe directory holds no regular files. Upload a directory with at least one file
Rule extraction_refusedWith --extract, a supported document is malformed or over the extraction limits; details.reason says which. Fix or remove it, or upload without --extract
Rule extraction_output_collisionWith --extract, an uploaded file sits at or around extracted/P.json. Rename or remove the file details.collides_with names
Rule extraction_reportWith --extract, spec already sets metadata.extraction. Remove it; the upload writes the report
WYRD_LOADER_400_INVALID_ENVELOPEThe source is missing or unreadable, or a file changed during the upload. Check the path and run the upload again

Upload is not atomic. Interrupted after it starts sending, it can leave the version declared but not yet Active; run the same upload again, which follows the version rules above.

Mount the uploaded files by exact reference in an Agent’s workspace, here the extraction of one handbook:

handbook-review.yaml yaml
# Judges a question against the uploaded staff-handbook 1.0.0, mounted at a
# path the judge's prompt names.
apiVersion: wyrd/v1
kind: Verifier
metadata:
  name: handbook-review
  version: 1.0.0
spec:
  description: The staff handbook answers the question, with a cited line.
  inputs:
    question: {type: json, schema: {type: string}}
  requirements:
    - id: handbook-answers
      statement: The handbook answers the question
      check:
        kind: agent
        agent: ./handbook-judge.yaml
        inputs: {question: inputs.question}
        workspace:
          /inputs/handbook.json:
            artifact:
              card: {kind: Artifact, name: staff-handbook, version: 1.0.0}
              path: extracted/handbook.md.json

Without path, every file of the version is mounted. The judge reads with fs.read and cites the attachment_<n> slot the tool reports. Uploading a new version never changes what this Verifier mounts. Agent tools and authorization lists the file tools and the grants each call needs.

Bind the Verifier to the Service it checks:

yaml
verified_by:
  - verifier: ./handbook-review.yaml

Then call it with the evidence the judgment rests on, here the question:

python
from wyrd import WyrdState

run = WyrdState.declare("support-reply.yaml").run()

# The judge's fs.read call runs as your credential, which needs a tool:invoke grant for fs.read.
result = run.observe.verify("handbook-review", question="How do outages escalate?")
judged = result.requirements[0]
print(judged["outcome"], judged["reason"])  # the reason quotes the handbook line it found
for citation in judged.get("citations", []):
    print(citation["slot"], citation["locator"])  # attachment_<n> and the cited lines

A pass without a citation counts as inconclusive. Put exact rules, such as an amount limit, in assert or code requirements of the same Verifier, so they never depend on the model; decide combines every requirement’s verdict. Your application reads the verdict and decides what to do, such as holding a payment. verify waits for the judge; submit queues it instead, as verify and submit explains.

Grouped session

Judge a whole conversation after it ends. Each reply is verified on its own; a session Verifier reads every reply’s evidence and outcome once the group goes quiet.

whole-conversation.yaml yaml
apiVersion: wyrd/v1
kind: Verifier
metadata: {name: whole-conversation, version: 1.0.0}
spec:
  description: Every reply of a conversation passed reply-rules.
  inputs:
    # Every reply-rules answer in the group, in order.
    replies: verifiers.reply-rules.inputs.answer
    # Every no-guarantee outcome in the group: passed, failed, or inconclusive.
    outcomes: verifiers.reply-rules.requirements.no-guarantee.outcome
  # List order: recorded_at (the default) is the order Wyrd received each reply;
  # inputs.replies.<field> orders by a field of each reply instead.
  order_by: recorded_at
  requirements:
    - id: every-reply-passed
      statement: No reply in the conversation promised a guaranteed refund
      # each checks every item; one failed reply fails the conversation
      check: {kind: assert, each: "inputs.outcomes[*]", operator: equals, expected: passed}

Bind it beside reply-rules with a group_end Trigger:

yaml
verified_by:
  - verifier: ./reply-rules.yaml
  - verifier: ./whole-conversation.yaml
    runs_on: {kind: group_end, after: 10m}  # judge 10 minutes after the last reply

New replies in the group push the session check back. Its verdict is stored in bifrost.verification.results with the group_id.

Drift

Compare recent values with a baseline on a schedule. Your code records samples; the schedule judges the window.

latency-drift.yaml yaml
apiVersion: wyrd/v1
kind: Verifier
metadata: {name: latency-drift, version: 1.0.0}
spec:
  description: Request latency stays distributed like the healthy baseline.
  inputs:
    latency: {type: samples, features: [latency]}  # values your code records over time
  requirements:
    - id: latency-stable
      statement: Request latency is distributed like the healthy baseline
      check:
        kind: drift
        on: inputs.latency
        method: Psi  # Population Stability Index: how far a distribution moved
        signal:
          kind: Distribution
          baseline_ref: {kind: Data, name: latency-baseline, version: 1.0.0}  # what healthy looks like
          features: [latency]
        condition: {kind: Statistical}
        profile:
          kind: Psi
          binning_strategy: {kind: EqualWidth, n_bins: 10}  # compare across 10 equal-width buckets
          threshold: {kind: Fixed, value: 0.25}  # fail when PSI is above 0.25

Bind it with a schedule:

yaml
verified_by:
  - verifier: ./latency-drift.yaml
    runs_on: {kind: schedule, cron: "0 0 * * *"}  # judge the day's samples at midnight

A drift Verifier refuses verify; only the schedule judges it.

Next

How verification works explains steps, verdicts, and where each check runs.

Last updated