Make your agent fix its own documentation

Put heading and prose requirements in perch.yaml, then have your agent run perch check after editing Markdown. The demo results show vague headings failing and their replacements passing.

Heading rules

- name: docs-headings-name-things
  where: "docs/**/*.md"
  ensure: >-
    Read the first six headings. Each names a task or a topic.
    Vague narrative headings such as "What we learned" or
    "One issue, opened up" break this rule.

Literal prose

- name: docs-no-anthropomorphism
  where: "docs/**/*.md"
  ensure: >-
    Read the first four prose paragraphs after front matter.
    Describe program behavior literally. A question having
    its own business, a result speaking for a method, or a
    rule deciding to stop hedging breaks this rule. Statements
    about a model generating text or returning a probability
    are literal behavior.

Replace “the first pass speaks for the method’s shape” with “the first pass checks method structure.”

Command output

- name: docs-show-output
  where: "docs/**/*.md"
  ensure: >-
    Find the first fenced block that runs a real perch command.
    Its output appears in that block or the next one. Skip
    installation commands, environment setup, and command
    syntax with placeholders. A command without output breaks
    this rule. A page with no such command satisfies it.

Copy output from the command you ran, including failures.

Check an edit

After setting up Perch, save the three rules and check the edited file:

perch check docs/install.md \
  --rules docs-headings-name-things,docs-no-anthropomorphism,docs-show-output

check reads the current file from disk. Put this step in the agent’s instructions after documentation edits. Exit 3 indicates findings to inspect; exit 1 or 2 means the check could not complete.

Compare controls

Try a known good heading such as “Installation” and a bad one such as “A journey into getting started.” Keep the rest of the page fixed. Repeat the comparison for a literal sentence and its anthropomorphic version.

If a rule gives both passages similar scores, reword the question before relying on it. Force LLMs to write better documentation describes the historical control results that led to bounded rules.