Semantic linting with Jev

This rule shows how a semantic lint fails on the broken function and passes on the guarded function.

In the same way you use a linter like ruff or flake8 to check your code meets style requirements, you can use Perch to check your code meets semantic requirements. For example, in this page we show a rule “Refunded orders cannot be refunded again.” and how Perch with Jev can find the broken function without a guard (80% probability), and then find the guarded function passes. All of the commands shown below and their results are presented for you to try.

Perch for agents: how to add a check

Agentic tools are notoriously difficult to instruct. They might follow your instructions on one edit, and then fall off on the next. One approach is to have the agent run a check after it has made changes. Here we show how to have the agent run a Perch check after it has changed the refund function. The function below correctly refunds an order once, but if it is called again it will refund the order twice.

def refund(order, payment):
    payment.refund(order.total)
    order.status = "refunded"
    return order

First, we write the requirement into a perch.yaml file, scoping it to the code we want to check using where and each.

- name: refund-once
  where: refund.py
  each: method
  ensure: Refunded orders cannot be refunded again.

Then we have the agent run perch check to see if the code meets the requirement.

$ perch check refund.py::refund --rules refund-once
refund.py:1  refund
1 check, 1 broken.

  Confidence  Rule         Description
         80%  refund-once  Refunded orders cannot be refunded again.

perch check checks uncommitted code and returns exit code 3, which we can use to have the agent inspect the findings. If it fixes the problem, then the same command would return 1 check, nothing to report. The full example including the correct function is at the bottom of this page.

What is semantic linting?

A semantic lint is a linter that can answer semantic questions about your code. For example, it can tell you if there is a code path that meets a particular requirement. In this example, both functions include the word “refund” but only one guards against double refunding.

In Perch, semantic lints are plain English requirements scoped to pieces of matching code, using where and each. When you run a check, Perch uses TypeSafe’s Jev to evaluate the method and return a probability. Rather than asserting that the function is correct or incorrect, Perch returns a finding for you to inspect. You can always write a regression test that calls refund twice to be sure.

How does Perch use Jev?

For each method, Perch sends the method itself and all the functions that the method has access to, including imports and other functions down the call graph. This gives Jev the context it needs to answer questions about whether an error bubbles up, or whether a refund happens before it is guarded. These requests are the same as for built-in defect and security questions.

Jev returns typed answers, including probabilities, categories and severity scores. Perch turns these answers into findings that you can filter, inspect or use to fail your CI. For more information on the format of the request and limits to the context, see the scan reference.

Checking your agent’s work: documentation

Rules can apply to any file, so you can use them to check that your agent has written good documentation. It is important to be specific in your rules: rather than asking for “good documentation”, you might ask for the first six headings to be judged.

In the example below, the heading “Getting started on your journey” does not meet the requirement, but “Installation” does.

A note on variability

If you were to send the same request again, the model might give a different answer. To avoid this, perch scan will reuse previous answers if the code, context, questions and model configuration are the same. If you want to ask the same question again, you can use perch check instead.

When you are writing rules for agents, you should test them on code that you know meets or doesn’t meet the rule. If the rule doesn’t differentiate between the two, you should try writing a different rule. Remember that you can always write regression tests to be sure that your code is correct.

For an existing application, the business-rule example saves Frappe CRM’s validation requirements as checks for later agent edits. The permission-check example applies the same workflow to an ownership boundary.

Getting started with agents

If you’re new to Perch, you can follow the quick start guide. To add Perch instructions to your agent of choice, you can run:

perch setup codex

We also have instructions for Claude Code, Cursor and pi. For more information, see the agent setup guide. Perch sends the relevant code to TypeSafe for analysis and saves the findings locally as part of each scan.

Full example

The guarded function returns early when the order is already refunded:

def refund(order, payment):
    if order.status == "refunded":
        return order
    payment.refund(order.total)
    order.status = "refunded"
    return order

The same check now passes:

$ perch check refund.py::refund --rules refund-once
refund.py:1  refund
1 check, nothing to report.

Both functions and the command output were captured on September 20; the refund example walks through the same rule step by step.