Skip to content

Python review checks

Goal

Decide whether a change may be offered as a pull request, and when it may not, say exactly which check failed and where. The review runs the toolchain gates in a blocking order and then makes the judgements a tool cannot, returning a pass or block verdict with the violations behind it.

The plugin owns the generic review phase — when it runs, the verdict artifact, and that a block stops the pull request. This recipe owns the Python-specific part: which gates run, in what order, and what a linter structurally cannot check.

Opinion

The gates run in a blocking order and the first failure stops the rest. A change that does not format cannot usefully be type-checked; a change that does not import cannot usefully be tested. Running everything and reporting a wall of failures hides which one is the cause.

A clean tool run is not a review. Every rule below the toolchain exists because no linter expresses it: whether logic crept into a console script, whether a dependency was added that the design did not decide, whether an exception is caught and dropped.

A swallowed exception is a blocking finding, not a style note. except Exception: pass, or a catch that logs and continues where the caller needed the failure, is how a tool reports success having done nothing.

Shell and path sinks are read directly. subprocess with shell=True and any interpolated value is an injection. A path built from external input and opened without being resolved and checked against its intended root is a traversal. Both are read in the diff, not inferred from a tool's silence.

A dependency the design did not decide blocks. Not because the package is bad, but because the posture was a decision and this is where it is enforced.

# noqa and # type: ignore need a reason. An unexplained suppression is a check deleted. With a reason it is a decision; without one it is an unreviewed exemption.

The project's pyproject.toml names the gates; where it names none, this recipe does. Python has no gofmt — no single formatter and linter that ships with the interpreter and settles the question — and the tools that fill the gap have genuinely turned over: Black and Flake8 were the answer for years and are now largely served by Ruff's ruff format and ruff check, and the type checker is an open field rather than a default: mypy and pyright have both been production tools for years, Meta's pyrefly reached a stable 1.0 and is the default checker on Instagram's codebase, and Astral's ty is still beta on 0.0.x with no stable API. So a review that hard-codes a toolchain will, on a real project, run tools the project deliberately did not choose and report a wall of findings the project already decided against. Deferring to [tool.*] in pyproject.toml is the correct default, and it is a position, not an omission.

That deferral has one limit, and this is where it is closed. A project with no declared toolchain gets nothing from a rule that says "run what the project declares" — the gates would resolve to no commands and the review would pass on silence. So where pyproject.toml declares no formatter, linter or type checker, run this floor and record that it was the recipe's choice rather than the project's: ruff format --check for formatting (it exits non-zero on a file that would be reformatted and writes nothing, which is what a review needs), ruff check for linting, mypy for typing at the strictness the design recorded, and pytest for tests. Ruff earns the floor by being one binary configured from one pyproject.toml section for both jobs; mypy earns it by being maintained in the python organisation and configured from [tool.mypy], where strict is a single switch — not by being the fastest, which it is not. A project that has chosen otherwise overrides all of this, and a project that has chosen pyright or pyrefly has chosen well; the floor exists for the project that has chosen nothing.

The dependency audit is pip-audit, and that one is named outright. No churn argument applies to it: it is maintained in the pypa organisation, it reads the Python Packaging Advisory Database through the PyPI JSON API, and it audits an environment, a requirements file, or a project directory directly. There is no equivalent second option whose existence would make naming it presumptuous, so it is named rather than deferred.

.py is not in the change-scoping floor, which is why this recipe declares it. The review command scopes its change-scoped gates by file extension, unioning the framework's declared extensions onto a framework-neutral floor of .php .js .mjs .cjs .ts .tsx .vue. No Python extension is in that floor, so without the ## Code-quality extensions declaration below, a pure-Python change would filter down to an empty file list and every change-scoped gate would skip itself — a run that looks clean because it examined nothing.

This framework declares no change-impact globs, and the absence is deliberate. Change-impact globs route a changed file to the e2e and visual-regression gates. A Python CLI binds neither phase — it has no rendered or browser surface those harnesses target — so declaring globs here would route to gates that cannot fire. A CLI's end-to-end shape is a test tier, not a phase: it lives in the implement recipe as the entry-point tier and is checked here under the test gate.

Preconditions

  • The change is complete and committed or stageable, so the diff scope is knowable.
  • The project's toolchain is installable, and its configuration lives in pyproject.toml.
  • The architecture artifact is available when the change had a design phase.

Input contract

code_path: string             # absolute path to the project root (the dir with pyproject.toml)
changed: [string]             # optional; the changed files or diff scope to review;
                              # when absent, derived from version control
architecture: string          # optional; path to the component map the change should conform to
new_code_only: boolean        # optional; default true. Scope the conformance checks to the
                              # change rather than to the whole project's history.

Sequence

If invoked in dry-run mode, report the gates and the commands it would run, and change nothing. Dry-run is required.

Steps 1 to 3 run the tool the project declares in pyproject.toml. Where it declares none, run the floor named in Opinion and record in the verdict that the tool was the recipe's choice, so a reader can tell an enforced project standard from a substituted default.

  1. Formatter, over the diff scope, in a mode that does not write. It must list nothing. A formatting difference is a block, because everything after it reads against a moving target. Where the project declares no formatter, ruff format --check: it exits non-zero on any file it would reformat and leaves the working tree alone, which is the only shape a review may use.

  2. Linter, over the diff scope. It must report nothing. Where a rule is suppressed, the suppression carries a reason; an unexplained # noqa is itself a finding. Where the project declares no linter, ruff check — and without a fix flag, because a review that repairs what it found has destroyed the evidence.

  3. Type checker, at the strictness the design recorded. It must report nothing over the changed scope. A new # type: ignore without a reason is a finding. Where the project declares no type checker, mypy with the design's strictness — which in [tool.mypy] is the strict switch when the design recorded strict.

  4. Tests. They must pass. Where the project declares multiple target Python versions, the review records which one this run used and whether the others were checked.

  5. Dependency audit. Run pip-audit over the project's resolved dependencies and report known advisories that are reachable from this change. An advisory in an unreachable transitive path is recorded, not blocked on. Unlike the gates above, this one is not deferred to the project's declaration: it is named because it has no live alternative to defer between.

  6. Conformance reads over the diff. Each is a read, not a tool: logic added to a console script; a dependency added the design did not decide; an exception caught and dropped; work moved to import time; shell=True with an interpolated value; a path from external input opened without being resolved against its root; a test that shells out where it should import; a public signature left unannotated against the recorded posture.

  7. Return the verdict. Pass, or block with the specific violations — file, line, and which check. A block names what to change; a verdict that says "issues found" is not a review.

Data flow

input:  code_path, changed (or VC-derived), architecture (optional), new_code_only (optional)
step 1: formatter result over the diff scope — clean or the files it would change
step 2: linter result, plus any suppression lacking a reason
step 3: type checker result at the recorded strictness, plus unexplained ignores
step 4: test result, and which Python version ran
step 5: dependency advisories, split into reachable and not
step 6: conformance findings, one per rule, each with file and line
step 7: pass, or block with the violation list
output: verdict and findings, returned to the caller. The plugin's review phase records it.

Code-quality extensions

code_quality_extensions: [".py", ".pyi", ".toml"]

Without this declaration a pure-Python change filters to an empty list against the framework-neutral floor (.php .js .mjs .cjs .ts .tsx .vue) and every change-scoped gate skips itself — a clean-looking run that examined nothing.

.pyi is in because a stub file is the declared public typing surface under PEP 561: an edit to one changes the contract the type-checker gate reads, and it is code in every sense the gates care about even though no runtime executes it.

.toml is the judgement call, and it goes in for the same reason Go declares .mod. pyproject.toml is where the dependency posture, the target versions and the tool configuration are decided, and a change that touches only it — adding a dependency the design did not decide, which is a blocking finding above — would otherwise scope to an empty list and be judged by nothing. The honest cost is that this filter matches extensions, not paths, so it cannot scope in the manifest while scoping out a generated lockfile the way Go excludes .sum: a PEP 751 pylock file is also .toml and will be pulled in. Treat a lockfile diff as evidence of the dependency decision, not as code to judge on its own terms — and note that the same limitation cuts the other way for suppressions, which is the subject of the next paragraph.

# noqa and # type: ignore are the checks most worth watching and are reachable by no extension declaration at all, because they are inline comments rather than files. The unexplained-suppression rule in the Sequence is a read of the diff by the reviewer for exactly that reason; no value this block could hold would reach it.

Check commands

Five rows — coding-standards, static-analysis, security, duplication, design-metrics — each a command or a named statement that this framework has none. {paths} expands to one argv token per file in the caller's file list, relative to the project root. pip-audit audits the current environment's installed distributions, not a file list, so its row carries no {paths}; it exits 1 when it finds a known vulnerability and 0 otherwise.

check_commands:
  - id: coding-standards
    argv: ["ruff", "check", "{paths}"]
  - id: static-analysis
    argv: ["mypy"]
  - id: security
    argv: ["pip-audit"]
  - id: duplication
    absent: >-
      The floor names no duplication tool. pylint's duplicate-code checker exists and is
      a project's choice to enable under [tool.pylint]; this recipe does not choose it.
  - id: design-metrics
    absent: >-
      The floor names no design-metrics tool. Ruff's C901 rule measures cyclomatic
      complexity alone, and only when a project selects it; radon and xenon are a
      project's choice. This recipe selects no rule set for a project.

The first two rows are the floor, not the project's choice. Opinion says the project's pyproject.toml names the linter and the type checker and this recipe names them only where it does not. A block a script reads cannot express that deferral, so the rows carry the floor, and a project that declares pyright, pyrefly or another linter under [tool.*] is one where these two rows run a tool the project did not choose. Record the substitution where it happens, the way step 1 to 3 record which tool ran. ruff format --check has no row of its own: formatting is the first gate in the Sequence and is not one of the three checks this block answers.

The mypy row takes no {paths}, because paths on its command line replace the project's scope. [tool.mypy] files is where a project says what is typed to which strictness, and a path argument overrides it. Run with {paths} against a project declaring files = ["src"] and strict = true, the row handed mypy 2.3.1 the tests as well and got thirteen errors in six test files that the project had scoped out on purpose, while bare mypy on the same tree reported Success: no issues found in 14 source files. mypy also parses whatever it is handed, so a pyproject.toml in the list — this recipe scopes .toml in — reports Name "project" is not defined. Bare mypy reads files and strict from the project and neither problem arises. Where [tool.mypy] declares no files, mypy exits 2 with Missing target module, package, files, or command: unmet, with the reason in the output, and the fix is a line in pyproject.toml. Ruff has neither problem: ruff check 0.15 given a .toml reports All checks passed! and honours per-file-ignores.

pip-audit reads the environment it runs in. Run from the project's virtualenv it audits that environment's installed distributions, prints No known vulnerabilities found on standard error, exits 0, and lists the project's own distribution as skipped because it is not on PyPI. Run from another interpreter it audits that one instead, so the caller's PATH decides what is audited.

Surface commands

Five rows, all absent. Surface commands are the suites review runs over a framework's user-visible surfaces, and a Python CLI has none: its interface is a console script, and its end-to-end shape is the entry-point and subprocess levels chosen in python-cli/test-authoring.md, not a phase. The rows are declared absent rather than left out because a present block with absent rows is how review knows a framework has no surfaces, while a missing block is a heading it could not find.

surface_commands:
  - id: e2e-preflight
    absent: >-
      There is no e2e suite here, so there is no site readiness to check before one runs.
  - id: e2e
    absent: >-
      A CLI's end-to-end coverage is the entry-point or subprocess level the test-authoring recipe chooses, run by the suite row of `python-cli/test-execution.md`; there is no browser surface for a separate e2e suite to drive.
  - id: visual-regression
    absent: >-
      No rendered surface exists to screenshot, so there is no baseline to diff against.
  - id: visual-regression-accept
    absent: >-
      There is no visual-regression suite, so there is no baseline to accept.
  - id: visual-parity
    absent: >-
      This framework names no parity harness.
  - id: visual-parity-accept
    absent: >-
      There is no parity suite, so there is no baseline to accept.

State-awareness contract

The recipe reads the project and runs its declared tools. It changes no file, fixes nothing it finds, and installs nothing beyond what the project already declares. A review that repairs what it is reviewing has removed the thing it was meant to report.

Verifier

After the recipe runs, verify:

  1. The gates ran in the stated order, and the first failure stopped the rest rather than being buried in a combined report.
  2. The formatter, linter and type checker each reported nothing over the diff scope, or the verdict is a block naming what they reported. Each records which tool ran and whether it came from the project's pyproject.toml or from this recipe's floor, and the formatter ran in a mode that writes nothing.
  3. Tests passed, and the verdict records which Python version ran and whether the other declared targets were checked.
  4. The dependency audit ran, and any advisory is recorded as reachable or not.
  5. Every conformance rule was read against the diff, and each finding carries a file and line.
  6. Every suppression introduced by the change carries a reason.
  7. The verdict is pass or block; a block lists the specific violations rather than a summary.
  8. The project is unchanged by the review.

This recipe ships no executable verifier of its own — the checks above are the agent-driven protocol; the plugin's review phase owns the verdict artifact and the block that stops the pull request.

References

External origins (referenced, not authored here)

Source Used for
The project's pyproject.toml The authority on which formatter, linter and type checker run and at what strictness — the declaration steps 1 to 3 defer to before the floor applies
Ruff (ruff format --check, ruff check) The formatting and linting floor for a project that declares neither — one binary for both jobs, and the --check behaviour (non-zero exit on a file it would reformat, nothing written) that makes it usable inside a read-only review
mypy ([tool.mypy], the strict switch) The typing floor for a project that declares no checker, and the single config switch the design's strict posture maps onto
pyright, pyrefly, and ty The alternatives that make the type checker an open field rather than a default, and the reason the gate defers to the project's declaration — pyrefly stable at 1.0 and Meta's default checker, ty still beta on 0.0.x with no stable API
pip-audit (maintained in the pypa organisation) The dependency audit — named rather than deferred; it reads the Python Packaging Advisory Database through the PyPI JSON API
PEP 561 What a declared typing posture and a shipped py.typed marker commit the package to at review time, and why a .pyi stub is part of the reviewable surface
The component map from the design phase The recorded decision the conformance reads in step 6 measure against, rather than reconstructing an intent from the code

Plugin-side generic mechanism (ai-dev-assistant)

The stack-neutral review phase this recipe binds Python into — when the checks run, the verdict artifact they emit, how the ## Code-quality extensions declaration above is consumed for change-scoping, and what a block does to the pull request — is documented in the plugin itself, not duplicated here. The recipe supplies only the Python-specific gate set, the blocking order, and the conformance reads a linter cannot express.

Note that, unlike the PHP CLI recipe under this root, this one does not defer linter execution to the code-quality-tools plugin: that plugin detects Drupal and Next.js projects and lints PHP and JavaScript file extensions, and has no Python arm. The commands above are therefore named directly.