Check Run Agents
Define custom AI checks in markdown. Macroscope runs them on every PR with access to your codebase, git history, and integrations.
Finding correctness bugs is the highest-stakes job of code review — and it's what Macroscope does on every PR out of the box. But code review was never just about bugs.
Every team has rules beyond correctness: style guides, architecture boundaries, event tracking invariants, ticket-to-code traceability. These rules live in tribal knowledge, enforced inconsistently by whoever reviews the PR — or through a patchwork of non-agentic CI/CD workflows.
Check Run Agents fix this.
What are Check Run Agents
A Check Run Agent is a markdown file in your repository that defines a custom AI agent. Drop it in .macroscope/check-run-agents/, write your instructions, and merge it into your default branch. Macroscope runs it as a GitHub check run on every PR that matches the agent's file scope.
Each agent gets its own GitHub Checks automatically. You scope each agent to the files it should care about, give it the tools it needs, and write the instructions like you would in a skill.
The instructions you write and the tools you enable define what the agent does. Most commonly, teams configure agents to take action by posting inline review comments on the PR and updating PR metadata like labels. But agents can take any action their tools allow — create GitHub issues, file Linear tickets, post Slack messages, consult your feature flag system, etc. Set the conclusion to failure and the agent blocks merges — the same way a failing test would.
Anatomy of a check run agent
Every agent is a single .md file with optional YAML frontmatter and a markdown body.
Here's a minimal example — a security review scoped to your API layer:
---
title: Security Review
model: claude-sonnet-4-6
tools:
- browse_code
- web_tools
include:
- "src/api/**"
---
Review this PR for security vulnerabilities. Focus on:
- SQL injection via string concatenation
- Hardcoded secrets or API keys
- Unsafe deserialization
- Missing input validation at API boundaries
Do not flag issues in test files. Do not flag pre-existing
issues that this PR does not introduce or modify.
This file creates a GitHub check run called "Macroscope - Security Review" that runs on every PR that touches files in src/api/, powered by Claude Sonnet 4.6, with access to browse the codebase and search the web for CVE references. PRs that don't touch that path? The agent doesn't run at all.
Here's what each frontmatter field controls (see the full reference for the latest supported fields and tools):
The markdown body after the frontmatter is the agent's instructions. Write them like you'd write a runbook: what to look for, what to ignore, how to format findings, what severity levels mean. The more specific you are, the better the agent performs.
Tools and integrations
The default tools let an agent browse code, read git history, query the GitHub API, and post PR comments. That alone covers style guide enforcement and architecture boundary checks.
But the real power is in the extended tools. Each one you add unlocks a different class of review.
Use cases
The flexibility of "markdown instructions + configurable tools + file scoping" makes the design space large. Here are the patterns we see teams using:
Style guide enforcement — Paste your coding standards into the instructions, scope to the relevant language or directory. We run six of these internally — one per language — each scoped to its own paths. The web team's agent knows about React patterns, barrel imports, and component structure. The backend agent knows about Go error handling, gRPC conventions, and database access patterns.
Architecture boundary enforcement — "The frontend must never query the database directly. All data access goes through the admin service." "Service A must not import from Service B." Rules like these are impossible to enforce with linters but trivial to describe in English.
Ticket requirement verification — Connect your issue tracker, and the agent reads linked tickets to verify the PR implements what was specified. Catches requirement drift before it reaches production — especially valuable when coding agents are writing the implementation.
Event tracking protection — Scope to files that contain analytics calls. The agent detects when tracking events are removed, renamed, or have their properties changed, and alerts the data team before the PR merges.
Content and SEO checks — Scope to your marketing site or docs directory. Verify that blog posts have complete metadata, that image references point to files that actually exist, that SEO titles target real search queries.
Compliance and security — Verify that sensitive code paths include audit logging. Check that new API endpoints have authentication middleware. Flag hardcoded secrets. Connect web_tools to cross-reference against CVE databases.
The pattern is always the same: if a human reviewer would catch it by reading the diff and checking some context, an agent can do it on every PR, every time.
Pricing
Check Run Agents are billed through agent credits. Every workspace gets 1,000 free credits per month. Beyond that, credits cost $0.01 each. The credit formula is straightforward: the raw LLM cost of running the agent, plus a 5% markup, divided by $0.01.
Agent credit usage is visible in your workspace billing settings, with per-agent and per-PR breakdowns available via CSV export.
Get started
Create a .macroscope/check-run-agents/ directory in your repo's default branch. Add a .md file with your first agent's instructions. Macroscope picks it up automatically on the next PR.
