prr-cli

prr is a Node.js CLI for running parallel code reviews and concept brainstorms against OpenAI-compatible language models, including NVIDIA-hosted and OpenAI backends.

Why multiple reviewers?

Different models often produce code that looks similarly correct on the surface, but they reach that result through different internal strengths, assumptions, and blind spots.

prr uses that diversity on purpose: by collecting multiple independent reviews, you increase the chance that one model notices an edge case, risk, or alternative approach that another model would miss.

Requirements

• Node.js 18+
• provider API key set in the environment or a local .env

Install

If you cloned this repository and want to run prr from source:

npm install
node bin/prr.js doctor

From a repo checkout, run commands as node bin/prr.js ....

After the package is published to npm, you can also install the CLI globally:

npm install -g prr-cli
prr doctor

Fastest way to try prr: NVIDIA quick start

Most examples in this repo work well with a free NVIDIA Build account.

1. Create a free account on NVIDIA: https://build.nvidia.com/
2. After login, go to https://build.nvidia.com/settings/api-keys and Generate API Key
3. Copy .env.example to .env, set NVIDIA_API_KEY, then run node bin/prr.js doctor

API Keys

prr reads API keys from the configured environment variable name, defaulting to NVIDIA_API_KEY.

For development you can either export the variable in your shell or place it in a local .env file that is loaded automatically at runtime. .env* is already ignored by git.

See .env.example for a starter file with the expected key names.

Security notes

• API keys are read from the configured environment variable or local .env; they are not written into review payloads or persisted artifacts.
• File-context tool access is restricted to workspace-relative paths and blocks ignored locations such as .git and node_modules.
• See SECURITY.md for the short release-facing security notes.

Examples:

bash / zsh

export NVIDIA_API_KEY='...'
node bin/prr.js doctor

PowerShell

$env:NVIDIA_API_KEY="..."
node bin/prr.js doctor

Windows cmd.exe

set NVIDIA_API_KEY=...
node bin\prr.js doctor

Local .env

NVIDIA_API_KEY=your-key-here

Usage

The examples below assume prr is available on your PATH. From a cloned repo, use node bin/prr.js in place of prr.

prr review src/file.js
prr review src/**/*.js
prr review --changed
prr review --execution sequential
prr review --changed --dry-run
prr review --resume
prr review --resume 2026-03-04T10-05-00-000Z
prr review src/cli.js --dry-run --verbose
prr brainstorm ./concept-brief.md
prr brainstorm ./concept-brief.md --dry-run --verbose
prr brainstorm ./concept-brief.md --no-synthesis
prr brainstorm ./concept-brief.md --config .prrrc.brainstorm.js --json
prr doctor
prr doctor --request
prr review examples/yai-timeout/yai-timeout.js --context-file ./review-context.md
prr review --config .prrrc.js src --json
prr experiment run src/**/*.js --preset smoke
prr auto run --changed --json
prr auto run --changed --strategy recommended

Default behavior:

• expands files, directories, and glob patterns
• runs reviewers sequentially by default
• starts from git diff context when available instead of embedding full files up front
• lets reviewers fetch file contents on demand through tool calls
• prefers concrete bug and edge-case findings over generic advice
• spaces request starts to remain under 40 requests per minute
• retries rate-limited reviewer calls with exponential backoff
• prints Markdown to stdout unless --json is provided

Use prr doctor before a live run when provider failures are unclear. It checks:

• whether the configured API key env name is the default one or a custom override
• whether the key was found from the process environment or .env
• DNS resolution for the configured endpoint host

Use prr doctor --request for a minimal live API probe against the configured OpenAI-compatible endpoint.

Configuration

Project-level configuration can live in .prrrc.js or .prrrc.json. You can also pass an explicit file path with --config.

Example:

module.exports = {
  provider: 'nvidia-free',
  endpoint: 'https://integrate.api.nvidia.com/v1',
  apiKeyEnv: 'NVIDIA_API_KEY',
  contextMode: 'diff',
  reviewers: [
    {
      name: 'Custom Reviewer',
      provider: 'openai',
      endpoint: 'https://api.openai.com/v1',
      apiKeyEnv: 'OPENAI_API_KEY',
      mode: 'defects',
      model: 'some/model',
      prompt: 'Review this code for concrete bugs and risks with custom rules.',
      maxOutputTokens: 2048,
      useTools: true,
      maxToolRounds: 6,
      structuredOutputMode: 'json_schema'
    },
    {
      name: 'Optional Brainstorm',
      provider: 'openai',
      endpoint: 'https://api.openai.com/v1',
      apiKeyEnv: 'OPENAI_API_KEY',
      mode: 'suggestions',
      model: 'some/model',
      prompt: 'Suggest at most 3 grounded follow-up improvements. Do not repeat defects. Include a clear benefit and tradeoff for each suggestion.',
      useTools: true,
      maxToolRounds: 4,
      allowSuggestions: true,
      maxSuggestions: 3,
      structuredOutputMode: 'json_schema'
    }
  ],
  brainstormers: [
    {
      name: 'Scope Challenger',
      model: 'meta/llama-3.1-70b-instruct',
      structuredOutputMode: 'json_object',
      prompt: 'Challenge the concept scope and identify missing constraints.',
      useTools: false
    },
    {
      name: 'Validation Planner',
      model: 'meta/llama-3.1-405b-instruct',
      structuredOutputMode: 'json_object',
      prompt: 'Turn the concept into validation work before implementation starts.',
      useTools: false
    }
  ],
  brainstormSynthesis: {
    name: 'Core Coder',
    model: 'meta/llama-3.1-405b-instruct',
    structuredOutputMode: 'json_object',
    prompt: 'Synthesize the brainstorm panel into a grounded recommendation and revised next brief.',
    useTools: false
  }
};

See .prrrc.example.js for a starter file.

Configuration notes:

• provider: descriptive provider label, for example nvidia-free or deepseek
• endpoint: OpenAI-compatible base URL; may be set globally or per reviewer
• apiKeyEnv: which environment variable contains the API key; may be set globally or per reviewer
• contextMode: diff or full
• contextMaxChars: max inline chars for diff/preview content before truncation (default 10000)
• executionMode: sequential or parallel
• reviewSharing: when true, sequential reviewers can call get_available_reviews to inspect earlier reviewer output and reduce duplicate claims
• timeoutMs: provider request timeout in milliseconds
• structuredOutput: set to false to force plain-text parsing even when the provider preset supports structured responses
• structuredOutputMode: override with json_schema or json_object when the provider supports it
• --context-file: inject an optional markdown context file into the review request
• maxOutputTokens: optional per-reviewer cap; omit it to let the provider decide
• useTools: whether the reviewer may fetch file contents on demand
• maxToolRounds: optional per-reviewer cap for tool-call rounds
• mode: reviewer lane, either defects or suggestions (default defects)
• allowSuggestions: when true, structured reviewers may return a separate suggestions array in addition to defect findings (mode: 'suggestions' turns this on automatically)
• maxSuggestions: optional cap for reviewer suggestions when allowSuggestions is enabled
• brainstormers: separate model panel for prr brainstorm, useful for broader or cheaper no-tools model fans
• brainstormSynthesis: optional single-model synthesis stage that runs after brainstormers and produces a tighter recommendation plus a revised markdown next brief

reviewSharing is only effective in sequential mode. In parallel mode, reviewers run concurrently and do not have prior results to inspect. When reviewers run with useTools: false, increase contextMaxChars if you need more inline source context for larger files. mode: 'suggestions' makes the reviewer an explicit improvement lane with suggestion-focused prompting and metadata. Structured output still gives the cleanest separation because it can return a dedicated suggestions array. Recommended pattern: keep normal reviewers in mode: 'defects', then add a separate optional reviewer in mode: 'suggestions' with a low maxSuggestions cap. For GPT-5-family models, maxOutputTokens maps to max_completion_tokens, which includes reasoning tokens as well as visible output. If you set it too low, the model can spend the whole budget reasoning and return an empty visible response. Prefer leaving it unset or keeping it comfortably above the expected final output size.

Reviewer transport overrides inherit from the global config by default, but each reviewer can override:

• provider
• endpoint
• apiKeyEnv
• timeoutMs
• structuredOutputMode
• mode
• allowSuggestions
• maxSuggestions

That means one review run can mix free NVIDIA-hosted models with paid OpenAI-compatible backends. For brainstorming, it is often cleaner to keep a dedicated config such as .prrrc.brainstorm.js and pass it with --config.

Brainstorm Mode

prr brainstorm <brief.md> runs a concept-review panel against a markdown brief instead of source files.

• input is a markdown concept note, design draft, or implementation plan
• models come from brainstormers, separate from normal reviewers
• an optional brainstormSynthesis reviewer can consolidate the panel into a prioritized recommendation and revised markdown next brief
• no file selection, diff parsing, or tool calls are required
• this makes it practical to fan out across many cheaper NVIDIA-hosted models

Typical flow:

• draft concept-brief.md
• run prr brainstorm concept-brief.md
• review the optional synthesis output and merge the concrete findings back into the brief
• re-run until the concept is tighter
• start implementation and switch to normal prr review

Output

The default report is Markdown with:

• timestamp
• one section per brainstormer
• combined concept findings sections
• optional synthesis section with priorities and a draft next brief
• summary totals for brainstorm items, synthesis status, errors, and duration

Use --json for machine-readable output suitable for automation. When synthesis is enabled, nextBrief is normalized as markdown text so it can be copied back into the source brief or saved directly for another brainstorm round.

Brainstorm runs are persisted under .prr/brainstorms/<run-id>/ with the same request.json, progress.json, per-reviewer results, and final result.json pattern as review runs.

Each normal review run is persisted under .prr/reviews/<run-id>/ with:

• request.json
• progress.json
• one JSON file per reviewer result
• result.json

Use --resume to continue the latest incomplete review run, or --resume <run-id> to continue a specific run. Resume checks that the current command resolves to the same review request before continuing.

Use --dry-run to inspect what would happen without calling the provider:

• selected files
• resolved config and execution mode
• optional markdown context file
• context snippets that would be sent
• reviewer prompts
• actual batching plan

Add --verbose to include the full built request messages in the dry-run output.

Review Quality

The default prompts are tuned to reduce common multi-agent review noise:

• prefer concrete defects, regressions, and edge cases
• avoid style-only preferences
• avoid duplicate findings
• say insufficient context instead of guessing

When a provider preset supports structured output, prr asks for machine-readable findings first and falls back to plain-text parsing otherwise. This keeps the workflow easier to validate and reduces markdown-style filler in review results.

prr also builds a deterministic combined-findings view after each run. This deduplicates overlapping findings locally across reviewers before any optional checker/judge stage, so repeated reports do not need another model call just to collapse duplicates.

Use --context-file when the reviewer needs short factual context that is not obvious from the code or diff alone, for example intended behavior, constraints, or already-known tradeoffs.

Example:

prr review examples/yai-timeout/yai-timeout.js \
  --context-file ./review-context.md

A good context file should stay concise and focus on:

• what changed or what to double-check
• expected behavior or constraints that are easy to miss
• intentional tradeoffs that should not be re-raised as defects

Stop rule for review loops:

• stop when no high-severity findings remain and two consecutive rounds add no materially new concrete defects
• treat repeated low/info or policy-only findings as closure signal, not as mandatory code churn
• record intentional tradeoffs and rejected findings before closing the loop

Context Strategy

The default review path is diff-first. If the target files are inside a Git work tree, prr sends diff context first and lets the model request file contents or line ranges only when needed. This reduces prompt size and helps avoid truncating large reviews on smaller context-window models.

Changed Files

For day-to-day use you can target only modified files:

prr review --changed
prr review --changed --since origin/main

This includes tracked diffs plus untracked files inside the current working tree.

Example Target

The repository includes a first real benchmark target under examples/yai-timeout. It is a small VanillaScript timing utility that is useful for tuning prompts, context strategy, and reviewer quality on realistic but focused code.

Useful commands:

prr review examples/yai-timeout/yai-timeout.js --dry-run
prr review examples/yai-timeout/yai-timeout.js
prr review examples/yai-timeout/yai-timeout.js --context-file ./review-context.md
prr experiment run examples/yai-timeout/yai-timeout.js --preset smoke

Early live benchmarks on YaiTimeout showed that a short, focused context note improved review quality and reduced token waste compared with a plain run.

The example is now considered closed for this repository cycle: no unresolved high-severity defects remain, and later rounds mostly repeated low-priority or speculative findings.

Auto Mode

prr auto run is the first proof-of-concept orchestration step. It currently:

• resolves explicit files or Git-changed files
• prefers the latest experiment recommendation when available
• swaps tool-enabled recommendations to the verified elite list when the recommended model is not tool-verified
• otherwise runs the configured reviewers
• falls back to the top elite tool-capable reviewer when recommendation runs fail with connection-only errors
• applies the local cleanup and quality scoring pass
• emits a combined JSON or text summary

Experimentation

prr includes an experiment runner for finding workable defaults across models, context strategies, and prompt styles.

prr experiment run src/**/*.js --preset smoke
prr experiment run src/**/*.js --preset standard --role security --json

Experiment runs are stored under .prr/experiments/ by default as:

• results.jsonl: one record per scenario
• summary.json: aggregated metrics and a recommended scenario

Use this to compare:

• diff-first plus tools vs diff-only
• model choices per review role
• prompt styles such as concise vs actionable

Recommendations are quality-weighted, not just count-weighted. The runner now deduplicates findings and penalizes generic low-signal advice so verbose but weak outputs do not automatically win.

Development

npm run lint
npm test

CI

GitHub Actions runs lint and tests on Node.js 18 and 20 for pushes to main and pull requests.

Implementation details and planned architecture live in docs/spec.md. Contributor expectations live in AGENTS.md.