playbooks

home / skills / openai / openai-agents-python / test-coverage-improver

test-coverage-improver skill

/.codex/skills/test-coverage-improver

This skill analyzes test coverage results, identifies gaps, proposes high-impact tests, and coordinates approval before implementation.

npx playbooks add skill openai/openai-agents-python --skill test-coverage-improver

Review the files below or copy the command above to add this skill to your agents.

Files (1)
SKILL.md
2.6 KB
---
name: test-coverage-improver
description: 'Improve test coverage in the OpenAI Agents Python repository: run `make coverage`, inspect coverage artifacts, identify low-coverage files, propose high-impact tests, and confirm with the user before writing tests.'
---

# Test Coverage Improver

## Overview

Use this skill whenever coverage needs assessment or improvement (coverage regressions, failing thresholds, or user requests for stronger tests). It runs the coverage suite, analyzes results, highlights the biggest gaps, and prepares test additions while confirming with the user before changing code.

## Quick Start

1. From the repo root run `make coverage` to regenerate `.coverage` data and `coverage.xml`.
2. Collect artifacts: `.coverage` and `coverage.xml`, plus the console output from `coverage report -m` for drill-downs.
3. Summarize coverage: total percentages, lowest files, and uncovered lines/paths.
4. Draft test ideas per file: scenario, behavior under test, expected outcome, and likely coverage gain.
5. Ask the user for approval to implement the proposed tests; pause until they agree.
6. After approval, write the tests in `tests/`, rerun `make coverage`, and then run `$code-change-verification` before marking work complete.

## Workflow Details

- **Run coverage**: Execute `make coverage` at repo root. Avoid watch flags and keep prior coverage artifacts only if comparing trends.
- **Parse summaries efficiently**:
  - Prefer the console output from `coverage report -m` for file-level totals; fallback to `coverage.xml` for tooling or spreadsheets.
  - Use `uv run coverage html` to generate `htmlcov/index.html` if you need an interactive drill-down.
- **Prioritize targets**:
  - Public APIs or shared utilities in `src/agents/` before examples or docs.
  - Files with low statement coverage or newly added code at 0%.
  - Recent bug fixes or risky code paths (error handling, retries, timeouts, concurrency).
- **Design impactful tests**:
  - Hit uncovered paths: error cases, boundary inputs, optional flags, and cancellation/timeouts.
  - Cover combinational logic rather than trivial happy paths.
  - Place tests under `tests/` and avoid flaky async timing.
- **Coordinate with the user**: Present a numbered, concise list of proposed test additions and expected coverage gains. Ask explicitly before editing code or fixtures.
- **After implementation**: Rerun coverage, report the updated summary, and note any remaining low-coverage areas.

## Notes

- Keep any added comments or code in English.
- Do not create `scripts/`, `references/`, or `assets/` unless needed later.
- If coverage artifacts are missing or stale, rerun `pnpm test:coverage` instead of guessing.

Overview

This skill improves test coverage for the OpenAI Agents Python repository by running the coverage suite, analyzing results, and proposing high-impact tests. It identifies low-coverage files, drafts targeted test scenarios, and confirms proposed changes with the user before writing tests. The goal is measurable coverage gains while avoiding flaky or invasive edits.

How this skill works

Run make coverage at the repository root to produce .coverage and coverage.xml and capture coverage report -m output. Parse file-level totals and uncovered lines, prioritize public APIs and risky code paths, and draft concise test proposals that target uncovered branches and error conditions. Present a numbered list of proposed tests and expected coverage gains, wait for user approval, then implement tests under tests/, rerun coverage, and report the updated summary.

When to use it

  • Coverage regression detected in CI or locally
  • Need to meet or raise a coverage threshold
  • New or refactored code lacks tests
  • During a release or bug-fix cycle to validate behavior
  • When preparing contributor PRs that should include tests

Best practices

  • Always run make coverage from the repo root to regenerate artifacts
  • Focus on public APIs and shared utilities before example or docs files
  • Target uncovered branches: error handling, boundary inputs, optional flags, and concurrency paths
  • Draft tests to avoid flaky timing; prefer deterministic async patterns and mocking
  • Present proposed tests with expected coverage impact and get explicit user approval before editing code

Example use cases

  • Detect and propose tests for files with 0% coverage after a new feature merge
  • Add tests that exercise error handling and retry logic in shared utilities
  • Draft and implement tests to cover optional flags and edge-case input parsing
  • Validate that a recent bugfix has unit tests that exercise the failing path
  • Raise overall project coverage to meet CI gating thresholds

FAQ

What artifacts do I need to provide?

Provide .coverage, coverage.xml, and the console output from coverage report -m; include htmlcov if you generated it for drill-downs.

Can I write tests before approval?

No. This workflow requires explicit user approval of proposed tests before implementing code changes.