home / skills / benchflow-ai / skillsbench / analyze-ci

analyze-ci skill

safe

/tasks/fix-build-agentops/environment/skills/analyze-ci

This skill analyzes failed GitHub Action jobs in a PR to produce a concise root-cause summary and actionable log snippets.

npx playbooks add skill benchflow-ai/skillsbench --skill analyze-ci

Review the files below or copy the command above to add this skill to your agents.

Files (1)

SKILL.md

958 B

---
name: analyze-ci
description: Analyze failed GitHub Action jobs for a pull request.
allowed-tools:
  - Bash(uv run skills analyze-ci:*)
---

# Analyze CI Failures

This skill analyzes logs from failed GitHub Action jobs using Claude.

## Prerequisites

- **GitHub Token**: Auto-detected via `gh auth token`, or set `GITHUB_TOKEN` env var

## Usage

```bash
# Analyze all failed jobs in a PR
uv run skills analyze-ci <pr_url>

# Analyze specific job URLs directly
uv run skills analyze-ci <job_url> [job_url ...]

# Show debug info (tokens and costs)
uv run skills analyze-ci <pr_url> --debug
```

Output: A concise failure summary with root cause, error messages, test names, and relevant log snippets.

## Examples

```bash
# Analyze CI failures for a PR
uv run skills analyze-ci https://github.com/mlflow/mlflow/pull/19601

# Analyze specific job URLs directly
uv run skills analyze-ci https://github.com/mlflow/mlflow/actions/runs/12345/job/67890
```

Overview

This skill analyzes failed GitHub Actions jobs for a pull request and produces concise, actionable failure summaries. It extracts root causes, key error messages, failing test names, and relevant log snippets to speed triage. The tool uses available GitHub tokens to fetch job logs and applies automated analysis to highlight the most important information.

How this skill works

Given a PR URL or one or more job URLs, the skill fetches the failed job logs from GitHub Actions using the configured GitHub token. It scans logs for errors, stack traces, and test failures, then groups and prioritizes findings to produce a short root-cause summary and supporting excerpts. Optional debug output shows tokens and cost estimates when requested.

When to use it

Triage failing CI on a pull request to find the likely root cause quickly.
When multiple jobs fail and you need a prioritized list of failures to investigate.
To extract failing test names and error snippets for bug reports or issue comments.
When you want a concise summary to share with teammates instead of raw logs.
When debugging flaky tests across similar job runs to identify patterns.

Best practices

Provide the PR URL to analyze all failed jobs in a single run for comprehensive insights.
Supply specific job URLs when you only want analysis of individual failures.
Ensure a GitHub token is available (gh auth token or GITHUB_TOKEN) for log access.
Use the debug flag sparingly to reveal tokens and cost info only when necessary.
Review the extracted log snippets before applying fixes to confirm context.

Example use cases

Run against a newly opened PR to get a quick list of failing tests and root causes.
Point at a single job URL when a specific workflow run fails to get focused diagnostics.
Generate a concise summary to paste into a PR comment explaining why CI failed.
Compare failures across multiple jobs to detect configuration or environment regressions.

FAQ

What credentials are required?

A GitHub token is required to fetch Actions logs. The skill auto-detects a token from gh auth token or reads GITHUB_TOKEN from the environment.

Can it analyze all jobs in a PR?

Yes. Supplying a PR URL triggers analysis of all failed jobs associated with that pull request.