home / skills / yeachan-heo / oh-my-claudecode / ultraqa

ultraqa skill

/skills/ultraqa

This skill automates QA cycling, running tests, builds, lint, and typechecks until the quality goal is met, with autonomous diagnosis and fixes.

npx playbooks add skill yeachan-heo/oh-my-claudecode --skill ultraqa

Review the files below or copy the command above to add this skill to your agents.

Files (1)
SKILL.md
4.2 KB
---
name: ultraqa
description: QA cycling workflow - test, verify, fix, repeat until goal met
---

# UltraQA Skill

[ULTRAQA ACTIVATED - AUTONOMOUS QA CYCLING]

## Overview

You are now in **ULTRAQA** mode - an autonomous QA cycling workflow that runs until your quality goal is met.

**Cycle**: qa-tester → architect verification → fix → repeat

## Goal Parsing

Parse the goal from arguments. Supported formats:

| Invocation | Goal Type | What to Check |
|------------|-----------|---------------|
| `/oh-my-claudecode:ultraqa --tests` | tests | All test suites pass |
| `/oh-my-claudecode:ultraqa --build` | build | Build succeeds with exit 0 |
| `/oh-my-claudecode:ultraqa --lint` | lint | No lint errors |
| `/oh-my-claudecode:ultraqa --typecheck` | typecheck | No TypeScript errors |
| `/oh-my-claudecode:ultraqa --custom "pattern"` | custom | Custom success pattern in output |

If no structured goal provided, interpret the argument as a custom goal.

## Cycle Workflow

### Cycle N (Max 5)

1. **RUN QA**: Execute verification based on goal type
   - `--tests`: Run the project's test command
   - `--build`: Run the project's build command
   - `--lint`: Run the project's lint command
   - `--typecheck`: Run the project's type check command
   - `--custom`: Run appropriate command and check for pattern
   - `--interactive`: Use qa-tester for interactive CLI/service testing:
     ```
     Task(subagent_type="oh-my-claudecode:qa-tester", model="sonnet", prompt="TEST:
     Goal: [describe what to verify]
     Service: [how to start]
     Test cases: [specific scenarios to verify]")
     ```

2. **CHECK RESULT**: Did the goal pass?
   - **YES** → Exit with success message
   - **NO** → Continue to step 3

3. **ARCHITECT DIAGNOSIS**: Spawn architect to analyze failure
   ```
   Task(subagent_type="oh-my-claudecode:architect", model="opus", prompt="DIAGNOSE FAILURE:
   Goal: [goal type]
   Output: [test/build output]
   Provide root cause and specific fix recommendations.")
   ```

4. **FIX ISSUES**: Apply architect's recommendations
   ```
   Task(subagent_type="oh-my-claudecode:executor", model="sonnet", prompt="FIX:
   Issue: [architect diagnosis]
   Files: [affected files]
   Apply the fix precisely as recommended.")
   ```

5. **REPEAT**: Go back to step 1

## Exit Conditions

| Condition | Action |
|-----------|--------|
| **Goal Met** | Exit with success: "ULTRAQA COMPLETE: Goal met after N cycles" |
| **Cycle 5 Reached** | Exit with diagnosis: "ULTRAQA STOPPED: Max cycles. Diagnosis: ..." |
| **Same Failure 3x** | Exit early: "ULTRAQA STOPPED: Same failure detected 3 times. Root cause: ..." |
| **Environment Error** | Exit: "ULTRAQA ERROR: [tmux/port/dependency issue]" |

## Observability

Output progress each cycle:
```
[ULTRAQA Cycle 1/5] Running tests...
[ULTRAQA Cycle 1/5] FAILED - 3 tests failing
[ULTRAQA Cycle 1/5] Architect diagnosing...
[ULTRAQA Cycle 1/5] Fixing: auth.test.ts - missing mock
[ULTRAQA Cycle 2/5] Running tests...
[ULTRAQA Cycle 2/5] PASSED - All 47 tests pass
[ULTRAQA COMPLETE] Goal met after 2 cycles
```

## State Tracking

Track state in `.omc/ultraqa-state.json`:
```json
{
  "active": true,
  "goal_type": "tests",
  "goal_pattern": null,
  "cycle": 1,
  "max_cycles": 5,
  "failures": ["3 tests failing: auth.test.ts"],
  "started_at": "2024-01-18T12:00:00Z",
  "session_id": "uuid"
}
```

## Cancellation

User can cancel with `/oh-my-claudecode:cancel` which clears the state file.

## Important Rules

1. **PARALLEL when possible** - Run diagnosis while preparing potential fixes
2. **TRACK failures** - Record each failure to detect patterns
3. **EARLY EXIT on pattern** - 3x same failure = stop and surface
4. **CLEAR OUTPUT** - User should always know current cycle and status
5. **CLEAN UP** - Clear state file on completion or cancellation

## STATE CLEANUP ON COMPLETION

**IMPORTANT: Delete state files on completion - do NOT just set `active: false`**

When goal is met OR max cycles reached OR exiting early:

```bash
# Delete ultraqa state file
rm -f .omc/state/ultraqa-state.json
```

This ensures clean state for future sessions. Stale state files with `active: false` should not be left behind.

---

Begin ULTRAQA cycling now. Parse the goal and start cycle 1.

Overview

This skill provides an autonomous QA cycling workflow that runs test, verify, fix, and repeat cycles until a defined quality goal is met. It coordinates tester, architect, and executor agents to diagnose failures and apply precise fixes across up to five cycles. The skill tracks state, outputs clear cycle progress, and enforces early-exit and cleanup rules to avoid stale sessions.

How this skill works

The skill parses a goal (tests, build, lint, typecheck, or a custom pattern) and runs the appropriate verification command. On failure it spawns an architect agent to diagnose the root cause and an executor agent to apply recommended fixes, then repeats verification. It records failures, detects repeated failure patterns, enforces max cycles and early exits, and deletes the session state file on completion or cancellation.

When to use it

  • Automate CI-style recovery when tests or builds intermittently fail.
  • Iterate automatically on lint or typecheck regressions until resolved.
  • Run repeated verification+fix cycles for complex regressions that need diagnosis and targeted code edits.
  • Apply a custom success pattern for nonstandard checks or service-level testing.
  • Use as a developer productivity tool to reduce manual triage of frequent failures.

Best practices

  • Define a clear, executable goal (use --tests, --build, --lint, --typecheck, or --custom with a pattern).
  • Limit side effects: ensure test/build commands are deterministic and reproducible in CI-like environments.
  • Allow architect recommendations to be specific and minimal — prefer targeted edits over broad refactors.
  • Enable parallel diagnosis and fix preparation to reduce cycle time when possible.
  • Ensure cleanup policies and state deletion are accepted by your workspace scripts to avoid stale state files.

Example use cases

  • Run ultraqa --tests to iterate until all test suites pass after a complex merge.
  • Use ultraqa --build in a release pipeline to auto-fix and reattempt build failures caused by minor config issues.
  • Invoke ultraqa --lint to apply quick style fixes and repeat until no lint errors remain.
  • Start ultraqa --typecheck to resolve TypeScript errors with architect-proposed fixes across multiple files.
  • Run ultraqa --custom "DEPLOY_READY" to verify a deployable artifact contains a custom success marker.

FAQ

How many cycles will it attempt?

By default it repeats up to 5 cycles and then stops with a final diagnosis if the goal is not met.

When does it exit early?

It stops early if the same failure is detected three times or if an environment error (ports, tmux, dependencies) prevents progress.