home / skills / yonatangross / orchestkit / webapp-testing

webapp-testing skill

/plugins/ork/skills/webapp-testing

This skill enables autonomous end-to-end web application testing with Planner, Generator, and Healer agents for planning, generating, and self-healing

npx playbooks add skill yonatangross/orchestkit --skill webapp-testing

Review the files below or copy the command above to add this skill to your agents.

Files (9)
SKILL.md
5.2 KB
---
name: Webapp Testing
description: Use when testing web applications with AI-assisted Playwright. Webapp testing covers autonomous test agents for planning, generating, and self-healing tests.
context: fork
agent: test-generator
version: 1.2.0
author: OrchestKit AI Agent Hub
tags: [playwright, testing, e2e, automation, agents, 2026]
user-invocable: false
---
Autonomous end-to-end testing with Playwright's three specialized agents for planning, generating, and self-healing tests automatically.

## The Three Agents

1. **Planner** - Explores app and creates test plans
2. **Generator** - Writes Playwright tests with best practices
3. **Healer** - Fixes failing tests automatically

## Quick Setup (CC 2.1.6)

```bash
# 1. Install Playwright
npm install --save-dev @playwright/test
npx playwright install
```

```json
// 2. Create/update .mcp.json in project root
{
  "mcpServers": {
    "playwright": {
      "command": "npx",
      "args": ["-y", "@playwright/mcp@latest"]
    }
  }
}
```

```bash
# 3. Initialize agents (after restarting Claude Code session)
npx playwright init-agents --loop=claude

# 4. Create tests/seed.spec.ts (required for Planner)
```

**Requirements:** VS Code v1.105+ (Oct 9, 2025)

## Agent Workflow

```
1. PLANNER   --> Explores app --> Creates specs/checkout.md
                 (uses seed.spec.ts)
                      |
                      v
2. GENERATOR --> Reads spec --> Tests live app --> Outputs tests/checkout.spec.ts
                 (verifies selectors actually work)
                      |
                      v
3. HEALER    --> Runs tests --> Fixes failures --> Updates selectors/waits
                 (self-healing)
```

## Directory Structure

```
your-project/
├── specs/              <- Planner outputs (Markdown plans)
├── tests/              <- Generator outputs (Playwright tests)
│   └── seed.spec.ts    <- Required: Planner learns from this
├── playwright.config.ts
└── .mcp.json           <- MCP server config (CC 2.1.6)
```

## Key Concepts

**seed.spec.ts is required** - Planner executes this to learn:
- Environment setup (fixtures, hooks)
- Authentication flow
- Available UI elements

**Generator validates live** - Doesn't just translate Markdown, actually tests app to verify selectors work.

**Healer auto-fixes** - When UI changes break tests, Healer replays, finds new selectors, patches tests.

See `references/` for detailed agent patterns and commands.

## Bundled Resources

- `assets/playwright-test-template.ts` - Playwright test template with BasePage, ApiMocker, and CustomAssertions
- `references/playwright-setup.md` - Playwright setup and configuration
- `references/visual-regression.md` - Visual regression testing patterns

## Related Skills

- `e2e-testing` - Core end-to-end testing patterns with Playwright
- `msw-mocking` - Mock Service Worker for API mocking in tests
- `a11y-testing` - Accessibility testing integration with Playwright
- `vcr-http-recording` - HTTP recording for deterministic test playback

## Key Decisions

| Decision | Choice | Rationale |
|----------|--------|-----------|
| Agent Architecture | Planner/Generator/Healer | Separation of concerns for autonomous testing |
| Seed Requirement | seed.spec.ts mandatory | Planner needs context to learn app patterns |
| Selector Strategy | Semantic locators | More resilient to UI changes than CSS selectors |
| Self-Healing | Healer agent | Reduces test maintenance burden automatically |

## Capability Details

### playwright-setup
**Keywords:** playwright, setup, install, configure, mcp
**Solves:**
- How do I set up Playwright testing?
- Install Playwright MCP server
- Configure test environment
- Initialize Playwright agents with Claude

### test-planning
**Keywords:** test plan, scenarios, user flows, test cases, planner agent
**Solves:**
- How do I create a test plan?
- Use Planner agent to explore app
- Identify test scenarios automatically
- Plan user flow testing with seed.spec.ts

### test-generation
**Keywords:** generate tests, write tests, playwright code, selectors, generator agent
**Solves:**
- How do I generate Playwright tests?
- Use Generator agent to write test code
- Create semantic locators that validate live
- Write tests with best practices

### test-healing
**Keywords:** fix tests, failing tests, self-heal, maintenance, healer agent
**Solves:**
- How do I fix failing tests automatically?
- Use Healer agent to update broken selectors
- Maintain test suite after UI changes
- Self-healing test automation

### agent-workflow
**Keywords:** planner generator healer, test workflow, autonomous testing, playwright agents
**Solves:**
- How do the three Playwright agents work together?
- Complete testing workflow with agents
- Planner -> Generator -> Healer pipeline
- Autonomous test creation and maintenance

### visual-regression
**Keywords:** visual regression, screenshot, toHaveScreenshot, snapshot, VRT, baseline, pixel diff, visual testing
**Solves:**
- How do I set up visual regression testing with Playwright?
- Replace Percy with Playwright native screenshots
- Configure toHaveScreenshot thresholds
- Handle cross-platform screenshot differences
- Mask dynamic content in screenshots
- Set up visual regression in CI/CD
- Update screenshot baselines
- Debug failed visual comparisons

Overview

This skill provides an autonomous end-to-end webapp testing pipeline using Playwright with three specialized agents: Planner, Generator, and Healer. It streamlines test planning, code generation, and self-healing so teams can create resilient Playwright suites with minimal manual upkeep. The package includes templates, configuration patterns, and instructions to run agents in a Claude Code environment.

How this skill works

The Planner explores your running app using a required seed.spec.ts to learn fixtures, auth flows, and UI structure, then emits human-readable specs. The Generator reads those specs and produces Playwright tests, verifying selectors against the live app as it writes code. The Healer runs tests, detects failures, and automatically patches selectors and waits to restore passing tests without manual edits.

When to use it

  • Bootstrapping end-to-end test coverage for a new or existing web app
  • Automating test creation when product UI changes frequently
  • Reducing maintenance by enabling self-healing tests
  • Integrating Playwright into a Claude Code development workflow
  • Adding visual regression baselines and native Playwright screenshot checks

Best practices

  • Always include a representative seed.spec.ts so the Planner can learn environment setup and auth patterns
  • Use semantic locators (data-test/id or role-based selectors) for more resilient tests
  • Run Generator against a stable dev or staging instance to validate selectors live
  • Keep tests small and focused on user flows to simplify healing logic
  • Store visual baseline snapshots and configure thresholds to avoid flaky VRT failures

Example use cases

  • Auto-generate checkout flow tests by having Planner explore the storefront and Generator emit tests/checkout.spec.ts
  • Detect and fix selector regressions after a UI refactor using the Healer to patch failing tests automatically
  • Integrate visual regression by adding toHaveScreenshot checks created from the Playwright test template
  • Use the Playwright test template with BasePage and ApiMocker to convert manual smoke checks into repeatable tests
  • Add the MCP server config (.mcp.json) and init-agents step to bring agents online in CI or local development

FAQ

Do I need a specific Playwright or Claude Code version?

Install Playwright via npm and run npx playwright install. The skill targets Claude Code patterns; follow the provided MCP server config and agent init commands for compatibility.

Why is seed.spec.ts required?

The Planner uses seed.spec.ts to learn project fixtures, authentication, and common UI patterns. It provides context so generated specs match your app conventions and environment.

How does self-healing avoid introducing incorrect fixes?

The Healer replays failing tests, searches for robust semantic locators, and prioritizes fixes that restore test intent. You should review healed changes in code review to confirm behavior matches expectations.