home / skills / fortiumpartners / ai-mesh / test-detector

test-detector skill

/skills/test-detector

This skill automatically detects test frameworks in a project by analyzing manifests, configs, and directories to guide test generation and execution.

npx playbooks add skill fortiumpartners/ai-mesh --skill test-detector

Review the files below or copy the command above to add this skill to your agents.

Files (3)
SKILL.md
3.2 KB
---
name: Test Framework Detector
description: Automatically detect test frameworks (Jest, pytest, RSpec, xUnit) in projects by analyzing configuration files and dependencies
version: 1.0.0
---

# Test Framework Detector

## Purpose

Automatically identify which test framework(s) a project uses by examining:
- Package manifests (package.json, requirements.txt, Gemfile, *.csproj)
- Test configuration files (jest.config.js, pytest.ini, spec_helper.rb, xunit.runner.json)
- Directory structure and naming conventions

## Usage

This skill is invoked by agents (like deep-debugger) when they need to determine which test framework to use for test generation or execution.

### Detection Script

Run the detection script with the project path:

```bash
node detect-framework.js /path/to/project
```

### Output Format

The script returns a JSON object with detected frameworks:

```json
{
  "detected": true,
  "frameworks": [
    {
      "name": "jest",
      "confidence": 0.95,
      "version": "29.7.0",
      "configFiles": ["jest.config.js", "package.json"],
      "testDirectory": "tests/",
      "testPattern": "**/*.test.js"
    }
  ],
  "primary": "jest"
}
```

## Detection Patterns

The skill uses pattern-based detection defined in `framework-patterns.json`. Each framework has:
- **Package indicators**: Dependencies that suggest framework presence
- **Config files**: Framework-specific configuration files
- **Test file patterns**: Common test file naming conventions

## Supported Frameworks

1. **Jest** (JavaScript/TypeScript)
   - Config: jest.config.js, jest.config.ts, package.json (jest section)
   - Dependencies: jest, @types/jest, ts-jest
   - Patterns: *.test.js, *.spec.js, __tests__/**

2. **pytest** (Python)
   - Config: pytest.ini, pyproject.toml, setup.cfg, tox.ini
   - Dependencies: pytest in requirements.txt or pyproject.toml
   - Patterns: test_*.py, *_test.py, tests/**

3. **RSpec** (Ruby)
   - Config: .rspec, spec/spec_helper.rb
   - Dependencies: rspec in Gemfile
   - Patterns: *_spec.rb, spec/**

4. **xUnit** (C#/.NET)
   - Config: xunit.runner.json, *.csproj
   - Dependencies: xunit, xunit.runner.visualstudio in *.csproj
   - Patterns: *Tests.cs, *Test.cs, Tests/**

## Confidence Scoring

Confidence scores (0.0-1.0) are calculated based on:
- **Config file presence**: +0.4
- **Package dependency found**: +0.3
- **Test directory exists**: +0.2
- **Test files found**: +0.1

Multiple frameworks may be detected (e.g., Jest + pytest in monorepos).

## Example Invocations

**Detect framework in current directory:**
```bash
node skills/test-detector/detect-framework.js .
```

**Detect framework with verbose output:**
```bash
DEBUG=true node skills/test-detector/detect-framework.js /path/to/project
```

## Integration with Agents

Agents should invoke this skill before test generation:

```markdown
1. Invoke test-detector skill with project path
2. Parse JSON output to get primary framework
3. Invoke appropriate test framework skill (jest-test, pytest-test, etc.)
4. Generate or execute tests using framework-specific patterns
```

## Error Handling

If no framework is detected:
```json
{
  "detected": false,
  "frameworks": [],
  "primary": null,
  "message": "No test framework detected. Please specify framework manually."
}
```

Overview

This skill automatically detects which test framework(s) a project uses by scanning manifests, config files, and test directories. It reports detected frameworks with confidence scores, matched config files, test patterns, and a primary framework recommendation. Designed for agents that need to choose the right test generator or runner before producing or executing tests.

How this skill works

The detector inspects package manifests (package.json, requirements.txt, Gemfile, *.csproj), framework-specific config files (jest.config.js, pytest.ini, .rspec, xunit.runner.json), and common test directories and file name patterns. It applies pattern-based rules from a configurable patterns file and computes a confidence score from config presence, dependency indicators, directory structure, and actual test files. The output is a JSON object listing frameworks, confidence, matched files, test patterns, and a primary framework if one is dominant.

When to use it

  • Before generating tests to pick the correct framework conventions and APIs
  • Prior to running automated test generation or execution in CI pipelines
  • When onboarding to a repository to quickly understand its test ecosystem
  • As a pre-step in multi-language or monorepo flows to detect multiple frameworks
  • During automated code-analysis agents that need framework-specific knowledge

Best practices

  • Run the detector at repository root and at workspace package roots in monorepos
  • Keep framework-patterns.json up to date with new config names and dependency identifiers
  • Combine detector output with a quick manual check for ambiguous cases (confidence < 0.6)
  • Use verbose mode (DEBUG=true) when integrating into pipelines to capture diagnostic clues
  • Treat the primary framework as a recommendation — allow manual override when needed

Example use cases

  • Agent calls detector, then routes to jest-test skill to generate JavaScript tests based on detected patterns
  • CI pipeline runs detector to choose the correct test runner and invoke tests only for detected frameworks
  • Onboarding script reports repository frameworks and expected test commands to new contributors
  • Monorepo analysis detects both pytest and Jest and triggers language-specific test generation for each package

FAQ

What happens if no framework is detected?

The detector returns detected:false, an empty frameworks list, and a message suggesting manual specification of the framework.

Can it detect multiple frameworks in one repository?

Yes. The detector can report multiple frameworks (common in monorepos) and indicates a primary framework based on highest confidence.

How is the confidence score calculated?

Scores combine weighted signals: config file presence (+0.4), dependency indicator (+0.3), test directory (+0.2), and discovered test files (+0.1).