home / skills / shaul1991 / shaul-agents-plugin / qa-analyst

qa-analyst skill

needs review

This skill helps QA analysts perform performance and quality metric analysis, load testing, and reporting for API endpoints.

npx playbooks add skill shaul1991/shaul-agents-plugin --skill qa-analyst

Review the files below or copy the command above to add this skill to your agents.

Files (1)

SKILL.md

2.0 KB

---
name: qa-analyst
description: QA Analyst Agent. 성능 분석, 부하 테스트, 품질 메트릭 분석을 담당합니다. 성능, 부하(load), 분석, 메트릭 관련 요청 시 사용됩니다.
allowed-tools: Bash(curl:*), Bash(docker:*), Bash(npm:*), Read, Grep
---

# QA Analyst Agent

## 역할
성능 분석 및 품질 메트릭 관리를 담당합니다.

## 성능 분석 도구

### 1. 응답 시간 측정
```bash
# 단일 요청
time curl -sf https://api-nest.shaul.link/health/live

# 여러 요청 평균
for i in {1..10}; do
  curl -sf -o /dev/null -w "%{time_total}\n" https://api-nest.shaul.link/health/live
done | awk '{sum+=$1} END {print "Average:", sum/NR, "seconds"}'
```

### 2. 부하 테스트 (ab, wrk)
```bash
# Apache Bench
ab -n 1000 -c 100 https://api-nest.shaul.link/health/live

# wrk (더 정교한 테스트)
wrk -t4 -c100 -d30s https://api-nest.shaul.link/health/live
```

### 3. 메모리/CPU 모니터링
```bash
docker stats --no-stream --filter "name=nest-api"
```

## 성능 지표

### 응답 시간
| 등급 | 기준 |
|------|------|
| 좋음 | < 100ms |
| 보통 | 100-500ms |
| 나쁨 | > 500ms |

### 처리량
| 등급 | 기준 |
|------|------|
| 좋음 | > 1000 req/s |
| 보통 | 500-1000 req/s |
| 나쁨 | < 500 req/s |

### 에러율
| 등급 | 기준 |
|------|------|
| 좋음 | < 0.1% |
| 보통 | 0.1-1% |
| 나쁨 | > 1% |

## 분석 보고서 형식

```markdown
## 성능 분석 보고서

### 테스트 환경
- 날짜: YYYY-MM-DD
- 환경: Dev/Prod
- 도구: ab/wrk

### 결과 요약
- 평균 응답 시간: XXms
- 처리량: XXX req/s
- 에러율: X.X%

### 상세 분석
[분석 내용]

### 권고사항
[개선 제안]
```

## 품질 대시보드

### 주요 메트릭
1. **가용성**: Uptime 비율
2. **응답성**: 평균/P95/P99 응답 시간
3. **신뢰성**: 에러율
4. **확장성**: 동시 처리 능력

### 모니터링 체크포인트
- [ ] 헬스체크 응답 확인
- [ ] 응답 시간 정상 범위
- [ ] 에러 로그 없음
- [ ] 리소스 사용량 정상

Overview

This skill is a QA Analyst Agent that performs performance analysis, load testing, and quality-metric evaluation for web services and APIs. It produces concise performance reports, recommends improvements, and monitors key runtime metrics. Use it to validate response time, throughput, error rates, and resource usage under realistic conditions.

How this skill works

The agent runs synthetic checks and load tests (single requests, repeated samples, Apache Bench, wrk) to measure response time and throughput. It collects runtime metrics such as CPU and memory from containers or hosts and calculates error rates and percentile latencies. Results are summarized into a standard performance report with findings and actionable recommendations.

When to use it

Before and after deployments to verify performance regressions.
When investigating slow responses, high error rates, or capacity limits.
To validate scalability and concurrency targets with load tests.
To establish baseline metrics and SLA targets for availability and latency.
During capacity planning or incident postmortems.

Best practices

Measure in environments that mirror production as closely as possible (data, network, config).
Run multiple samples and percentiles (P95/P99) rather than relying on averages alone.
Correlate load-test results with resource metrics (CPU, memory, I/O) to find bottlenecks.
Gradually increase load to identify knee points and avoid false conclusions from abrupt stress tests.
Record test environment, tool versions, and repeatable commands for reproducible results.

Example use cases

Validate API latency and throughput after a code change using curl loops and wrk.
Run a stress test to determine maximum concurrent connections and identify scaling limits.
Monitor container CPU/memory during load to correlate performance degradation with resource exhaustion.
Produce a short performance report (environment, summary metrics, detailed analysis, recommendations) for stakeholders.
Set up a checklist for routine monitoring: health endpoints, response time thresholds, error logs, and resource usage.

FAQ

What tools does the agent use for load testing?

It uses lightweight approaches like curl for sampling, Apache Bench (ab) for basic load, and wrk for more advanced workloads.

Which metrics should I prioritize?

Prioritize availability (uptime), response-time percentiles (P95/P99), throughput (req/s), and error rate; correlate with CPU/memory for root cause analysis.