playbooks

home / skills / openclaw / skills / data-analysis

data-analysis skill

/skills/ivangdavila/data-analysis

This skill guides data analysis with rigorous methodology, clarifying decisions, assessing uncertainty, and highlighting actionable next steps for data-driven

npx playbooks add skill openclaw/skills --skill data-analysis

Review the files below or copy the command above to add this skill to your agents.

Files (4)
SKILL.md
2.9 KB
---
name: Data Analysis
description: Turn raw data into decisions with statistical rigor, proper methodology, and awareness of analytical pitfalls.
---

## When to Load

User asks about: analyzing data, finding patterns, understanding metrics, testing hypotheses, cohort analysis, A/B testing, churn analysis, statistical significance.

## Core Principle

Analysis without a decision is just arithmetic. Always clarify: **What would change if this analysis shows X vs Y?**

## Methodology First

Before touching data:
1. **What decision** is this analysis supporting?
2. **What would change your mind?** (the real question)
3. **What data do you actually have** vs what you wish you had?
4. **What timeframe** is relevant?

## Statistical Rigor Checklist

- [ ] Sample size sufficient? (small N = wide confidence intervals)
- [ ] Comparison groups fair? (same time period, similar conditions)
- [ ] Multiple comparisons? (20 tests = 1 "significant" by chance)
- [ ] Effect size meaningful? (statistically significant ≠ practically important)
- [ ] Uncertainty quantified? ("12-18% lift" not just "15% lift")

## Analytical Pitfalls to Catch

| Pitfall | What it looks like | How to avoid |
|---------|-------------------|--------------|
| Simpson's Paradox | Trend reverses when you segment | Always check by key dimensions |
| Survivorship bias | Only analyzing current users | Include churned/failed in dataset |
| Comparing unequal periods | Feb (28d) vs March (31d) | Normalize to per-day or same-length windows |
| p-hacking | Testing until something is "significant" | Pre-register hypotheses or adjust for multiple comparisons |
| Correlation in time series | Both went up = "related" | Check if controlling for time removes relationship |
| Aggregating percentages | Averaging percentages directly | Re-calculate from underlying totals |

For detailed examples of each pitfall, see `pitfalls.md`.

## Approach Selection

| Question type | Approach | Key output |
|---------------|----------|------------|
| "Is X different from Y?" | Hypothesis test | p-value + effect size + CI |
| "What predicts Z?" | Regression/correlation | Coefficients + R² + residual check |
| "How do users behave over time?" | Cohort analysis | Retention curves by cohort |
| "Are these groups different?" | Segmentation | Profiles + statistical comparison |
| "What's unusual?" | Anomaly detection | Flagged points + context |

For technique details and when to use each, see `techniques.md`.

## Output Standards

1. **Lead with the insight**, not the methodology
2. **Quantify uncertainty** — ranges, not point estimates
3. **State limitations** — what this analysis can't tell you
4. **Recommend next steps** — what would strengthen the conclusion

## Red Flags to Escalate

- User wants to "prove" a predetermined conclusion
- Sample size too small for reliable inference
- Data quality issues that invalidate analysis
- Confounders that can't be controlled for

Overview

This skill turns raw data into actionable decisions using statistical rigor, clear methodology, and awareness of common analytical pitfalls. It focuses on producing insight-first results with quantified uncertainty, stated limitations, and recommended next steps. The goal is to support real decisions, not just generate numbers.

How this skill works

Before any computation, the skill clarifies the decision the analysis must inform, the counterfactual that would change the decision, available data, and the relevant timeframe. It applies appropriate methods — hypothesis tests, regressions, cohort analysis, segmentation, or anomaly detection — and reports effect sizes, confidence intervals, and caveats. Outputs lead with the key insight, quantify uncertainty, and include actionable recommendations and escalation flags when results are unreliable.

When to use it

  • Deciding whether a change produced a real lift (A/B testing, significance, effect size)
  • Understanding drivers of an outcome (regression, correlation, predictive features)
  • Tracking user behavior over time (cohort retention, lifecycle analysis)
  • Detecting anomalies or unusual events in metrics
  • Segmenting users to compare groups or tailor actions
  • Assessing churn, lifetime value, or other critical business metrics

Best practices

  • Start with the decision: define what would change based on results
  • Pre-specify hypotheses and timeframes to avoid p-hacking
  • Quantify uncertainty: report confidence intervals and effect sizes
  • Check for common biases (Simpson’s paradox, survivorship bias, confounders)
  • Normalize comparisons (per-day, per-user) and control for multiple tests

Example use cases

  • A/B test analysis that reports lift, CI, and whether the effect is practically meaningful
  • Cohort retention analysis to prioritize product improvements for at-risk segments
  • Churn analysis identifying leading indicators and recommended interventions
  • Regression to identify predictors of conversion with residual diagnostics
  • Anomaly detection that flags spikes and provides contextual likely causes

FAQ

What if my sample size is small?

Small samples produce wide confidence intervals; flag the uncertainty, avoid strong claims, and recommend more data or pooled analysis where appropriate.

How do you avoid false positives when running many tests?

Pre-register key hypotheses, adjust p-values for multiple comparisons (Bonferroni, Benjamini–Hochberg), and emphasize effect sizes over isolated p-values.