home / skills / plurigrid / asi / norvidize

norvidize skill

/skills/norvidize

This skill extracts and audits claims from documentation, code, and tests to populate the norvid claim tracking system.

npx playbooks add skill plurigrid/asi --skill norvidize

Review the files below or copy the command above to add this skill to your agents.

Files (1)
SKILL.md
4.9 KB
---
name: norvidize
description: Extract and audit claims for norvid tracking system
---

# Norvidize Skill

Extract claims from documentation, code, and tests into the norvid claim tracking system.

## When to Use

- After completing a significant feature
- When auditing documentation accuracy
- Before releases to ensure claims match reality
- When docs/tests/specs diverge from implementation

## Claim Registry Location

```
crates/basin-norvid/src/claims.rs
```

## Assurance Levels (lowest to highest)

| Level | Meaning | Evidence Required |
|-------|---------|-------------------|
| Mentioned | Just referenced somewhere | Any doc |
| Designed | Has design doc | docs/design/*.md |
| Specified | Has sigil spec | *.sigil file |
| Implemented | Code exists | Rust file + symbol |
| Tested | Has passing tests | Test file + test name |
| Proven | Has Lean proof | Lean file + theorem |

## Claim Categories

- **Feature**: User-visible functionality
- **Property**: System guarantee (linearizability, durability)
- **Guarantee**: Promise to users
- **Optimization**: Performance improvement
- **Invariant**: Internal system invariant
- **Decision**: Architectural choice with rationale
- **Omission**: Deliberate non-feature

## Evidence Types

```rust
// Documentation
Evidence::doc("docs/CLAIMS.md")

// Rust implementation
Evidence::rust("crates/foo/src/bar.rs", "SymbolName")

// Test file
Evidence::test("crates/foo/src/bar.rs", "test_prefix")

// Sigil specification
Evidence::sigil("sigil/specs/foo.sigil")

// External (benchmark, paper)
Evidence::external("benchmarks/harnesses/redis/src/main.rs")
```

## Adding Claims

```rust
registry.register(
    Claim::new("CLM_UNIQUE_ID", "Short description")
        .with_category(Category::Feature)
        .with_component(Component::Slate)
        .with_assurance(Assurance::Implemented)
        .with_evidence(Assurance::Designed, Evidence::doc("docs/design/foo.md"))
        .with_evidence(Assurance::Implemented, Evidence::rust("crates/slate/src/lib.rs", "FooStruct"))
);
```

## Extraction Process

### 1. Find Candidate Claims

Scan these sources for claim candidates:

```bash
# Explicit claims in docs
grep -r "CLM-\|CLAIM\|GUARANTEE\|INVARIANT" docs/

# Test names suggest guarantees
grep -r "#\[test\]" -A 2 crates/ | grep "fn test_"

# Performance claims
grep -r "ops/s\|latency\|throughput" docs/

# Sigil specs
find sigil/ -name "*.sigil" -exec grep -l "claim\|invariant\|property" {} \;
```

### 2. Categorize and Deduplicate

Group into tiers:
- **Tier 1 (SPEC_*)**: Claims with sigil specs - highest value
- **Tier 2 (TEST_*)**: Claims with test evidence
- **Tier 3**: Design-level claims (docs only)

### 3. Verify Evidence Paths

Before adding at Tested/Implemented level, verify paths exist:

```bash
# Check file exists
test -f "crates/basin-redis/src/crdt.rs" && echo "OK"

# Check symbol exists
grep -l "EntityStore" crates/basin-meta/src/indexes/*.rs
```

### 4. Add to Registry

Add claims to `claims.rs` in appropriate section:
- SPEC_* claims near line 1720
- TEST_* claims after specs
- PERF_* claims in performance section
- Product claims (CLM_STASH, etc.) with products

## CI Verification

The CI test filters claims at Tested/Proven level and verifies:
1. Evidence file exists
2. Symbol/test found in file

```bash
# Run verification
graft test -p basin-norvid test_all_claims_verified
```

## Common Path Mappings

| Old/Wrong Path | Correct Path |
|----------------|--------------|
| `crates/stash/` | `crates/basin-redis/` |
| `crates/shelf/` | `crates/basin-mesh/` or `crates/spool-core/` |
| `crates/basin-jepsen/` | `tests/basin-jepsen/` |
| `products/nfs/tests/` | `crates/nfs/tests/` |

## Downgrade Strategy

If evidence path is uncertain, downgrade to Designed level:

```rust
// Was Tested but can't verify path
.with_assurance(Assurance::Designed)
.with_evidence(Assurance::Designed, Evidence::doc("docs/CLAIMS.md"))
```

Upgrade later when evidence is confirmed.

## Parallel Audit Pattern

For bulk verification, spawn sonnet agents:

```
Task: Audit PERF_* claims
- Check benchmark files exist
- Verify they actually measure what claim says
- Report which claims have valid evidence

Task: Audit product claims (CLM_STASH, etc.)
- Find actual test locations
- Verify symbol names
- Report path corrections needed
```

## Anti-Patterns

- **Don't add Tested claims without verifying paths** - CI will fail
- **Don't embed large values in claim IDs** - Keep IDs short like `CLM_STASH`
- **Don't skip Designed level** - Even Tested claims should have design evidence
- **Don't guess component names** - Check `Component` enum in identity.rs

## Quick Reference

```bash
# Find existing claims
grep -n "Claim::new" crates/basin-norvid/src/claims.rs | head -20

# Check CI status
graft test -p basin-norvid test_all_claims_verified 2>&1 | grep -E "FAILED|passed"

# Find test files for a crate
find crates/basin-meta -name "*.rs" -exec grep -l "#\[test\]" {} \;

# Count claims by category
grep -E "Category::" crates/basin-norvid/src/claims.rs | sort | uniq -c
```

Overview

This skill extracts, organizes, and audits claims for the norvid tracking system to ensure documentation, specs, tests, and implementation align. It helps convert candidate statements from docs, tests, sigil specs, and code into structured claims with evidence and assurance levels. Use it to reduce drift between what the system promises and what it actually provides.

How this skill works

The skill scans source artifacts (documentation, Rust code, tests, and sigil specs) to find candidate claims, classifies them by category and assurance level, and verifies evidence paths before adding entries to the claim registry at crates/basin-norvid/src/claims.rs. It supports downgrades when evidence can't be verified, and includes routines for bulk audits, CI verification checks, and mapping common path mistakes.

When to use it

  • After finishing a significant feature to record its guarantees
  • When auditing documentation or spec accuracy before a release
  • If tests, docs, and implementation start to diverge
  • During bulk verification of performance or product claims
  • When preparing CI checks that validate claim evidence

Best practices

  • Prioritize tiered evidence: spec (sigil) > test > design docs
  • Always verify file paths and symbol/test names before marking Tested/Implemented
  • Keep claim IDs short and stable (eg. CLM_STASH)
  • Add design-level evidence even when upgrading directly to Tested
  • Downgrade unsure claims to Designed and upgrade later with confirmed evidence

Example use cases

  • Scan docs and tests to generate SPEC_* and TEST_* claim candidates for the registry
  • Run a parallel audit to validate PERF_* claims against benchmark harnesses
  • Correct common path mappings (old crate paths -> correct crate or tests path)
  • Prepare a release checklist that ensures every user-facing feature has matching claim evidence
  • Automate CI verification that ensures evidence files and symbols exist for Tested claims

FAQ

Where are claims stored?

Claims are registered in crates/basin-norvid/src/claims.rs.

What assurance levels are supported?

Levels: Mentioned, Designed, Specified, Implemented, Tested, Proven, each requiring progressively stronger evidence.

What if I can't find a test or symbol path?

Downgrade the claim to Designed and add design evidence; schedule a follow-up to locate or add the missing artifacts.