home / skills / tursodatabase / turso / mvcc

mvcc skill

/.claude/skills/mvcc

This skill explains experimental MVCC in Turso, covering snapshot isolation, versioning, limitations, and how it differs from WAL.

npx playbooks add skill tursodatabase/turso --skill mvcc

Review the files below or copy the command above to add this skill to your agents.

Files (1)
SKILL.md
2.7 KB
---
name: mvcc
description: Overview of Experimental MVCC feature - snapshot isolation, versioning, limitations
---
# MVCC Guide (Experimental)

Multi-Version Concurrency Control. **Work in progress, not production-ready.**

**CRITICAL**: Ignore MVCC when debugging unless the bug is MVCC-specific.

## Enabling MVCC

```sql
PRAGMA journal_mode = 'experimental_mvcc';
```

Runtime configuration, not a compile-time feature flag. Per-database setting.

## How It Works

Standard WAL: single version per page, readers see snapshot at read mark time.

MVCC: multiple row versions, snapshot isolation. Each transaction sees consistent snapshot at begin time.

### Key Differences from WAL

| Aspect | WAL | MVCC |
|--------|-----|------|
| Write granularity | Every commit writes full pages | Affected rows only
| Readers/Writers | Don't block each other | Don't block each other |
| Persistence | `.db-wal` | `.db-log` (logical log) |
| Isolation | Snapshot (page-level) | Snapshot (row-level) |

### Versioning

Each row version tracks:
- `begin` - timestamp when visible
- `end` - timestamp when deleted/replaced
- `btree_resident` - existed before MVCC enabled

## Architecture

```
Database
  └─ mv_store: MvStore
      ├─ rows: SkipMap<RowID, Vec<RowVersion>>
      ├─ txs: SkipMap<TxID, Transaction>
      ├─ Storage (.db-log file)
      └─ CheckpointStateMachine
```

**Per-connection**: `mv_tx` tracks current MVCC transaction.

**Shared**: `MvStore` with lock-free `crossbeam_skiplist` structures.

## Key Files

- `core/mvcc/mod.rs` - Module overview
- `core/mvcc/database/mod.rs` - Main implementation (~3000 lines)
- `core/mvcc/cursor.rs` - Merged MVCC + B-tree cursor
- `core/mvcc/persistent_storage/logical_log.rs` - Disk format
- `core/mvcc/database/checkpoint_state_machine.rs` - Checkpoint logic

## Checkpointing

Flushes row versions to B-tree periodically.

```sql
PRAGMA mvcc_checkpoint_threshold = <pages>;
```

Process: acquire lock → begin pager txn → write rows → commit → truncate log → fsync → release.

## Current Limitations

**Not implemented:**
- Garbage collection (old versions accumulate)
- Recovery from logical log on restart

**Known issues:**
- Checkpoint blocks other transactions, even reads!
- No spilling to disk; memory use concerns

## Testing

```bash
# Run MVCC-specific tests
cargo test mvcc

# TCL tests with MVCC
make test-mvcc
```

Use `#[turso_macros::test(mvcc)]` attribute for MVCC-enabled tests.

```rust
#[turso_macros::test(mvcc)]
fn test_something() {
    // runs with MVCC enabled
}
```

## References

- `core/mvcc/mod.rs` documents data anomalies (dirty reads, lost updates, etc.)
- Snapshot isolation vs serializability: MVCC provides the former, not the latter

Overview

This skill explains the experimental MVCC feature for Turso, an in-process SQL database compatible with SQLite. It summarizes snapshot isolation, row versioning, architecture, checkpointing, and current limitations. The content is practical and targeted at developers evaluating or testing MVCC.

How this skill works

MVCC implements snapshot isolation by keeping multiple versions of rows so each transaction sees a consistent snapshot at its begin time. Versions record begin and end timestamps and a flag for rows that existed before MVCC was enabled. MVCC stores logical changes in a .db-log file and merges versions into the B-tree during checkpoints.

When to use it

  • When you need snapshot isolation at the row level for concurrent readers and writers.
  • When evaluating experimental MVCC behavior in Turso for development or tests.
  • When testing multi-version semantics, version visibility, or cursor merging with B-tree data.
  • When running MVCC-specific unit or TCL tests in a controlled environment.
  • Not for production use due to current limitations and incomplete recovery/GC.

Best practices

  • Enable MVCC per database with PRAGMA journal_mode = 'experimental_mvcc' during testing only.
  • Ignore MVCC complexity when debugging unrelated bugs; focus on MVCC-specific issues only when they reproduce under MVCC.
  • Use MVCC test attributes (#[turso_macros::test(mvcc)]) to ensure tests run with MVCC enabled.
  • Set a checkpoint threshold (PRAGMA mvcc_checkpoint_threshold) to control when versions are flushed to the B-tree during tests.
  • Monitor memory usage since there is no spilling to disk or garbage collection yet.

Example use cases

  • Run cargo test mvcc to execute MVCC-specific Rust tests and validate versioning behavior.
  • Use make test-mvcc to run end-to-end TCL tests against the MVCC runtime.
  • Simulate snapshot isolation scenarios to observe begin/end version visibility and isolation anomalies.
  • Experiment with checkpoint thresholds to see how checkpointing merges versions and truncates the logical log.
  • Inspect files like core/mvcc/database/mod.rs and logical_log.rs to understand internal formats and checkpoint state machine.

FAQ

Is MVCC ready for production?

No. MVCC is experimental and not production-ready due to missing garbage collection, incomplete recovery from the logical log, and other known issues.

How do I enable MVCC?

Enable it at runtime per database with PRAGMA journal_mode = 'experimental_mvcc'.

Will MVCC block readers or writers?

No. Like WAL, MVCC does not block readers or writers; it provides snapshot isolation at the row level.