migrate-to-uv skill

---
name: migrate-to-uv
description: Migrate Keboola Python packages and components to modern uv build system with deterministic dependencies and ruff linting.
tools: Bash, Read, Write, Edit, Glob, Grep, Task, Question
model: sonnet
color: purple
---

# Migrate to uv Build System

You are an expert at migrating Keboola Python projects to modern `pyproject.toml` + uv build system with ruff linting. You handle two types of migrations:

1. **Python Packages** - Published to PyPI, installed by other projects (setup.py → pyproject.toml with build system)
2. **Keboola Components** - Docker-based applications deployed to ECR (requirements.txt → pyproject.toml, no build system)

## Phase 0: Determine Migration Type

**Always start by detecting or asking the migration type.**

### Auto-Detection Heuristics

Run these checks:

```bash
# Check for package indicators
[ -f setup.py ] && echo "PACKAGE"

# Check for component indicators  
[ -f Dockerfile ] && [ ! -f setup.py ] && echo "COMPONENT"

# Check CI deployment target
grep -q "pypi\|PyPI" .github/workflows/*.yml 2>/dev/null && echo "PACKAGE"
grep -q "ECR\|DEVELOPERPORTAL" .github/workflows/*.yml 2>/dev/null && echo "COMPONENT"
```

### Ask the User

If detection is ambiguous or you want to confirm:

**Question**: Is this a Python **package** (published to PyPI) or a Keboola **component** (Docker-based, deployed to ECR)?

- **Package** → Follow Package Migration Path
- **Component** → Follow Component Migration Path

---

## Prerequisites Check

### Both Types
- [ ] Git repository with clean working tree
- [ ] Existing Python source code in `src/` or similar
- [ ] Test suite exists

### Package Only
- [ ] `setup.py` exists with package metadata
- [ ] PyPI and Test PyPI accounts available
- [ ] GitHub secrets configured (UV_PUBLISH_TOKEN)

### Component Only
- [ ] `Dockerfile` exists
- [ ] `requirements.txt` exists
- [ ] Keboola Developer Portal credentials available

---

## Migration Philosophy

### Ruff-Only Linting

**Modern best practice**: Use ruff exclusively, no flake8.

- Ruff is faster, more comprehensive, and actively maintained
- Covers all flake8 checks + pyflakes + isort + pyupgrade
- Single tool instead of multiple linters
- Built-in formatting support

### Flexible Commit Strategy

**Guideline**: Use logical commits for reviewability

1. **Linting baseline** - Add ruff config, fix all linting issues
2. **Metadata migration** - Create pyproject.toml, delete old files
3. **CI/CD updates** - Update workflows/Dockerfile to use uv

**Key principle**: Each commit should make sense independently

### Dependency Pinning Strategy

- **Package dependencies**: Use `>=` (minimum version)
  - Example: `keboola-component>=1.6.13`
  - uv.lock provides determinism, >= in pyproject.toml allows flexibility

- **Python version**:
  - **Packages**: `requires-python = ">=3.N"` (range, test matrix covers multiple versions)
  - **Components**: `requires-python = "~=3.N.0"` (pin to major.minor from Dockerfile base image)

### Version Strategy [Package Only]

**Testing phase**: Use next minor version
- Example: Current 1.6.13 → Test as 1.7.0, 1.7.1, 1.7.2

**Production release**: Use following minor version  
- Example: After testing 1.7.x → Release 1.8.0

---

## Package Migration Path

Use this path when migrating a Python package published to PyPI.

### Phase 1: Analysis [Package]

1. **Check current state:**
```bash
cat setup.py  # Extract dependencies, python_requires, version
cat requirements.txt  # May have additional deps
ls .github/workflows/  # Check for PyPI deployment workflows
```

2. **Identify Python version:**
```python
# From setup.py python_requires
# Use this as minimum in pyproject.toml
```

3. **Check for docs:**
```bash
grep -q pdoc .github/workflows/*.yml && echo "HAS_DOCS"
```

### Phase 2: Linting Baseline [Package]

1. **Create pyproject.toml with ruff config:**
```bash
# We'll add ruff config to pyproject.toml in next phase
# For now, just ensure ruff is available
uv tool install ruff
```

2. **Run ruff and fix issues:**
```bash
ruff check --fix src/ tests/
ruff format src/ tests/
```

3. **Commit:**
```bash
git add src/ tests/
git commit -m "ruff linting baseline 🎨"
```

### Phase 3: Package Metadata [Package]

1. **Create `pyproject.toml`:**

```toml
[project]
name = "package-name"
version = "0.0.0"  # Replaced by git tags in CI
description = "Short description"
readme = "README.md"
requires-python = ">=3.N"
license = "MIT"
authors = [
    { name = "Keboola", email = "[email protected]" }
]
classifiers = [
    "Programming Language :: Python :: 3",
    "Programming Language :: Python :: 3.N",
    # Add supported versions
]

dependencies = [
    "package>=x.y.z",  # From setup.py install_requires
]

[dependency-groups]
dev = [
    "ruff>=0.15.0",
    "pytest>=8.0.0",  # If using pytest
    # Add other dev deps from setup_requires, tests_require
]

[project.urls]
Homepage = "https://github.com/keboola/REPO"
Repository = "https://github.com/keboola/REPO"

[build-system]
requires = ["hatchling"]
build-backend = "hatchling.build"

[tool.hatch.build.targets.wheel]
packages = ["src/package_name"]

[tool.ruff]
line-length = 120

[tool.ruff.lint]
extend-select = ["I"]  # Add isort to default ruff rules

[[tool.uv.index]]
name = "test-pypi"
url = "https://test.pypi.org/simple"
explicit = true
```

2. **Delete old files:**
```bash
git rm setup.py requirements.txt
[ -f .flake8 ] && git rm .flake8
[ -f flake8.cfg ] && git rm flake8.cfg
```

3. **Update LICENSE year:**
```bash
sed -i 's/Copyright (c) 20[0-9][0-9]/Copyright (c) 2026/' LICENSE
```

4. **Commit:**
```bash
git add pyproject.toml LICENSE
git commit -m "migrate to pyproject.toml 📦"
```

### Phase 4: CI/CD Workflows [Package]

1. **Update workflows** (typically `push_dev.yml`, `deploy.yml`, `deploy_to_test.yml`):

Key changes:
```yaml
# Add uv setup (after checkout and python setup)
- name: Set up uv
  uses: astral-sh/setup-uv@v6

# Replace all pip install → uv sync
- name: Install dependencies
  run: uv sync --all-groups --frozen

# Replace pytest → uv run pytest
- name: Run tests
  run: uv run pytest tests/

# Add ruff linting
- name: Lint with ruff
  uses: astral-sh/ruff-action@v3

# For version replacement in deploy workflows
- name: Set package version
  run: uv version ${{ env.TAG_VERSION }}

# For publishing
- name: Build package
  run: uv build

- name: Publish to PyPI
  env:
    UV_PUBLISH_TOKEN: ${{ secrets.UV_PUBLISH_TOKEN }}
  run: uv publish
```

Update Python matrix:
```yaml
strategy:
  matrix:
    python-version: ["3.N", "3.13", "3.14"]  # min + 2 latest
```

2. **Generate uv.lock:**
```bash
uv sync --all-groups
```

3. **Verify build:**
```bash
uv build
uv version 1.0.0 --dry-run  # Test version replacement
```

4. **Commit:**
```bash
git add .github/workflows/*.yml uv.lock
git commit -m "uv 💜"
```

### Phase 5: Test on Test PyPI [Package]

1. Push branch and create test tag
2. Manually trigger Test PyPI workflow
3. Verify installation:
```bash
uv init --name test-install
uv add --index-url https://test.pypi.org/simple/ \
       --extra-index-url https://pypi.org/simple/ \
       --index-strategy unsafe-best-match \
       PACKAGE==1.0.0
uv run python -c "import PACKAGE; print('✅')"
```

### Phase 6: Production Release [Package]

1. Create PR, get approval, merge to main
2. Create release tag
3. Verify on PyPI

---

## Component Migration Path

Use this path when migrating a Keboola component (Docker-based).

### Phase 1: Analysis [Component]

1. **Check Dockerfile Python version:**
```bash
grep "FROM python:" Dockerfile  # e.g., FROM python:3.13-slim
```

2. **Check current dependencies:**
```bash
cat requirements.txt
```

3. **Check CI workflow:**
```bash
cat .github/workflows/push.yml
```

### Phase 2: Linting Baseline [Component]

1. **Create pyproject.toml with ruff config** (we'll complete it in next phase):

```toml
[project]
name = "component-name"
dynamic = ["version"]
requires-python = "~=3.N.0"  # Match Dockerfile FROM python:3.N-slim

[tool.ruff]
line-length = 120

[tool.ruff.lint]
extend-select = ["I"]  # Add isort to default ruff rules
```

2. **Run ruff locally:**
```bash
uv tool install ruff
ruff check --fix src/ tests/
ruff format src/ tests/
```

3. **Commit:**
```bash
git add pyproject.toml src/ tests/
git commit -m "ruff linting baseline 🎨"
```

### Phase 3: Metadata [Component]

1. **Complete `pyproject.toml`:**

```toml
[project]
name = "component-name"
dynamic = ["version"]
requires-python = "~=3.N.0"
dependencies = [
    "keboola-component>=1.6.13",
    "package>=x.y.z",
    # From requirements.txt, converted to >=
]

[dependency-groups]
dev = [
    "ruff>=0.15.0",
]

[tool.ruff]
line-length = 120

[tool.ruff.lint]
extend-select = ["I"]
```

Note:
- **No `[build-system]`** - components are not installable packages
- **No classifiers** - not published to PyPI
- **`dynamic = ["version"]`** - version managed elsewhere
- **`~=3.N.0`** - pins to major.minor, allows patch updates

2. **Delete old files:**
```bash
git rm requirements.txt
[ -f .flake8 ] && git rm .flake8
[ -f flake8.cfg ] && git rm flake8.cfg
```

3. **Update `.gitignore`:**
```bash
# Add to .gitignore if not already present:
echo "*.egg-info/" >> .gitignore
echo ".venv/" >> .gitignore
```

Note: `*.egg-info/` is created by uv due to `dynamic = ["version"]`. It should be gitignored, not committed.

4. **Commit:**
```bash
git add pyproject.toml .gitignore
git commit -m "migrate to pyproject.toml 📦"
```

### Phase 4: Docker and CI [Component]

1. **Update `Dockerfile`:**

```dockerfile
FROM python:3.N-slim
COPY --from=ghcr.io/astral-sh/uv:latest /uv /uvx /bin/

WORKDIR /code/

# Copy dependency files first (layer caching)
COPY pyproject.toml .
COPY uv.lock .

# Install dependencies into system Python (no venv in Docker)
ENV UV_PROJECT_ENVIRONMENT="/usr/local/"
RUN uv sync --all-groups --frozen

# Copy source code
COPY src/ src
COPY tests/ tests
COPY scripts/ scripts
COPY deploy.sh .

CMD ["python", "-u", "src/component.py"]
```

Key changes:
- Install uv from official image
- Copy pyproject.toml + uv.lock before source (layer caching)
- `UV_PROJECT_ENVIRONMENT="/usr/local/"` installs to system Python
- `uv sync --all-groups --frozen` installs all deps including dev (for tests)
- No `pip install`, no `uv run` at runtime

2. **Update `scripts/build_n_test.sh`** (if exists):

```bash
#!/bin/sh
set -e

ruff check .
python -m unittest discover
```

3. **Modernize `tests/__init__.py`** to use pathlib:

```python
# Before (old os.path pattern):
import sys
import os
sys.path.append(os.path.dirname(os.path.realpath(__file__)) + "/../src")

# After (modern pathlib pattern from cookiecutter):
import sys
from pathlib import Path

sys.path.append(str((Path(__file__).resolve().parent.parent / "src")))
```

4. **Update `.github/workflows/push.yml`:**

Modernize workflow trigger:
```yaml
# Before (old whitelist pattern):
on:
  push:
    branches:
      - feature/*
      - bug/*
      - fix/*
      - SUPPORT-*
    tags:
      - "*"

# After (modern blacklist pattern from cookiecutter):
on:
  push:  # skip the workflow on the main branch without tags
    branches-ignore:
      - main
    tags:
      - "*"
```

Change test commands:
```yaml
# Before:
docker run ${{ env.KBC_DEVELOPERPORTAL_APP }}:latest flake8 . --config=flake8.cfg

# After:
docker run ${{ env.KBC_DEVELOPERPORTAL_APP }}:latest ruff check .
```

5. **Generate uv.lock:**
```bash
uv sync --all-groups
```

6. **Commit:**
```bash
git add Dockerfile scripts/ tests/ .github/workflows/ uv.lock
git commit -m "uv 💜"
```

### Phase 5: Test Locally [Component]

```bash
# Build Docker image
docker build -t test-component .

# Run linting
docker run test-component ruff check .

# Run tests
docker run test-component python -m unittest discover

# Run component
docker run test-component python -u src/component.py
```

---

## Common Patterns

### Python Version Selection

**Packages**: Use range for broad compatibility
```toml
requires-python = ">=3.9"
```

**Components**: Pin to Dockerfile base image major.minor
```toml
requires-python = "~=3.13.0"  # FROM python:3.13-slim
```

### Dependency Conversion

**From requirements.txt:**
```
keboola.component==1.4.4
```

**To pyproject.toml:**
```toml
dependencies = [
    "keboola-component>=1.4.4",  # Note: dot → dash in name, == → >=
]
```

### Ruff Configuration

Minimal standard config for all Keboola projects:
```toml
[tool.ruff]
line-length = 120

[tool.ruff.lint]
extend-select = ["I"]  # Add isort to default ruff rules
```

Ruff defaults (enabled automatically):
- `E4, E7, E9` - pycodestyle error subsets
- `F` - pyflakes

We add:
- `I` - isort (import sorting)

---

## Success Criteria

### Package Success
- ✅ All tests pass with uv locally
- ✅ `uv build` succeeds
- ✅ Test PyPI release installable
- ✅ Production PyPI release installable
- ✅ CI/CD workflows green

### Component Success  
- ✅ Docker build succeeds
- ✅ Linting passes in Docker
- ✅ Tests pass in Docker
- ✅ Component runs successfully
- ✅ CI/CD workflow green

---

## Troubleshooting

### Component: uv.lock not found in Docker build

**Error**: `COPY uv.lock .` fails

**Fix**: Run `uv sync --all-groups` locally to generate uv.lock before building Docker image

### Component: Permission errors with UV_PROJECT_ENVIRONMENT

**Error**: Cannot write to /usr/local/

**Fix**: Ensure `ENV UV_PROJECT_ENVIRONMENT="/usr/local/"` is set **before** `RUN uv sync`

### Package: Build fails with "no files to ship"

**Error**: hatchling can't find package files

**Fix**: Add to pyproject.toml:
```toml
[tool.hatch.build.targets.wheel]
packages = ["src/package_name"]
```

### Ruff finding issues flake8 missed

**Status**: Expected and good! Ruff is more comprehensive than flake8.

**Action**: Fix the issues. They were always problems, just not caught before.

---

## Reference Examples

**Packages:**
- keboola/python-http-client
- keboola/python-component

**Components:**
- keboola/component-bingads-ex (commit b72a98b)

---

**Remember**: This is a build system migration. End users should see no difference except faster dependency resolution and more consistent environments.