home / skills / jeremylongshore / claude-code-plugins-plus-skills / distributed-training-setup

distributed-training-setup skill

safe

/skills/07-ml-training/distributed-training-setup

This skill provides automated guidance and production-ready configurations for distributed training setup, accelerating ML workflow deployment.

npx playbooks add skill jeremylongshore/claude-code-plugins-plus-skills --skill distributed-training-setup

Review the files below or copy the command above to add this skill to your agents.

Files (1)

SKILL.md

2.2 KB

---
name: "distributed-training-setup"
description: |
  Configure distributed training setup operations. Auto-activating skill for ML Training.
  Triggers on: distributed training setup, distributed training setup
  Part of the ML Training skill category. Use when working with distributed training setup functionality. Trigger with phrases like "distributed training setup", "distributed setup", "distributed".
allowed-tools: "Read, Write, Edit, Bash(python:*), Bash(pip:*)"
version: 1.0.0
license: MIT
author: "Jeremy Longshore <[email protected]>"
---

# Distributed Training Setup

## Overview

This skill provides automated assistance for distributed training setup tasks within the ML Training domain.

## When to Use

This skill activates automatically when you:
- Mention "distributed training setup" in your request
- Ask about distributed training setup patterns or best practices
- Need help with machine learning training skills covering data preparation, model training, hyperparameter tuning, and experiment tracking.

## Instructions

1. Provides step-by-step guidance for distributed training setup
2. Follows industry best practices and patterns
3. Generates production-ready code and configurations
4. Validates outputs against common standards

## Examples

**Example: Basic Usage**
Request: "Help me with distributed training setup"
Result: Provides step-by-step guidance and generates appropriate configurations


## Prerequisites

- Relevant development environment configured
- Access to necessary tools and services
- Basic understanding of ml training concepts


## Output

- Generated configurations and code
- Best practice recommendations
- Validation results


## Error Handling

| Error | Cause | Solution |
|-------|-------|----------|
| Configuration invalid | Missing required fields | Check documentation for required parameters |
| Tool not found | Dependency not installed | Install required tools per prerequisites |
| Permission denied | Insufficient access | Verify credentials and permissions |


## Resources

- Official documentation for related tools
- Best practices guides
- Community examples and tutorials

## Related Skills

Part of the **ML Training** skill category.
Tags: ml, training, pytorch, tensorflow, sklearn

Overview

This skill automates the setup and validation of distributed training environments for machine learning workloads. It guides configuration, generates production-ready code and infra snippets, and enforces common standards so distributed jobs run reliably and reproducibly. Use it to speed deployment of PyTorch, TensorFlow, or mixed-framework training across clusters or cloud instances.

How this skill works

The skill inspects your target architecture (single-node multi-GPU, multi-node, or managed cluster) and the tooling you prefer (MPI, NCCL, Horovod, PyTorch DDP, Kubernetes, or cloud-managed training). It produces step-by-step setup plans, config files (launch scripts, scheduler specs, container options), and example training code or entrypoints. It also runs validation checks for common misconfigurations and suggests fixes for permissions, networking, and dependency issues.

When to use it

You need a repeatable multi-node or multi-GPU training deployment.
Preparing launch scripts, container images, or scheduler configs for distributed jobs.
Validating network, GPU visibility, and inter-node communication settings before large runs.
Converting single-node training code to use DDP/Horovod/MPI patterns.
Creating reproducible experiment environments for hyperparameter sweeps or large-scale jobs.

Best practices

Start from a small pilot (1–2 nodes) to validate orchestration and communication before scaling up.
Use deterministic environment specs: locked container images, pinned package versions, and clear entrypoint scripts.
Prefer framework-native distributed APIs (e.g., PyTorch DDP) for efficiency and simpler debugging.
Automate logging and experiment tracking (e.g., WandB, MLflow) and attach health checks for node/GPU utilization.
Validate permissions and network ports early; ensure NCCL/P2P ports and SSH (if used) are open and authenticated.

Example use cases

Generate Kubernetes Job and Service manifests for a distributed PyTorch DDP training cluster.
Produce a launch script and Dockerfile for multi-GPU training with NCCL tuning and environment variables set.
Convert single-process TensorFlow training script into a TF.distribute-compatible multi-worker script and provide TF_CONFIG examples.
Create a scheduler batch script (Slurm) with node allocation, GPU binding, and job-array support for hyperparameter sweeps.
Run pre-launch checks that validate GPU visibility, NCCL topology, and SSH key access across nodes.

FAQ

What inputs do you need to generate a setup?

Provide target architecture (single-node/multi-node), framework (PyTorch/TensorFlow), cluster type (Kubernetes, Slurm, cloud), GPU counts, and any preferred tooling (DDP, Horovod, NCCL).

Can you validate permission and network issues?

Yes. The skill lists common permission failures, required ports and environment vars, and suggests commands or config changes to resolve them.