playbooks

home / skills / davila7 / claude-code-templates / fine-tuning-llama-factory

fine-tuning-llama-factory skill

/cli-tool/components/skills/ai-research/fine-tuning-llama-factory

This skill guides you through llama-factory for fine-tuning LLMs with no-code WebUI, enabling multi-model support and advanced QLoRA setups.

This is most likely a fork of the llama-factory skill from orchestra-research
npx playbooks add skill davila7/claude-code-templates --skill fine-tuning-llama-factory

Review the files below or copy the command above to add this skill to your agents.

Files (6)
SKILL.md
2.4 KB
---
name: llama-factory
description: Expert guidance for fine-tuning LLMs with LLaMA-Factory - WebUI no-code, 100+ models, 2/3/4/5/6/8-bit QLoRA, multimodal support
version: 1.0.0
author: Orchestra Research
license: MIT
tags: [Fine-Tuning, LLaMA Factory, LLM, WebUI, No-Code, QLoRA, LoRA, Multimodal, HuggingFace, Llama, Qwen, Gemma]
dependencies: [llmtuner, torch, transformers, datasets, peft, accelerate, gradio]
---

# Llama-Factory Skill

Comprehensive assistance with llama-factory development, generated from official documentation.

## When to Use This Skill

This skill should be triggered when:
- Working with llama-factory
- Asking about llama-factory features or APIs
- Implementing llama-factory solutions
- Debugging llama-factory code
- Learning llama-factory best practices

## Quick Reference

### Common Patterns

*Quick reference patterns will be added as you use the skill.*

## Reference Files

This skill includes comprehensive documentation in `references/`:

- **_images.md** -  Images documentation
- **advanced.md** - Advanced documentation
- **getting_started.md** - Getting Started documentation
- **other.md** - Other documentation

Use `view` to read specific reference files when detailed information is needed.

## Working with This Skill

### For Beginners
Start with the getting_started or tutorials reference files for foundational concepts.

### For Specific Features
Use the appropriate category reference file (api, guides, etc.) for detailed information.

### For Code Examples
The quick reference section above contains common patterns extracted from the official docs.

## Resources

### references/
Organized documentation extracted from official sources. These files contain:
- Detailed explanations
- Code examples with language annotations
- Links to original documentation
- Table of contents for quick navigation

### scripts/
Add helper scripts here for common automation tasks.

### assets/
Add templates, boilerplate, or example projects here.

## Notes

- This skill was automatically generated from official documentation
- Reference files preserve the structure and examples from source docs
- Code examples include language detection for better syntax highlighting
- Quick reference patterns are extracted from common usage examples in the docs

## Updating

To refresh this skill with updated documentation:
1. Re-run the scraper with the same configuration
2. The skill will be rebuilt with the latest information

Overview

This skill provides expert guidance for fine-tuning LLMs using LLaMA-Factory with a WebUI no-code experience, support for 100+ models, QLoRA quantization at 2/3/4/5/6/8-bit, and multimodal capabilities. It bundles configuration patterns, troubleshooting steps, and curated examples to speed model customization and deployment. The content is organized for both beginners and advanced users to find targeted instructions and scripts.

How this skill works

The skill inspects common configuration and workflow needs for LLaMA-Factory: dataset preparation, quantization choices, training parameters, and WebUI operations. It surfaces concise commands, recommended settings for QLoRA and mixed-precision, and checks for common failure modes during fine-tuning. Users can consult the referenced guides and scripts to apply changes, monitor jobs, and validate model behavior.

When to use it

  • You want to fine-tune a LLaMA-family model without writing training code using the WebUI no-code flow.
  • You need guidance choosing QLoRA bit-width (2/3/4/5/6/8) and memory-versus-accuracy trade-offs.
  • You are setting up multimodal inputs (text + images) or integrating a model into a pipeline.
  • You need troubleshooting steps for common training failures, quantization issues, or WebUI errors.
  • You want example scripts and configuration patterns for reproducible experiments.

Best practices

  • Start with small, well-curated datasets to validate training loops before scaling up.
  • Choose the lowest quantization bit-width that fits memory constraints, then evaluate accuracy.
  • Use mixed-precision and gradient checkpointing to reduce VRAM when training larger models.
  • Validate checkpoints with held-out prompts and multimodal examples to detect regressions early.
  • Keep training configs and seed values versioned for reproducibility and auditing.

Example use cases

  • No-code fine-tuning of a 7B model via the WebUI to adapt it for domain-specific customer support.
  • Experimenting with 4-bit QLoRA to compress a 13B model while maintaining acceptable performance.
  • Adding image inputs to a language model to build a multimodal assistant for document review.
  • Automating repeated experiments with provided helper scripts to compare quantization strategies.
  • Debugging a failed training run using the troubleshooting checklist and log inspection steps.

FAQ

Which QLoRA bit-width should I try first?

Start with 4-bit for a good balance of memory savings and accuracy, then try 3- and 2-bit if you need further compression and can accept potential quality loss.

Can I fine-tune multimodal models with the WebUI?

Yes. The WebUI supports multimodal workflows; prepare aligned text-image pairs and validate with multimodal evaluation prompts.