[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"skill:01M72FK4XYE6YA3V7R8MSW89DB":3,"skill-same:01M72FK4XYE6YA3V7R8MSW89DB":82},{"skillNo":4,"name":5,"nameZh":6,"description":7,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":16,"categories":17,"keywords":18,"publishedAt":19,"linkOnly":20,"markdownContent":21,"markdownBody":22,"frontMatter":23,"refName":43,"commitSha":44,"skillDirectory":45,"forksCount":14,"repositoryHtmlUrl":46,"repositoryHomepageUrl":6,"ownerHtmlUrl":47,"ownerBlogUrl":6,"ownerDescription":48,"updatedAt":49,"install":50,"resources":63},"01M72FK4XYE6YA3V7R8MSW89DB","implementing-llms-litgpt",null,"Implements and trains LLMs using Lightning AI's LitGPT with 20+ pretrained architectures (Llama, Gemma, Phi, Qwen, Mistral). Use when need clean model implementations, educational understanding of architectures, or production fine-tuning with LoRA\u002FQLoRA. Single-file implementations, no abstraction layers.","cocyuhao\u002Fai-dev-tooling-research-archive","cocyuhao","https:\u002F\u002Favatars.githubusercontent.com\u002Fu\u002F269083740?v=4","01M72F908ET5R76KP9NXS6Y2GJ","Hao Yu","01M72F909376KCNJVKG8MEPY2Q",0,"MIT","https:\u002F\u002Fgithub.com\u002Fcocyuhao\u002Fai-dev-tooling-research-archive\u002Fblob\u002Fmain\u002Fclaude-code-templates\u002Fcli-tool\u002Fcomponents\u002Fskills\u002Fai-research\u002Fmodel-architecture-litgpt\u002FSKILL.md",[],[],"2026-11-10 18:39:20",false,"---\nname: implementing-llms-litgpt\ndescription: Implements and trains LLMs using Lightning AI's LitGPT with 20+ pretrained architectures (Llama, Gemma, Phi, Qwen, Mistral). Use when need clean model implementations, educational understanding of architectures, or production fine-tuning with LoRA\u002FQLoRA. Single-file implementations, no abstraction layers.\nversion: 1.0.0\nauthor: Orchestra Research\nlicense: MIT\ntags: [Model Architecture, LitGPT, Lightning AI, LLM Implementation, LoRA, QLoRA, Fine-Tuning, Llama, Gemma, Phi, Mistral, Educational]\ndependencies: [litgpt, torch, transformers]\n---\n\n# LitGPT - Clean LLM Implementations\n\n## Quick start\n\nLitGPT provides 20+ pretrained LLM implementations with clean, readable code and production-ready training workflows.\n\n**Installation**:\n```bash\npip install 'litgpt[extra]'\n```\n\n**Load and use any model**:\n```python\nfrom litgpt import LLM\n\n# Load pretrained model\nllm = LLM.load(\"microsoft\u002Fphi-2\")\n\n# Generate text\nresult = llm.generate(\n    \"What is the capital of France?\",\n    max_new_tokens=50,\n    temperature=0.7\n)\nprint(result)\n```\n\n**List available models**:\n```bash\nlitgpt download list\n```\n\n## Common workflows\n\n### Workflow 1: Fine-tune on custom dataset\n\nCopy this checklist:\n\n```\nFine-Tuning Setup:\n- [ ] Step 1: Download pretrained model\n- [ ] Step 2: Prepare dataset\n- [ ] Step 3: Configure training\n- [ ] Step 4: Run fine-tuning\n```\n\n**Step 1: Download pretrained model**\n\n```bash\n# Download Llama 3 8B\nlitgpt download meta-llama\u002FMeta-Llama-3-8B\n\n# Download Phi-2 (smaller, faster)\nlitgpt download microsoft\u002Fphi-2\n\n# Download Gemma 2B\nlitgpt download google\u002Fgemma-2b\n```\n\nModels are saved to `checkpoints\u002F` directory.\n\n**Step 2: Prepare dataset**\n\nLitGPT supports multiple formats:\n\n**Alpaca format** (instruction-response):\n```json\n[\n  {\n    \"instruction\": \"What is the capital of France?\",\n    \"input\": \"\",\n    \"output\": \"The capital of France is Paris.\"\n  },\n  {\n    \"instruction\": \"Translate to Spanish: Hello, how are you?\",\n    \"input\": \"\",\n    \"output\": \"Hola, ¿cómo estás?\"\n  }\n]\n```\n\nSave as `data\u002Fmy_dataset.json`.\n\n**Step 3: Configure training**\n\n```bash\n# Full fine-tuning (requires 40GB+ GPU for 7B models)\nlitgpt finetune \\\n  meta-llama\u002FMeta-Llama-3-8B \\\n  --data JSON \\\n  --data.json_path data\u002Fmy_dataset.json \\\n  --train.max_steps 1000 \\\n  --train.learning_rate 2e-5 \\\n  --train.micro_batch_size 1 \\\n  --train.global_batch_size 16\n\n# LoRA fine-tuning (efficient, 16GB GPU)\nlitgpt finetune_lora \\\n  microsoft\u002Fphi-2 \\\n  --data JSON \\\n  --data.json_path data\u002Fmy_dataset.json \\\n  --lora_r 16 \\\n  --lora_alpha 32 \\\n  --lora_dropout 0.05 \\\n  --train.max_steps 1000 \\\n  --train.learning_rate 1e-4\n```\n\n**Step 4: Run fine-tuning**\n\nTraining saves checkpoints to `out\u002Ffinetune\u002F` automatically.\n\nMonitor training:\n```bash\n# View logs\ntail -f out\u002Ffinetune\u002Flogs.txt\n\n# TensorBoard (if using --train.logger_name tensorboard)\ntensorboard --logdir out\u002Ffinetune\u002Flightning_logs\n```\n\n### Workflow 2: LoRA fine-tuning on single GPU\n\nMost memory-efficient option.\n\n```\nLoRA Training:\n- [ ] Step 1: Choose base model\n- [ ] Step 2: Configure LoRA parameters\n- [ ] Step 3: Train with LoRA\n- [ ] Step 4: Merge LoRA weights (optional)\n```\n\n**Step 1: Choose base model**\n\nFor limited GPU memory (12-16GB):\n- **Phi-2** (2.7B) - Best quality\u002Fsize tradeoff\n- **Llama 3 1B** - Smallest, fastest\n- **Gemma 2B** - Good reasoning\n\n**Step 2: Configure LoRA parameters**\n\n```bash\nlitgpt finetune_lora \\\n  microsoft\u002Fphi-2 \\\n  --data JSON \\\n  --data.json_path data\u002Fmy_dataset.json \\\n  --lora_r 16 \\          # LoRA rank (8-64, higher=more capacity)\n  --lora_alpha 32 \\      # LoRA scaling (typically 2×r)\n  --lora_dropout 0.05 \\  # Prevent overfitting\n  --lora_query true \\    # Apply LoRA to query projection\n  --lora_key false \\     # Usually not needed\n  --lora_value true \\    # Apply LoRA to value projection\n  --lora_projection true \\  # Apply LoRA to output projection\n  --lora_mlp false \\     # Usually not needed\n  --lora_head false      # Usually not needed\n```\n\nLoRA rank guide:\n- `r=8`: Lightweight, 2-4MB adapters\n- `r=16`: Standard, good quality\n- `r=32`: High capacity, use for complex tasks\n- `r=64`: Maximum quality, 4× larger adapters\n\n**Step 3: Train with LoRA**\n\n```bash\nlitgpt finetune_lora \\\n  microsoft\u002Fphi-2 \\\n  --data JSON \\\n  --data.json_path data\u002Fmy_dataset.json \\\n  --lora_r 16 \\\n  --train.epochs 3 \\\n  --train.learning_rate 1e-4 \\\n  --train.micro_batch_size 4 \\\n  --train.global_batch_size 32 \\\n  --out_dir out\u002Fphi2-lora\n\n# Memory usage: ~8-12GB for Phi-2 with LoRA\n```\n\n**Step 4: Merge LoRA weights** (optional)\n\nMerge LoRA adapters into base model for deployment:\n\n```bash\nlitgpt merge_lora \\\n  out\u002Fphi2-lora\u002Ffinal \\\n  --out_dir out\u002Fphi2-merged\n```\n\nNow use merged model:\n```python\nfrom litgpt import LLM\nllm = LLM.load(\"out\u002Fphi2-merged\")\n```\n\n### Workflow 3: Pretrain from scratch\n\nTrain new model on your domain data.\n\n```\nPretraining:\n- [ ] Step 1: Prepare pretraining dataset\n- [ ] Step 2: Configure model architecture\n- [ ] Step 3: Set up multi-GPU training\n- [ ] Step 4: Launch pretraining\n```\n\n**Step 1: Prepare pretraining dataset**\n\nLitGPT expects tokenized data. Use `prepare_dataset.py`:\n\n```bash\npython scripts\u002Fprepare_dataset.py \\\n  --source_path data\u002Fmy_corpus.txt \\\n  --checkpoint_dir checkpoints\u002Ftokenizer \\\n  --destination_path data\u002Fpretrain \\\n  --split train,val\n```\n\n**Step 2: Configure model architecture**\n\nEdit config file or use existing:\n\n```python\n# config\u002Fpythia-160m.yaml\nmodel_name: pythia-160m\nblock_size: 2048\nvocab_size: 50304\nn_layer: 12\nn_head: 12\nn_embd: 768\nrotary_percentage: 0.25\nparallel_residual: true\nbias: true\n```\n\n**Step 3: Set up multi-GPU training**\n\n```bash\n# Single GPU\nlitgpt pretrain \\\n  --config config\u002Fpythia-160m.yaml \\\n  --data.data_dir data\u002Fpretrain \\\n  --train.max_tokens 10_000_000_000\n\n# Multi-GPU with FSDP\nlitgpt pretrain \\\n  --config config\u002Fpythia-1b.yaml \\\n  --data.data_dir data\u002Fpretrain \\\n  --devices 8 \\\n  --train.max_tokens 100_000_000_000\n```\n\n**Step 4: Launch pretraining**\n\nFor large-scale pretraining on cluster:\n\n```bash\n# Using SLURM\nsbatch --nodes=8 --gpus-per-node=8 \\\n  pretrain_script.sh\n\n# pretrain_script.sh content:\nlitgpt pretrain \\\n  --config config\u002Fpythia-1b.yaml \\\n  --data.data_dir \u002Fshared\u002Fdata\u002Fpretrain \\\n  --devices 8 \\\n  --num_nodes 8 \\\n  --train.global_batch_size 512 \\\n  --train.max_tokens 300_000_000_000\n```\n\n### Workflow 4: Convert and deploy model\n\nExport LitGPT models for production.\n\n```\nModel Deployment:\n- [ ] Step 1: Test inference locally\n- [ ] Step 2: Quantize model (optional)\n- [ ] Step 3: Convert to GGUF (for llama.cpp)\n- [ ] Step 4: Deploy with API\n```\n\n**Step 1: Test inference locally**\n\n```python\nfrom litgpt import LLM\n\nllm = LLM.load(\"out\u002Fphi2-lora\u002Ffinal\")\n\n# Single generation\nprint(llm.generate(\"What is machine learning?\"))\n\n# Streaming\nfor token in llm.generate(\"Explain quantum computing\", stream=True):\n    print(token, end=\"\", flush=True)\n\n# Batch inference\nprompts = [\"Hello\", \"Goodbye\", \"Thank you\"]\nresults = [llm.generate(p) for p in prompts]\n```\n\n**Step 2: Quantize model** (optional)\n\nReduce model size with minimal quality loss:\n\n```bash\n# 8-bit quantization (50% size reduction)\nlitgpt convert_lit_checkpoint \\\n  out\u002Fphi2-lora\u002Ffinal \\\n  --dtype bfloat16 \\\n  --quantize bnb.nf4\n\n# 4-bit quantization (75% size reduction)\nlitgpt convert_lit_checkpoint \\\n  out\u002Fphi2-lora\u002Ffinal \\\n  --quantize bnb.nf4-dq  # Double quantization\n```\n\n**Step 3: Convert to GGUF** (for llama.cpp)\n\n```bash\npython scripts\u002Fconvert_lit_checkpoint.py \\\n  --checkpoint_path out\u002Fphi2-lora\u002Ffinal \\\n  --output_path models\u002Fphi2.gguf \\\n  --model_name microsoft\u002Fphi-2\n```\n\n**Step 4: Deploy with API**\n\n```python\nfrom fastapi import FastAPI\nfrom litgpt import LLM\n\napp = FastAPI()\nllm = LLM.load(\"out\u002Fphi2-lora\u002Ffinal\")\n\n@app.post(\"\u002Fgenerate\")\ndef generate(prompt: str, max_tokens: int = 100):\n    result = llm.generate(\n        prompt,\n        max_new_tokens=max_tokens,\n        temperature=0.7\n    )\n    return {\"response\": result}\n\n# Run: uvicorn api:app --host 0.0.0.0 --port 8000\n```\n\n## When to use vs alternatives\n\n**Use LitGPT when:**\n- Want to understand LLM architectures (clean, readable code)\n- Need production-ready training recipes\n- Educational purposes or research\n- Prototyping new model ideas\n- Lightning ecosystem user\n\n**Use alternatives instead:**\n- **Axolotl\u002FTRL**: More fine-tuning features, YAML configs\n- **Megatron-Core**: Maximum performance for >70B models\n- **HuggingFace Transformers**: Broadest model support\n- **vLLM**: Inference-only (no training)\n\n## Common issues\n\n**Issue: Out of memory during fine-tuning**\n\nUse LoRA instead of full fine-tuning:\n```bash\n# Instead of litgpt finetune (requires 40GB+)\nlitgpt finetune_lora  # Only needs 12-16GB\n```\n\nOr enable gradient checkpointing:\n```bash\nlitgpt finetune_lora \\\n  ... \\\n  --train.gradient_accumulation_iters 4  # Accumulate gradients\n```\n\n**Issue: Training too slow**\n\nEnable Flash Attention (built-in, automatic on compatible hardware):\n```python\n# Already enabled by default on Ampere+ GPUs (A100, RTX 30\u002F40 series)\n# No configuration needed\n```\n\nUse smaller micro-batch and accumulate:\n```bash\n--train.micro_batch_size 1 \\\n--train.global_batch_size 32 \\\n--train.gradient_accumulation_iters 32  # Effective batch=32\n```\n\n**Issue: Model not loading**\n\nCheck model name:\n```bash\n# List all available models\nlitgpt download list\n\n# Download if not exists\nlitgpt download meta-llama\u002FMeta-Llama-3-8B\n```\n\nVerify checkpoints directory:\n```bash\nls checkpoints\u002F\n# Should see: meta-llama\u002FMeta-Llama-3-8B\u002F\n```\n\n**Issue: LoRA adapters too large**\n\nReduce LoRA rank:\n```bash\n--lora_r 8  # Instead of 16 or 32\n```\n\nApply LoRA to fewer layers:\n```bash\n--lora_query true \\\n--lora_value true \\\n--lora_projection false \\  # Disable this\n--lora_mlp false  # And this\n```\n\n## Advanced topics\n\n**Supported architectures**: See [references\u002Fsupported-models.md](references\u002Fsupported-models.md) for complete list of 20+ model families with sizes and capabilities.\n\n**Training recipes**: See [references\u002Ftraining-recipes.md](references\u002Ftraining-recipes.md) for proven hyperparameter configurations for pretraining and fine-tuning.\n\n**FSDP configuration**: See [references\u002Fdistributed-training.md](references\u002Fdistributed-training.md) for multi-GPU training with Fully Sharded Data Parallel.\n\n**Custom architectures**: See [references\u002Fcustom-models.md](references\u002Fcustom-models.md) for implementing new model architectures in LitGPT style.\n\n## Hardware requirements\n\n- **GPU**: NVIDIA (CUDA 11.8+), AMD (ROCm), Apple Silicon (MPS)\n- **Memory**:\n  - Inference (Phi-2): 6GB\n  - LoRA fine-tuning (7B): 16GB\n  - Full fine-tuning (7B): 40GB+\n  - Pretraining (1B): 24GB\n- **Storage**: 5-50GB per model (depending on size)\n\n## Resources\n\n- GitHub: https:\u002F\u002Fgithub.com\u002FLightning-AI\u002Flitgpt\n- Docs: https:\u002F\u002Flightning.ai\u002Fdocs\u002Flitgpt\n- Tutorials: https:\u002F\u002Flightning.ai\u002Fdocs\u002Flitgpt\u002Ftutorials\n- Model zoo: 20+ pretrained architectures (Llama, Gemma, Phi, Qwen, Mistral, Mixtral, Falcon, etc.)\n\n\n","# LitGPT - Clean LLM Implementations\n\n## Quick start\n\nLitGPT provides 20+ pretrained LLM implementations with clean, readable code and production-ready training workflows.\n\n**Installation**:\n```bash\npip install 'litgpt[extra]'\n```\n\n**Load and use any model**:\n```python\nfrom litgpt import LLM\n\n# Load pretrained model\nllm = LLM.load(\"microsoft\u002Fphi-2\")\n\n# Generate text\nresult = llm.generate(\n    \"What is the capital of France?\",\n    max_new_tokens=50,\n    temperature=0.7\n)\nprint(result)\n```\n\n**List available models**:\n```bash\nlitgpt download list\n```\n\n## Common workflows\n\n### Workflow 1: Fine-tune on custom dataset\n\nCopy this checklist:\n\n```\nFine-Tuning Setup:\n- [ ] Step 1: Download pretrained model\n- [ ] Step 2: Prepare dataset\n- [ ] Step 3: Configure training\n- [ ] Step 4: Run fine-tuning\n```\n\n**Step 1: Download pretrained model**\n\n```bash\n# Download Llama 3 8B\nlitgpt download meta-llama\u002FMeta-Llama-3-8B\n\n# Download Phi-2 (smaller, faster)\nlitgpt download microsoft\u002Fphi-2\n\n# Download Gemma 2B\nlitgpt download google\u002Fgemma-2b\n```\n\nModels are saved to `checkpoints\u002F` directory.\n\n**Step 2: Prepare dataset**\n\nLitGPT supports multiple formats:\n\n**Alpaca format** (instruction-response):\n```json\n[\n  {\n    \"instruction\": \"What is the capital of France?\",\n    \"input\": \"\",\n    \"output\": \"The capital of France is Paris.\"\n  },\n  {\n    \"instruction\": \"Translate to Spanish: Hello, how are you?\",\n    \"input\": \"\",\n    \"output\": \"Hola, ¿cómo estás?\"\n  }\n]\n```\n\nSave as `data\u002Fmy_dataset.json`.\n\n**Step 3: Configure training**\n\n```bash\n# Full fine-tuning (requires 40GB+ GPU for 7B models)\nlitgpt finetune \\\n  meta-llama\u002FMeta-Llama-3-8B \\\n  --data JSON \\\n  --data.json_path data\u002Fmy_dataset.json \\\n  --train.max_steps 1000 \\\n  --train.learning_rate 2e-5 \\\n  --train.micro_batch_size 1 \\\n  --train.global_batch_size 16\n\n# LoRA fine-tuning (efficient, 16GB GPU)\nlitgpt finetune_lora \\\n  microsoft\u002Fphi-2 \\\n  --data JSON \\\n  --data.json_path data\u002Fmy_dataset.json \\\n  --lora_r 16 \\\n  --lora_alpha 32 \\\n  --lora_dropout 0.05 \\\n  --train.max_steps 1000 \\\n  --train.learning_rate 1e-4\n```\n\n**Step 4: Run fine-tuning**\n\nTraining saves checkpoints to `out\u002Ffinetune\u002F` automatically.\n\nMonitor training:\n```bash\n# View logs\ntail -f out\u002Ffinetune\u002Flogs.txt\n\n# TensorBoard (if using --train.logger_name tensorboard)\ntensorboard --logdir out\u002Ffinetune\u002Flightning_logs\n```\n\n### Workflow 2: LoRA fine-tuning on single GPU\n\nMost memory-efficient option.\n\n```\nLoRA Training:\n- [ ] Step 1: Choose base model\n- [ ] Step 2: Configure LoRA parameters\n- [ ] Step 3: Train with LoRA\n- [ ] Step 4: Merge LoRA weights (optional)\n```\n\n**Step 1: Choose base model**\n\nFor limited GPU memory (12-16GB):\n- **Phi-2** (2.7B) - Best quality\u002Fsize tradeoff\n- **Llama 3 1B** - Smallest, fastest\n- **Gemma 2B** - Good reasoning\n\n**Step 2: Configure LoRA parameters**\n\n```bash\nlitgpt finetune_lora \\\n  microsoft\u002Fphi-2 \\\n  --data JSON \\\n  --data.json_path data\u002Fmy_dataset.json \\\n  --lora_r 16 \\          # LoRA rank (8-64, higher=more capacity)\n  --lora_alpha 32 \\      # LoRA scaling (typically 2×r)\n  --lora_dropout 0.05 \\  # Prevent overfitting\n  --lora_query true \\    # Apply LoRA to query projection\n  --lora_key false \\     # Usually not needed\n  --lora_value true \\    # Apply LoRA to value projection\n  --lora_projection true \\  # Apply LoRA to output projection\n  --lora_mlp false \\     # Usually not needed\n  --lora_head false      # Usually not needed\n```\n\nLoRA rank guide:\n- `r=8`: Lightweight, 2-4MB adapters\n- `r=16`: Standard, good quality\n- `r=32`: High capacity, use for complex tasks\n- `r=64`: Maximum quality, 4× larger adapters\n\n**Step 3: Train with LoRA**\n\n```bash\nlitgpt finetune_lora \\\n  microsoft\u002Fphi-2 \\\n  --data JSON \\\n  --data.json_path data\u002Fmy_dataset.json \\\n  --lora_r 16 \\\n  --train.epochs 3 \\\n  --train.learning_rate 1e-4 \\\n  --train.micro_batch_size 4 \\\n  --train.global_batch_size 32 \\\n  --out_dir out\u002Fphi2-lora\n\n# Memory usage: ~8-12GB for Phi-2 with LoRA\n```\n\n**Step 4: Merge LoRA weights** (optional)\n\nMerge LoRA adapters into base model for deployment:\n\n```bash\nlitgpt merge_lora \\\n  out\u002Fphi2-lora\u002Ffinal \\\n  --out_dir out\u002Fphi2-merged\n```\n\nNow use merged model:\n```python\nfrom litgpt import LLM\nllm = LLM.load(\"out\u002Fphi2-merged\")\n```\n\n### Workflow 3: Pretrain from scratch\n\nTrain new model on your domain data.\n\n```\nPretraining:\n- [ ] Step 1: Prepare pretraining dataset\n- [ ] Step 2: Configure model architecture\n- [ ] Step 3: Set up multi-GPU training\n- [ ] Step 4: Launch pretraining\n```\n\n**Step 1: Prepare pretraining dataset**\n\nLitGPT expects tokenized data. Use `prepare_dataset.py`:\n\n```bash\npython scripts\u002Fprepare_dataset.py \\\n  --source_path data\u002Fmy_corpus.txt \\\n  --checkpoint_dir checkpoints\u002Ftokenizer \\\n  --destination_path data\u002Fpretrain \\\n  --split train,val\n```\n\n**Step 2: Configure model architecture**\n\nEdit config file or use existing:\n\n```python\n# config\u002Fpythia-160m.yaml\nmodel_name: pythia-160m\nblock_size: 2048\nvocab_size: 50304\nn_layer: 12\nn_head: 12\nn_embd: 768\nrotary_percentage: 0.25\nparallel_residual: true\nbias: true\n```\n\n**Step 3: Set up multi-GPU training**\n\n```bash\n# Single GPU\nlitgpt pretrain \\\n  --config config\u002Fpythia-160m.yaml \\\n  --data.data_dir data\u002Fpretrain \\\n  --train.max_tokens 10_000_000_000\n\n# Multi-GPU with FSDP\nlitgpt pretrain \\\n  --config config\u002Fpythia-1b.yaml \\\n  --data.data_dir data\u002Fpretrain \\\n  --devices 8 \\\n  --train.max_tokens 100_000_000_000\n```\n\n**Step 4: Launch pretraining**\n\nFor large-scale pretraining on cluster:\n\n```bash\n# Using SLURM\nsbatch --nodes=8 --gpus-per-node=8 \\\n  pretrain_script.sh\n\n# pretrain_script.sh content:\nlitgpt pretrain \\\n  --config config\u002Fpythia-1b.yaml \\\n  --data.data_dir \u002Fshared\u002Fdata\u002Fpretrain \\\n  --devices 8 \\\n  --num_nodes 8 \\\n  --train.global_batch_size 512 \\\n  --train.max_tokens 300_000_000_000\n```\n\n### Workflow 4: Convert and deploy model\n\nExport LitGPT models for production.\n\n```\nModel Deployment:\n- [ ] Step 1: Test inference locally\n- [ ] Step 2: Quantize model (optional)\n- [ ] Step 3: Convert to GGUF (for llama.cpp)\n- [ ] Step 4: Deploy with API\n```\n\n**Step 1: Test inference locally**\n\n```python\nfrom litgpt import LLM\n\nllm = LLM.load(\"out\u002Fphi2-lora\u002Ffinal\")\n\n# Single generation\nprint(llm.generate(\"What is machine learning?\"))\n\n# Streaming\nfor token in llm.generate(\"Explain quantum computing\", stream=True):\n    print(token, end=\"\", flush=True)\n\n# Batch inference\nprompts = [\"Hello\", \"Goodbye\", \"Thank you\"]\nresults = [llm.generate(p) for p in prompts]\n```\n\n**Step 2: Quantize model** (optional)\n\nReduce model size with minimal quality loss:\n\n```bash\n# 8-bit quantization (50% size reduction)\nlitgpt convert_lit_checkpoint \\\n  out\u002Fphi2-lora\u002Ffinal \\\n  --dtype bfloat16 \\\n  --quantize bnb.nf4\n\n# 4-bit quantization (75% size reduction)\nlitgpt convert_lit_checkpoint \\\n  out\u002Fphi2-lora\u002Ffinal \\\n  --quantize bnb.nf4-dq  # Double quantization\n```\n\n**Step 3: Convert to GGUF** (for llama.cpp)\n\n```bash\npython scripts\u002Fconvert_lit_checkpoint.py \\\n  --checkpoint_path out\u002Fphi2-lora\u002Ffinal \\\n  --output_path models\u002Fphi2.gguf \\\n  --model_name microsoft\u002Fphi-2\n```\n\n**Step 4: Deploy with API**\n\n```python\nfrom fastapi import FastAPI\nfrom litgpt import LLM\n\napp = FastAPI()\nllm = LLM.load(\"out\u002Fphi2-lora\u002Ffinal\")\n\n@app.post(\"\u002Fgenerate\")\ndef generate(prompt: str, max_tokens: int = 100):\n    result = llm.generate(\n        prompt,\n        max_new_tokens=max_tokens,\n        temperature=0.7\n    )\n    return {\"response\": result}\n\n# Run: uvicorn api:app --host 0.0.0.0 --port 8000\n```\n\n## When to use vs alternatives\n\n**Use LitGPT when:**\n- Want to understand LLM architectures (clean, readable code)\n- Need production-ready training recipes\n- Educational purposes or research\n- Prototyping new model ideas\n- Lightning ecosystem user\n\n**Use alternatives instead:**\n- **Axolotl\u002FTRL**: More fine-tuning features, YAML configs\n- **Megatron-Core**: Maximum performance for >70B models\n- **HuggingFace Transformers**: Broadest model support\n- **vLLM**: Inference-only (no training)\n\n## Common issues\n\n**Issue: Out of memory during fine-tuning**\n\nUse LoRA instead of full fine-tuning:\n```bash\n# Instead of litgpt finetune (requires 40GB+)\nlitgpt finetune_lora  # Only needs 12-16GB\n```\n\nOr enable gradient checkpointing:\n```bash\nlitgpt finetune_lora \\\n  ... \\\n  --train.gradient_accumulation_iters 4  # Accumulate gradients\n```\n\n**Issue: Training too slow**\n\nEnable Flash Attention (built-in, automatic on compatible hardware):\n```python\n# Already enabled by default on Ampere+ GPUs (A100, RTX 30\u002F40 series)\n# No configuration needed\n```\n\nUse smaller micro-batch and accumulate:\n```bash\n--train.micro_batch_size 1 \\\n--train.global_batch_size 32 \\\n--train.gradient_accumulation_iters 32  # Effective batch=32\n```\n\n**Issue: Model not loading**\n\nCheck model name:\n```bash\n# List all available models\nlitgpt download list\n\n# Download if not exists\nlitgpt download meta-llama\u002FMeta-Llama-3-8B\n```\n\nVerify checkpoints directory:\n```bash\nls checkpoints\u002F\n# Should see: meta-llama\u002FMeta-Llama-3-8B\u002F\n```\n\n**Issue: LoRA adapters too large**\n\nReduce LoRA rank:\n```bash\n--lora_r 8  # Instead of 16 or 32\n```\n\nApply LoRA to fewer layers:\n```bash\n--lora_query true \\\n--lora_value true \\\n--lora_projection false \\  # Disable this\n--lora_mlp false  # And this\n```\n\n## Advanced topics\n\n**Supported architectures**: See [references\u002Fsupported-models.md](references\u002Fsupported-models.md) for complete list of 20+ model families with sizes and capabilities.\n\n**Training recipes**: See [references\u002Ftraining-recipes.md](references\u002Ftraining-recipes.md) for proven hyperparameter configurations for pretraining and fine-tuning.\n\n**FSDP configuration**: See [references\u002Fdistributed-training.md](references\u002Fdistributed-training.md) for multi-GPU training with Fully Sharded Data Parallel.\n\n**Custom architectures**: See [references\u002Fcustom-models.md](references\u002Fcustom-models.md) for implementing new model architectures in LitGPT style.\n\n## Hardware requirements\n\n- **GPU**: NVIDIA (CUDA 11.8+), AMD (ROCm), Apple Silicon (MPS)\n- **Memory**:\n  - Inference (Phi-2): 6GB\n  - LoRA fine-tuning (7B): 16GB\n  - Full fine-tuning (7B): 40GB+\n  - Pretraining (1B): 24GB\n- **Storage**: 5-50GB per model (depending on size)\n\n## Resources\n\n- GitHub: https:\u002F\u002Fgithub.com\u002FLightning-AI\u002Flitgpt\n- Docs: https:\u002F\u002Flightning.ai\u002Fdocs\u002Flitgpt\n- Tutorials: https:\u002F\u002Flightning.ai\u002Fdocs\u002Flitgpt\u002Ftutorials\n- Model zoo: 20+ pretrained architectures (Llama, Gemma, Phi, Qwen, Mistral, Mixtral, Falcon, etc.)\n\n\n",{"name":5,"tags":24,"author":37,"license":15,"version":38,"description":7,"dependencies":39},[25,26,27,28,29,30,31,32,33,34,35,36],"Model Architecture","LitGPT","Lightning AI","LLM Implementation","LoRA","QLoRA","Fine-Tuning","Llama","Gemma","Phi","Mistral","Educational","Orchestra Research","1.0.0",[40,41,42],"litgpt","torch","transformers","main","f4a24bd1ab047411d4ae57ee9a96701a7706e779","claude-code-templates\u002Fcli-tool\u002Fcomponents\u002Fskills\u002Fai-research\u002Fmodel-architecture-litgpt","https:\u002F\u002Fgithub.com\u002Fcocyuhao\u002Fai-dev-tooling-research-archive","https:\u002F\u002Fgithub.com\u002Fcocyuhao","Beijing Wuzi University","2026-05-12 10:57:02",{"source":8,"skillName":5,"commands":51,"prompt":61,"zipAvailable":62},[52,55,58],{"tool":53,"command":54},"npx","npx skills add cocyuhao\u002Fai-dev-tooling-research-archive --skill implementing-llms-litgpt",{"tool":56,"command":57},"bunx","bunx skills add cocyuhao\u002Fai-dev-tooling-research-archive --skill implementing-llms-litgpt",{"tool":59,"command":60},"pnpm","pnpm dlx skills add cocyuhao\u002Fai-dev-tooling-research-archive --skill implementing-llms-litgpt","安装技能，技能地址：https:\u002F\u002Fgithub.com\u002Fcocyuhao\u002Fai-dev-tooling-research-archive\u002Fblob\u002Fmain\u002Fclaude-code-templates\u002Fcli-tool\u002Fcomponents\u002Fskills\u002Fai-research\u002Fmodel-architecture-litgpt\u002FSKILL.md",true,[64,70,74,78],{"path":65,"resourceType":66,"resourceTypeDesc":67,"sizeBytes":68,"sourceHtmlUrl":69},"references\u002Fcustom-models.md","REFERENCE","参考文档",15655,"https:\u002F\u002Fgithub.com\u002Fcocyuhao\u002Fai-dev-tooling-research-archive\u002Fblob\u002Fmain\u002Fclaude-code-templates\u002Fcli-tool\u002Fcomponents\u002Fskills\u002Fai-research\u002Fmodel-architecture-litgpt\u002Freferences\u002Fcustom-models.md",{"path":71,"resourceType":66,"resourceTypeDesc":67,"sizeBytes":72,"sourceHtmlUrl":73},"references\u002Fdistributed-training.md",11099,"https:\u002F\u002Fgithub.com\u002Fcocyuhao\u002Fai-dev-tooling-research-archive\u002Fblob\u002Fmain\u002Fclaude-code-templates\u002Fcli-tool\u002Fcomponents\u002Fskills\u002Fai-research\u002Fmodel-architecture-litgpt\u002Freferences\u002Fdistributed-training.md",{"path":75,"resourceType":66,"resourceTypeDesc":67,"sizeBytes":76,"sourceHtmlUrl":77},"references\u002Fsupported-models.md",7935,"https:\u002F\u002Fgithub.com\u002Fcocyuhao\u002Fai-dev-tooling-research-archive\u002Fblob\u002Fmain\u002Fclaude-code-templates\u002Fcli-tool\u002Fcomponents\u002Fskills\u002Fai-research\u002Fmodel-architecture-litgpt\u002Freferences\u002Fsupported-models.md",{"path":79,"resourceType":66,"resourceTypeDesc":67,"sizeBytes":80,"sourceHtmlUrl":81},"references\u002Ftraining-recipes.md",10978,"https:\u002F\u002Fgithub.com\u002Fcocyuhao\u002Fai-dev-tooling-research-archive\u002Fblob\u002Fmain\u002Fclaude-code-templates\u002Fcli-tool\u002Fcomponents\u002Fskills\u002Fai-research\u002Fmodel-architecture-litgpt\u002Freferences\u002Ftraining-recipes.md",[83,91,99,106,113,120],{"skillNo":84,"name":85,"nameZh":6,"description":86,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":87,"categories":88,"keywords":89,"publishedAt":90},"01M72JQ593HE3SRJD3446CCKA8","web-access","所有联网操作必须通过此 skill 处理，包括：搜索、网页抓取、登录后操作、网络交互等。 触发场景：用户要求搜索信息、查看网页内容、访问需要登录的网站、操作网页界面、抓取社交媒体内容（小红书、微博、推特等）、读取动态渲染页面、以及任何需要真实浏览器环境的网络任务。","https:\u002F\u002Fgithub.com\u002Fcocyuhao\u002Fai-dev-tooling-research-archive\u002Fblob\u002Fmain\u002Fweb-access\u002FSKILL.md",[],[],"2026-11-10 22:39:42",{"skillNo":92,"name":93,"nameZh":6,"description":94,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":95,"sourceHtmlUrl":96,"categories":97,"keywords":98,"publishedAt":90},"01M72JKXJYFT2RBJ2HP5ZRA853","last30days","Multi-query social research across Reddit, X, YouTube, TikTok, Instagram, Hacker News, Polymarket, and the web. Intelligent query planning with Gemini\u002FOpenAI fallback.","https:\u002F\u002Fgithub.com\u002Fcocyuhao\u002Fai-dev-tooling-research-archive\u002Fblob\u002Fmain\u002Flast30days-skill\u002FLICENSE","https:\u002F\u002Fgithub.com\u002Fcocyuhao\u002Fai-dev-tooling-research-archive\u002Fblob\u002Fmain\u002Flast30days-skill\u002FSKILL.md",[],[],{"skillNo":100,"name":101,"nameZh":6,"description":102,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":103,"categories":104,"keywords":105,"publishedAt":90},"01M72JKMRQSD0ZY66KQ863HVY5","writing-plans","Use when you have a spec or requirements for a multi-step task. Creates comprehensive implementation plans with bite-sized tasks, exact file paths, and complete code examples.","https:\u002F\u002Fgithub.com\u002Fcocyuhao\u002Fai-dev-tooling-research-archive\u002Fblob\u002Fmain\u002Fhermes-agent\u002Fskills\u002Fsoftware-development\u002Fwriting-plans\u002FSKILL.md",[],[],{"skillNo":107,"name":108,"nameZh":6,"description":109,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":110,"categories":111,"keywords":112,"publishedAt":90},"01M72JKHCV8DT6VWSD4KS5YWT9","test-driven-development","Use when implementing any feature or bugfix, before writing implementation code. Enforces RED-GREEN-REFACTOR cycle with test-first approach.","https:\u002F\u002Fgithub.com\u002Fcocyuhao\u002Fai-dev-tooling-research-archive\u002Fblob\u002Fmain\u002Fhermes-agent\u002Fskills\u002Fsoftware-development\u002Ftest-driven-development\u002FSKILL.md",[],[],{"skillNo":114,"name":115,"nameZh":6,"description":116,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":117,"categories":118,"keywords":119,"publishedAt":90},"01M72JKD7VX7EQM38R704F3KT1","systematic-debugging","Use when encountering any bug, test failure, or unexpected behavior. 4-phase root cause investigation — NO fixes without understanding the problem first.","https:\u002F\u002Fgithub.com\u002Fcocyuhao\u002Fai-dev-tooling-research-archive\u002Fblob\u002Fmain\u002Fhermes-agent\u002Fskills\u002Fsoftware-development\u002Fsystematic-debugging\u002FSKILL.md",[],[],{"skillNo":121,"name":122,"nameZh":6,"description":123,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":124,"categories":125,"keywords":126,"publishedAt":90},"01M72JKA9XXB92FS8JHPCYW8CM","subagent-driven-development","Use when executing implementation plans with independent tasks. Dispatches fresh delegate_task per task with two-stage review (spec compliance then code quality).","https:\u002F\u002Fgithub.com\u002Fcocyuhao\u002Fai-dev-tooling-research-archive\u002Fblob\u002Fmain\u002Fhermes-agent\u002Fskills\u002Fsoftware-development\u002Fsubagent-driven-development\u002FSKILL.md",[],[]]