[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"skill:01M77T5161F625C67R0NF78HBJ":3,"skill-same:01M77T5161F625C67R0NF78HBJ":61},{"skillNo":4,"name":5,"nameZh":6,"description":7,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":16,"categories":17,"keywords":18,"publishedAt":19,"linkOnly":20,"markdownContent":21,"markdownBody":22,"frontMatter":23,"refName":38,"commitSha":39,"skillDirectory":40,"forksCount":41,"repositoryHtmlUrl":42,"repositoryHomepageUrl":43,"ownerHtmlUrl":44,"ownerBlogUrl":6,"ownerDescription":45,"updatedAt":46,"install":47,"resources":60},"01M77T5161F625C67R0NF78HBJ","rocm_vllm_deployment",null,"Production-ready vLLM deployment on AMD ROCm GPUs. Combines environment auto-check, model parameter detection, Docker Compose deployment, health verification, and functional testing with comprehensive logging and security best practices.","Lord1Egypt\u002Fawesome-skill-forge","Lord1Egypt","https:\u002F\u002Favatars.githubusercontent.com\u002Fu\u002F99746359?v=4","01M7592YDNT0XK9WJ0T2MQVPDN","Mohamed Mounir Ali AL Keshita","01M75GRFNHC8VZNWEB8DKDE433",3,"MIT","https:\u002F\u002Fgithub.com\u002FLord1Egypt\u002Fawesome-skill-forge\u002Fblob\u002Fmaster\u002Fcommunity\u002Fclawhub\u002Fr\u002Frocm-vllm-deployment\u002FSKILL.md",[],[],"2026-11-15 14:19:17",false,"---\nname: rocm_vllm_deployment\ndescription: Production-ready vLLM deployment on AMD ROCm GPUs. Combines environment auto-check, model parameter detection, Docker Compose deployment, health verification, and functional testing with comprehensive logging and security best practices.\nversion: 1.0.0\nauthor: Alex He \u003Cheye_dev@163.com>\ntimeout: 3600s\nplatform: Linux (AMD GPU ROCm)\ntags:\n  - LLM\n  - Deployment\n  - AMD\n  - ROCm\n  - Docker Compose\n  - vLLM\n  - Automation\n  - EnvCheck\n  - AutoRepair\n---\n\n# ROCm vLLM Deployment Skill\n\nProduction-ready automation for deploying vLLM inference services on AMD ROCm GPUs using Docker Compose.\n\n## Features\n\n- Environment Auto-Check - Detects and repairs missing dependencies\n- Model Parameter Detection - Auto-reads config.json for optimal settings\n- VRAM Estimation - Calculates memory requirements before deployment\n- Secure Token Handling - Never writes tokens to compose files\n- **Structured Output** - All logs and test results saved per-model\n- **Deployment Reports** - Human-readable summary for each deployment\n- Health Verification - Automated health checks and functional tests\n- Troubleshooting Guide - Common issues and solutions\n\n## Environment Prerequisites\n\n**Recommended (for production):** Add to `~\u002F.bash_profile`:\n\n```bash\n# HuggingFace authentication token (required for gated models)\nexport HF_TOKEN=\"hf_xxxxxxxxxxxxxxxxxxxxxxxxxxxxxx\"\n\n# Model cache directory (optional)\nexport HF_HOME=\"$HOME\u002Fmodels\"\n\n# Apply changes\nsource ~\u002F.bash_profile\n```\n\n**Not required for testing:** The skill will proceed without these set:\n- **HF_TOKEN**: Optional — public models work without it; gated models fail at download with clear error\n- **HF_HOME**: Optional — defaults to `\u002Froot\u002F.cache\u002Fhuggingface\u002Fhub`\n\n### Environment Variable Detection\n\n**Priority Order:**\n1. **Explicit parameter** (highest) — Provided in task\u002Frequest (e.g., `hf_token: \"xxx\"`)\n2. **Environment variable** — Already set in shell or from parent process\n3. **~\u002F.bash_profile** — Source to load variables\n4. **Default value** (lowest) — HF_HOME defaults to `\u002Froot\u002F.cache\u002Fhuggingface\u002Fhub`\n\n| Variable | Required | If Missing |\n|----------|----------|------------|\n| `HF_TOKEN` | **Conditional** | Continue without token (public models work; gated models fail at download with clear error) |\n| `HF_HOME` | No | **Warning + Default** — Use `\u002Froot\u002F.cache\u002Fhuggingface\u002Fhub` |\n\n**Philosophy:** Fail fast for configuration errors, fail at download time for authentication errors.\n\n---\n\n## Helper Scripts\n\n**Location:** `\u003Cskill-dir>\u002Fscripts\u002F`\n\n### check-env.sh\n\nValidate and load environment variables before deployment.\n\n**Usage:**\n```bash\n# Basic check (HF_TOKEN optional, HF_HOME optional with default)\n.\u002Fscripts\u002Fcheck-env.sh\n\n# Strict mode (HF_HOME required, fails if not set)\n.\u002Fscripts\u002Fcheck-env.sh --strict\n\n# Quiet mode (minimal output, for automation)\n.\u002Fscripts\u002Fcheck-env.sh --quiet\n\n# Test with environment variables\nHF_TOKEN=\"hf_xxx\" HF_HOME=\"\u002Fmodels\" .\u002Fscripts\u002Fcheck-env.sh\n```\n\n**Exit Codes:**\n| Code | Meaning |\n|------|---------|\n| 0 | Environment check completed (variables loaded or defaulted) |\n| 2 | Critical error (e.g., cannot source ~\u002F.bash_profile) |\n\n**Note:** This script is optional. You can also directly run `source ~\u002F.bash_profile`.\n\n---\n\n### generate-report.sh\n\nGenerate human-readable deployment report after successful deployment.\n\n**Usage:**\n```bash\n.\u002Fscripts\u002Fgenerate-report.sh \u003Cmodel-id> \u003Ccontainer-name> \u003Cport> \u003Cstatus> [model-load-time] [memory-used]\n\n# Example:\n.\u002Fscripts\u002Fgenerate-report.sh \\\n  \"Qwen-Qwen3-0.6B\" \\\n  \"vllm-qwen3-0-6b\" \\\n  \"8001\" \\\n  \"✅ Success\" \\\n  \"3.6\" \\\n  \"1.2\"\n```\n\n**Parameters:**\n| Parameter | Required | Description |\n|-----------|----------|-------------|\n| `model-id` | Yes | Model ID (with `\u002F` replaced by `-`) |\n| `container-name` | Yes | Docker container name |\n| `port` | Yes | Host port for API endpoint |\n| `status` | Yes | Deployment status (e.g., \"✅ Success\") |\n| `model-load-time` | No | Model loading time in seconds |\n| `memory-used` | No | Memory consumption in GiB |\n\n**Output:** `$HOME\u002Fvllm-compose\u002F\u003Cmodel-id>\u002FDEPLOYMENT_REPORT.md`\n\n**Exit Codes:**\n| Code | Meaning |\n|------|---------|\n| 0 | Report generated successfully |\n| 1 | Missing required parameters |\n| 2 | Output directory not found |\n\n**Integration:** This script is automatically called in **Phase 7** of the deployment workflow.\n\n---\n\n## Input Schema\n\n| Parameter | Type | Required | Default | Description |\n|-----------|------|----------|---------|-------------|\n| model_id | String | Yes | - | HuggingFace model ID |\n| docker_image | String | No | rocm\u002Fvllm-dev:nightly | vLLM Docker image |\n| tensor_parallel_size | Integer | No | 1 | Number of GPUs |\n| port | Integer | No | 9999 | API server port |\n| hf_home | String | No | `${HF_HOME}` or `\u002Froot\u002F.cache\u002Fhuggingface\u002Fhub` | Model cache directory |\n| hf_token | Secret | Conditional | `${HF_TOKEN}` | HuggingFace token (optional for public models, required for gated models) |\n| max_model_len | Integer | No | Auto-detect | Maximum sequence length |\n| gpu_memory_utilization | Float | No | 0.85 | GPU memory utilization |\n| auto_install | Boolean | No | true | Auto-install dependencies |\n| log_level | String | No | INFO | Logging verbosity |\n\n## Output Structure\n\n**All deployment artifacts MUST be saved to:**\n```\n$HOME\u002Fvllm-compose\u002F\u003Cmodel-id-slash-to-dash>\u002F\n```\n\nConvert model ID to directory name by replacing `\u002F` with `-`:\n- `openai\u002Fgpt-oss-20b` → `$HOME\u002Fvllm-compose\u002Fopenai-gpt-oss-20b\u002F`\n- `Qwen\u002FQwen3-Coder-Next-FP8` → `$HOME\u002Fvllm-compose\u002FQwen-Qwen3-Coder-Next-FP8\u002F`\n\n**Per-model directory structure:**\n```\n$HOME\u002Fvllm-compose\u002F\u003Cmodel-id>\u002F\n├── deployment.log          # Full deployment logs (stdout + stderr)\n├── test-results.json       # Functional test results (JSON format)\n├── docker-compose.yml      # Generated Docker Compose file\n├── .env                    # HF_TOKEN environment (chmod 600, optional)\n└── DEPLOYMENT_REPORT.md    # Human-readable deployment summary\n```\n\n**File requirements:**\n- `deployment.log` — Capture ALL container logs during deployment\n- `test-results.json` — Save API response from functional test request\n- `DEPLOYMENT_REPORT.md` — Generated in Phase 7\n- All three files MUST exist before marking deployment as complete\n\n## Execution Workflow\n\n### Phase 0: Environment Check & Auto-Repair\n\n**Step 0.1: Load Environment Variables**\n\n```bash\n# Source ~\u002F.bash_profile to load HF_HOME and HF_TOKEN\nsource ~\u002F.bash_profile\n\n# If HF_HOME is not defined, it defaults to \u002Froot\u002F.cache\u002Fhuggingface\u002Fhub\n```\n\nIf HF_HOME is not defined in ~\u002F.bash_profile, it defaults to `\u002Froot\u002F.cache\u002Fhuggingface\u002Fhub`.\n\n**Step 0.2: Create Output Directory**\n- Create: `$HOME\u002Fvllm-compose\u002F\u003Cmodel-id>\u002F`\n\n**Step 0.3: Initialize Logging**\n- All output → `$HOME\u002Fvllm-compose\u002F\u003Cmodel-id>\u002Fdeployment.log`\n\n**Step 0.4: System Checks**\n- Detect OS and package manager\n- Check Python, pip, huggingface_hub\n- Check Docker, docker compose\n- Check ROCm tools (rocm-smi\u002Famd-smi)\n- Check GPU access (\u002Fdev\u002Fkfd, \u002Fdev\u002Fdri)\n- Check disk space (20GB minimum)\n\n### Phase 1: Model Download\n\n**Use HF_HOME from Phase 0 (environment variable or default):**\n\n```bash\n# Download model to HF_HOME\nhuggingface-cli download \u003Cmodel_id> --local-dir \"$HF_HOME\u002Fhub\u002Fmodels--\u003Corg>--\u003Cmodel>\"\n\n# Or use snapshot_download via Python:\npython -c \"from huggingface_hub import snapshot_download; snapshot_download(repo_id='\u003Cmodel_id>', cache_dir='$HF_HOME')\"\n```\n\n**Authentication Handling:**\n\n| Scenario | Behavior |\n|----------|----------|\n| Public model + no token | ✅ Download succeeds |\n| Public model + token provided | ✅ Download succeeds |\n| Gated model + no token | ❌ Download fails with \"authentication required\" error |\n| Gated model + invalid token | ❌ Download fails with \"invalid token\" error |\n| Gated model + valid token | ✅ Download succeeds |\n\n**On Authentication Failure:**\n```bash\necho \"ERROR: Model download failed - authentication required\"\necho \"This model requires a valid HF_TOKEN.\"\necho \"\"\necho \"Please add to ~\u002F.bash_profile:\"\necho \"  export HF_TOKEN=\\\"hf_xxxxxxxxxxxxxxxxxxxxxxxxxxxxxx\\\"\"\necho \"Then run: source ~\u002F.bash_profile\"\nexit 1\n```\n\n- Locate model path in HF cache: `$HF_HOME\u002Fhub\u002Fmodels--\u003Corg>--\u003Cmodel-name>\u002F`\n- Log download progress to `deployment.log`\n\n### Phase 2: Model Parameter Detection\n- Read config.json from model\n- Auto-detect: max_model_len, hidden_size, num_attention_heads, num_hidden_layers, vocab_size, dtype\n- Validate TP size divides attention heads\n- Estimate VRAM requirement\n\n### Phase 3: Docker Compose Configuration\n\n**Generate files in output directory:**\n\n- **docker-compose.yml** → `$HOME\u002Fvllm-compose\u002F\u003Cmodel-id>\u002Fdocker-compose.yml`\n  - Mount HF_HOME as volume (read-only for models)\n  - NO hardcoded tokens in compose file\n\n- **.env** → `$HOME\u002Fvllm-compose\u002F\u003Cmodel-id>\u002F.env` (optional)\n  - Contains: `HF_TOKEN=\u003Cvalue>`\n  - Permissions: `chmod 600`\n  - Only created if user explicitly requests persistent token storage\n\n**Volume mount example:**\n```yaml\nvolumes:\n  - ${HF_HOME}:\u002Froot\u002F.cache\u002Fhuggingface\u002Fhub:ro\n  - \u002Fdev\u002Fkfd:\u002Fdev\u002Fkfd\n  - \u002Fdev\u002Fdri:\u002Fdev\u002Fdri\n```\n\n**Important:** Docker Compose reads `${HF_HOME}` from the host environment at runtime. Before running docker compose, source ~\u002F.bash_profile: `source ~\u002F.bash_profile`\n\n\n### Phase 4: Container Launch\n\n**Important:** Before deploying, pull the latest image to ensure updates:\n```bash\ndocker pull rocm\u002Fvllm-dev:nightly\n```\n\n**Note:** Default port is 9999. Before running docker compose, check if port is available: `ss -tlnp | grep :\u003Cport>`. If port is in use, specify a different port in docker-compose.yml.\n\n- Pass HF_TOKEN at runtime: HF_TOKEN=$HF_TOKEN docker compose up -d\n- Wait for container initialization\n\n### Phase 5: Health Verification\n- Check container status\n- Test \u002Fhealth endpoint\n- Test \u002Fv1\u002Fmodels endpoint\n\n### Phase 6: Functional Testing\n- Run completion test via `\u002Fv1\u002Fchat\u002Fcompletions` API\n- **Save response to:** `$HOME\u002Fvllm-compose\u002F\u003Cmodel-id>\u002Ftest-results.json`\n- Verify response contains valid completion\n- **Log deployment complete** → Append to `deployment.log`\n- **Deployment is complete only when both files exist:**\n  - `deployment.log`\n  - `test-results.json`\n\n### Phase 7: Deployment Report\n\n**Generate human-readable deployment report using the helper script.**\n\n**Step 7.1: Extract Deployment Metrics**\n\n```bash\n# Parse deployment.log for metrics\nMODEL_LOAD_TIME=$(grep -o \"model loading took [0-9.]* seconds\" deployment.log | grep -o '[0-9.]*' || echo \"N\u002FA\")\nMEMORY_USED=$(grep -o \"took [0-9.]* GiB memory\" deployment.log | grep -o '[0-9.]*' || echo \"N\u002FA\")\n```\n\n**Step 7.2: Generate Report**\n\n```bash\n# Execute the report generation script\n\u003Cskill-dir>\u002Fscripts\u002Fgenerate-report.sh \\\n  \"\u003Cmodel-id>\" \\\n  \"\u003Ccontainer-name>\" \\\n  \"\u003Cport>\" \\\n  \"\u003Cstatus>\" \\\n  \"$MODEL_LOAD_TIME\" \\\n  \"$MEMORY_USED\"\n\n# Example:\n.\u002Fscripts\u002Fgenerate-report.sh \\\n  \"Qwen-Qwen3-0.6B\" \\\n  \"vllm-qwen3-0-6b\" \\\n  \"8001\" \\\n  \"✅ Success\" \\\n  \"3.6\" \\\n  \"1.2\"\n```\n\n**Output:** `$HOME\u002Fvllm-compose\u002F\u003Cmodel-id>\u002FDEPLOYMENT_REPORT.md`\n\n**Report Contents:**\n- Output structure verification (file checklist)\n- Deployment summary table (health, test, metrics)\n- Test results (request\u002Fresponse preview)\n- Environment configuration\n- Quick commands for operations\n\n**Completion Criteria:**\n- `DEPLOYMENT_REPORT.md` exists in output directory\n- Report contains all required sections\n- All file checks show ✅\n\n## Security Best Practices\n\n1. **Never commit tokens to version control** — Add `.env` to `.gitignore`\n2. **Use .env files with chmod 600** — Restrict access to owner only\n3. **Mask tokens in logs** — Show only first 10 chars: `${TOKEN:0:10}...`\n4. **Pass tokens at runtime** — `HF_TOKEN=$HF_TOKEN docker compose up -d`\n5. **Store tokens in ~\u002F.bash_profile** — For production environments, set `HF_TOKEN` in user's shell config\n6. **Set token for gated models** — HF_TOKEN is validated at download time; set in ~\u002F.bash_profile for production\n\n## Troubleshooting\n\n### Environment Variables\n\n| Issue | Solution |\n|-------|----------|\n| `HF_TOKEN not set` | Add `export HF_TOKEN=\"hf_xxx\"` to `~\u002F.bash_profile`, then `source ~\u002F.bash_profile`. Or provide via parameter. |\n| `HF_HOME not set` | defaults to `\u002Froot\u002F.cache\u002Fhuggingface\u002Fhub`. For production, add `export HF_HOME=\"\u002Fpath\"` to `~\u002F.bash_profile`. |\n| `~\u002F.bash_profile not found` | Create `~\u002F.bash_profile` and add environment variables. |\n| `Changes not taking effect` | Run `source ~\u002F.bash_profile` or restart terminal. |\n| `HF_TOKEN provided but download still fails` | Token may be invalid or lack access to the model. Verify token at https:\u002F\u002Fhuggingface.co\u002Fsettings\u002Ftokens |\n\n### Model Download\n\n| Issue | Solution |\n|-------|----------|\n| `Authentication required` (gated model) | Set `HF_TOKEN` in `~\u002F.bash_profile` or provide via parameter. Ensure token has access to the model. |\n| `Model not found` | Verify model ID is correct (case-sensitive). Check model exists on HuggingFace. |\n| `Download timeout` | Check network connection. Large models may take time. |\n\n### Deployment\n\n| Issue | Solution |\n|-------|----------|\n| hf CLI not found | `pip install huggingface_hub` |\n| Docker Compose fails | Use `docker compose` (no hyphen) |\n| GPU access fails | Add user to `render` group: `sudo usermod -aG render $USER` |\n| Port in use | Change `port` parameter |\n| OOM | Reduce `gpu_memory_utilization` |\n\n## Cleanup\n\n```bash\ncd $HOME\u002Fvllm-compose\u002F\u003Cmodel-id>\ndocker compose down\n```\n\n## Status Check\n\n**Check deployment status and logs:**\n\n```bash\n# View deployment directory\nls -la $HOME\u002Fvllm-compose\u002F\u003Cmodel-id>\u002F\n\n# View live logs\ntail -f $HOME\u002Fvllm-compose\u002F\u003Cmodel-id>\u002Fdeployment.log\n\n# View test results\ncat $HOME\u002Fvllm-compose\u002F\u003Cmodel-id>\u002Ftest-results.json\n\n# Check container status\ndocker ps | grep \u003Cmodel-id>\n\n# Verify environment variables\necho \"HF_TOKEN: ${HF_TOKEN:0:10}...\"\necho \"HF_HOME: $HF_HOME\"\n```\n\n## Quick Start (Production)\n\n**Step 1: Add environment variables to ~\u002F.bash_profile**\n\n```bash\n# Required: HuggingFace token\nexport HF_TOKEN=\"hf_xxxxxxxxxxxxxxxxxxxxxxxxxxxxxx\"\n\n# Recommended: Custom model storage path (production)\nexport HF_HOME=\"\u002Fdata\u002Fmodels\u002Fhuggingface\"\n\n# Apply changes\nsource ~\u002F.bash_profile\n```\n\n**Step 2: Verify environment is ready**\n\n```bash\n# Source ~\u002F.bash_profile to load variables\nsource ~\u002F.bash_profile\n\n# Expected output:\n# === Environment Ready ===\n# Summary:\n#   HF_TOKEN: hf_xxxxxx...\n#   HF_HOME:  \u002Fdata\u002Fmodels\u002Fhuggingface\n```\n\n**Step 3: Run deployment**\n\n```bash\n# The skill will automatically:\n# 1. Source ~\u002F.bash_profile to load HF_HOME and HF_TOKEN\n# 2. Use HF_TOKEN and HF_HOME from environment (or ~\u002F.bash_profile, or defaults)\n# 3. Proceed without token for public models\n# 4. Fail at download time with clear error if gated model requires token\n```\n\n## Version History\n\n| Version | Changes |\n|---------|---------|\n| 1.0.0 | Initial release |\n\n","# ROCm vLLM Deployment Skill\n\nProduction-ready automation for deploying vLLM inference services on AMD ROCm GPUs using Docker Compose.\n\n## Features\n\n- Environment Auto-Check - Detects and repairs missing dependencies\n- Model Parameter Detection - Auto-reads config.json for optimal settings\n- VRAM Estimation - Calculates memory requirements before deployment\n- Secure Token Handling - Never writes tokens to compose files\n- **Structured Output** - All logs and test results saved per-model\n- **Deployment Reports** - Human-readable summary for each deployment\n- Health Verification - Automated health checks and functional tests\n- Troubleshooting Guide - Common issues and solutions\n\n## Environment Prerequisites\n\n**Recommended (for production):** Add to `~\u002F.bash_profile`:\n\n```bash\n# HuggingFace authentication token (required for gated models)\nexport HF_TOKEN=\"hf_xxxxxxxxxxxxxxxxxxxxxxxxxxxxxx\"\n\n# Model cache directory (optional)\nexport HF_HOME=\"$HOME\u002Fmodels\"\n\n# Apply changes\nsource ~\u002F.bash_profile\n```\n\n**Not required for testing:** The skill will proceed without these set:\n- **HF_TOKEN**: Optional — public models work without it; gated models fail at download with clear error\n- **HF_HOME**: Optional — defaults to `\u002Froot\u002F.cache\u002Fhuggingface\u002Fhub`\n\n### Environment Variable Detection\n\n**Priority Order:**\n1. **Explicit parameter** (highest) — Provided in task\u002Frequest (e.g., `hf_token: \"xxx\"`)\n2. **Environment variable** — Already set in shell or from parent process\n3. **~\u002F.bash_profile** — Source to load variables\n4. **Default value** (lowest) — HF_HOME defaults to `\u002Froot\u002F.cache\u002Fhuggingface\u002Fhub`\n\n| Variable | Required | If Missing |\n|----------|----------|------------|\n| `HF_TOKEN` | **Conditional** | Continue without token (public models work; gated models fail at download with clear error) |\n| `HF_HOME` | No | **Warning + Default** — Use `\u002Froot\u002F.cache\u002Fhuggingface\u002Fhub` |\n\n**Philosophy:** Fail fast for configuration errors, fail at download time for authentication errors.\n\n---\n\n## Helper Scripts\n\n**Location:** `\u003Cskill-dir>\u002Fscripts\u002F`\n\n### check-env.sh\n\nValidate and load environment variables before deployment.\n\n**Usage:**\n```bash\n# Basic check (HF_TOKEN optional, HF_HOME optional with default)\n.\u002Fscripts\u002Fcheck-env.sh\n\n# Strict mode (HF_HOME required, fails if not set)\n.\u002Fscripts\u002Fcheck-env.sh --strict\n\n# Quiet mode (minimal output, for automation)\n.\u002Fscripts\u002Fcheck-env.sh --quiet\n\n# Test with environment variables\nHF_TOKEN=\"hf_xxx\" HF_HOME=\"\u002Fmodels\" .\u002Fscripts\u002Fcheck-env.sh\n```\n\n**Exit Codes:**\n| Code | Meaning |\n|------|---------|\n| 0 | Environment check completed (variables loaded or defaulted) |\n| 2 | Critical error (e.g., cannot source ~\u002F.bash_profile) |\n\n**Note:** This script is optional. You can also directly run `source ~\u002F.bash_profile`.\n\n---\n\n### generate-report.sh\n\nGenerate human-readable deployment report after successful deployment.\n\n**Usage:**\n```bash\n.\u002Fscripts\u002Fgenerate-report.sh \u003Cmodel-id> \u003Ccontainer-name> \u003Cport> \u003Cstatus> [model-load-time] [memory-used]\n\n# Example:\n.\u002Fscripts\u002Fgenerate-report.sh \\\n  \"Qwen-Qwen3-0.6B\" \\\n  \"vllm-qwen3-0-6b\" \\\n  \"8001\" \\\n  \"✅ Success\" \\\n  \"3.6\" \\\n  \"1.2\"\n```\n\n**Parameters:**\n| Parameter | Required | Description |\n|-----------|----------|-------------|\n| `model-id` | Yes | Model ID (with `\u002F` replaced by `-`) |\n| `container-name` | Yes | Docker container name |\n| `port` | Yes | Host port for API endpoint |\n| `status` | Yes | Deployment status (e.g., \"✅ Success\") |\n| `model-load-time` | No | Model loading time in seconds |\n| `memory-used` | No | Memory consumption in GiB |\n\n**Output:** `$HOME\u002Fvllm-compose\u002F\u003Cmodel-id>\u002FDEPLOYMENT_REPORT.md`\n\n**Exit Codes:**\n| Code | Meaning |\n|------|---------|\n| 0 | Report generated successfully |\n| 1 | Missing required parameters |\n| 2 | Output directory not found |\n\n**Integration:** This script is automatically called in **Phase 7** of the deployment workflow.\n\n---\n\n## Input Schema\n\n| Parameter | Type | Required | Default | Description |\n|-----------|------|----------|---------|-------------|\n| model_id | String | Yes | - | HuggingFace model ID |\n| docker_image | String | No | rocm\u002Fvllm-dev:nightly | vLLM Docker image |\n| tensor_parallel_size | Integer | No | 1 | Number of GPUs |\n| port | Integer | No | 9999 | API server port |\n| hf_home | String | No | `${HF_HOME}` or `\u002Froot\u002F.cache\u002Fhuggingface\u002Fhub` | Model cache directory |\n| hf_token | Secret | Conditional | `${HF_TOKEN}` | HuggingFace token (optional for public models, required for gated models) |\n| max_model_len | Integer | No | Auto-detect | Maximum sequence length |\n| gpu_memory_utilization | Float | No | 0.85 | GPU memory utilization |\n| auto_install | Boolean | No | true | Auto-install dependencies |\n| log_level | String | No | INFO | Logging verbosity |\n\n## Output Structure\n\n**All deployment artifacts MUST be saved to:**\n```\n$HOME\u002Fvllm-compose\u002F\u003Cmodel-id-slash-to-dash>\u002F\n```\n\nConvert model ID to directory name by replacing `\u002F` with `-`:\n- `openai\u002Fgpt-oss-20b` → `$HOME\u002Fvllm-compose\u002Fopenai-gpt-oss-20b\u002F`\n- `Qwen\u002FQwen3-Coder-Next-FP8` → `$HOME\u002Fvllm-compose\u002FQwen-Qwen3-Coder-Next-FP8\u002F`\n\n**Per-model directory structure:**\n```\n$HOME\u002Fvllm-compose\u002F\u003Cmodel-id>\u002F\n├── deployment.log          # Full deployment logs (stdout + stderr)\n├── test-results.json       # Functional test results (JSON format)\n├── docker-compose.yml      # Generated Docker Compose file\n├── .env                    # HF_TOKEN environment (chmod 600, optional)\n└── DEPLOYMENT_REPORT.md    # Human-readable deployment summary\n```\n\n**File requirements:**\n- `deployment.log` — Capture ALL container logs during deployment\n- `test-results.json` — Save API response from functional test request\n- `DEPLOYMENT_REPORT.md` — Generated in Phase 7\n- All three files MUST exist before marking deployment as complete\n\n## Execution Workflow\n\n### Phase 0: Environment Check & Auto-Repair\n\n**Step 0.1: Load Environment Variables**\n\n```bash\n# Source ~\u002F.bash_profile to load HF_HOME and HF_TOKEN\nsource ~\u002F.bash_profile\n\n# If HF_HOME is not defined, it defaults to \u002Froot\u002F.cache\u002Fhuggingface\u002Fhub\n```\n\nIf HF_HOME is not defined in ~\u002F.bash_profile, it defaults to `\u002Froot\u002F.cache\u002Fhuggingface\u002Fhub`.\n\n**Step 0.2: Create Output Directory**\n- Create: `$HOME\u002Fvllm-compose\u002F\u003Cmodel-id>\u002F`\n\n**Step 0.3: Initialize Logging**\n- All output → `$HOME\u002Fvllm-compose\u002F\u003Cmodel-id>\u002Fdeployment.log`\n\n**Step 0.4: System Checks**\n- Detect OS and package manager\n- Check Python, pip, huggingface_hub\n- Check Docker, docker compose\n- Check ROCm tools (rocm-smi\u002Famd-smi)\n- Check GPU access (\u002Fdev\u002Fkfd, \u002Fdev\u002Fdri)\n- Check disk space (20GB minimum)\n\n### Phase 1: Model Download\n\n**Use HF_HOME from Phase 0 (environment variable or default):**\n\n```bash\n# Download model to HF_HOME\nhuggingface-cli download \u003Cmodel_id> --local-dir \"$HF_HOME\u002Fhub\u002Fmodels--\u003Corg>--\u003Cmodel>\"\n\n# Or use snapshot_download via Python:\npython -c \"from huggingface_hub import snapshot_download; snapshot_download(repo_id='\u003Cmodel_id>', cache_dir='$HF_HOME')\"\n```\n\n**Authentication Handling:**\n\n| Scenario | Behavior |\n|----------|----------|\n| Public model + no token | ✅ Download succeeds |\n| Public model + token provided | ✅ Download succeeds |\n| Gated model + no token | ❌ Download fails with \"authentication required\" error |\n| Gated model + invalid token | ❌ Download fails with \"invalid token\" error |\n| Gated model + valid token | ✅ Download succeeds |\n\n**On Authentication Failure:**\n```bash\necho \"ERROR: Model download failed - authentication required\"\necho \"This model requires a valid HF_TOKEN.\"\necho \"\"\necho \"Please add to ~\u002F.bash_profile:\"\necho \"  export HF_TOKEN=\\\"hf_xxxxxxxxxxxxxxxxxxxxxxxxxxxxxx\\\"\"\necho \"Then run: source ~\u002F.bash_profile\"\nexit 1\n```\n\n- Locate model path in HF cache: `$HF_HOME\u002Fhub\u002Fmodels--\u003Corg>--\u003Cmodel-name>\u002F`\n- Log download progress to `deployment.log`\n\n### Phase 2: Model Parameter Detection\n- Read config.json from model\n- Auto-detect: max_model_len, hidden_size, num_attention_heads, num_hidden_layers, vocab_size, dtype\n- Validate TP size divides attention heads\n- Estimate VRAM requirement\n\n### Phase 3: Docker Compose Configuration\n\n**Generate files in output directory:**\n\n- **docker-compose.yml** → `$HOME\u002Fvllm-compose\u002F\u003Cmodel-id>\u002Fdocker-compose.yml`\n  - Mount HF_HOME as volume (read-only for models)\n  - NO hardcoded tokens in compose file\n\n- **.env** → `$HOME\u002Fvllm-compose\u002F\u003Cmodel-id>\u002F.env` (optional)\n  - Contains: `HF_TOKEN=\u003Cvalue>`\n  - Permissions: `chmod 600`\n  - Only created if user explicitly requests persistent token storage\n\n**Volume mount example:**\n```yaml\nvolumes:\n  - ${HF_HOME}:\u002Froot\u002F.cache\u002Fhuggingface\u002Fhub:ro\n  - \u002Fdev\u002Fkfd:\u002Fdev\u002Fkfd\n  - \u002Fdev\u002Fdri:\u002Fdev\u002Fdri\n```\n\n**Important:** Docker Compose reads `${HF_HOME}` from the host environment at runtime. Before running docker compose, source ~\u002F.bash_profile: `source ~\u002F.bash_profile`\n\n\n### Phase 4: Container Launch\n\n**Important:** Before deploying, pull the latest image to ensure updates:\n```bash\ndocker pull rocm\u002Fvllm-dev:nightly\n```\n\n**Note:** Default port is 9999. Before running docker compose, check if port is available: `ss -tlnp | grep :\u003Cport>`. If port is in use, specify a different port in docker-compose.yml.\n\n- Pass HF_TOKEN at runtime: HF_TOKEN=$HF_TOKEN docker compose up -d\n- Wait for container initialization\n\n### Phase 5: Health Verification\n- Check container status\n- Test \u002Fhealth endpoint\n- Test \u002Fv1\u002Fmodels endpoint\n\n### Phase 6: Functional Testing\n- Run completion test via `\u002Fv1\u002Fchat\u002Fcompletions` API\n- **Save response to:** `$HOME\u002Fvllm-compose\u002F\u003Cmodel-id>\u002Ftest-results.json`\n- Verify response contains valid completion\n- **Log deployment complete** → Append to `deployment.log`\n- **Deployment is complete only when both files exist:**\n  - `deployment.log`\n  - `test-results.json`\n\n### Phase 7: Deployment Report\n\n**Generate human-readable deployment report using the helper script.**\n\n**Step 7.1: Extract Deployment Metrics**\n\n```bash\n# Parse deployment.log for metrics\nMODEL_LOAD_TIME=$(grep -o \"model loading took [0-9.]* seconds\" deployment.log | grep -o '[0-9.]*' || echo \"N\u002FA\")\nMEMORY_USED=$(grep -o \"took [0-9.]* GiB memory\" deployment.log | grep -o '[0-9.]*' || echo \"N\u002FA\")\n```\n\n**Step 7.2: Generate Report**\n\n```bash\n# Execute the report generation script\n\u003Cskill-dir>\u002Fscripts\u002Fgenerate-report.sh \\\n  \"\u003Cmodel-id>\" \\\n  \"\u003Ccontainer-name>\" \\\n  \"\u003Cport>\" \\\n  \"\u003Cstatus>\" \\\n  \"$MODEL_LOAD_TIME\" \\\n  \"$MEMORY_USED\"\n\n# Example:\n.\u002Fscripts\u002Fgenerate-report.sh \\\n  \"Qwen-Qwen3-0.6B\" \\\n  \"vllm-qwen3-0-6b\" \\\n  \"8001\" \\\n  \"✅ Success\" \\\n  \"3.6\" \\\n  \"1.2\"\n```\n\n**Output:** `$HOME\u002Fvllm-compose\u002F\u003Cmodel-id>\u002FDEPLOYMENT_REPORT.md`\n\n**Report Contents:**\n- Output structure verification (file checklist)\n- Deployment summary table (health, test, metrics)\n- Test results (request\u002Fresponse preview)\n- Environment configuration\n- Quick commands for operations\n\n**Completion Criteria:**\n- `DEPLOYMENT_REPORT.md` exists in output directory\n- Report contains all required sections\n- All file checks show ✅\n\n## Security Best Practices\n\n1. **Never commit tokens to version control** — Add `.env` to `.gitignore`\n2. **Use .env files with chmod 600** — Restrict access to owner only\n3. **Mask tokens in logs** — Show only first 10 chars: `${TOKEN:0:10}...`\n4. **Pass tokens at runtime** — `HF_TOKEN=$HF_TOKEN docker compose up -d`\n5. **Store tokens in ~\u002F.bash_profile** — For production environments, set `HF_TOKEN` in user's shell config\n6. **Set token for gated models** — HF_TOKEN is validated at download time; set in ~\u002F.bash_profile for production\n\n## Troubleshooting\n\n### Environment Variables\n\n| Issue | Solution |\n|-------|----------|\n| `HF_TOKEN not set` | Add `export HF_TOKEN=\"hf_xxx\"` to `~\u002F.bash_profile`, then `source ~\u002F.bash_profile`. Or provide via parameter. |\n| `HF_HOME not set` | defaults to `\u002Froot\u002F.cache\u002Fhuggingface\u002Fhub`. For production, add `export HF_HOME=\"\u002Fpath\"` to `~\u002F.bash_profile`. |\n| `~\u002F.bash_profile not found` | Create `~\u002F.bash_profile` and add environment variables. |\n| `Changes not taking effect` | Run `source ~\u002F.bash_profile` or restart terminal. |\n| `HF_TOKEN provided but download still fails` | Token may be invalid or lack access to the model. Verify token at https:\u002F\u002Fhuggingface.co\u002Fsettings\u002Ftokens |\n\n### Model Download\n\n| Issue | Solution |\n|-------|----------|\n| `Authentication required` (gated model) | Set `HF_TOKEN` in `~\u002F.bash_profile` or provide via parameter. Ensure token has access to the model. |\n| `Model not found` | Verify model ID is correct (case-sensitive). Check model exists on HuggingFace. |\n| `Download timeout` | Check network connection. Large models may take time. |\n\n### Deployment\n\n| Issue | Solution |\n|-------|----------|\n| hf CLI not found | `pip install huggingface_hub` |\n| Docker Compose fails | Use `docker compose` (no hyphen) |\n| GPU access fails | Add user to `render` group: `sudo usermod -aG render $USER` |\n| Port in use | Change `port` parameter |\n| OOM | Reduce `gpu_memory_utilization` |\n\n## Cleanup\n\n```bash\ncd $HOME\u002Fvllm-compose\u002F\u003Cmodel-id>\ndocker compose down\n```\n\n## Status Check\n\n**Check deployment status and logs:**\n\n```bash\n# View deployment directory\nls -la $HOME\u002Fvllm-compose\u002F\u003Cmodel-id>\u002F\n\n# View live logs\ntail -f $HOME\u002Fvllm-compose\u002F\u003Cmodel-id>\u002Fdeployment.log\n\n# View test results\ncat $HOME\u002Fvllm-compose\u002F\u003Cmodel-id>\u002Ftest-results.json\n\n# Check container status\ndocker ps | grep \u003Cmodel-id>\n\n# Verify environment variables\necho \"HF_TOKEN: ${HF_TOKEN:0:10}...\"\necho \"HF_HOME: $HF_HOME\"\n```\n\n## Quick Start (Production)\n\n**Step 1: Add environment variables to ~\u002F.bash_profile**\n\n```bash\n# Required: HuggingFace token\nexport HF_TOKEN=\"hf_xxxxxxxxxxxxxxxxxxxxxxxxxxxxxx\"\n\n# Recommended: Custom model storage path (production)\nexport HF_HOME=\"\u002Fdata\u002Fmodels\u002Fhuggingface\"\n\n# Apply changes\nsource ~\u002F.bash_profile\n```\n\n**Step 2: Verify environment is ready**\n\n```bash\n# Source ~\u002F.bash_profile to load variables\nsource ~\u002F.bash_profile\n\n# Expected output:\n# === Environment Ready ===\n# Summary:\n#   HF_TOKEN: hf_xxxxxx...\n#   HF_HOME:  \u002Fdata\u002Fmodels\u002Fhuggingface\n```\n\n**Step 3: Run deployment**\n\n```bash\n# The skill will automatically:\n# 1. Source ~\u002F.bash_profile to load HF_HOME and HF_TOKEN\n# 2. Use HF_TOKEN and HF_HOME from environment (or ~\u002F.bash_profile, or defaults)\n# 3. Proceed without token for public models\n# 4. Fail at download time with clear error if gated model requires token\n```\n\n## Version History\n\n| Version | Changes |\n|---------|---------|\n| 1.0.0 | Initial release |\n\n",{"name":5,"tags":24,"author":34,"timeout":35,"version":36,"platform":37,"description":7},[25,26,27,28,29,30,31,32,33],"LLM","Deployment","AMD","ROCm","Docker Compose","vLLM","Automation","EnvCheck","AutoRepair","Alex He \u003Cheye_dev@163.com>","3600s","1.0.0","Linux (AMD GPU ROCm)","master","4aa004da461d820f4ea5499e58eb8d196cf99016","community\u002Fclawhub\u002Fr\u002Frocm-vllm-deployment",0,"https:\u002F\u002Fgithub.com\u002FLord1Egypt\u002Fawesome-skill-forge","https:\u002F\u002Fskill-explorer-rho.vercel.app","https:\u002F\u002Fgithub.com\u002FLord1Egypt","Unbreakable. Unstoppable. Unmatched⚡.\r\n Speaker of machine tongues 🤖  | World-builder of decentralized dreams 🌐 | Defender of the digital realm🛡️.","2026-06-10 08:21:01",{"source":8,"skillName":5,"commands":48,"prompt":58,"zipAvailable":59},[49,52,55],{"tool":50,"command":51},"npx","npx skills add Lord1Egypt\u002Fawesome-skill-forge --skill rocm_vllm_deployment",{"tool":53,"command":54},"bunx","bunx skills add Lord1Egypt\u002Fawesome-skill-forge --skill rocm_vllm_deployment",{"tool":56,"command":57},"pnpm","pnpm dlx skills add Lord1Egypt\u002Fawesome-skill-forge --skill rocm_vllm_deployment","安装技能，技能地址：https:\u002F\u002Fgithub.com\u002FLord1Egypt\u002Fawesome-skill-forge\u002Fblob\u002Fmaster\u002Fcommunity\u002Fclawhub\u002Fr\u002Frocm-vllm-deployment\u002FSKILL.md",true,[],[62,69,76,83,90,97],{"skillNo":63,"name":64,"nameZh":6,"description":65,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":66,"categories":67,"keywords":68,"publishedAt":19},"01M785WKJB99G3STR1NF9NRTJW","youtube-content","YouTube transcripts to summaries, threads, blogs.","https:\u002F\u002Fgithub.com\u002FLord1Egypt\u002Fawesome-skill-forge\u002Fblob\u002Fmaster\u002Fskills\u002Fyoutube-content\u002FSKILL.md",[],[],{"skillNo":70,"name":71,"nameZh":6,"description":72,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":73,"categories":74,"keywords":75,"publishedAt":19},"01M785WEFHB0G2S9YA6EPR3H0E","xurl","X\u002FTwitter via xurl CLI: post, search, DM, media, v2 API.","https:\u002F\u002Fgithub.com\u002FLord1Egypt\u002Fawesome-skill-forge\u002Fblob\u002Fmaster\u002Fskills\u002Fxurl\u002FSKILL.md",[],[],{"skillNo":77,"name":78,"nameZh":6,"description":79,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":80,"categories":81,"keywords":82,"publishedAt":19},"01M785W98EDN4EFJXJPMQEHCRC","touchdesigner-mcp","Control a running TouchDesigner instance via twozero MCP — create operators, set parameters, wire connections, execute Python, build real-time visuals. 36 native tools.","https:\u002F\u002Fgithub.com\u002FLord1Egypt\u002Fawesome-skill-forge\u002Fblob\u002Fmaster\u002Fskills\u002Ftouchdesigner-mcp\u002FSKILL.md",[],[],{"skillNo":84,"name":85,"nameZh":6,"description":86,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":87,"categories":88,"keywords":89,"publishedAt":19},"01M785W5S9FP3QBH2V191E025P","test-driven-development","TDD: enforce RED-GREEN-REFACTOR, tests before code.","https:\u002F\u002Fgithub.com\u002FLord1Egypt\u002Fawesome-skill-forge\u002Fblob\u002Fmaster\u002Fskills\u002Ftest-driven-development\u002FSKILL.md",[],[],{"skillNo":91,"name":92,"nameZh":6,"description":93,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":94,"categories":95,"keywords":96,"publishedAt":19},"01M785W2AS08NSSH9YATCV6FJB","teams-meeting-pipeline","Operate the Teams meeting summary pipeline via Hermes CLI — summarize meetings, inspect pipeline status, replay jobs, manage Microsoft Graph subscriptions.","https:\u002F\u002Fgithub.com\u002FLord1Egypt\u002Fawesome-skill-forge\u002Fblob\u002Fmaster\u002Fskills\u002Fteams-meeting-pipeline\u002FSKILL.md",[],[],{"skillNo":98,"name":99,"nameZh":6,"description":100,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":101,"categories":102,"keywords":103,"publishedAt":19},"01M785VXKW4T34BTP1SCHW1R7F","systematic-debugging","4-phase root cause debugging: understand bugs before fixing.","https:\u002F\u002Fgithub.com\u002FLord1Egypt\u002Fawesome-skill-forge\u002Fblob\u002Fmaster\u002Fskills\u002Fsystematic-debugging\u002FSKILL.md",[],[]]