[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"skill:01M73166NZJS30AS3C4NXTTPMA":3,"skill-same:01M73166NZJS30AS3C4NXTTPMA":63},{"skillNo":4,"name":5,"nameZh":6,"description":7,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":16,"categories":17,"keywords":18,"publishedAt":19,"linkOnly":20,"markdownContent":21,"markdownBody":22,"frontMatter":23,"refName":41,"commitSha":42,"skillDirectory":43,"forksCount":44,"repositoryHtmlUrl":45,"repositoryHomepageUrl":46,"ownerHtmlUrl":47,"ownerBlogUrl":6,"ownerDescription":6,"updatedAt":48,"install":49,"resources":62},"01M73166NZJS30AS3C4NXTTPMA","heartmula",null,"HeartMuLa: Suno-like song generation from lyrics + tags.","StarryCod\u002Fcogitum","StarryCod","https:\u002F\u002Favatars.githubusercontent.com\u002Fu\u002F200964623?v=4","01M7311AM3RY259SYKGHWBEDN0","OpenReason","01M7311AMRC1PBSTYZWSD1AQFJ",12,"MIT","https:\u002F\u002Fgithub.com\u002FStarryCod\u002Fcogitum\u002Fblob\u002Fmaster\u002Fcogitum\u002Fdata\u002Fskills\u002Fmedia\u002Fheartmula\u002FSKILL.md",[],[],"2026-11-11 15:11:14",false,"---\nname: heartmula\ndescription: \"HeartMuLa: Suno-like song generation from lyrics + tags.\"\nversion: 1.0.0\nplatforms: [linux, macos, windows]\nmetadata:\n  cogitum:\n    tags: [music, audio, generation, ai, heartmula, heartcodec, lyrics, songs]\n    related_skills: [audiocraft]\n---\n\n# HeartMuLa - Open-Source Music Generation\n\n## Overview\nHeartMuLa is a family of open-source music foundation models (Apache-2.0) that generates music conditioned on lyrics and tags, with multilingual support. Generates full songs from lyrics + tags. Comparable to Suno for open-source. Includes:\n- **HeartMuLa** - Music language model (3B\u002F7B) for generation from lyrics + tags\n- **HeartCodec** - 12.5Hz music codec for high-fidelity audio reconstruction\n- **HeartTranscriptor** - Whisper-based lyrics transcription\n- **HeartCLAP** - Audio-text alignment model\n\n## When to Use\n- User wants to generate music\u002Fsongs from text descriptions\n- User wants an open-source Suno alternative\n- User wants local\u002Foffline music generation\n- User asks about HeartMuLa, heartlib, or AI music generation\n\n## Hardware Requirements\n- **Minimum**: 8GB VRAM with `--lazy_load true` (loads\u002Funloads models sequentially)\n- **Recommended**: 16GB+ VRAM for comfortable single-GPU usage\n- **Multi-GPU**: Use `--mula_device cuda:0 --codec_device cuda:1` to split across GPUs\n- 3B model with lazy_load peaks at ~6.2GB VRAM\n\n## Installation Steps\n\n### 1. Clone Repository\n```bash\ncd ~\u002F  # or desired directory\ngit clone https:\u002F\u002Fgithub.com\u002FHeartMuLa\u002Fheartlib.git\ncd heartlib\n```\n\n### 2. Create Virtual Environment (Python 3.10 required)\n```bash\nuv venv --python 3.10 .venv\n. .venv\u002Fbin\u002Factivate\nuv pip install -e .\n```\n\n### 3. Fix Dependency Compatibility Issues\n\n**IMPORTANT**: As of Feb 2026, the pinned dependencies have conflicts with newer packages. Apply these fixes:\n\n```bash\n# Upgrade datasets (old version incompatible with current pyarrow)\nuv pip install --upgrade datasets\n\n# Upgrade transformers (needed for huggingface-hub 1.x compatibility)\nuv pip install --upgrade transformers\n```\n\n### 4. Patch Source Code (Required for transformers 5.x)\n\n**Patch 1 - RoPE cache fix** in `src\u002Fheartlib\u002Fheartmula\u002Fmodeling_heartmula.py`:\n\nIn the `setup_caches` method of the `HeartMuLa` class, add RoPE reinitialization after the `reset_caches` try\u002Fexcept block and before the `with device:` block:\n\n```python\n# Re-initialize RoPE caches that were skipped during meta-device loading\nfrom torchtune.models.llama3_1._position_embeddings import Llama3ScaledRoPE\nfor module in self.modules():\n    if isinstance(module, Llama3ScaledRoPE) and not module.is_cache_built:\n        module.rope_init()\n        module.to(device)\n```\n\n**Why**: `from_pretrained` creates model on meta device first; `Llama3ScaledRoPE.rope_init()` skips cache building on meta tensors, then never rebuilds after weights are loaded to real device.\n\n**Patch 2 - HeartCodec loading fix** in `src\u002Fheartlib\u002Fpipelines\u002Fmusic_generation.py`:\n\nAdd `ignore_mismatched_sizes=True` to ALL `HeartCodec.from_pretrained()` calls (there are 2: the eager load in `__init__` and the lazy load in the `codec` property).\n\n**Why**: VQ codebook `initted` buffers have shape `[1]` in checkpoint vs `[]` in model. Same data, just scalar vs 0-d tensor. Safe to ignore.\n\n### 5. Download Model Checkpoints\n```bash\ncd heartlib  # project root\nhf download --local-dir '.\u002Fckpt' 'HeartMuLa\u002FHeartMuLaGen'\nhf download --local-dir '.\u002Fckpt\u002FHeartMuLa-oss-3B' 'HeartMuLa\u002FHeartMuLa-oss-3B-happy-new-year'\nhf download --local-dir '.\u002Fckpt\u002FHeartCodec-oss' 'HeartMuLa\u002FHeartCodec-oss-20260123'\n```\n\nAll 3 can be downloaded in parallel. Total size is several GB.\n\n## GPU \u002F CUDA\n\nHeartMuLa uses CUDA by default (`--mula_device cuda --codec_device cuda`). No extra setup needed if the user has an NVIDIA GPU with PyTorch CUDA support installed.\n\n- The installed `torch==2.4.1` includes CUDA 12.1 support out of the box\n- `torchtune` may report version `0.4.0+cpu` — this is just package metadata, it still uses CUDA via PyTorch\n- To verify GPU is being used, look for \"CUDA memory\" lines in the output (e.g. \"CUDA memory before unloading: 6.20 GB\")\n- **No GPU?** You can run on CPU with `--mula_device cpu --codec_device cpu`, but expect generation to be **extremely slow** (potentially 30-60+ minutes for a single song vs ~4 minutes on GPU). CPU mode also requires significant RAM (~12GB+ free). If the user has no NVIDIA GPU, recommend using a cloud GPU service (Google Colab free tier with T4, Lambda Labs, etc.) or the online demo at https:\u002F\u002Fheartmula.github.io\u002F instead.\n\n## Usage\n\n### Basic Generation\n```bash\ncd heartlib\n. .venv\u002Fbin\u002Factivate\npython .\u002Fexamples\u002Frun_music_generation.py \\\n  --model_path=.\u002Fckpt \\\n  --version=\"3B\" \\\n  --lyrics=\".\u002Fassets\u002Flyrics.txt\" \\\n  --tags=\".\u002Fassets\u002Ftags.txt\" \\\n  --save_path=\".\u002Fassets\u002Foutput.mp3\" \\\n  --lazy_load true\n```\n\n### Input Formatting\n\n**Tags** (comma-separated, no spaces):\n```\npiano,happy,wedding,synthesizer,romantic\n```\nor\n```\nrock,energetic,guitar,drums,male-vocal\n```\n\n**Lyrics** (use bracketed structural tags):\n```\n[Intro]\n\n[Verse]\nYour lyrics here...\n\n[Chorus]\nChorus lyrics...\n\n[Bridge]\nBridge lyrics...\n\n[Outro]\n```\n\n### Key Parameters\n| Parameter | Default | Description |\n|-----------|---------|-------------|\n| `--max_audio_length_ms` | 240000 | Max length in ms (240s = 4 min) |\n| `--topk` | 50 | Top-k sampling |\n| `--temperature` | 1.0 | Sampling temperature |\n| `--cfg_scale` | 1.5 | Classifier-free guidance scale |\n| `--lazy_load` | false | Load\u002Funload models on demand (saves VRAM) |\n| `--mula_dtype` | bfloat16 | Dtype for HeartMuLa (bf16 recommended) |\n| `--codec_dtype` | float32 | Dtype for HeartCodec (fp32 recommended for quality) |\n\n### Performance\n- RTF (Real-Time Factor) ≈ 1.0 — a 4-minute song takes ~4 minutes to generate\n- Output: MP3, 48kHz stereo, 128kbps\n\n## Pitfalls\n1. **Do NOT use bf16 for HeartCodec** — degrades audio quality. Use fp32 (default).\n2. **Tags may be ignored** — known issue (#90). Lyrics tend to dominate; experiment with tag ordering.\n3. **Triton not available on macOS** — Linux\u002FCUDA only for GPU acceleration.\n4. **RTX 5080 incompatibility** reported in upstream issues.\n5. The dependency pin conflicts require the manual upgrades and patches described above.\n\n## Links\n- Repo: https:\u002F\u002Fgithub.com\u002FHeartMuLa\u002Fheartlib\n- Models: https:\u002F\u002Fhuggingface.co\u002FHeartMuLa\n- Paper: https:\u002F\u002Farxiv.org\u002Fabs\u002F2601.10547\n- License: Apache-2.0\n","# HeartMuLa - Open-Source Music Generation\n\n## Overview\nHeartMuLa is a family of open-source music foundation models (Apache-2.0) that generates music conditioned on lyrics and tags, with multilingual support. Generates full songs from lyrics + tags. Comparable to Suno for open-source. Includes:\n- **HeartMuLa** - Music language model (3B\u002F7B) for generation from lyrics + tags\n- **HeartCodec** - 12.5Hz music codec for high-fidelity audio reconstruction\n- **HeartTranscriptor** - Whisper-based lyrics transcription\n- **HeartCLAP** - Audio-text alignment model\n\n## When to Use\n- User wants to generate music\u002Fsongs from text descriptions\n- User wants an open-source Suno alternative\n- User wants local\u002Foffline music generation\n- User asks about HeartMuLa, heartlib, or AI music generation\n\n## Hardware Requirements\n- **Minimum**: 8GB VRAM with `--lazy_load true` (loads\u002Funloads models sequentially)\n- **Recommended**: 16GB+ VRAM for comfortable single-GPU usage\n- **Multi-GPU**: Use `--mula_device cuda:0 --codec_device cuda:1` to split across GPUs\n- 3B model with lazy_load peaks at ~6.2GB VRAM\n\n## Installation Steps\n\n### 1. Clone Repository\n```bash\ncd ~\u002F  # or desired directory\ngit clone https:\u002F\u002Fgithub.com\u002FHeartMuLa\u002Fheartlib.git\ncd heartlib\n```\n\n### 2. Create Virtual Environment (Python 3.10 required)\n```bash\nuv venv --python 3.10 .venv\n. .venv\u002Fbin\u002Factivate\nuv pip install -e .\n```\n\n### 3. Fix Dependency Compatibility Issues\n\n**IMPORTANT**: As of Feb 2026, the pinned dependencies have conflicts with newer packages. Apply these fixes:\n\n```bash\n# Upgrade datasets (old version incompatible with current pyarrow)\nuv pip install --upgrade datasets\n\n# Upgrade transformers (needed for huggingface-hub 1.x compatibility)\nuv pip install --upgrade transformers\n```\n\n### 4. Patch Source Code (Required for transformers 5.x)\n\n**Patch 1 - RoPE cache fix** in `src\u002Fheartlib\u002Fheartmula\u002Fmodeling_heartmula.py`:\n\nIn the `setup_caches` method of the `HeartMuLa` class, add RoPE reinitialization after the `reset_caches` try\u002Fexcept block and before the `with device:` block:\n\n```python\n# Re-initialize RoPE caches that were skipped during meta-device loading\nfrom torchtune.models.llama3_1._position_embeddings import Llama3ScaledRoPE\nfor module in self.modules():\n    if isinstance(module, Llama3ScaledRoPE) and not module.is_cache_built:\n        module.rope_init()\n        module.to(device)\n```\n\n**Why**: `from_pretrained` creates model on meta device first; `Llama3ScaledRoPE.rope_init()` skips cache building on meta tensors, then never rebuilds after weights are loaded to real device.\n\n**Patch 2 - HeartCodec loading fix** in `src\u002Fheartlib\u002Fpipelines\u002Fmusic_generation.py`:\n\nAdd `ignore_mismatched_sizes=True` to ALL `HeartCodec.from_pretrained()` calls (there are 2: the eager load in `__init__` and the lazy load in the `codec` property).\n\n**Why**: VQ codebook `initted` buffers have shape `[1]` in checkpoint vs `[]` in model. Same data, just scalar vs 0-d tensor. Safe to ignore.\n\n### 5. Download Model Checkpoints\n```bash\ncd heartlib  # project root\nhf download --local-dir '.\u002Fckpt' 'HeartMuLa\u002FHeartMuLaGen'\nhf download --local-dir '.\u002Fckpt\u002FHeartMuLa-oss-3B' 'HeartMuLa\u002FHeartMuLa-oss-3B-happy-new-year'\nhf download --local-dir '.\u002Fckpt\u002FHeartCodec-oss' 'HeartMuLa\u002FHeartCodec-oss-20260123'\n```\n\nAll 3 can be downloaded in parallel. Total size is several GB.\n\n## GPU \u002F CUDA\n\nHeartMuLa uses CUDA by default (`--mula_device cuda --codec_device cuda`). No extra setup needed if the user has an NVIDIA GPU with PyTorch CUDA support installed.\n\n- The installed `torch==2.4.1` includes CUDA 12.1 support out of the box\n- `torchtune` may report version `0.4.0+cpu` — this is just package metadata, it still uses CUDA via PyTorch\n- To verify GPU is being used, look for \"CUDA memory\" lines in the output (e.g. \"CUDA memory before unloading: 6.20 GB\")\n- **No GPU?** You can run on CPU with `--mula_device cpu --codec_device cpu`, but expect generation to be **extremely slow** (potentially 30-60+ minutes for a single song vs ~4 minutes on GPU). CPU mode also requires significant RAM (~12GB+ free). If the user has no NVIDIA GPU, recommend using a cloud GPU service (Google Colab free tier with T4, Lambda Labs, etc.) or the online demo at https:\u002F\u002Fheartmula.github.io\u002F instead.\n\n## Usage\n\n### Basic Generation\n```bash\ncd heartlib\n. .venv\u002Fbin\u002Factivate\npython .\u002Fexamples\u002Frun_music_generation.py \\\n  --model_path=.\u002Fckpt \\\n  --version=\"3B\" \\\n  --lyrics=\".\u002Fassets\u002Flyrics.txt\" \\\n  --tags=\".\u002Fassets\u002Ftags.txt\" \\\n  --save_path=\".\u002Fassets\u002Foutput.mp3\" \\\n  --lazy_load true\n```\n\n### Input Formatting\n\n**Tags** (comma-separated, no spaces):\n```\npiano,happy,wedding,synthesizer,romantic\n```\nor\n```\nrock,energetic,guitar,drums,male-vocal\n```\n\n**Lyrics** (use bracketed structural tags):\n```\n[Intro]\n\n[Verse]\nYour lyrics here...\n\n[Chorus]\nChorus lyrics...\n\n[Bridge]\nBridge lyrics...\n\n[Outro]\n```\n\n### Key Parameters\n| Parameter | Default | Description |\n|-----------|---------|-------------|\n| `--max_audio_length_ms` | 240000 | Max length in ms (240s = 4 min) |\n| `--topk` | 50 | Top-k sampling |\n| `--temperature` | 1.0 | Sampling temperature |\n| `--cfg_scale` | 1.5 | Classifier-free guidance scale |\n| `--lazy_load` | false | Load\u002Funload models on demand (saves VRAM) |\n| `--mula_dtype` | bfloat16 | Dtype for HeartMuLa (bf16 recommended) |\n| `--codec_dtype` | float32 | Dtype for HeartCodec (fp32 recommended for quality) |\n\n### Performance\n- RTF (Real-Time Factor) ≈ 1.0 — a 4-minute song takes ~4 minutes to generate\n- Output: MP3, 48kHz stereo, 128kbps\n\n## Pitfalls\n1. **Do NOT use bf16 for HeartCodec** — degrades audio quality. Use fp32 (default).\n2. **Tags may be ignored** — known issue (#90). Lyrics tend to dominate; experiment with tag ordering.\n3. **Triton not available on macOS** — Linux\u002FCUDA only for GPU acceleration.\n4. **RTX 5080 incompatibility** reported in upstream issues.\n5. The dependency pin conflicts require the manual upgrades and patches described above.\n\n## Links\n- Repo: https:\u002F\u002Fgithub.com\u002FHeartMuLa\u002Fheartlib\n- Models: https:\u002F\u002Fhuggingface.co\u002FHeartMuLa\n- Paper: https:\u002F\u002Farxiv.org\u002Fabs\u002F2601.10547\n- License: Apache-2.0\n",{"name":5,"version":24,"metadata":25,"platforms":37,"description":7},"1.0.0",{"cogitum":26},{"tags":27,"related_skills":35},[28,29,30,31,5,32,33,34],"music","audio","generation","ai","heartcodec","lyrics","songs",[36],"audiocraft",[38,39,40],"linux","macos","windows","master","a99de210a5d82a24b7aa5020afa1f904f595d14c","cogitum\u002Fdata\u002Fskills\u002Fmedia\u002Fheartmula",0,"https:\u002F\u002Fgithub.com\u002FStarryCod\u002Fcogitum","https:\u002F\u002Fwww.npmjs.com\u002Fpackage\u002Fcogitum","https:\u002F\u002Fgithub.com\u002FStarryCod","2026-05-25 23:49:07",{"source":8,"skillName":5,"commands":50,"prompt":60,"zipAvailable":61},[51,54,57],{"tool":52,"command":53},"npx","npx skills add StarryCod\u002Fcogitum --skill heartmula",{"tool":55,"command":56},"bunx","bunx skills add StarryCod\u002Fcogitum --skill heartmula",{"tool":58,"command":59},"pnpm","pnpm dlx skills add StarryCod\u002Fcogitum --skill heartmula","安装技能，技能地址：https:\u002F\u002Fgithub.com\u002FStarryCod\u002Fcogitum\u002Fblob\u002Fmaster\u002Fcogitum\u002Fdata\u002Fskills\u002Fmedia\u002Fheartmula\u002FSKILL.md",true,[],[64,71,78,85,92,99],{"skillNo":65,"name":66,"nameZh":6,"description":67,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":68,"categories":69,"keywords":70,"publishedAt":19},"01M731AYJQFFAGEQ8T8NMPKM72","yuanbao","Yuanbao (元宝) groups: @mention users, query info\u002Fmembers.","https:\u002F\u002Fgithub.com\u002FStarryCod\u002Fcogitum\u002Fblob\u002Fmaster\u002Fcogitum\u002Fdata\u002Fskills\u002Fyuanbao\u002FSKILL.md",[],[],{"skillNo":72,"name":73,"nameZh":6,"description":74,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":75,"categories":76,"keywords":77,"publishedAt":19},"01M731AWAHBZQ4JG5HBRXSTEWP","tg-moderator","Telegram chat moderator\u002Fcompanion mode. No tools, conversational only.","https:\u002F\u002Fgithub.com\u002FStarryCod\u002Fcogitum\u002Fblob\u002Fmaster\u002Fcogitum\u002Fdata\u002Fskills\u002Ftg-moderator\u002FSKILL.md",[],[],{"skillNo":79,"name":80,"nameZh":6,"description":81,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":82,"categories":83,"keywords":84,"publishedAt":19},"01M731AS9N89S2E50MSW2Y9QYY","writing-plans","Write implementation plans: bite-sized tasks, paths, code.","https:\u002F\u002Fgithub.com\u002FStarryCod\u002Fcogitum\u002Fblob\u002Fmaster\u002Fcogitum\u002Fdata\u002Fskills\u002Fsoftware-development\u002Fwriting-plans\u002FSKILL.md",[],[],{"skillNo":86,"name":87,"nameZh":6,"description":88,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":89,"categories":90,"keywords":91,"publishedAt":19},"01M731ANESTYGBFV3XSAT0AS8Q","test-driven-development","TDD: enforce RED-GREEN-REFACTOR, tests before code.","https:\u002F\u002Fgithub.com\u002FStarryCod\u002Fcogitum\u002Fblob\u002Fmaster\u002Fcogitum\u002Fdata\u002Fskills\u002Fsoftware-development\u002Ftest-driven-development\u002FSKILL.md",[],[],{"skillNo":93,"name":94,"nameZh":6,"description":95,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":96,"categories":97,"keywords":98,"publishedAt":19},"01M731AG1KE7ZHM0P1R890ZVB0","systematic-debugging","4-phase root cause debugging: understand bugs before fixing.","https:\u002F\u002Fgithub.com\u002FStarryCod\u002Fcogitum\u002Fblob\u002Fmaster\u002Fcogitum\u002Fdata\u002Fskills\u002Fsoftware-development\u002Fsystematic-debugging\u002FSKILL.md",[],[],{"skillNo":100,"name":101,"nameZh":6,"description":102,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":103,"categories":104,"keywords":105,"publishedAt":19},"01M731ABWQ3DTCBY8E6WTYSBTT","subagent-driven-development","Execute plans via delegate_task subagents (2-stage review).","https:\u002F\u002Fgithub.com\u002FStarryCod\u002Fcogitum\u002Fblob\u002Fmaster\u002Fcogitum\u002Fdata\u002Fskills\u002Fsoftware-development\u002Fsubagent-driven-development\u002FSKILL.md",[],[]]