[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"skill:01M7ECV2SF22DZ834ENZPDWNVA":3,"skill-same:01M7ECV2SF22DZ834ENZPDWNVA":49},{"skillNo":4,"name":5,"nameZh":6,"description":6,"descriptionZh":6,"repositoryFullName":7,"ownerLogin":8,"ownerAvatarUrl":9,"ownerAccountNo":10,"ownerDisplayName":11,"repositoryNo":12,"starsCount":13,"licenseSpdxId":14,"licenseFileUrl":6,"sourceHtmlUrl":15,"categories":16,"keywords":17,"publishedAt":18,"linkOnly":19,"markdownContent":20,"markdownBody":20,"frontMatter":21,"refName":22,"commitSha":23,"skillDirectory":24,"forksCount":13,"repositoryHtmlUrl":25,"repositoryHomepageUrl":6,"ownerHtmlUrl":26,"ownerBlogUrl":6,"ownerDescription":27,"updatedAt":28,"install":29,"resources":42},"01M7ECV2SF22DZ834ENZPDWNVA","heartmula",null,"jonahthan433\u002FCortexOS","jonahthan433","https:\u002F\u002Favatars.githubusercontent.com\u002Fu\u002F211465495?v=4","01M7ECH9HPHGA89DCXHHJBTS3C","jonah.ai","01M7ECH9JCEM45DZMWPT7148S1",0,"Apache-2.0","https:\u002F\u002Fgithub.com\u002Fjonahthan433\u002FCortexOS\u002Fblob\u002Fmain\u002Fskills\u002Fimported\u002Fmedia\u002Fheartmula\u002FSKILL.md",[],[],"2026-11-15 14:19:17",false,"# heartmula\n\n# HeartMuLa - Open-Source Music Generation\n\n## Overview\nHeartMuLa is a family of open-source music foundation models (Apache-2.0) that generates music conditioned on lyrics and tags, with multilingual support. Generates full songs from lyrics + tags. Comparable to Suno for open-source. Includes:\n- **HeartMuLa** - Music language model (3B\u002F7B) for generation from lyrics + tags\n- **HeartCodec** - 12.5Hz music codec for high-fidelity audio reconstruction\n- **HeartTranscriptor** - Whisper-based lyrics transcription\n- **HeartCLAP** - Audio-text alignment model\n\n## When to Use\n- User wants to generate music\u002Fsongs from text descriptions\n- User wants an open-source Suno alternative\n- User wants local\u002Foffline music generation\n- User asks about HeartMuLa, heartlib, or AI music generation\n\n## Hardware Requirements\n- **Minimum**: 8GB VRAM with `--lazy_load true` (loads\u002Funloads models sequentially)\n- **Recommended**: 16GB+ VRAM for comfortable single-GPU usage\n- **Multi-GPU**: Use `--mula_device cuda:0 --codec_device cuda:1` to split across GPUs\n- 3B model with lazy_load peaks at ~6.2GB VRAM\n\n## Installation Steps\n\n### 1. Clone Repository\n```bash\ncd ~\u002F  # or desired directory\ngit clone https:\u002F\u002Fgithub.com\u002FHeartMuLa\u002Fheartlib.git\ncd heartlib\n```\n\n### 2. Create Virtual Environment (Python 3.10 required)\n```bash\nuv venv --python 3.10 .venv\n. .venv\u002Fbin\u002Factivate\nuv pip install -e .\n```\n\n### 3. Fix Dependency Compatibility Issues\n\n**IMPORTANT**: As of Feb 2026, the pinned dependencies have conflicts with newer packages. Apply these fixes:\n\n```bash\n# Upgrade datasets (old version incompatible with current pyarrow)\nuv pip install --upgrade datasets\n\n# Upgrade transformers (needed for huggingface-hub 1.x compatibility)\nuv pip install --upgrade transformers\n```\n\n### 4. Patch Source Code (Required for transformers 5.x)\n\n**Patch 1 - RoPE cache fix** in `src\u002Fheartlib\u002Fheartmula\u002Fmodeling_heartmula.py`:\n\nIn the `setup_caches` method of the `HeartMuLa` class, add RoPE reinitialization after the `reset_caches` try\u002Fexcept block and before the `with device:` block:\n\n```python\n# Re-initialize RoPE caches that were skipped during meta-device loading\nfrom torchtune.models.llama3_1._position_embeddings import Llama3ScaledRoPE\nfor module in self.modules():\n    if isinstance(module, Llama3ScaledRoPE) and not module.is_cache_built:\n        module.rope_init()\n        module.to(device)\n```\n\n**Why**: `from_pretrained` creates model on meta device first; `Llama3ScaledRoPE.rope_init()` skips cache building on meta tensors, then never rebuilds after weights are loaded to real device.\n\n**Patch 2 - HeartCodec loading fix** in `src\u002Fheartlib\u002Fpipelines\u002Fmusic_generation.py`:\n\nAdd `ignore_mismatched_sizes=True` to ALL `HeartCodec.from_pretrained()` calls (there are 2: the eager load in `__init__` and the lazy load in the `codec` property).\n\n**Why**: VQ codebook `initted` buffers have shape `[1]` in checkpoint vs `[]` in model. Same data, just scalar vs 0-d tensor. Safe to ignore.\n\n### 5. Download Model Checkpoints\n```bash\ncd heartlib  # project root\nhf download --local-dir '.\u002Fckpt' 'HeartMuLa\u002FHeartMuLaGen'\nhf download --local-dir '.\u002Fckpt\u002FHeartMuLa-oss-3B' 'HeartMuLa\u002FHeartMuLa-oss-3B-happy-new-year'\nhf download --local-dir '.\u002Fckpt\u002FHeartCodec-oss' 'HeartMuLa\u002FHeartCodec-oss-20260123'\n```\n\nAll 3 can be downloaded in parallel. Total size is several GB.\n\n## GPU \u002F CUDA\n\nHeartMuLa uses CUDA by default (`--mula_device cuda --codec_device cuda`). No extra setup needed if the user has an NVIDIA GPU with PyTorch CUDA support installed.\n\n- The installed `torch==2.4.1` includes CUDA 12.1 support out of the box\n- `torchtune` may report version `0.4.0+cpu` — this is just package metadata, it still uses CUDA via PyTorch\n- To verify GPU is being used, look for \"CUDA memory\" lines in the output (e.g. \"CUDA memory before unloading: 6.20 GB\")\n- **No GPU?** You can run on CPU with `--mula_device cpu --codec_device cpu`, but expect generation to be **extremely slow** (potentially 30-60+ minutes for a single song vs ~4 minutes on GPU). CPU mode also requires significant RAM (~12GB+ free). If the user has no NVIDIA GPU, recommend using a cloud GPU service (Google Colab free tier with T4, Lambda Labs, etc.) or the online demo at https:\u002F\u002Fheartmula.github.io\u002F instead.\n\n## Usage\n\n### Basic Generation\n```bash\ncd heartlib\n. .venv\u002Fbin\u002Factivate\npython .\u002Fexamples\u002Frun_music_generation.py \\\n  --model_path=.\u002Fckpt \\\n  --version=\"3B\" \\\n  --lyrics=\".\u002Fassets\u002Flyrics.txt\" \\\n  --tags=\".\u002Fassets\u002Ftags.txt\" \\\n  --save_path=\".\u002Fassets\u002Foutput.mp3\" \\\n  --lazy_load true\n```\n\n### Input Formatting\n\n**Tags** (comma-separated, no spaces):\n```\npiano,happy,wedding,synthesizer,romantic\n```\nor\n```\nrock,energetic,guitar,drums,male-vocal\n```\n\n**Lyrics** (use bracketed structural tags):\n```\n[Intro]\n\n[Verse]\nYour lyrics here...\n\n[Chorus]\nChorus lyrics...\n\n[Bridge]\nBridge lyrics...\n\n[Outro]\n```\n\n### Key Parameters\n| Parameter | Default | Description |\n|-----------|---------|-------------|\n| `--max_audio_length_ms` | 240000 | Max length in ms (240s = 4 min) |\n| `--topk` | 50 | Top-k sampling |\n| `--temperature` | 1.0 | Sampling temperature |\n| `--cfg_scale` | 1.5 | Classifier-free guidance scale |\n| `--lazy_load` | false | Load\u002Funload models on demand (saves VRAM) |\n| `--mula_dtype` | bfloat16 | Dtype for HeartMuLa (bf16 recommended) |\n| `--codec_dtype` | float32 | Dtype for HeartCodec (fp32 recommended for quality) |\n\n### Performance\n- RTF (Real-Time Factor) ≈ 1.0 — a 4-minute song takes ~4 minutes to generate\n- Output: MP3, 48kHz stereo, 128kbps\n\n## Pitfalls\n1. **Do NOT use bf16 for HeartCodec** — degrades audio quality. Use fp32 (default).\n2. **Tags may be ignored** — known issue (#90). Lyrics tend to dominate; experiment with tag ordering.\n3. **Triton not available on macOS** — Linux\u002FCUDA only for GPU acceleration.\n4. **RTX 5080 incompatibility** reported in upstream issues.\n5. The dependency pin conflicts require the manual upgrades and patches described above.\n\n## Links\n- Repo: https:\u002F\u002Fgithub.com\u002FHeartMuLa\u002Fheartlib\n- Models: https:\u002F\u002Fhuggingface.co\u002FHeartMuLa\n- Paper: https:\u002F\u002Farxiv.org\u002Fabs\u002F2601.10547\n- License: Apache-2.0\n",{},"main","38b32c99e7ed0b8e215d638e92cfa452c1ec3079","skills\u002Fimported\u002Fmedia\u002Fheartmula","https:\u002F\u002Fgithub.com\u002Fjonahthan433\u002FCortexOS","https:\u002F\u002Fgithub.com\u002Fjonahthan433","AI Consultant","2026-10-24 05:13:39",{"source":7,"skillName":5,"commands":30,"prompt":40,"zipAvailable":41},[31,34,37],{"tool":32,"command":33},"npx","npx skills add jonahthan433\u002FCortexOS --skill heartmula",{"tool":35,"command":36},"bunx","bunx skills add jonahthan433\u002FCortexOS --skill heartmula",{"tool":38,"command":39},"pnpm","pnpm dlx skills add jonahthan433\u002FCortexOS --skill heartmula","安装技能，技能地址：https:\u002F\u002Fgithub.com\u002Fjonahthan433\u002FCortexOS\u002Fblob\u002Fmain\u002Fskills\u002Fimported\u002Fmedia\u002Fheartmula\u002FSKILL.md",true,[43],{"path":44,"resourceType":45,"resourceTypeDesc":46,"sizeBytes":47,"sourceHtmlUrl":48},"skill.json","REFERENCE","参考文档",321,"https:\u002F\u002Fgithub.com\u002Fjonahthan433\u002FCortexOS\u002Fblob\u002Fmain\u002Fskills\u002Fimported\u002Fmedia\u002Fheartmula\u002Fskill.json",[50,56,62,68,74,80],{"skillNo":51,"name":52,"nameZh":6,"description":6,"descriptionZh":6,"repositoryFullName":7,"ownerLogin":8,"ownerAvatarUrl":9,"ownerAccountNo":10,"ownerDisplayName":11,"repositoryNo":12,"starsCount":13,"licenseSpdxId":14,"licenseFileUrl":6,"sourceHtmlUrl":53,"categories":54,"keywords":55,"publishedAt":18},"01M7EDQZDQX3Q8Y7GCQX6KN9V7","opportunity-brief","https:\u002F\u002Fgithub.com\u002Fjonahthan433\u002FCortexOS\u002Fblob\u002Fmain\u002Fskills\u002Fsales\u002Fopportunity-brief\u002FSKILL.md",[],[],{"skillNo":57,"name":58,"nameZh":6,"description":6,"descriptionZh":6,"repositoryFullName":7,"ownerLogin":8,"ownerAvatarUrl":9,"ownerAccountNo":10,"ownerDisplayName":11,"repositoryNo":12,"starsCount":13,"licenseSpdxId":14,"licenseFileUrl":6,"sourceHtmlUrl":59,"categories":60,"keywords":61,"publishedAt":18},"01M7EDQYA0V9Y99VKBREJJ3MRT","morning-intel","https:\u002F\u002Fgithub.com\u002Fjonahthan433\u002FCortexOS\u002Fblob\u002Fmain\u002Fskills\u002Fresearch\u002Fmorning-intel\u002FSKILL.md",[],[],{"skillNo":63,"name":64,"nameZh":6,"description":6,"descriptionZh":6,"repositoryFullName":7,"ownerLogin":8,"ownerAvatarUrl":9,"ownerAccountNo":10,"ownerDisplayName":11,"repositoryNo":12,"starsCount":13,"licenseSpdxId":14,"licenseFileUrl":6,"sourceHtmlUrl":65,"categories":66,"keywords":67,"publishedAt":18},"01M7EDQVX1JVY0B6PDHKKGM19G","decision-brief","https:\u002F\u002Fgithub.com\u002Fjonahthan433\u002FCortexOS\u002Fblob\u002Fmain\u002Fskills\u002Fresearch\u002Fdecision-brief\u002FSKILL.md",[],[],{"skillNo":69,"name":70,"nameZh":6,"description":6,"descriptionZh":6,"repositoryFullName":7,"ownerLogin":8,"ownerAvatarUrl":9,"ownerAccountNo":10,"ownerDisplayName":11,"repositoryNo":12,"starsCount":13,"licenseSpdxId":14,"licenseFileUrl":6,"sourceHtmlUrl":71,"categories":72,"keywords":73,"publishedAt":18},"01M7EDQV1GENZAC2SZSD8KZ3GM","weekly-review","https:\u002F\u002Fgithub.com\u002Fjonahthan433\u002FCortexOS\u002Fblob\u002Fmain\u002Fskills\u002Fproductivity\u002Fweekly-review\u002FSKILL.md",[],[],{"skillNo":75,"name":76,"nameZh":6,"description":6,"descriptionZh":6,"repositoryFullName":7,"ownerLogin":8,"ownerAvatarUrl":9,"ownerAccountNo":10,"ownerDisplayName":11,"repositoryNo":12,"starsCount":13,"licenseSpdxId":14,"licenseFileUrl":6,"sourceHtmlUrl":77,"categories":78,"keywords":79,"publishedAt":18},"01M7EDQT2KETJ1Y3CGVN9EKW2J","system-health","https:\u002F\u002Fgithub.com\u002Fjonahthan433\u002FCortexOS\u002Fblob\u002Fmain\u002Fskills\u002Fops\u002Fsystem-health\u002FSKILL.md",[],[],{"skillNo":81,"name":82,"nameZh":6,"description":6,"descriptionZh":6,"repositoryFullName":7,"ownerLogin":8,"ownerAvatarUrl":9,"ownerAccountNo":10,"ownerDisplayName":11,"repositoryNo":12,"starsCount":13,"licenseSpdxId":14,"licenseFileUrl":6,"sourceHtmlUrl":83,"categories":84,"keywords":85,"publishedAt":18},"01M7EDQQN2JK3XPWMS1M4T0F3K","backup-restore","https:\u002F\u002Fgithub.com\u002Fjonahthan433\u002FCortexOS\u002Fblob\u002Fmain\u002Fskills\u002Fops\u002Fbackup-restore\u002FSKILL.md",[],[]]