[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"skill:01M7EDFFJZCXQ825MS1C6KBKVX":3,"skill-same:01M7EDFFJZCXQ825MS1C6KBKVX":70},{"skillNo":4,"name":5,"nameZh":6,"description":7,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":16,"categories":17,"keywords":18,"publishedAt":19,"linkOnly":20,"markdownContent":21,"markdownBody":22,"frontMatter":23,"refName":30,"commitSha":31,"skillDirectory":32,"forksCount":14,"repositoryHtmlUrl":33,"repositoryHomepageUrl":33,"ownerHtmlUrl":34,"ownerBlogUrl":6,"ownerDescription":6,"updatedAt":35,"install":36,"resources":49},"01M7EDFFJZCXQ825MS1C6KBKVX","bio-genome-annotation-repeat-annotation",null,"Discovers, classifies, and masks repetitive elements and transposable elements with RepeatModeler2 (de novo family library), RepeatMasker (masking against a library), EDTA (plant\u002Fstructural TEs), or EarlGrey (auto-curating wrapper), and quantifies TE expression from RNA-seq with TEtranscripts\u002FSQuIRE. Covers de-novo-library-as-curation-project, soft-vs-hard masking, the domesticated-gene over-masking massacre, Dfam-vs-RepBase, TE classification (Class I\u002FII, family-vs-copy), Kimura repeat landscapes, LAI, and the RNA-seq multimapping problem. Use when masking repeats before gene prediction, building a TE library for a non-model genome, or analyzing transposable-element content or expression.","Mr-Nobody-Anonymous\u002FAll-skills","Mr-Nobody-Anonymous","https:\u002F\u002Favatars.githubusercontent.com\u002Fu\u002F184253119?v=4","01M7AEXFJHKTYQW5R0ZCQQTWFV","anonymous","01M7C00M3GQR4QKP9GZA42GFAK",0,"MIT","https:\u002F\u002Fgithub.com\u002FMr-Nobody-Anonymous\u002FAll-skills\u002Fblob\u002Fmain\u002Fawesome_skills\u002Fbiology\u002Fbio-genome-annotation-repeat-annotation\u002FSKILL.md",[],[],"2026-11-15 14:19:17",false,"---\nname: bio-genome-annotation-repeat-annotation\ndescription: Discovers, classifies, and masks repetitive elements and transposable elements with RepeatModeler2 (de novo family library), RepeatMasker (masking against a library), EDTA (plant\u002Fstructural TEs), or EarlGrey (auto-curating wrapper), and quantifies TE expression from RNA-seq with TEtranscripts\u002FSQuIRE. Covers de-novo-library-as-curation-project, soft-vs-hard masking, the domesticated-gene over-masking massacre, Dfam-vs-RepBase, TE classification (Class I\u002FII, family-vs-copy), Kimura repeat landscapes, LAI, and the RNA-seq multimapping problem. Use when masking repeats before gene prediction, building a TE library for a non-model genome, or analyzing transposable-element content or expression.\ntool_type: cli\nprimary_tool: RepeatMasker\nsource: \"https:\u002F\u002Fgithub.com\u002FGPTomics\u002FbioSkills\"\nsource_repository: \"GPTomics\u002FbioSkills\"\nsource_path: \"genome-annotation\u002Frepeat-annotation\u002FSKILL.md\"\nlicense: \"MIT\"\nimported_at: \"2026-09-20\"\n---\n\n## Version Compatibility\n\nReference examples tested with: RepeatModeler 2.0.5+, RepeatMasker 4.1.5+, EDTA 2.1+, EarlGrey 4.0+, TEtranscripts 2.2+, matplotlib 3.8+, pandas 2.2+.\n\nBefore using code patterns, verify installed versions match. If versions differ:\n- CLI: `\u003Ctool> --version` then `\u003Ctool> --help` to confirm flags\n- Python: `pip show \u003Cpackage>` then `help(module.function)` to check signatures\n\nThe **library database version matters as much as the binary**: RepeatMasker now ships with Dfam (open); RepBase has been paywalled since May 2019, so any pipeline that \"requires RepBase\" is a reproducibility\u002Faccess hazard - record the Dfam release and library provenance. If code throws an error, introspect the installed tool and adapt rather than retrying.\n\n# Repeat and Transposable Element Annotation\n\n**\"Mask repeats in my genome assembly\"** -> Build a de novo repeat-family library, annotate copies genome-wide, and soft-mask them as a prerequisite for gene prediction.\n- CLI: `RepeatModeler -database mydb -LTRStruct` (library), `RepeatMasker -lib lib.fa -xsmall assembly.fa` (soft-mask), or `EarlGrey`\u002F`EDTA.pl` (wrappers)\n\n## The Single Most Important Modern Insight -- The Library Is the Experiment, and the Assembly Caps It\n\nTwo load-bearing truths the masker hides:\n\n1. **De novo library construction is a curation research project, not a button.** A RepeatModeler2 run emits `mydb-families.fa` overnight - a *draft of a draft*: consensi are routinely 5'-truncated (L1 looks 1.5 kb when the active element is 6 kb), boundary-bled into flanking unique sequence, chimeric (two families merged), and 30-60% \"Unknown\" on a non-model genome. The dominant error in published TE annotations is the *library*, not the masker engine. Crucially, **masking percentage is robust to a bad library** (a chimeric consensus still masks roughly the right real estate), so the headline number survives while everything downstream rots: inflated family counts, wrong classification, distorted age landscapes, and - the killer - host-gene-contaminated consensi that silently mask real genes. Masking + gross % can use an automated library; any *per-family biological claim* (this family is young\u002Factive\u002Fnovel) needs curation (Goubert 2022 *Mob DNA* 13:7; TE-Aid; MCHelper).\n\n2. **Annotation quality is capped by assembly quality.** Short-read de Bruijn assemblers *collapse* near-identical TE copies and *drop* the youngest (most identical, most biologically active) ones, so short-read assemblies systematically under-count TEs and bias the age distribution toward \"old\" - which masquerades as the real signal \"this lineage has no recent activity.\" Software cannot recover what the assembler threw away. Always ask what assembly a \"% repeat\" came from; HiFi\u002FT2T raised the ceiling (LAI measures it) but T2T satellite\u002Fcentromere repeats still exceed what the standard TE toolchain can annotate.\n\n## Tool Taxonomy\n\n| Tool | Citation | Role | When |\n|------|----------|------|------|\n| RepeatModeler2 | Flynn 2020 *PNAS* | de novo family discovery -> consensus library | discover genome-specific families (run `-LTRStruct`) |\n| RepeatMasker | Smit\u002FHubley\u002FGreen (software) | annotate\u002Fmask a genome **against** a library | the masking step; does not discover families |\n| EarlGrey | Baril 2024 *MBE* | wraps RepeatModeler2 + auto consensus-elongation + RepeatMasker + plots | **non-model default**; minimal hand-work |\n| EDTA | Ou 2019 *Genome Biol* | structural LTR\u002FTIR\u002FHelitron discovery + filtering | **plant \u002F structurally-rich** genomes |\n| LTR_retriever | Ou & Jiang 2018 *Plant Physiol* | isolate intact LTR-RTs; feeds LAI | LTR focus \u002F assembly-quality (LAI) |\n| TRF | Benson 1999 *NAR* | tandem\u002Fsatellite repeats | a different algorithm class from TE maskers |\n| RepeatClassifier \u002F DeepTE \u002F TERL | Flynn 2020; Yan 2020 | classify unknown consensi | attack the \"Unknown\" fraction (validate; can mislabel) |\n\nDfam (open) vs RepBase (paywalled since 2019) is the database schism - modern open pipelines build on Dfam + de novo. Engine `-e`: `rmblast` (default, consensus FASTA) vs `nhmmer` (Dfam profile HMMs, more sensitive for ancient repeats, slower) - the same genome reads a higher % with HMM detection.\n\n## Decision Tree by Scenario\n\n| Scenario | Recommended | Why |\n|----------|-------------|-----|\n| Non-model eukaryote, defensible answer, minimal hand-work | EarlGrey | RepeatModeler2 + auto-curation + clean outputs |\n| Plant \u002F structurally-rich TE genome | EDTA | best-in-class LTR\u002FTIR\u002FHelitron structural annotation + host-gene filtering |\n| Well-covered vertebrate, just need masking | RepeatMasker `-species` against Dfam | curated families already exist |\n| Mask before gene prediction | RepeatMasker `-xsmall` (soft) + decontaminated library | predictors need soft-masking |\n| Publication-grade TE biology claim | de novo -> manual curation (Goubert protocol, TE-Aid) | automated library is the start, not the end |\n| Tandem\u002Fsatellite\u002Fcentromeric repeats | TRF + satellite tools (not RepeatMasker) | library-based TE tools don't see tandem arrays |\n| TE expression from RNA-seq | -> TEtranscripts\u002FSQuIRE (EM multimapper handling) | see expression section |\n| TE insertion polymorphisms from reads | -> variant-calling (MELT\u002FTEPID) | out of scope here |\n\n## RepeatModeler2 -> RepeatMasker (the canonical pair)\n\n```bash\n# 1. De novo family discovery -> mydb-families.fa\nBuildDatabase -name mydb assembly.fa\nRepeatModeler -database mydb -threads 16 -LTRStruct   # -LTRStruct enables the LTR structural pipeline\n\n# 2. (Recommended) decontaminate the library against host proteins, then UNION with Dfam clade\n#    -> pull any consensus whose best hit is a host gene with no transposase\u002FRT\u002Fintegrase domain\n\n# 3. Soft-mask against (custom library) for gene prediction\nRepeatMasker -lib mydb-families.fa -xsmall -gff -e rmblast -pa 16 -dir rm_out assembly.fa\n```\n\n`-xsmall` = **soft-mask (lowercase)** - the key flag, the one people get wrong. Default `.masked` output hard-masks with N; `-x` masks with X. `-nolow` skips low-complexity\u002Fsimple repeats (often wanted before gene prediction - see below). Outputs: `.masked`, `.out`, `.tbl` (summary %), `.align` (needed for the landscape).\n\n## Soft vs Hard Masking (Critical, and the Over-Masking Massacre)\n\n- **Gene prediction needs SOFT-masking.** Modern predictors (AUGUSTUS\u002FGeneMark\u002FBRAKER) avoid *nucleating* models in lowercase but let exons extend into repeats - real genes have TE-derived exons and TE-filled introns. **Hard-masking (N) destroys sequence**: any gene overlapping a repeat is truncated or never called, and the predictor reports nothing - no error, no log. The gene is simply absent.\n- **Over-aggressive soft-masking is also a failure.** A gene whose promoter sits in an LTR, or a young gene inside a recent TE burst, gets suppressed because the predictor won't start a model in a heavily-lowercased locus. Before gene prediction, soft-mask *interspersed repeats only* and skip low-complexity (`-nolow`) - simple repeats overlap real coding microsatellites and low-complexity protein domains.\n- **The domesticated-gene massacre.** A de novo library contains fragments of real multicopy gene families because they *look* repetitive - and masking them deletes the genome's most interesting genes. Named casualties: **RAG1\u002FRAG2** (domesticated Transib transposase; V(D)J recombination), **syncytins** (captured retroviral env; placentation), **SETMAR\u002FMetnase** (Hsmar1 mariner + SET domain; DNA repair), **CENP-B** (pogo transposase; centromere), and the **KRAB-ZNF arrays** (~350+ primate zinc-finger genes that exist *to repress TEs* - masking them deletes the genome's anti-TE machinery). Defense: **decontaminate the library against a protein DB before masking** (EDTA does a version; RepeatModeler2 does not by default). Treat any gene model falling entirely inside a masked \"TE\" as a flag to investigate, not a finished call.\n\n## TE Classification (Wicker-compatible)\n\n- **Class I (retrotransposons, copy-and-paste via RNA + RT):** LTR-RTs (Ty3\u002FGypsy, Ty1\u002FCopia - dominate large plant genomes), LINEs (autonomous, often 5'-truncated), SINEs (non-autonomous, e.g. Alu).\n- **Class II (DNA transposons, mostly cut-and-paste):** TIR superfamilies (hAT, Tc1\u002FMariner, CACTA, PIF\u002FHarbinger, Mutator), Helitrons (rolling-circle, can capture host genes), MITEs (non-autonomous TIR derivatives, EDTA reclassifies ≤600 bp).\n- **Family vs copy:** a *family* is one consensus in the library (~thousands); a *copy\u002Finsertion* is one genomic locus matching it (millions). \"% genome masked\" counts copies; classification is family-level; \"10,000 TEs\" is ambiguous between the two.\n- **Full-length vs decayed:** most copies are dead, truncated, point-mutated relics; only a tiny fraction are intact. **Solo-LTR : full-length ratio** is real biology (recombinational LTR-RT removal rate), measurable only if the assembly resolved full-length elements. Wicker 2007 *Nat Rev Genet* is the reference scheme; present Gypsy\u002FCopia and the ICTV Metaviridae\u002FPseudoviridae names both.\n\n## Repeat Statistics and Age Landscape with Python\n\n**Goal:** Summarize masked content by class and plot the Kimura-divergence landscape (a relative within-genome age readout).\n\n**Approach:** Parse the RepeatMasker `.out` file, group by class for bp and genome fraction, then histogram percent divergence stratified by major TE class (x = divergence-from-consensus ~ relative age).\n\n```python\nimport pandas as pd\n\ndef parse_repeatmasker_out(out_file):\n    records = []\n    with open(out_file) as f:\n        for i, line in enumerate(f):\n            if i \u003C 3:\n                continue\n            parts = line.split()\n            if len(parts) \u003C 15:\n                continue\n            records.append({'perc_div': float(parts[1]), 'seqid': parts[4],\n                            'repeat_class': parts[10], 'length': int(parts[6]) - int(parts[5]) + 1})\n    return pd.DataFrame(records)\n\ndef repeat_summary(rm_df, genome_size):\n    by_class = rm_df.groupby('repeat_class')['length'].sum().sort_values(ascending=False)\n    total = rm_df['length'].sum()\n    print(f'Total masked: {total\u002Fgenome_size:.1%} of genome (a LOWER bound; ancient copies decay past detection)')\n    return by_class \u002F genome_size * 100\n```\n\nThe landscape is **right-censored** - the most ancient TEs decayed past alignment detection, so \"no old activity\" can mean \"old activity is invisible.\" A sharp left (low-divergence) peak is a recent\u002Fongoing burst; treat *presence* of a recent peak as informative and *absence* of an old hump cautiously. A truncated\u002Fchimeric consensus distorts the whole x-axis (another reason curation matters); never compare landscapes across genomes annotated with different libraries.\n\n## TE Expression from RNA-seq (the Multimapping Minefield)\n\nA read from a young high-copy family maps equally to hundreds of near-identical loci. **Unique-only mapping** (standard RNA-seq QC) discards most TE signal and biases toward old, uniquely-mappable copies - measuring the *least* active elements. Use EM\u002Fprobabilistic reassignment: **TEtranscripts\u002FTElocal** (Jin 2015), **SQuIRE** (Yang 2019), **Telescope** (Bendall 2019). Subfamily-level (TEtranscripts: \"L1 went up\", high power, no locus) vs locus-level (SQuIRE\u002FTElocal\u002FTelescope: \"this HERV-K on chr7 is on\", noisy, mappability-sensitive) changes the conclusion, not just the resolution. The dominant false positive: **a TE in an intron or downstream of an expressed gene is not \"expressed\"** - read-through\u002Fintron-retention piles reads on it; distinguish autonomous transcription from passenger signal by strand and continuity (TEspeX filters embedded-TE reads). Be skeptical of any \"TEs reactivated in disease\u002Faging\" headline that used unique-only mapping.\n\n## Per-Method Failure Modes\n\n### Hard-masking before gene prediction\n**Trigger:** running RepeatMasker without `-xsmall` (default hard-masks with N). **Mechanism:** masked sequence is destroyed. **Symptom:** genes overlapping repeats silently absent from the GFF. **Fix:** `-xsmall`; hand a soft-masked genome to the predictor.\n\n### Host-gene-contaminated library\n**Trigger:** masking with an uncurated de novo library. **Mechanism:** multicopy gene families look repetitive and enter the library. **Symptom:** suspiciously few NLR\u002FZNF\u002FOR genes; domesticated genes (RAG1, CENP-B) missing. **Fix:** BLAST the library against a protein DB; drop consensi hitting host genes with no TE domain.\n\n### Trusting % repeat from a short-read assembly\n**Trigger:** comparing TE content across studies\u002Fassemblies. **Mechanism:** short reads collapse\u002Fdrop young copies; % depends on library+engine+assembly. **Symptom:** \"low TE, all ancient\" or non-comparable cross-study tables. **Fix:** check LAI\u002Fassembly type; report method + assembly with every number; never compare published % across papers.\n\n### Discarding multimappers in TE RNA-seq\n**Trigger:** unique-only TE quantification. **Mechanism:** young high-copy families are not uniquely mappable. **Symptom:** most TE signal lost, bias to old elements. **Fix:** EM tools (TEtranscripts\u002FSQuIRE\u002FTelescope); separate read-through from autonomous transcription.\n\n### \"Unknown\" passed off as a result\n**Trigger:** shipping a 40%-Unknown library without inspection. **Mechanism:** classification is the hardest, last, most-skipped step. **Symptom:** weak biological annotation; possible gene-family contamination hiding in Unknown. **Fix:** RepeatClassifier\u002FDeepTE to triage; curate; note DB-coverage limits.\n\n## Quantitative Thresholds\n\n| Threshold | Source | Rationale |\n|-----------|--------|-----------|\n| `-xsmall` soft-mask before gene prediction | predictor requirement | hard-mask truncates repeat-overlapping genes |\n| TE content scales with genome size (human ~50%, maize ~85%, Arabidopsis ~20-25%, fungi ~1-20%) | clade norms (approx) | main driver of the C-value enigma; sanity-check vs genome size |\n| \"Unknown\" ~\u003C15% (mammal) vs 30-50% (non-model) | DB coverage | high Unknown bounds biological claims; very low on non-model = over-assignment |\n| LAI \u003C10 draft \u002F 10-20 reference \u002F >20 gold | Ou 2018 *NAR* | LTR-RT-resolution metric; only valid for LTR-rich genomes |\n| Report library + engine + assembly with any % | reproducibility | % masked is non-comparable across methods |\n| 80-80-80 (≥80% id over ≥80% length over ≥80 bp) | Wicker lineage | dereplication threshold, NOT a quality check |\n\n## Common Errors\n\n| Error \u002F symptom | Cause | Solution |\n|-----------------|-------|----------|\n| Gene prediction finds too few genes | hard-masked, or over-masked low-complexity | `-xsmall`; `-nolow` before gene prediction |\n| Suspiciously few NLR\u002FZNF\u002FOR genes | uncurated library masked gene families | decontaminate library against a protein DB |\n| Low masking percentage | novel repeats absent from DB | run RepeatModeler2 first; union de novo + Dfam |\n| RepeatModeler very slow | normal for large genomes | `-threads`; consider EDTA (plants) or EarlGrey |\n| \"TE re-activated\" result looks too clean | unique-only mapping \u002F read-through | EM tools; check strand + continuity from neighbor |\n| Cross-study % repeat disagree | different library\u002Fengine\u002Fassembly | re-annotate uniformly; report method |\n\n## References\n\n- Flynn JM, et al. 2020. RepeatModeler2 for automated genomic discovery of transposable element families. *PNAS* 117:9451-9457.\n- Storer J, et al. 2021. The Dfam community resource of transposable element families, sequence models, and genome annotations. *Mob DNA* 12:2.\n- Ou S, et al. 2019. Benchmarking transposable element annotation methods for creation of a streamlined, comprehensive pipeline (EDTA). *Genome Biol* 20:275.\n- Ou S, Jiang N. 2018. LTR_retriever: a highly accurate and sensitive program for identification of long terminal repeat retrotransposons. *Plant Physiol* 176:1410-1422.\n- Ou S, Chen J, Jiang N. 2018. Assessing genome assembly quality using the LTR Assembly Index (LAI). *Nucleic Acids Res* 46:e126.\n- Wicker T, et al. 2007. A unified classification system for eukaryotic transposable elements. *Nat Rev Genet* 8:973-982.\n- Baril T, Galbraith J, Hayward A. 2024. Earl Grey: a fully automated user-friendly transposable element annotation and analysis pipeline. *Mol Biol Evol* 41:msae068.\n- Goubert C, et al. 2022. A beginner's guide to manual curation of transposable elements. *Mob DNA* 13:7.\n- Jin Y, et al. 2015. TEtranscripts: a package for including transposable elements in differential expression analysis of RNA-seq datasets. *Bioinformatics* 31:3593-3599.\n- Yang WR, et al. 2019. SQuIRE reveals locus-specific regulation of interspersed repeat expression. *Nucleic Acids Res* 47:e27.\n- Bendall ML, et al. 2019. Telescope: characterization of the retrotranscriptome by accurate estimation of transposable element expression. *PLoS Comput Biol* 15:e1006453.\n- Benson G. 1999. Tandem repeats finder: a program to analyze DNA sequences. *Nucleic Acids Res* 27:573-580.\n\n## Related Skills\n\n- eukaryotic-gene-prediction - Receives the soft-masked genome; over-masking is a shared failure\n- annotation-qc - LAI and repeat-content sanity in the assembly-to-annotation handoff\n- genome-assembly\u002Fassembly-qc - Assembly type sets the TE-annotation ceiling (LAI)\n- differential-expression\u002Fdeseq2-basics - Differential TE expression from TEtranscripts\u002FSQuIRE counts\n- copy-number\u002Frecurrent-cnv - Segmental duplications, a distinct phenomenon from interspersed repeats\n","## Version Compatibility\n\nReference examples tested with: RepeatModeler 2.0.5+, RepeatMasker 4.1.5+, EDTA 2.1+, EarlGrey 4.0+, TEtranscripts 2.2+, matplotlib 3.8+, pandas 2.2+.\n\nBefore using code patterns, verify installed versions match. If versions differ:\n- CLI: `\u003Ctool> --version` then `\u003Ctool> --help` to confirm flags\n- Python: `pip show \u003Cpackage>` then `help(module.function)` to check signatures\n\nThe **library database version matters as much as the binary**: RepeatMasker now ships with Dfam (open); RepBase has been paywalled since May 2019, so any pipeline that \"requires RepBase\" is a reproducibility\u002Faccess hazard - record the Dfam release and library provenance. If code throws an error, introspect the installed tool and adapt rather than retrying.\n\n# Repeat and Transposable Element Annotation\n\n**\"Mask repeats in my genome assembly\"** -> Build a de novo repeat-family library, annotate copies genome-wide, and soft-mask them as a prerequisite for gene prediction.\n- CLI: `RepeatModeler -database mydb -LTRStruct` (library), `RepeatMasker -lib lib.fa -xsmall assembly.fa` (soft-mask), or `EarlGrey`\u002F`EDTA.pl` (wrappers)\n\n## The Single Most Important Modern Insight -- The Library Is the Experiment, and the Assembly Caps It\n\nTwo load-bearing truths the masker hides:\n\n1. **De novo library construction is a curation research project, not a button.** A RepeatModeler2 run emits `mydb-families.fa` overnight - a *draft of a draft*: consensi are routinely 5'-truncated (L1 looks 1.5 kb when the active element is 6 kb), boundary-bled into flanking unique sequence, chimeric (two families merged), and 30-60% \"Unknown\" on a non-model genome. The dominant error in published TE annotations is the *library*, not the masker engine. Crucially, **masking percentage is robust to a bad library** (a chimeric consensus still masks roughly the right real estate), so the headline number survives while everything downstream rots: inflated family counts, wrong classification, distorted age landscapes, and - the killer - host-gene-contaminated consensi that silently mask real genes. Masking + gross % can use an automated library; any *per-family biological claim* (this family is young\u002Factive\u002Fnovel) needs curation (Goubert 2022 *Mob DNA* 13:7; TE-Aid; MCHelper).\n\n2. **Annotation quality is capped by assembly quality.** Short-read de Bruijn assemblers *collapse* near-identical TE copies and *drop* the youngest (most identical, most biologically active) ones, so short-read assemblies systematically under-count TEs and bias the age distribution toward \"old\" - which masquerades as the real signal \"this lineage has no recent activity.\" Software cannot recover what the assembler threw away. Always ask what assembly a \"% repeat\" came from; HiFi\u002FT2T raised the ceiling (LAI measures it) but T2T satellite\u002Fcentromere repeats still exceed what the standard TE toolchain can annotate.\n\n## Tool Taxonomy\n\n| Tool | Citation | Role | When |\n|------|----------|------|------|\n| RepeatModeler2 | Flynn 2020 *PNAS* | de novo family discovery -> consensus library | discover genome-specific families (run `-LTRStruct`) |\n| RepeatMasker | Smit\u002FHubley\u002FGreen (software) | annotate\u002Fmask a genome **against** a library | the masking step; does not discover families |\n| EarlGrey | Baril 2024 *MBE* | wraps RepeatModeler2 + auto consensus-elongation + RepeatMasker + plots | **non-model default**; minimal hand-work |\n| EDTA | Ou 2019 *Genome Biol* | structural LTR\u002FTIR\u002FHelitron discovery + filtering | **plant \u002F structurally-rich** genomes |\n| LTR_retriever | Ou & Jiang 2018 *Plant Physiol* | isolate intact LTR-RTs; feeds LAI | LTR focus \u002F assembly-quality (LAI) |\n| TRF | Benson 1999 *NAR* | tandem\u002Fsatellite repeats | a different algorithm class from TE maskers |\n| RepeatClassifier \u002F DeepTE \u002F TERL | Flynn 2020; Yan 2020 | classify unknown consensi | attack the \"Unknown\" fraction (validate; can mislabel) |\n\nDfam (open) vs RepBase (paywalled since 2019) is the database schism - modern open pipelines build on Dfam + de novo. Engine `-e`: `rmblast` (default, consensus FASTA) vs `nhmmer` (Dfam profile HMMs, more sensitive for ancient repeats, slower) - the same genome reads a higher % with HMM detection.\n\n## Decision Tree by Scenario\n\n| Scenario | Recommended | Why |\n|----------|-------------|-----|\n| Non-model eukaryote, defensible answer, minimal hand-work | EarlGrey | RepeatModeler2 + auto-curation + clean outputs |\n| Plant \u002F structurally-rich TE genome | EDTA | best-in-class LTR\u002FTIR\u002FHelitron structural annotation + host-gene filtering |\n| Well-covered vertebrate, just need masking | RepeatMasker `-species` against Dfam | curated families already exist |\n| Mask before gene prediction | RepeatMasker `-xsmall` (soft) + decontaminated library | predictors need soft-masking |\n| Publication-grade TE biology claim | de novo -> manual curation (Goubert protocol, TE-Aid) | automated library is the start, not the end |\n| Tandem\u002Fsatellite\u002Fcentromeric repeats | TRF + satellite tools (not RepeatMasker) | library-based TE tools don't see tandem arrays |\n| TE expression from RNA-seq | -> TEtranscripts\u002FSQuIRE (EM multimapper handling) | see expression section |\n| TE insertion polymorphisms from reads | -> variant-calling (MELT\u002FTEPID) | out of scope here |\n\n## RepeatModeler2 -> RepeatMasker (the canonical pair)\n\n```bash\n# 1. De novo family discovery -> mydb-families.fa\nBuildDatabase -name mydb assembly.fa\nRepeatModeler -database mydb -threads 16 -LTRStruct   # -LTRStruct enables the LTR structural pipeline\n\n# 2. (Recommended) decontaminate the library against host proteins, then UNION with Dfam clade\n#    -> pull any consensus whose best hit is a host gene with no transposase\u002FRT\u002Fintegrase domain\n\n# 3. Soft-mask against (custom library) for gene prediction\nRepeatMasker -lib mydb-families.fa -xsmall -gff -e rmblast -pa 16 -dir rm_out assembly.fa\n```\n\n`-xsmall` = **soft-mask (lowercase)** - the key flag, the one people get wrong. Default `.masked` output hard-masks with N; `-x` masks with X. `-nolow` skips low-complexity\u002Fsimple repeats (often wanted before gene prediction - see below). Outputs: `.masked`, `.out`, `.tbl` (summary %), `.align` (needed for the landscape).\n\n## Soft vs Hard Masking (Critical, and the Over-Masking Massacre)\n\n- **Gene prediction needs SOFT-masking.** Modern predictors (AUGUSTUS\u002FGeneMark\u002FBRAKER) avoid *nucleating* models in lowercase but let exons extend into repeats - real genes have TE-derived exons and TE-filled introns. **Hard-masking (N) destroys sequence**: any gene overlapping a repeat is truncated or never called, and the predictor reports nothing - no error, no log. The gene is simply absent.\n- **Over-aggressive soft-masking is also a failure.** A gene whose promoter sits in an LTR, or a young gene inside a recent TE burst, gets suppressed because the predictor won't start a model in a heavily-lowercased locus. Before gene prediction, soft-mask *interspersed repeats only* and skip low-complexity (`-nolow`) - simple repeats overlap real coding microsatellites and low-complexity protein domains.\n- **The domesticated-gene massacre.** A de novo library contains fragments of real multicopy gene families because they *look* repetitive - and masking them deletes the genome's most interesting genes. Named casualties: **RAG1\u002FRAG2** (domesticated Transib transposase; V(D)J recombination), **syncytins** (captured retroviral env; placentation), **SETMAR\u002FMetnase** (Hsmar1 mariner + SET domain; DNA repair), **CENP-B** (pogo transposase; centromere), and the **KRAB-ZNF arrays** (~350+ primate zinc-finger genes that exist *to repress TEs* - masking them deletes the genome's anti-TE machinery). Defense: **decontaminate the library against a protein DB before masking** (EDTA does a version; RepeatModeler2 does not by default). Treat any gene model falling entirely inside a masked \"TE\" as a flag to investigate, not a finished call.\n\n## TE Classification (Wicker-compatible)\n\n- **Class I (retrotransposons, copy-and-paste via RNA + RT):** LTR-RTs (Ty3\u002FGypsy, Ty1\u002FCopia - dominate large plant genomes), LINEs (autonomous, often 5'-truncated), SINEs (non-autonomous, e.g. Alu).\n- **Class II (DNA transposons, mostly cut-and-paste):** TIR superfamilies (hAT, Tc1\u002FMariner, CACTA, PIF\u002FHarbinger, Mutator), Helitrons (rolling-circle, can capture host genes), MITEs (non-autonomous TIR derivatives, EDTA reclassifies ≤600 bp).\n- **Family vs copy:** a *family* is one consensus in the library (~thousands); a *copy\u002Finsertion* is one genomic locus matching it (millions). \"% genome masked\" counts copies; classification is family-level; \"10,000 TEs\" is ambiguous between the two.\n- **Full-length vs decayed:** most copies are dead, truncated, point-mutated relics; only a tiny fraction are intact. **Solo-LTR : full-length ratio** is real biology (recombinational LTR-RT removal rate), measurable only if the assembly resolved full-length elements. Wicker 2007 *Nat Rev Genet* is the reference scheme; present Gypsy\u002FCopia and the ICTV Metaviridae\u002FPseudoviridae names both.\n\n## Repeat Statistics and Age Landscape with Python\n\n**Goal:** Summarize masked content by class and plot the Kimura-divergence landscape (a relative within-genome age readout).\n\n**Approach:** Parse the RepeatMasker `.out` file, group by class for bp and genome fraction, then histogram percent divergence stratified by major TE class (x = divergence-from-consensus ~ relative age).\n\n```python\nimport pandas as pd\n\ndef parse_repeatmasker_out(out_file):\n    records = []\n    with open(out_file) as f:\n        for i, line in enumerate(f):\n            if i \u003C 3:\n                continue\n            parts = line.split()\n            if len(parts) \u003C 15:\n                continue\n            records.append({'perc_div': float(parts[1]), 'seqid': parts[4],\n                            'repeat_class': parts[10], 'length': int(parts[6]) - int(parts[5]) + 1})\n    return pd.DataFrame(records)\n\ndef repeat_summary(rm_df, genome_size):\n    by_class = rm_df.groupby('repeat_class')['length'].sum().sort_values(ascending=False)\n    total = rm_df['length'].sum()\n    print(f'Total masked: {total\u002Fgenome_size:.1%} of genome (a LOWER bound; ancient copies decay past detection)')\n    return by_class \u002F genome_size * 100\n```\n\nThe landscape is **right-censored** - the most ancient TEs decayed past alignment detection, so \"no old activity\" can mean \"old activity is invisible.\" A sharp left (low-divergence) peak is a recent\u002Fongoing burst; treat *presence* of a recent peak as informative and *absence* of an old hump cautiously. A truncated\u002Fchimeric consensus distorts the whole x-axis (another reason curation matters); never compare landscapes across genomes annotated with different libraries.\n\n## TE Expression from RNA-seq (the Multimapping Minefield)\n\nA read from a young high-copy family maps equally to hundreds of near-identical loci. **Unique-only mapping** (standard RNA-seq QC) discards most TE signal and biases toward old, uniquely-mappable copies - measuring the *least* active elements. Use EM\u002Fprobabilistic reassignment: **TEtranscripts\u002FTElocal** (Jin 2015), **SQuIRE** (Yang 2019), **Telescope** (Bendall 2019). Subfamily-level (TEtranscripts: \"L1 went up\", high power, no locus) vs locus-level (SQuIRE\u002FTElocal\u002FTelescope: \"this HERV-K on chr7 is on\", noisy, mappability-sensitive) changes the conclusion, not just the resolution. The dominant false positive: **a TE in an intron or downstream of an expressed gene is not \"expressed\"** - read-through\u002Fintron-retention piles reads on it; distinguish autonomous transcription from passenger signal by strand and continuity (TEspeX filters embedded-TE reads). Be skeptical of any \"TEs reactivated in disease\u002Faging\" headline that used unique-only mapping.\n\n## Per-Method Failure Modes\n\n### Hard-masking before gene prediction\n**Trigger:** running RepeatMasker without `-xsmall` (default hard-masks with N). **Mechanism:** masked sequence is destroyed. **Symptom:** genes overlapping repeats silently absent from the GFF. **Fix:** `-xsmall`; hand a soft-masked genome to the predictor.\n\n### Host-gene-contaminated library\n**Trigger:** masking with an uncurated de novo library. **Mechanism:** multicopy gene families look repetitive and enter the library. **Symptom:** suspiciously few NLR\u002FZNF\u002FOR genes; domesticated genes (RAG1, CENP-B) missing. **Fix:** BLAST the library against a protein DB; drop consensi hitting host genes with no TE domain.\n\n### Trusting % repeat from a short-read assembly\n**Trigger:** comparing TE content across studies\u002Fassemblies. **Mechanism:** short reads collapse\u002Fdrop young copies; % depends on library+engine+assembly. **Symptom:** \"low TE, all ancient\" or non-comparable cross-study tables. **Fix:** check LAI\u002Fassembly type; report method + assembly with every number; never compare published % across papers.\n\n### Discarding multimappers in TE RNA-seq\n**Trigger:** unique-only TE quantification. **Mechanism:** young high-copy families are not uniquely mappable. **Symptom:** most TE signal lost, bias to old elements. **Fix:** EM tools (TEtranscripts\u002FSQuIRE\u002FTelescope); separate read-through from autonomous transcription.\n\n### \"Unknown\" passed off as a result\n**Trigger:** shipping a 40%-Unknown library without inspection. **Mechanism:** classification is the hardest, last, most-skipped step. **Symptom:** weak biological annotation; possible gene-family contamination hiding in Unknown. **Fix:** RepeatClassifier\u002FDeepTE to triage; curate; note DB-coverage limits.\n\n## Quantitative Thresholds\n\n| Threshold | Source | Rationale |\n|-----------|--------|-----------|\n| `-xsmall` soft-mask before gene prediction | predictor requirement | hard-mask truncates repeat-overlapping genes |\n| TE content scales with genome size (human ~50%, maize ~85%, Arabidopsis ~20-25%, fungi ~1-20%) | clade norms (approx) | main driver of the C-value enigma; sanity-check vs genome size |\n| \"Unknown\" ~\u003C15% (mammal) vs 30-50% (non-model) | DB coverage | high Unknown bounds biological claims; very low on non-model = over-assignment |\n| LAI \u003C10 draft \u002F 10-20 reference \u002F >20 gold | Ou 2018 *NAR* | LTR-RT-resolution metric; only valid for LTR-rich genomes |\n| Report library + engine + assembly with any % | reproducibility | % masked is non-comparable across methods |\n| 80-80-80 (≥80% id over ≥80% length over ≥80 bp) | Wicker lineage | dereplication threshold, NOT a quality check |\n\n## Common Errors\n\n| Error \u002F symptom | Cause | Solution |\n|-----------------|-------|----------|\n| Gene prediction finds too few genes | hard-masked, or over-masked low-complexity | `-xsmall`; `-nolow` before gene prediction |\n| Suspiciously few NLR\u002FZNF\u002FOR genes | uncurated library masked gene families | decontaminate library against a protein DB |\n| Low masking percentage | novel repeats absent from DB | run RepeatModeler2 first; union de novo + Dfam |\n| RepeatModeler very slow | normal for large genomes | `-threads`; consider EDTA (plants) or EarlGrey |\n| \"TE re-activated\" result looks too clean | unique-only mapping \u002F read-through | EM tools; check strand + continuity from neighbor |\n| Cross-study % repeat disagree | different library\u002Fengine\u002Fassembly | re-annotate uniformly; report method |\n\n## References\n\n- Flynn JM, et al. 2020. RepeatModeler2 for automated genomic discovery of transposable element families. *PNAS* 117:9451-9457.\n- Storer J, et al. 2021. The Dfam community resource of transposable element families, sequence models, and genome annotations. *Mob DNA* 12:2.\n- Ou S, et al. 2019. Benchmarking transposable element annotation methods for creation of a streamlined, comprehensive pipeline (EDTA). *Genome Biol* 20:275.\n- Ou S, Jiang N. 2018. LTR_retriever: a highly accurate and sensitive program for identification of long terminal repeat retrotransposons. *Plant Physiol* 176:1410-1422.\n- Ou S, Chen J, Jiang N. 2018. Assessing genome assembly quality using the LTR Assembly Index (LAI). *Nucleic Acids Res* 46:e126.\n- Wicker T, et al. 2007. A unified classification system for eukaryotic transposable elements. *Nat Rev Genet* 8:973-982.\n- Baril T, Galbraith J, Hayward A. 2024. Earl Grey: a fully automated user-friendly transposable element annotation and analysis pipeline. *Mol Biol Evol* 41:msae068.\n- Goubert C, et al. 2022. A beginner's guide to manual curation of transposable elements. *Mob DNA* 13:7.\n- Jin Y, et al. 2015. TEtranscripts: a package for including transposable elements in differential expression analysis of RNA-seq datasets. *Bioinformatics* 31:3593-3599.\n- Yang WR, et al. 2019. SQuIRE reveals locus-specific regulation of interspersed repeat expression. *Nucleic Acids Res* 47:e27.\n- Bendall ML, et al. 2019. Telescope: characterization of the retrotranscriptome by accurate estimation of transposable element expression. *PLoS Comput Biol* 15:e1006453.\n- Benson G. 1999. Tandem repeats finder: a program to analyze DNA sequences. *Nucleic Acids Res* 27:573-580.\n\n## Related Skills\n\n- eukaryotic-gene-prediction - Receives the soft-masked genome; over-masking is a shared failure\n- annotation-qc - LAI and repeat-content sanity in the assembly-to-annotation handoff\n- genome-assembly\u002Fassembly-qc - Assembly type sets the TE-annotation ceiling (LAI)\n- differential-expression\u002Fdeseq2-basics - Differential TE expression from TEtranscripts\u002FSQuIRE counts\n- copy-number\u002Frecurrent-cnv - Segmental duplications, a distinct phenomenon from interspersed repeats\n",{"name":5,"source":24,"license":15,"tool_type":25,"description":7,"imported_at":26,"source_path":27,"primary_tool":28,"source_repository":29},"https:\u002F\u002Fgithub.com\u002FGPTomics\u002FbioSkills","cli","2026-09-20","genome-annotation\u002Frepeat-annotation\u002FSKILL.md","RepeatMasker","GPTomics\u002FbioSkills","main","2d1750717747097b602acf30bd09a7cbd4b4c6ea","awesome_skills\u002Fbiology\u002Fbio-genome-annotation-repeat-annotation","https:\u002F\u002Fgithub.com\u002FMr-Nobody-Anonymous\u002FAll-skills","https:\u002F\u002Fgithub.com\u002FMr-Nobody-Anonymous","2026-11-09 11:45:36",{"source":8,"skillName":5,"commands":37,"prompt":47,"zipAvailable":48},[38,41,44],{"tool":39,"command":40},"npx","npx skills add Mr-Nobody-Anonymous\u002FAll-skills --skill bio-genome-annotation-repeat-annotation",{"tool":42,"command":43},"bunx","bunx skills add Mr-Nobody-Anonymous\u002FAll-skills --skill bio-genome-annotation-repeat-annotation",{"tool":45,"command":46},"pnpm","pnpm dlx skills add Mr-Nobody-Anonymous\u002FAll-skills --skill bio-genome-annotation-repeat-annotation","安装技能，技能地址：https:\u002F\u002Fgithub.com\u002FMr-Nobody-Anonymous\u002FAll-skills\u002Fblob\u002Fmain\u002Fawesome_skills\u002Fbiology\u002Fbio-genome-annotation-repeat-annotation\u002FSKILL.md",true,[50,56,60,64],{"path":51,"resourceType":52,"resourceTypeDesc":53,"sizeBytes":54,"sourceHtmlUrl":55},"examples\u002Frepeat_annotation.sh","SCRIPT","脚本",1985,"https:\u002F\u002Fgithub.com\u002FMr-Nobody-Anonymous\u002FAll-skills\u002Fblob\u002Fmain\u002Fawesome_skills\u002Fbiology\u002Fbio-genome-annotation-repeat-annotation\u002Fexamples\u002Frepeat_annotation.sh",{"path":57,"resourceType":52,"resourceTypeDesc":53,"sizeBytes":58,"sourceHtmlUrl":59},"examples\u002Frepeat_stats.py",4993,"https:\u002F\u002Fgithub.com\u002FMr-Nobody-Anonymous\u002FAll-skills\u002Fblob\u002Fmain\u002Fawesome_skills\u002Fbiology\u002Fbio-genome-annotation-repeat-annotation\u002Fexamples\u002Frepeat_stats.py",{"path":61,"resourceType":52,"resourceTypeDesc":53,"sizeBytes":62,"sourceHtmlUrl":63},"examples\u002Fte_expression.py",3181,"https:\u002F\u002Fgithub.com\u002FMr-Nobody-Anonymous\u002FAll-skills\u002Fblob\u002Fmain\u002Fawesome_skills\u002Fbiology\u002Fbio-genome-annotation-repeat-annotation\u002Fexamples\u002Fte_expression.py",{"path":65,"resourceType":66,"resourceTypeDesc":67,"sizeBytes":68,"sourceHtmlUrl":69},"usage-guide.md","REFERENCE","参考文档",4401,"https:\u002F\u002Fgithub.com\u002FMr-Nobody-Anonymous\u002FAll-skills\u002Fblob\u002Fmain\u002Fawesome_skills\u002Fbiology\u002Fbio-genome-annotation-repeat-annotation\u002Fusage-guide.md",[71,79,86,93,100,107],{"skillNo":72,"name":73,"nameZh":6,"description":74,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":75,"categories":76,"keywords":77,"publishedAt":78},"01M7FFFZPC2025JQQHPHDX154J","dei-assessment","Assess diversity, equity, and inclusion with metrics. TRIGGERS - Use when user needs help with dei-assessment related tasks.","https:\u002F\u002Fgithub.com\u002FMr-Nobody-Anonymous\u002FAll-skills\u002Fblob\u002Fmain\u002Fawesome_skills\u002Fhr-people\u002Fdei-assessment\u002FSKILL.md",[],[],"2026-11-16 10:05:34",{"skillNo":80,"name":81,"nameZh":6,"description":82,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":83,"categories":84,"keywords":85,"publishedAt":78},"01M7FFFWN8H4WA7A6Y398QKB6J","dei-action-plan","Create DEI action plans with goals and initiatives. TRIGGERS - Use when user needs help with dei-action-plan related tasks.","https:\u002F\u002Fgithub.com\u002FMr-Nobody-Anonymous\u002FAll-skills\u002Fblob\u002Fmain\u002Fawesome_skills\u002Fhr-people\u002Fdei-action-plan\u002FSKILL.md",[],[],{"skillNo":87,"name":88,"nameZh":6,"description":89,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":90,"categories":91,"keywords":92,"publishedAt":78},"01M7FFFSHTM0QZRVZ2NRJTJ0PY","culture-fit-assessment","Design culture add assessment processes. TRIGGERS - Use when user needs help with culture-fit-assessment related tasks.","https:\u002F\u002Fgithub.com\u002FMr-Nobody-Anonymous\u002FAll-skills\u002Fblob\u002Fmain\u002Fawesome_skills\u002Fhr-people\u002Fculture-fit-assessment\u002FSKILL.md",[],[],{"skillNo":94,"name":95,"nameZh":6,"description":96,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":97,"categories":98,"keywords":99,"publishedAt":78},"01M7FFFMA12RKEQA38KHJSXXPS","cowork-to-code-bridge","Use an already-installed, independently verified cowork-to-code bridge to run narrowly approved actions on the user's own macOS, Linux, or WSL2 machine through a local file queue.","https:\u002F\u002Fgithub.com\u002FMr-Nobody-Anonymous\u002FAll-skills\u002Fblob\u002Fmain\u002Fawesome_skills\u002Fhr-people\u002Fcowork-to-code-bridge\u002FSKILL.md",[],[],{"skillNo":101,"name":102,"nameZh":6,"description":103,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":104,"categories":105,"keywords":106,"publishedAt":78},"01M7FFFH1F6M8YRGW5V487GH1K","counter-offer-response","Draft counter-offer responses for retention. TRIGGERS - Use when user needs help with counter-offer-response related tasks.","https:\u002F\u002Fgithub.com\u002FMr-Nobody-Anonymous\u002FAll-skills\u002Fblob\u002Fmain\u002Fawesome_skills\u002Fhr-people\u002Fcounter-offer-response\u002FSKILL.md",[],[],{"skillNo":108,"name":109,"nameZh":6,"description":110,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":111,"categories":112,"keywords":113,"publishedAt":78},"01M7FFFDGEHV8Y7J9N40HZXW52","continuous-feedback-system","Design continuous feedback systems. TRIGGERS - Use when user needs help with continuous-feedback-system related tasks.","https:\u002F\u002Fgithub.com\u002FMr-Nobody-Anonymous\u002FAll-skills\u002Fblob\u002Fmain\u002Fawesome_skills\u002Fhr-people\u002Fcontinuous-feedback-system\u002FSKILL.md",[],[]]