[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"skill:01M7EE8ECSG2FPW3GYCSJ1ND7B":3,"skill-same:01M7EE8ECSG2FPW3GYCSJ1ND7B":66},{"skillNo":4,"name":5,"nameZh":6,"description":7,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":16,"categories":17,"keywords":18,"publishedAt":19,"linkOnly":20,"markdownContent":21,"markdownBody":22,"frontMatter":23,"refName":30,"commitSha":31,"skillDirectory":32,"forksCount":14,"repositoryHtmlUrl":33,"repositoryHomepageUrl":33,"ownerHtmlUrl":34,"ownerBlogUrl":6,"ownerDescription":6,"updatedAt":35,"install":36,"resources":49},"01M7EE8ECSG2FPW3GYCSJ1ND7B","bio-metagenomics-abundance",null,"Turns shotgun classifier output into a defensible abundance table with Bracken Bayesian re-estimation, then compositional treatment (CLR, zero handling), library-size normalization, reference-frame differential abundance, and optional absolute quantification. Covers why a relative-abundance change is not a change, why Bracken read fractions and MetaPhlAn percentages are different physical quantities, the silent -r read-length bias, the genome-size confound no library-size method fixes, and the rarefaction debate. Use when estimating species abundance from a Kraken2 report, normalizing a community count table, choosing a compositional transform, or converting relative to absolute load. For classification see kraken-classification; for diversity\u002Fordination\u002FDA mechanics see metagenome-visualization.","Mr-Nobody-Anonymous\u002FAll-skills","Mr-Nobody-Anonymous","https:\u002F\u002Favatars.githubusercontent.com\u002Fu\u002F184253119?v=4","01M7AEXFJHKTYQW5R0ZCQQTWFV","anonymous","01M7C00M3GQR4QKP9GZA42GFAK",0,"MIT","https:\u002F\u002Fgithub.com\u002FMr-Nobody-Anonymous\u002FAll-skills\u002Fblob\u002Fmain\u002Fawesome_skills\u002Fbiology\u002Fbio-metagenomics-abundance\u002FSKILL.md",[],[],"2026-11-15 14:19:17",false,"---\nname: bio-metagenomics-abundance\ndescription: Turns shotgun classifier output into a defensible abundance table with Bracken Bayesian re-estimation, then compositional treatment (CLR, zero handling), library-size normalization, reference-frame differential abundance, and optional absolute quantification. Covers why a relative-abundance change is not a change, why Bracken read fractions and MetaPhlAn percentages are different physical quantities, the silent -r read-length bias, the genome-size confound no library-size method fixes, and the rarefaction debate. Use when estimating species abundance from a Kraken2 report, normalizing a community count table, choosing a compositional transform, or converting relative to absolute load. For classification see kraken-classification; for diversity\u002Fordination\u002FDA mechanics see metagenome-visualization.\ntool_type: mixed\nprimary_tool: Bracken\nsource: \"https:\u002F\u002Fgithub.com\u002FGPTomics\u002FbioSkills\"\nsource_repository: \"GPTomics\u002FbioSkills\"\nsource_path: \"metagenomics\u002Fabundance-estimation\u002FSKILL.md\"\nlicense: \"MIT\"\nimported_at: \"2026-09-20\"\n---\n\n## Version Compatibility\n\nReference examples tested with: Bracken 2.9+, Kraken2 2.1.3+, pandas 2.2+, scikit-bio 0.6+, R zCompositions 1.5+.\n\nBefore using code patterns, verify installed versions match. If versions differ:\n- CLI: `bracken -h`, `bracken-build -h` to confirm flags and defaults\n- Python: `pip show \u003Cpackage>` then `help(module.function)` to check signatures\n- R: `packageVersion('zCompositions')` then `?cmultRepl` to verify parameters\n\nIf code throws ImportError, AttributeError, or TypeError, introspect the installed\npackage and adapt the example to match the actual API rather than retrying.\n\nThe Bracken `databaseRLENmers.kmer_distrib` is built per database at a fixed read length and MUST match both the Kraken2 database and the actual (post-trim) read length; `-r` is not auto-detected and a mismatch silently biases every species fraction. Bracken needs the default Kraken2 report format, not mpa-style.\n\n# Abundance Estimation\n\n**\"How much of each taxon is in my sample?\"** -> Re-estimate species reads with Bracken, then treat the table as a composition - because the sequencer fixed the total, so the numbers are relative and a change in one forces apparent changes in the rest.\n- CLI: `bracken -d DB -i kraken.kreport -o out.bracken -w out.bracken.kreport -r 150 -l S -t 10`\n\nScope: Bracken mechanics plus what happens to the numbers afterward - estimand choice, compositional transforms, normalization, reference-frame DA logic, absolute conversion. Read classification -> kraken-classification, metaphlan-profiling. Diversity\u002Fordination\u002FDA tool mechanics -> metagenome-visualization. Generic 16S diversity -> the microbiome category.\n\n## The Single Most Important Modern Insight -- A Relative-Abundance Change Is Not a Change\n\nA shotgun abundance table is a composition: the sequencer fixes the total number of reads, not the sample's microbial load. So every number is relative to every other, and \"taxon X went up 2-fold\" is undefined without a reference frame or an external load anchor. Without one, a single blooming taxon makes every other taxon look depleted (the blooming-taxon illusion), and a Pearson correlation of two taxa's proportions is biased negative by arithmetic, not biology. Bracken is step one of about six: classify -> re-estimate -> compositional transform -> normalize -> reference-frame test -> (optional) absolute conversion. Two corollaries:\n\n1. **Bracken percent and MetaPhlAn percent are different physical quantities.** Bracken is a fraction of READS (large-genome biased, approximately fraction of DNA); MetaPhlAn is a genome-size-normalized marker abundance (approximately fraction of cells). They legitimately disagree 2-3x for the same sample. Picking one chooses an estimand (DNA vs cells), not the \"more accurate tool.\"\n2. **No library-size normalizer fixes the genome-size confound.** TSS\u002FCSS\u002FTMM\u002FGMPR all operate on the count table and inherit the large-genome bias; only a coverage-based estimand (CoverM) or a genome-normalized profiler (MetaPhlAn) removes it.\n\n## What Quantity Is Being Estimated?\n\n| Estimand | Definition | Bias \u002F use |\n|----------|------------|-----------|\n| Read count | raw reads to a taxon | library-size and genome-size dependent; never compare raw across samples |\n| Relative abundance | read count \u002F total | compositional (sums to 1); still genome-size biased |\n| Coverage abundance | reads x readlen \u002F genome size | removes large-genome bias; ~ fraction of genomes\u002Fcells (CoverM) |\n| Cell fraction | fraction of organisms | needs coverage + an assumption of one genome per cell; polyploidy\u002Fgrowth bias it |\n| Absolute load | composition x external total | the only basis for \"increased\u002Fdecreased\" (flow\u002FqPCR\u002Fspike-in) |\n\n## Bracken: Step One, Not the Finish Line\n\nBracken redistributes reads stranded at the genus\u002Ffamily node (where shared k-mers stopped Kraken) down to species, using a database-derived expectation of where length-L reads classify. It does not classify and it does not add precision.\n\n```bash\nbracken-build -d \"$KRAKEN_DB\" -t 8 -k 35 -l 150   # one-time; -k MUST equal the Kraken2 build k (35)\nbracken -d \"$KRAKEN_DB\" -i kraken.kreport -o out.bracken -w out.bracken.kreport \\\n    -r 150 \\   # MUST equal the bracken-build -l AND the actual post-trim read length (not auto-detected)\n    -l S -t 10 # species level; -t drops taxa with fewer than 10 clade-level reads (strict \u003C) before redistribution\n```\n\nOutput columns: `name, taxonomy_id, taxonomy_lvl, kraken_assigned_reads, added_reads, new_est_reads, fraction_total_reads`. `fraction_total_reads` is a fraction of classified-and-retained reads, not of all input - so two samples with different unclassified (e.g. host) fractions have non-comparable denominators. `combine_bracken_outputs.py --files *.bracken -o matrix.tsv` builds a taxa-by-sample matrix.\n\n### Bracken's defining failure mode\nBracken can only redistribute among species already in the database. A true organism absent from the database has its reads parked by Kraken at the shared genus node, and Bracken hands them to the database-present congeners - confidently fabricating or inflating those species. Distrust any species with high `added_reads` but tiny `kraken_assigned_reads`; gate presence on upstream unique-minimizer evidence (kraken-classification) and a coverage breadth check.\n\n## Treat the Table as a Composition\n\n**Goal:** Make the abundance table valid for multivariate stats and correlation by removing closure with a centered log-ratio, after handling zeros (log of zero is undefined).\n\n**Approach:** Impute count zeros with Bayesian-multiplicative replacement (preserves ratios), then CLR-transform; use Aitchison distance (Euclidean on CLR) downstream, never raw-proportion Pearson or Bray-Curtis for correlation.\n\n```python\nimport pandas as pd\nimport numpy as np\nfrom skbio.stats.composition import clr, multi_replace   # multiplicative_replacement was renamed multi_replace in skbio 0.6\n\ncounts = pd.read_csv('matrix.tsv', sep='\\t', index_col=0)   # taxa x samples (new_est_reads)\nmat = counts.T.values.astype(float)                          # samples x taxa for transform\nmat_nozero = multi_replace(mat \u002F mat.sum(axis=1, keepdims=True))\nclr_mat = clr(mat_nozero)                                    # closure removed; rows are CLR coordinates\nclr_df = pd.DataFrame(clr_mat, index=counts.columns, columns=counts.index)\n```\n\nFor sparse tables prefer R `zCompositions::cmultRepl()` (Bayesian-multiplicative, posterior-imputed) over a fixed +1 pseudocount, which is arbitrary, distorts ratios, and re-opens closure. Structural zeros (taxon genuinely absent) and sampling zeros (present below detection) are usually indistinguishable from the table - disclose the assumption rather than pretend otherwise.\n\n## Library-Size Normalization\n\n| Method | What it does | Shotgun applicability |\n|--------|--------------|-----------------------|\n| TSS (proportions) | divide by library size | IS the compositional closure; fine for viz, biased for DA\u002Fcorrelation |\n| Rarefaction | subsample to common depth | discards data; defensible for diversity, contested for DA (see below) |\n| CSS (metagenomeSeq) | scale by a cumulative-sum quantile | robust to dominant taxa; designed for marker surveys, usable on counts |\n| TMM (edgeR) | trimmed mean of M-values | \"most features unchanged\" often violated in microbiome; use with caution |\n| RLE (DESeq median-of-ratios) | geometric-mean reference | breaks on zeros (geometric mean -> 0); needs poscounts workaround |\n| GMPR | pairwise median ratios then geometric-mean | purpose-built for zero-inflated counts; good default size factor |\n\nNone of these fixes the genome-size confound - that needs a coverage estimand or a genome-normalized profiler.\n\n### The rarefaction debate (do not take a side; decide per analysis)\nMcMurdie & Holmes 2014 (*PLoS Comput Biol* 10:e1003531) showed rarefying for DIFFERENTIAL ABUNDANCE is statistically wasteful versus modeling library size. Schloss 2024 (*mSphere* 9:e00354-23 and the companion e00355-23) argues rarefaction is currently the best control of uneven effort for RICHNESS and COMMUNITY-DISTANCE analyses, where scaling alone does not remove the depth effect. They concern different downstream analyses: for DA do not rarefy (use a CoDA\u002Freference-frame method or a modeled size factor); for alpha\u002Fbeta diversity rarefaction is defensible. Treating \"rarefy: yes\u002Fno\" as one global switch is the mistake.\n\n## Reference-Frame Differential Abundance (logic here; mechanics in metagenome-visualization)\n\n\"Taxon X increased\" needs a reference frame because the total is fixed. Methods differ chiefly by their implicit frame: total-sum (naive, wrong), the geometric mean of all taxa (CLR \u002F ALDEx2), or an estimated per-sample sampling fraction (ANCOM-BC). ALDEx2 (Fernandes 2014 *Microbiome* 2:15) Monte-Carlo samples a Dirichlet posterior, CLR-transforms, and tests each draw; ANCOM-BC (Lin & Peddada 2020 *Nat Commun* 11:3514) estimates and corrects each sample's sampling fraction. Do not run naive t-tests or Wilcoxon on TSS proportions. Run the tools and read their output in metagenome-visualization.\n\n## Absolute Quantification: the Only Basis for \"Increased\u002FDecreased\"\n\nRelative methods recover the composition; absolute load = composition x an external total. Anchors: flow-cytometry cell counts (QMP, Vandeputte 2017 *Nature* 551:507, which showed apparent Crohn's \"increases\" were a microbial-load artifact - the absolute trajectory was opposite); per-taxon flow density (Props 2017 *ISME J* 11:584); a known spike-in organism added before extraction (SCML, Stammler 2016 *Microbiome* 4:28; a cellular spike also captures extraction bias a DNA spike misses); or total 16S qPCR copies (cheap, copy-number biased). Any \"X increased\" claim without an anchor or a reference-frame method is unsupported.\n\n## Per-Method Failure Modes\n\n### Bracken `-r` read-length mismatch\n**Trigger:** `-r 150` on trimmed ~120 bp reads, or a database built only for a different length. **Mechanism:** the redistribution prior is fragment-length specific and not auto-detected. **Symptom:** biased species fractions with no error if the `.kmer_distrib` exists, hard crash if not. **Fix:** `-r` = actual post-trim read length and a matching built distribution.\n\n### Pearson\u002FBray-Curtis on proportions\n**Trigger:** correlating two taxa's relative abundances, or Bray-Curtis distance for a \"who-co-occurs\" claim. **Mechanism:** closure biases proportion correlations negative and makes Euclidean\u002FBray-Curtis sub-compositionally incoherent. **Symptom:** spurious negative interactions; unstable clustering. **Fix:** CLR + Aitchison distance, or SparCC\u002Fproportionality for co-occurrence.\n\n### Relative fold-change reported as absolute\n**Trigger:** \"taxon X doubled\" from a relative table. **Mechanism:** one bloom deflates every other proportion. **Symptom:** whole-community \"depletion\" that is really one taxon rising. **Fix:** anchor to load (flow\u002FqPCR\u002Fspike-in) or use ANCOM-BC; state which frame.\n\n### RLE\u002FDESeq on a sparse table\n**Trigger:** DESeq2\u002FedgeR median-of-ratios on a species count table. **Mechanism:** the geometric-mean reference collapses to 0 when any feature has a zero. **Symptom:** degenerate size factors, errors, or nonsense fold-changes. **Fix:** GMPR or CSS; or DESeq2 with a poscounts estimator.\n\n## Quantitative Thresholds\n\n| Threshold | Source | Rationale |\n|-----------|--------|-----------|\n| Bracken `-k` = 35 | Lu 2017 *PeerJ Comput Sci* 3:e104 | must equal the Kraken2 database k-mer length |\n| Bracken `-r` = post-trim read length | Bracken docs | redistribution prior is fragment-length specific; silent bias otherwise |\n| Bracken `-t` 10 default | Bracken docs | redistribution floor; raise to suppress noise, too high deletes real rare taxa |\n| Multiplicative\u002FBayesian zero replacement over +1 | Martin-Fernandez 2015 *Stat Modelling* 15:134 | preserves ratios; fixed pseudocount distorts them and re-opens closure |\n| Rarefy for diversity, not for DA | McMurdie 2014; Schloss 2024 *mSphere* 9:e00354-23 | decision is per-analysis, not global |\n| Coverage breadth (`covered_fraction`) presence gate | Aroney 2025 *Bioinformatics* 41:btaf147 | high mean coverage over a few % of a genome is a conserved-region artifact |\n\n## Common Errors\n\n| Error \u002F symptom | Cause | Solution |\n|-----------------|-------|----------|\n| \"kmer_distrib not found\" | `-r` has no matching built distribution | `bracken-build -l \u003Creadlen>` or pick a DB shipping it |\n| Bracken rejects input | mpa-style or per-read file passed to `-i` | give the default Kraken2 `--report` |\n| Species with huge added_reads, tiny assigned | redistribution into a DB-present relative of an absent taxon | gate presence on unique minimizers + coverage breadth |\n| CLR returns -inf \u002F NaN | zeros not replaced before log-ratio | multiplicative or `cmultRepl` replacement first |\n| Cross-sample fractions not comparable | differing unclassified (host) fractions in the denominator | track classified fraction as a covariate; host-deplete upstream |\n| MetaPhlAn and Bracken disagree | different estimands (cells vs reads) | do not merge; pick one estimand and state it |\n\n## References\n\n- Lu J, Breitwieser FP, Thielen P, Salzberg SL. 2017. Bracken: estimating species abundance in metagenomics data. *PeerJ Comput Sci* 3:e104.\n- Gloor GB, Macklaim JM, Pawlowsky-Glahn V, Egozcue JJ. 2017. Microbiome datasets are compositional: and this is not optional. *Front Microbiol* 8:2224.\n- Quinn TP, Erb I, Richardson MF, Crowley TM. 2018. Understanding sequencing data as compositions: an outlook and review. *Bioinformatics* 34:2870-2878.\n- Fernandes AD, Reid JN, Macklaim JM, et al. 2014. Unifying the analysis of high-throughput sequencing datasets. *Microbiome* 2:15.\n- Lin H, Peddada SD. 2020. Analysis of compositions of microbiomes with bias correction. *Nat Commun* 11:3514.\n- Martin-Fernandez JA, Hron K, Templ M, Filzmoser P, Palarea-Albaladejo J. 2015. Bayesian-multiplicative treatment of count zeros in compositional data sets. *Stat Modelling* 15:134-158.\n- McMurdie PJ, Holmes S. 2014. Waste not, want not: why rarefying microbiome data is inadmissible. *PLoS Comput Biol* 10:e1003531.\n- Schloss PD. 2024. Rarefaction is currently the best approach to control for uneven sequencing effort in amplicon sequence analyses. *mSphere* 9:e00354-23.\n- Weiss S, Xu ZZ, Peddada S, et al. 2017. Normalization and microbial differential abundance strategies depend upon data characteristics. *Microbiome* 5:27.\n- Chen L, Reeve J, Zhang L, et al. 2018. GMPR: a robust normalization method for zero-inflated count data. *PeerJ* 6:e4600.\n- Vandeputte D, Kathagen G, D'hoe K, et al. 2017. Quantitative microbiome profiling links gut community variation to microbial load. *Nature* 551:507-511.\n- Stammler F, Glasner J, Hiergeist A, et al. 2016. Adjusting microbiome profiles for differences in microbial load by spike-in bacteria. *Microbiome* 4:28.\n- Aroney STN, Newell RJP, Nissen JN, Camargo AP, Tyson GW, Woodcroft BJ. 2025. CoverM: read alignment statistics for metagenomics. *Bioinformatics* 41:btaf147.\n\n## Related Skills\n\n- kraken-classification - Generates the Kraken2 report and owns the read-count-is-not-abundance reframe\n- metaphlan-profiling - Genome-size-normalized cell-fraction abundance; a different estimand\n- metagenome-visualization - Diversity, ordination, and differential-abundance tool mechanics\n- contamination-controls - Host-fraction and blank handling that affect the denominator\n- genome-assembly\u002Fmetagenome-assembly - Coverage-based MAG abundance via read mapping\n- workflows\u002Fmetagenomics-pipeline - End-to-end profiling and abundance\n","## Version Compatibility\n\nReference examples tested with: Bracken 2.9+, Kraken2 2.1.3+, pandas 2.2+, scikit-bio 0.6+, R zCompositions 1.5+.\n\nBefore using code patterns, verify installed versions match. If versions differ:\n- CLI: `bracken -h`, `bracken-build -h` to confirm flags and defaults\n- Python: `pip show \u003Cpackage>` then `help(module.function)` to check signatures\n- R: `packageVersion('zCompositions')` then `?cmultRepl` to verify parameters\n\nIf code throws ImportError, AttributeError, or TypeError, introspect the installed\npackage and adapt the example to match the actual API rather than retrying.\n\nThe Bracken `databaseRLENmers.kmer_distrib` is built per database at a fixed read length and MUST match both the Kraken2 database and the actual (post-trim) read length; `-r` is not auto-detected and a mismatch silently biases every species fraction. Bracken needs the default Kraken2 report format, not mpa-style.\n\n# Abundance Estimation\n\n**\"How much of each taxon is in my sample?\"** -> Re-estimate species reads with Bracken, then treat the table as a composition - because the sequencer fixed the total, so the numbers are relative and a change in one forces apparent changes in the rest.\n- CLI: `bracken -d DB -i kraken.kreport -o out.bracken -w out.bracken.kreport -r 150 -l S -t 10`\n\nScope: Bracken mechanics plus what happens to the numbers afterward - estimand choice, compositional transforms, normalization, reference-frame DA logic, absolute conversion. Read classification -> kraken-classification, metaphlan-profiling. Diversity\u002Fordination\u002FDA tool mechanics -> metagenome-visualization. Generic 16S diversity -> the microbiome category.\n\n## The Single Most Important Modern Insight -- A Relative-Abundance Change Is Not a Change\n\nA shotgun abundance table is a composition: the sequencer fixes the total number of reads, not the sample's microbial load. So every number is relative to every other, and \"taxon X went up 2-fold\" is undefined without a reference frame or an external load anchor. Without one, a single blooming taxon makes every other taxon look depleted (the blooming-taxon illusion), and a Pearson correlation of two taxa's proportions is biased negative by arithmetic, not biology. Bracken is step one of about six: classify -> re-estimate -> compositional transform -> normalize -> reference-frame test -> (optional) absolute conversion. Two corollaries:\n\n1. **Bracken percent and MetaPhlAn percent are different physical quantities.** Bracken is a fraction of READS (large-genome biased, approximately fraction of DNA); MetaPhlAn is a genome-size-normalized marker abundance (approximately fraction of cells). They legitimately disagree 2-3x for the same sample. Picking one chooses an estimand (DNA vs cells), not the \"more accurate tool.\"\n2. **No library-size normalizer fixes the genome-size confound.** TSS\u002FCSS\u002FTMM\u002FGMPR all operate on the count table and inherit the large-genome bias; only a coverage-based estimand (CoverM) or a genome-normalized profiler (MetaPhlAn) removes it.\n\n## What Quantity Is Being Estimated?\n\n| Estimand | Definition | Bias \u002F use |\n|----------|------------|-----------|\n| Read count | raw reads to a taxon | library-size and genome-size dependent; never compare raw across samples |\n| Relative abundance | read count \u002F total | compositional (sums to 1); still genome-size biased |\n| Coverage abundance | reads x readlen \u002F genome size | removes large-genome bias; ~ fraction of genomes\u002Fcells (CoverM) |\n| Cell fraction | fraction of organisms | needs coverage + an assumption of one genome per cell; polyploidy\u002Fgrowth bias it |\n| Absolute load | composition x external total | the only basis for \"increased\u002Fdecreased\" (flow\u002FqPCR\u002Fspike-in) |\n\n## Bracken: Step One, Not the Finish Line\n\nBracken redistributes reads stranded at the genus\u002Ffamily node (where shared k-mers stopped Kraken) down to species, using a database-derived expectation of where length-L reads classify. It does not classify and it does not add precision.\n\n```bash\nbracken-build -d \"$KRAKEN_DB\" -t 8 -k 35 -l 150   # one-time; -k MUST equal the Kraken2 build k (35)\nbracken -d \"$KRAKEN_DB\" -i kraken.kreport -o out.bracken -w out.bracken.kreport \\\n    -r 150 \\   # MUST equal the bracken-build -l AND the actual post-trim read length (not auto-detected)\n    -l S -t 10 # species level; -t drops taxa with fewer than 10 clade-level reads (strict \u003C) before redistribution\n```\n\nOutput columns: `name, taxonomy_id, taxonomy_lvl, kraken_assigned_reads, added_reads, new_est_reads, fraction_total_reads`. `fraction_total_reads` is a fraction of classified-and-retained reads, not of all input - so two samples with different unclassified (e.g. host) fractions have non-comparable denominators. `combine_bracken_outputs.py --files *.bracken -o matrix.tsv` builds a taxa-by-sample matrix.\n\n### Bracken's defining failure mode\nBracken can only redistribute among species already in the database. A true organism absent from the database has its reads parked by Kraken at the shared genus node, and Bracken hands them to the database-present congeners - confidently fabricating or inflating those species. Distrust any species with high `added_reads` but tiny `kraken_assigned_reads`; gate presence on upstream unique-minimizer evidence (kraken-classification) and a coverage breadth check.\n\n## Treat the Table as a Composition\n\n**Goal:** Make the abundance table valid for multivariate stats and correlation by removing closure with a centered log-ratio, after handling zeros (log of zero is undefined).\n\n**Approach:** Impute count zeros with Bayesian-multiplicative replacement (preserves ratios), then CLR-transform; use Aitchison distance (Euclidean on CLR) downstream, never raw-proportion Pearson or Bray-Curtis for correlation.\n\n```python\nimport pandas as pd\nimport numpy as np\nfrom skbio.stats.composition import clr, multi_replace   # multiplicative_replacement was renamed multi_replace in skbio 0.6\n\ncounts = pd.read_csv('matrix.tsv', sep='\\t', index_col=0)   # taxa x samples (new_est_reads)\nmat = counts.T.values.astype(float)                          # samples x taxa for transform\nmat_nozero = multi_replace(mat \u002F mat.sum(axis=1, keepdims=True))\nclr_mat = clr(mat_nozero)                                    # closure removed; rows are CLR coordinates\nclr_df = pd.DataFrame(clr_mat, index=counts.columns, columns=counts.index)\n```\n\nFor sparse tables prefer R `zCompositions::cmultRepl()` (Bayesian-multiplicative, posterior-imputed) over a fixed +1 pseudocount, which is arbitrary, distorts ratios, and re-opens closure. Structural zeros (taxon genuinely absent) and sampling zeros (present below detection) are usually indistinguishable from the table - disclose the assumption rather than pretend otherwise.\n\n## Library-Size Normalization\n\n| Method | What it does | Shotgun applicability |\n|--------|--------------|-----------------------|\n| TSS (proportions) | divide by library size | IS the compositional closure; fine for viz, biased for DA\u002Fcorrelation |\n| Rarefaction | subsample to common depth | discards data; defensible for diversity, contested for DA (see below) |\n| CSS (metagenomeSeq) | scale by a cumulative-sum quantile | robust to dominant taxa; designed for marker surveys, usable on counts |\n| TMM (edgeR) | trimmed mean of M-values | \"most features unchanged\" often violated in microbiome; use with caution |\n| RLE (DESeq median-of-ratios) | geometric-mean reference | breaks on zeros (geometric mean -> 0); needs poscounts workaround |\n| GMPR | pairwise median ratios then geometric-mean | purpose-built for zero-inflated counts; good default size factor |\n\nNone of these fixes the genome-size confound - that needs a coverage estimand or a genome-normalized profiler.\n\n### The rarefaction debate (do not take a side; decide per analysis)\nMcMurdie & Holmes 2014 (*PLoS Comput Biol* 10:e1003531) showed rarefying for DIFFERENTIAL ABUNDANCE is statistically wasteful versus modeling library size. Schloss 2024 (*mSphere* 9:e00354-23 and the companion e00355-23) argues rarefaction is currently the best control of uneven effort for RICHNESS and COMMUNITY-DISTANCE analyses, where scaling alone does not remove the depth effect. They concern different downstream analyses: for DA do not rarefy (use a CoDA\u002Freference-frame method or a modeled size factor); for alpha\u002Fbeta diversity rarefaction is defensible. Treating \"rarefy: yes\u002Fno\" as one global switch is the mistake.\n\n## Reference-Frame Differential Abundance (logic here; mechanics in metagenome-visualization)\n\n\"Taxon X increased\" needs a reference frame because the total is fixed. Methods differ chiefly by their implicit frame: total-sum (naive, wrong), the geometric mean of all taxa (CLR \u002F ALDEx2), or an estimated per-sample sampling fraction (ANCOM-BC). ALDEx2 (Fernandes 2014 *Microbiome* 2:15) Monte-Carlo samples a Dirichlet posterior, CLR-transforms, and tests each draw; ANCOM-BC (Lin & Peddada 2020 *Nat Commun* 11:3514) estimates and corrects each sample's sampling fraction. Do not run naive t-tests or Wilcoxon on TSS proportions. Run the tools and read their output in metagenome-visualization.\n\n## Absolute Quantification: the Only Basis for \"Increased\u002FDecreased\"\n\nRelative methods recover the composition; absolute load = composition x an external total. Anchors: flow-cytometry cell counts (QMP, Vandeputte 2017 *Nature* 551:507, which showed apparent Crohn's \"increases\" were a microbial-load artifact - the absolute trajectory was opposite); per-taxon flow density (Props 2017 *ISME J* 11:584); a known spike-in organism added before extraction (SCML, Stammler 2016 *Microbiome* 4:28; a cellular spike also captures extraction bias a DNA spike misses); or total 16S qPCR copies (cheap, copy-number biased). Any \"X increased\" claim without an anchor or a reference-frame method is unsupported.\n\n## Per-Method Failure Modes\n\n### Bracken `-r` read-length mismatch\n**Trigger:** `-r 150` on trimmed ~120 bp reads, or a database built only for a different length. **Mechanism:** the redistribution prior is fragment-length specific and not auto-detected. **Symptom:** biased species fractions with no error if the `.kmer_distrib` exists, hard crash if not. **Fix:** `-r` = actual post-trim read length and a matching built distribution.\n\n### Pearson\u002FBray-Curtis on proportions\n**Trigger:** correlating two taxa's relative abundances, or Bray-Curtis distance for a \"who-co-occurs\" claim. **Mechanism:** closure biases proportion correlations negative and makes Euclidean\u002FBray-Curtis sub-compositionally incoherent. **Symptom:** spurious negative interactions; unstable clustering. **Fix:** CLR + Aitchison distance, or SparCC\u002Fproportionality for co-occurrence.\n\n### Relative fold-change reported as absolute\n**Trigger:** \"taxon X doubled\" from a relative table. **Mechanism:** one bloom deflates every other proportion. **Symptom:** whole-community \"depletion\" that is really one taxon rising. **Fix:** anchor to load (flow\u002FqPCR\u002Fspike-in) or use ANCOM-BC; state which frame.\n\n### RLE\u002FDESeq on a sparse table\n**Trigger:** DESeq2\u002FedgeR median-of-ratios on a species count table. **Mechanism:** the geometric-mean reference collapses to 0 when any feature has a zero. **Symptom:** degenerate size factors, errors, or nonsense fold-changes. **Fix:** GMPR or CSS; or DESeq2 with a poscounts estimator.\n\n## Quantitative Thresholds\n\n| Threshold | Source | Rationale |\n|-----------|--------|-----------|\n| Bracken `-k` = 35 | Lu 2017 *PeerJ Comput Sci* 3:e104 | must equal the Kraken2 database k-mer length |\n| Bracken `-r` = post-trim read length | Bracken docs | redistribution prior is fragment-length specific; silent bias otherwise |\n| Bracken `-t` 10 default | Bracken docs | redistribution floor; raise to suppress noise, too high deletes real rare taxa |\n| Multiplicative\u002FBayesian zero replacement over +1 | Martin-Fernandez 2015 *Stat Modelling* 15:134 | preserves ratios; fixed pseudocount distorts them and re-opens closure |\n| Rarefy for diversity, not for DA | McMurdie 2014; Schloss 2024 *mSphere* 9:e00354-23 | decision is per-analysis, not global |\n| Coverage breadth (`covered_fraction`) presence gate | Aroney 2025 *Bioinformatics* 41:btaf147 | high mean coverage over a few % of a genome is a conserved-region artifact |\n\n## Common Errors\n\n| Error \u002F symptom | Cause | Solution |\n|-----------------|-------|----------|\n| \"kmer_distrib not found\" | `-r` has no matching built distribution | `bracken-build -l \u003Creadlen>` or pick a DB shipping it |\n| Bracken rejects input | mpa-style or per-read file passed to `-i` | give the default Kraken2 `--report` |\n| Species with huge added_reads, tiny assigned | redistribution into a DB-present relative of an absent taxon | gate presence on unique minimizers + coverage breadth |\n| CLR returns -inf \u002F NaN | zeros not replaced before log-ratio | multiplicative or `cmultRepl` replacement first |\n| Cross-sample fractions not comparable | differing unclassified (host) fractions in the denominator | track classified fraction as a covariate; host-deplete upstream |\n| MetaPhlAn and Bracken disagree | different estimands (cells vs reads) | do not merge; pick one estimand and state it |\n\n## References\n\n- Lu J, Breitwieser FP, Thielen P, Salzberg SL. 2017. Bracken: estimating species abundance in metagenomics data. *PeerJ Comput Sci* 3:e104.\n- Gloor GB, Macklaim JM, Pawlowsky-Glahn V, Egozcue JJ. 2017. Microbiome datasets are compositional: and this is not optional. *Front Microbiol* 8:2224.\n- Quinn TP, Erb I, Richardson MF, Crowley TM. 2018. Understanding sequencing data as compositions: an outlook and review. *Bioinformatics* 34:2870-2878.\n- Fernandes AD, Reid JN, Macklaim JM, et al. 2014. Unifying the analysis of high-throughput sequencing datasets. *Microbiome* 2:15.\n- Lin H, Peddada SD. 2020. Analysis of compositions of microbiomes with bias correction. *Nat Commun* 11:3514.\n- Martin-Fernandez JA, Hron K, Templ M, Filzmoser P, Palarea-Albaladejo J. 2015. Bayesian-multiplicative treatment of count zeros in compositional data sets. *Stat Modelling* 15:134-158.\n- McMurdie PJ, Holmes S. 2014. Waste not, want not: why rarefying microbiome data is inadmissible. *PLoS Comput Biol* 10:e1003531.\n- Schloss PD. 2024. Rarefaction is currently the best approach to control for uneven sequencing effort in amplicon sequence analyses. *mSphere* 9:e00354-23.\n- Weiss S, Xu ZZ, Peddada S, et al. 2017. Normalization and microbial differential abundance strategies depend upon data characteristics. *Microbiome* 5:27.\n- Chen L, Reeve J, Zhang L, et al. 2018. GMPR: a robust normalization method for zero-inflated count data. *PeerJ* 6:e4600.\n- Vandeputte D, Kathagen G, D'hoe K, et al. 2017. Quantitative microbiome profiling links gut community variation to microbial load. *Nature* 551:507-511.\n- Stammler F, Glasner J, Hiergeist A, et al. 2016. Adjusting microbiome profiles for differences in microbial load by spike-in bacteria. *Microbiome* 4:28.\n- Aroney STN, Newell RJP, Nissen JN, Camargo AP, Tyson GW, Woodcroft BJ. 2025. CoverM: read alignment statistics for metagenomics. *Bioinformatics* 41:btaf147.\n\n## Related Skills\n\n- kraken-classification - Generates the Kraken2 report and owns the read-count-is-not-abundance reframe\n- metaphlan-profiling - Genome-size-normalized cell-fraction abundance; a different estimand\n- metagenome-visualization - Diversity, ordination, and differential-abundance tool mechanics\n- contamination-controls - Host-fraction and blank handling that affect the denominator\n- genome-assembly\u002Fmetagenome-assembly - Coverage-based MAG abundance via read mapping\n- workflows\u002Fmetagenomics-pipeline - End-to-end profiling and abundance\n",{"name":5,"source":24,"license":15,"tool_type":25,"description":7,"imported_at":26,"source_path":27,"primary_tool":28,"source_repository":29},"https:\u002F\u002Fgithub.com\u002FGPTomics\u002FbioSkills","mixed","2026-09-20","metagenomics\u002Fabundance-estimation\u002FSKILL.md","Bracken","GPTomics\u002FbioSkills","main","2d1750717747097b602acf30bd09a7cbd4b4c6ea","awesome_skills\u002Fbiology\u002Fbio-metagenomics-abundance","https:\u002F\u002Fgithub.com\u002FMr-Nobody-Anonymous\u002FAll-skills","https:\u002F\u002Fgithub.com\u002FMr-Nobody-Anonymous","2026-11-09 11:45:36",{"source":8,"skillName":5,"commands":37,"prompt":47,"zipAvailable":48},[38,41,44],{"tool":39,"command":40},"npx","npx skills add Mr-Nobody-Anonymous\u002FAll-skills --skill bio-metagenomics-abundance",{"tool":42,"command":43},"bunx","bunx skills add Mr-Nobody-Anonymous\u002FAll-skills --skill bio-metagenomics-abundance",{"tool":45,"command":46},"pnpm","pnpm dlx skills add Mr-Nobody-Anonymous\u002FAll-skills --skill bio-metagenomics-abundance","安装技能，技能地址：https:\u002F\u002Fgithub.com\u002FMr-Nobody-Anonymous\u002FAll-skills\u002Fblob\u002Fmain\u002Fawesome_skills\u002Fbiology\u002Fbio-metagenomics-abundance\u002FSKILL.md",true,[50,56,60],{"path":51,"resourceType":52,"resourceTypeDesc":53,"sizeBytes":54,"sourceHtmlUrl":55},"examples\u002Fbracken_abundance.sh","SCRIPT","脚本",1575,"https:\u002F\u002Fgithub.com\u002FMr-Nobody-Anonymous\u002FAll-skills\u002Fblob\u002Fmain\u002Fawesome_skills\u002Fbiology\u002Fbio-metagenomics-abundance\u002Fexamples\u002Fbracken_abundance.sh",{"path":57,"resourceType":52,"resourceTypeDesc":53,"sizeBytes":58,"sourceHtmlUrl":59},"examples\u002Fcompositional_transform.py",2777,"https:\u002F\u002Fgithub.com\u002FMr-Nobody-Anonymous\u002FAll-skills\u002Fblob\u002Fmain\u002Fawesome_skills\u002Fbiology\u002Fbio-metagenomics-abundance\u002Fexamples\u002Fcompositional_transform.py",{"path":61,"resourceType":62,"resourceTypeDesc":63,"sizeBytes":64,"sourceHtmlUrl":65},"usage-guide.md","REFERENCE","参考文档",4657,"https:\u002F\u002Fgithub.com\u002FMr-Nobody-Anonymous\u002FAll-skills\u002Fblob\u002Fmain\u002Fawesome_skills\u002Fbiology\u002Fbio-metagenomics-abundance\u002Fusage-guide.md",[67,75,82,89,96,103],{"skillNo":68,"name":69,"nameZh":6,"description":70,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":71,"categories":72,"keywords":73,"publishedAt":74},"01M7FFFZPC2025JQQHPHDX154J","dei-assessment","Assess diversity, equity, and inclusion with metrics. TRIGGERS - Use when user needs help with dei-assessment related tasks.","https:\u002F\u002Fgithub.com\u002FMr-Nobody-Anonymous\u002FAll-skills\u002Fblob\u002Fmain\u002Fawesome_skills\u002Fhr-people\u002Fdei-assessment\u002FSKILL.md",[],[],"2026-11-16 10:05:34",{"skillNo":76,"name":77,"nameZh":6,"description":78,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":79,"categories":80,"keywords":81,"publishedAt":74},"01M7FFFWN8H4WA7A6Y398QKB6J","dei-action-plan","Create DEI action plans with goals and initiatives. TRIGGERS - Use when user needs help with dei-action-plan related tasks.","https:\u002F\u002Fgithub.com\u002FMr-Nobody-Anonymous\u002FAll-skills\u002Fblob\u002Fmain\u002Fawesome_skills\u002Fhr-people\u002Fdei-action-plan\u002FSKILL.md",[],[],{"skillNo":83,"name":84,"nameZh":6,"description":85,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":86,"categories":87,"keywords":88,"publishedAt":74},"01M7FFFSHTM0QZRVZ2NRJTJ0PY","culture-fit-assessment","Design culture add assessment processes. TRIGGERS - Use when user needs help with culture-fit-assessment related tasks.","https:\u002F\u002Fgithub.com\u002FMr-Nobody-Anonymous\u002FAll-skills\u002Fblob\u002Fmain\u002Fawesome_skills\u002Fhr-people\u002Fculture-fit-assessment\u002FSKILL.md",[],[],{"skillNo":90,"name":91,"nameZh":6,"description":92,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":93,"categories":94,"keywords":95,"publishedAt":74},"01M7FFFMA12RKEQA38KHJSXXPS","cowork-to-code-bridge","Use an already-installed, independently verified cowork-to-code bridge to run narrowly approved actions on the user's own macOS, Linux, or WSL2 machine through a local file queue.","https:\u002F\u002Fgithub.com\u002FMr-Nobody-Anonymous\u002FAll-skills\u002Fblob\u002Fmain\u002Fawesome_skills\u002Fhr-people\u002Fcowork-to-code-bridge\u002FSKILL.md",[],[],{"skillNo":97,"name":98,"nameZh":6,"description":99,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":100,"categories":101,"keywords":102,"publishedAt":74},"01M7FFFH1F6M8YRGW5V487GH1K","counter-offer-response","Draft counter-offer responses for retention. TRIGGERS - Use when user needs help with counter-offer-response related tasks.","https:\u002F\u002Fgithub.com\u002FMr-Nobody-Anonymous\u002FAll-skills\u002Fblob\u002Fmain\u002Fawesome_skills\u002Fhr-people\u002Fcounter-offer-response\u002FSKILL.md",[],[],{"skillNo":104,"name":105,"nameZh":6,"description":106,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":107,"categories":108,"keywords":109,"publishedAt":74},"01M7FFFDGEHV8Y7J9N40HZXW52","continuous-feedback-system","Design continuous feedback systems. TRIGGERS - Use when user needs help with continuous-feedback-system related tasks.","https:\u002F\u002Fgithub.com\u002FMr-Nobody-Anonymous\u002FAll-skills\u002Fblob\u002Fmain\u002Fawesome_skills\u002Fhr-people\u002Fcontinuous-feedback-system\u002FSKILL.md",[],[]]