[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"skill:01M7ED5KDGS9QAHSNZMD5SCQBH":3,"skill-same:01M7ED5KDGS9QAHSNZMD5SCQBH":62},{"skillNo":4,"name":5,"nameZh":6,"description":7,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":16,"categories":17,"keywords":18,"publishedAt":19,"linkOnly":20,"markdownContent":21,"markdownBody":22,"frontMatter":23,"refName":30,"commitSha":31,"skillDirectory":32,"forksCount":14,"repositoryHtmlUrl":33,"repositoryHomepageUrl":33,"ownerHtmlUrl":34,"ownerBlogUrl":6,"ownerDescription":6,"updatedAt":35,"install":36,"resources":49},"01M7ED5KDGS9QAHSNZMD5SCQBH","bio-expression-matrix-counts-ingest",null,"Imports gene expression count matrices from featureCounts, HTSeq, STAR ReadsPerGene, Salmon\u002Fkallisto via tximport or tximeta, RSEM, 10X Genomics MTX\u002FH5, AnnData H5AD, and RDS. Handles silent-miscounting traps (featureCounts -p v2.0.2 API break, STAR strandedness column choice, salmon NumReads-sum without tximport, RSEM non-integer expected_count, GENCODE _PAR_Y suffix, zero-length-transcript TPM divide-by-zero), and encodes the tximport countsFromAbundance decision tree with the \"lengthScaledTPM is not TPM\" warning. Use when assembling a gene-by-sample count matrix from aligner or quantifier output, importing salmon\u002Fkallisto for DESeq2 vs limma-voom, choosing strandedness column for STAR, debugging zero-count panics, or building tx2gene mapping.","Mr-Nobody-Anonymous\u002FAll-skills","Mr-Nobody-Anonymous","https:\u002F\u002Favatars.githubusercontent.com\u002Fu\u002F184253119?v=4","01M7AEXFJHKTYQW5R0ZCQQTWFV","anonymous","01M7C00M3GQR4QKP9GZA42GFAK",0,"MIT","https:\u002F\u002Fgithub.com\u002FMr-Nobody-Anonymous\u002FAll-skills\u002Fblob\u002Fmain\u002Fawesome_skills\u002Fbiology\u002Fbio-expression-matrix-counts-ingest\u002FSKILL.md",[],[],"2026-11-15 14:19:17",false,"---\nname: bio-expression-matrix-counts-ingest\ndescription: Imports gene expression count matrices from featureCounts, HTSeq, STAR ReadsPerGene, Salmon\u002Fkallisto via tximport or tximeta, RSEM, 10X Genomics MTX\u002FH5, AnnData H5AD, and RDS. Handles silent-miscounting traps (featureCounts -p v2.0.2 API break, STAR strandedness column choice, salmon NumReads-sum without tximport, RSEM non-integer expected_count, GENCODE _PAR_Y suffix, zero-length-transcript TPM divide-by-zero), and encodes the tximport countsFromAbundance decision tree with the \"lengthScaledTPM is not TPM\" warning. Use when assembling a gene-by-sample count matrix from aligner or quantifier output, importing salmon\u002Fkallisto for DESeq2 vs limma-voom, choosing strandedness column for STAR, debugging zero-count panics, or building tx2gene mapping.\ntool_type: mixed\nprimary_tool: tximport\nsource: \"https:\u002F\u002Fgithub.com\u002FGPTomics\u002FbioSkills\"\nsource_repository: \"GPTomics\u002FbioSkills\"\nsource_path: \"expression-matrix\u002Fcounts-ingest\u002FSKILL.md\"\nlicense: \"MIT\"\nimported_at: \"2026-09-20\"\n---\n\n## Version Compatibility\n\nReference examples tested with: pandas 2.2+, numpy 1.26+, scanpy 1.10+, anndata 0.10+, tximport 1.30+, tximeta 1.20+, GenomicFeatures 1.54+, Subread\u002FfeatureCounts 2.0.6+ (post-v2.0.2 API), STAR 2.7.10+, Salmon 1.10+, kallisto 0.48+, RSEM 1.3.3+, HTSeq 2.0+\n\nBefore using code patterns, verify installed versions match. If versions differ:\n- Python: `pip show \u003Cpackage>` then `help(module.function)` to check signatures\n- R: `packageVersion('\u003Cpkg>')` then `?function_name` to verify parameters\n- CLI: `\u003Ctool> --version` then `\u003Ctool> --help` to confirm flags\n\nIf code throws ImportError, AttributeError, or TypeError, introspect the installed\npackage and adapt the example to match the actual API rather than retrying.\n\n# Count Matrix Ingestion\n\n**\"Load my featureCounts \u002F Salmon \u002F STAR output into a count matrix\"** -> Parse the per-sample quantification, strip metadata, choose the correct strandedness or NumReads column, optionally apply length-bias correction via tximport, and return a gene-by-sample matrix appropriate for the downstream DE tool.\n\n## The Single Most Important Modern Insight -- `lengthScaledTPM` is NOT TPM; the naming has fooled many\n\n`tximport(..., countsFromAbundance='lengthScaledTPM')` returns a **count-scale matrix** (sum to library size) with the gene-length bias removed, intended as input to DE tools that cannot accept offsets (limma-voom). The word \"TPM\" in the option name has misled users into reporting these values as normalized abundance -- they are not.\n\nThe decision tree for `countsFromAbundance` (Soneson, Love, Robinson 2015 *F1000Res* 4:1521):\n\n| Option | Returns | Use with |\n|--------|---------|----------|\n| `'no'` (default) | Raw NumReads; length matrix passed as DESeq2\u002FedgeR offset | DESeq2 via `DESeqDataSetFromTximport`, edgeR via `DGEList` -- the correct path for DGE |\n| `'scaledTPM'` | TPM scaled to library size | DGE when downstream tool cannot accept offsets (rare) |\n| `'lengthScaledTPM'` | TPM scaled by avg transcript length AND library size | limma-voom for DGE; recommended modern default when offsets unavailable |\n| `'dtuScaledTPM'` | Per-transcript scaling by median isoform length | Differential Transcript Usage (DRIMSeq, DEXSeq) ONLY; requires `txOut=TRUE` |\n\nTwo adjacent traps with the same flavor of silent miscount:\n\n1. **featureCounts `-p` since v2.0.2 (March 2021) needs `--countReadPairs`**. Pre-v2.0.2, `-p` flagged paired-end AND counted pairs as fragments. Post-v2.0.2, `-p` only flags paired-end -- pairs are NOT counted as one fragment unless `--countReadPairs` is added. Old scripts run on new installs produce ~2x the expected counts (one count per mate). No warning. Always pass `-p --countReadPairs` together for paired-end.\n\n2. **STAR `ReadsPerGene.out.tab` column choice depends on library protocol**. Column 2 is unstranded; column 3 is forward-stranded; column 4 is reverse-stranded. Illumina TruSeq Stranded (the dominant kit, dUTP-based) is reverse-stranded -- column 4. Reading column 3 for TruSeq throws away ~95% of reads. Reading column 2 conflates antisense expression. Verify with RSeQC `infer_experiment.py` on a few BAMs before assembling the matrix.\n\n## Algorithmic Taxonomy\n\n| Source | Output structure | Read into | Caveats |\n|--------|------------------|-----------|---------|\n| featureCounts (Liao 2014) | TSV with 6 metadata cols + N sample cols | `read.delim` (R), `pd.read_csv(comment='#')` (Python) | -p\u002F--countReadPairs v2.0.2 break; -O double-counts; -s strandedness |\n| HTSeq-count (Anders 2015) | Per-sample 2-col TSV with `__` summary lines | Per-sample read, drop `__` rows, concat | `--mode` matters (union vs intersection-strict vs intersection-nonempty); `-s` strandedness |\n| STAR `--quantMode GeneCounts` | Per-sample 4-col `ReadsPerGene.out.tab` | Skip first 4 summary rows; pick column by strandedness | Column 4 = TruSeq Stranded |\n| Salmon `quant.sf` | Per-sample 5-col TSV (Name, Length, EffectiveLength, TPM, NumReads) | tximport (recommended) or manual NumReads sum (length-biased) | Selective alignment is the default since Salmon 1.0.0 (Srivastava 2020); the `--validateMappings` flag is now a no-op |\n| kallisto `abundance.tsv` \u002F `.h5` | Per-sample 5-col TSV; bootstraps in `.h5` | tximport (gene-level) or sleuth (transcript-level with bootstrap variance) | `kallisto quant -b 100` for sleuth |\n| RSEM `*.genes.results` \u002F `*.isoforms.results` | Per-sample TSV; expected_count is non-integer | tximport (`type='rsem'`) with `round()` via DESeq2; or manual | Zero-length transcripts cause `lengths > 0 is not TRUE` |\n| 10X Genomics CellRanger | filtered_feature_bc_matrix\u002F MTX dir OR `.h5` | scanpy `read_10x_mtx` \u002F `read_10x_h5`; Seurat `Read10X` | Single-cell convention: cells in rows |\n| AnnData `.h5ad` | scverse single-file binary | scanpy `read_h5ad`; in R via zellkonverter | `.X` vs `.layers['counts']` vs `.raw.X` semantics |\n| RDS (from R) | R-serialized object | `pyreadr` (Python); base R `readRDS` | For Seurat objects, convert first |\n\n## Decision Tree by Scenario\n\n| Scenario | Recommended approach |\n|----------|---------------------|\n| Bulk RNA-seq with featureCounts paired-end | `featureCounts -p --countReadPairs -s 2 -t exon -g gene_id` (TruSeq Stranded) |\n| Bulk RNA-seq with STAR | `--quantMode GeneCounts`; read column 4 for TruSeq Stranded |\n| Salmon\u002Fkallisto -> gene-level DGE with DESeq2 | `tximport(type='salmon', tx2gene)` + `DESeqDataSetFromTximport()` -- offsets handled |\n| Salmon -> gene-level DGE with limma-voom | `tximport(..., countsFromAbundance='lengthScaledTPM')` -- no offset |\n| Salmon\u002Fkallisto -> DTE (differential transcript expression) | `tximport(..., txOut=TRUE, countsFromAbundance='dtuScaledTPM')` for DRIMSeq\u002FDEXSeq; OR `edgeR::catchSalmon()` for the Baldoni 2024 framework |\n| kallisto with bootstraps -> sleuth (uncertainty-aware DE) | `sleuth_prep()` directly; do not go through tximport |\n| RSEM -> DESeq2 | `tximport(files, type='rsem', txIn=FALSE)` + `DESeqDataSetFromTximport()` |\n| Want automatic annotation provenance | `tximeta` (Love 2020 *PLoS Comp Biol* 16:e1007664) |\n| 3'-tagged library (10x bulk, QuantSeq) | `countsFromAbundance='no'` WITHOUT length offset -- length bias negligible |\n| 10X single-cell | `sc.read_10x_h5()` or `Read10X_h5()` |\n| Strandedness unknown | RSeQC `infer_experiment.py` on 1-2 BAMs BEFORE re-running quantification |\n\n## featureCounts -- The `-p` and `-O` Traps\n\n**Goal:** Run featureCounts correctly on paired-end stranded RNA-seq, then read the output without inheriting the metadata columns.\n\n**Approach:** CLI invocation with `-p --countReadPairs -s \u003Cstrandedness>`, plus optional `-O --fraction` for overlapping-gene handling; parse with pandas\u002Fread.delim stripping the 6 metadata columns.\n\n```bash\nfeatureCounts -T 8 -p --countReadPairs -s 2 \\\n    -t exon -g gene_id \\\n    -a annotation.gtf \\\n    -o featurecounts.txt \\\n    sample1.bam sample2.bam sample3.bam\n```\n\n```python\nimport pandas as pd\n\nfc = pd.read_csv('featurecounts.txt', sep='\\t', comment='#')\ncounts = fc.set_index('Geneid').iloc[:, 5:]\ncounts.columns = [c.replace('.bam', '').split('\u002F')[-1] for c in counts.columns]\n```\n\n```r\nfc \u003C- read.delim('featurecounts.txt', comment.char = '#', row.names = 1)\ncounts \u003C- fc[, 6:ncol(fc)]\ncolnames(counts) \u003C- gsub('.*\u002F|\\\\.bam$', '', colnames(counts))\n```\n\n| Flag | Meaning | Default | When to flip |\n|------|---------|---------|--------------|\n| `-p` | Input is paired-end | off | Always for paired-end -- AND add `--countReadPairs` |\n| `--countReadPairs` | Count fragment (pair) as one | off in v2.0.2+ | Always for paired-end (post-v2.0.2 API change) |\n| `-s 0|1|2` | Strandedness | 0 (unstranded) | `-s 2` for TruSeq Stranded; `-s 1` for forward kits |\n| `-O` | Allow multi-overlap | off | Off for typical DGE; on creates double-counts in overlapping genes |\n| `-M` | Count multi-mappers | off | Off for DGE; on with `--fraction` for fractional counting |\n| `-t` | Feature type | `exon` | Almost always `exon` |\n| `-g` | Group attribute | `gene_id` | `gene_id` for gene-level; `transcript_id` is wrong (use Salmon for transcript-level) |\n\n## STAR `--quantMode GeneCounts`\n\n**Goal:** Build a count matrix from STAR's per-sample 4-column output, choosing the correct strandedness column.\n\n**Approach:** Skip the first 4 summary rows; index column 0 (gene ID); pick column for strandedness; concat across samples.\n\n```python\nimport pandas as pd\nfrom pathlib import Path\n\ndef load_star_genecounts(filepaths, strandedness='reverse'):\n    '''Load STAR ReadsPerGene.out.tab files.\n    File columns (1-indexed): 1=gene_id, 2=unstranded, 3=forward, 4=reverse.\n    After read_csv with index_col=0, the three remaining columns are 0=unstranded, 1=forward, 2=reverse.\n    Illumina TruSeq Stranded is 'reverse'.\n    '''\n    col_map = {'unstranded': 0, 'forward': 1, 'reverse': 2}\n    col_idx = col_map[strandedness]\n    dfs = {}\n    for fp in filepaths:\n        sample = Path(fp).name.replace('_ReadsPerGene.out.tab', '')\n        df = pd.read_csv(fp, sep='\\t', header=None, index_col=0)\n        dfs[sample] = df.iloc[4:, col_idx]\n    return pd.DataFrame(dfs)\n```\n\nStrandedness verification: for a stranded library, the \"wrong\" strand column total should be \u003C5% of the \"right\" strand column total. If comparable, the library was unstranded or there was a kit \u002F config mix-up.\n\n```bash\ninfer_experiment.py -r annotation.bed -i sample.bam\n```\n\n## Salmon \u002F kallisto via tximport\n\n**Goal:** Import transcript-level Salmon or kallisto quantifications to gene-level counts with length-bias correction.\n\n**Approach:** Build a `tx2gene` mapping; call `tximport()` with the right `type` and `countsFromAbundance`; hand the result to DESeq2 or limma-voom.\n\n```r\nlibrary(tximport)\nlibrary(DESeq2)\n\ntx2gene \u003C- read.csv('tx2gene.csv')\n\nfiles \u003C- file.path('salmon_out', samples$id, 'quant.sf')\nnames(files) \u003C- samples$id\n\ntxi \u003C- tximport(files, type = 'salmon', tx2gene = tx2gene)\n\ndds \u003C- DESeqDataSetFromTximport(txi, colData = samples, design = ~ condition)\n```\n\nThe naive alternative -- summing NumReads to gene level -- is wrong when isoform usage varies across samples. NumReads is normalized against EffectiveLength per transcript; sum-then-DE introduces a length-by-condition bias indistinguishable from differential expression. tximport handles this by carrying the per-sample average transcript length matrix as a DESeq2\u002FedgeR offset.\n\nFor limma-voom (no offset mechanism):\n\n```r\ntxi \u003C- tximport(files, type = 'salmon', tx2gene = tx2gene,\n                countsFromAbundance = 'lengthScaledTPM')\ny \u003C- DGEList(counts = txi$counts)\nv \u003C- voom(y, design, plot = TRUE)\n```\n\nFor DTU (DRIMSeq, DEXSeq), use `dtuScaledTPM` with `txOut=TRUE`.\n\nFor 3'-tagged libraries (10x Chromium 3', QuantSeq), length bias is negligible; use `countsFromAbundance='no'` WITHOUT the length offset (`tximeta`'s argument or manual disable).\n\n## tximeta -- Automatic Provenance\n\n**Goal:** Import Salmon\u002Fkallisto with automatic linkage to the exact annotation release used in the index.\n\n**Approach:** `tximeta` (Love, Soneson, Hickey et al. 2020 *PLoS Comp Biol* 16:e1007664) inspects the Salmon index hash and pulls matching annotation and metadata; the resulting `SummarizedExperiment` carries the provenance.\n\n```r\nlibrary(tximeta)\n\ncoldata \u003C- data.frame(\n    names = samples$id,\n    files = file.path('salmon_out', samples$id, 'quant.sf'),\n    condition = samples$condition\n)\n\nse \u003C- tximeta(coldata)\ngse \u003C- summarizeToGene(se)\n\nlibrary(DESeq2)\ndds \u003C- DESeqDataSet(gse, design = ~ condition)\n```\n\nFor new projects, `tximeta` is the modern preference over hand-managed `tx2gene` -- it eliminates a class of \"wrong annotation\" bugs.\n\n## HTSeq-count\n\n```python\nimport pandas as pd\nfrom pathlib import Path\n\ndef load_htseq_counts(filepaths):\n    '''Load HTSeq count files, dropping the __no_feature etc. summary rows.'''\n    dfs = {}\n    for fp in filepaths:\n        sample = Path(fp).stem.replace('_counts', '')\n        df = pd.read_csv(fp, sep='\\t', header=None, index_col=0,\n                         names=['gene', 'count'])\n        df = df[~df.index.str.startswith('__')]\n        dfs[sample] = df['count']\n    return pd.DataFrame(dfs)\n```\n\nHTSeq overlap modes (Anders, Pyl, Huber 2015 *Bioinformatics* 31:166):\n\n| Mode | Behavior |\n|------|----------|\n| `union` (default) | Read assigned to union of overlapping features; >1 gene -> `__ambiguous` |\n| `intersection-strict` | Every base of read must overlap the same single feature |\n| `intersection-nonempty` | Intersection across positions; nonempty -> that gene wins |\n\n`-s yes|no|reverse`: TruSeq Stranded is `reverse`. The `--mode` and `-s` flags must match the library; mismatches are silent miscounts.\n\nThe `__no_feature` and `__ambiguous` totals are QC signals. If `__no_feature` > 30%: annotation incomplete, wrong reference, or strandedness misspecified. If `__ambiguous` > 10%: many overlapping gene annotations (common with comprehensive GTFs); consider `intersection-nonempty`.\n\n## RSEM expected_count\n\n```r\nlibrary(tximport)\nlibrary(DESeq2)\n\nfiles \u003C- file.path('rsem_out', paste0(samples$id, '.genes.results'))\nnames(files) \u003C- samples$id\ntxi \u003C- tximport(files, type = 'rsem', txIn = FALSE, txOut = FALSE)\n\ntxi$length[txi$length == 0] \u003C- 1\n\ndds \u003C- DESeqDataSetFromTximport(txi, colData = samples, design = ~ condition)\n```\n\nRSEM `expected_count` is non-integer (EM-derived). DESeq2 requires integers, but `DESeqDataSetFromTximport` rounds appropriately. The `txi$length == 0` substitution handles the \"Error: all(lengths > 0) is not TRUE\" panic from rRNA-filtered or zero-length transcripts.\n\nAvoid `round()` + `DESeqDataSetFromMatrix` -- that path loses the length correction.\n\n## kallisto + sleuth (Uncertainty-Aware DE)\n\n```bash\nkallisto quant -i index -o sample1_out -b 100 -t 8 read1_1.fq.gz read1_2.fq.gz\n```\n\n```r\nlibrary(sleuth)\n\ns2c \u003C- data.frame(sample = samples$id,\n                  condition = samples$condition,\n                  path = file.path('kallisto_out', samples$id))\n\nso \u003C- sleuth_prep(s2c, ~ condition,\n                  target_mapping = tx2gene,\n                  aggregation_column = 'gene_id',\n                  gene_mode = TRUE)\nso \u003C- sleuth_fit(so, ~ condition, 'full')\nso \u003C- sleuth_fit(so, ~ 1, 'reduced')\nso \u003C- sleuth_lrt(so, 'reduced', 'full')\nresults \u003C- sleuth_results(so, 'reduced:full')\n```\n\nSleuth (Pimentel et al. 2017 *Nat Methods* 14:687) uses the 100 bootstrap replicates from kallisto to decouple BIOLOGICAL variance (between-replicate) from INFERENTIAL variance (within-replicate, from EM uncertainty). For transcript-level analyses where quantification uncertainty matters (similar isoforms, low coverage), sleuth's response error linear model is more conservative than DESeq2 on plain counts. For well-quantified gene-level DGE, sleuth and DESeq2-via-tximport converge.\n\n## Salmon Selective Alignment\n\nSelective alignment (Srivastava 2020 *Genome Biol* 21:239) -- which combines fast pseudo-mapping with traditional alignment of seed extensions to improve quantification accuracy for transcripts with sequence similarity -- has been the **default** since Salmon 1.0.0. The historical `--validateMappings` flag that explicitly requested it is now a deprecated no-op; passing it does nothing.\n\n```bash\nsalmon quant -i index -l A \\\n    -1 read_1.fq.gz -2 read_2.fq.gz \\\n    --gcBias --seqBias \\\n    -p 8 -o sample_out\n```\n\n`--gcBias` corrects fragment GC bias; `--seqBias` corrects random hexamer priming bias. Both are recommended for any Salmon run from RNA-seq with biological condition variation. They affect EffectiveLength estimates and propagate through tximport.\n\n## 10X Genomics\n\n```python\nimport scanpy as sc\n\nadata = sc.read_10x_mtx('filtered_feature_bc_matrix\u002F')\nadata = sc.read_10x_h5('filtered_feature_bc_matrix.h5')\n```\n\n```r\nlibrary(Seurat)\nmat \u003C- Read10X(data.dir = 'filtered_feature_bc_matrix\u002F')\nmat \u003C- Read10X_h5('filtered_feature_bc_matrix.h5')\n```\n\n10X convention: cells in rows (AnnData) or cells in columns (Seurat after Read10X). The R\u002FPython convention differs by transpose.\n\n## Annotation Pre-Filtering -- rRNA, Mt-rRNA, Pseudogenes\n\n**Goal:** Drop biotypes that distort downstream normalization or are irrelevant to the question.\n\n**Approach:** Filter on `gene_biotype` from the GTF (Ensembl) or `gene_type` (GENCODE).\n\n```r\nlibrary(rtracklayer)\ngtf \u003C- import('Homo_sapiens.GRCh38.110.gtf.gz')\ngene_info \u003C- as.data.frame(gtf[gtf$type == 'gene',\n                                c('gene_id', 'gene_name', 'gene_biotype')])\n\nkeep_biotypes \u003C- c('protein_coding', 'lncRNA',\n                   'IG_C_gene', 'IG_D_gene', 'IG_J_gene', 'IG_V_gene',\n                   'TR_C_gene', 'TR_D_gene', 'TR_J_gene', 'TR_V_gene')\nkeep_genes \u003C- gene_info$gene_id[gene_info$gene_biotype %in% keep_biotypes]\ncounts_filt \u003C- counts[rownames(counts) %in% keep_genes, ]\n```\n\nMt-encoded protein-coding genes (MT-CO1, MT-ND1, ...) are a judgment call: include for tissue-specific work (heart, muscle); drop when mitochondrial fraction varies with cell stress and could confound normalization.\n\nIn single-cell, mitochondrial percentage is a STANDARD QC metric (cells with high %mito are stressed\u002Fdying); see `single-cell\u002Fpreprocessing`.\n\n## GENCODE vs Ensembl -- What Differs at the File Level\n\n| Difference | Ensembl | GENCODE |\n|------------|---------|---------|\n| chromosome naming | `1`, `2`, ..., `MT` | `chr1`, `chr2`, ..., `chrM` |\n| PAR gene encoding (releases 25-43) | Once on chrX | Both chrX and chrY with `_PAR_Y` suffix on chrY copies |\n| PAR gene encoding (releases 44+ \u002F Ensembl 110+) | Once on chrX | chrY copies get their own ENSG accessions; `_PAR_Y` retired |\n| Subset releases | Single release | `basic` (high-confidence subset) and `comprehensive` (everything) |\n\nCRITICAL: code that strips Ensembl version suffixes via `sub('\\\\..*', '', x)` ALSO strips the `_PAR_Y` tag in GENCODE 25-43, collapsing the chrY duplicate onto the chrX gene -- silently introducing duplicate row indices. Use `sub('\\\\.[0-9]+(_PAR_Y)?$', '\\\\1', x)` to preserve.\n\nBAM files aligned to GENCODE (`chr1`) cannot be quantified against Ensembl GTF (`1`) without rename. Mismatched naming causes `__no_feature` to dominate.\n\n## Filter Low-Count Genes\n\n```r\nlibrary(edgeR)\n\ny \u003C- DGEList(counts = counts, group = group)\nkeep \u003C- filterByExpr(y, design = model.matrix(~ condition, coldata))\ny \u003C- y[keep, , keep.lib.sizes = FALSE]\n```\n\n```python\nmin_counts, min_samples = 10, 3\nexpressed = (counts >= min_counts).sum(axis=1) >= min_samples\ncounts_filt = counts.loc[expressed]\n```\n\nSee `differential-expression\u002Fedger-basics` for `filterByExpr` semantics. DESeq2 has automatic independent filtering at `results()` time; manual pre-filter is speed-only.\n\n## Per-Method Failure Modes\n\n### featureCounts paired-end double-counted\n\n**Trigger:** Pipeline written against featureCounts pre-v2.0.2 used `-p` only; rerun on Subread 2.0.6 produces counts ~2x expected.\n\n**Mechanism:** Post-v2.0.2 `-p` flags paired-end but does NOT count pairs as fragments. Each mate is counted separately.\n\n**Symptom:** Library sizes ~2x what RNA-seq QC reports; downstream CPM compressed; \"more reads than expected\" panic.\n\n**Fix:** Add `--countReadPairs`. Re-run featureCounts.\n\n### STAR wrong strandedness column\n\n**Trigger:** TruSeq Stranded library quantified via STAR; user reads column 3 (forward); ~95% of reads dropped.\n\n**Mechanism:** TruSeq Stranded is reverse-stranded (column 4). Column 3 is forward-stranded; for a reverse library, almost no reads align in the forward orientation.\n\n**Symptom:** Library sizes ~5% of expected; almost no DE detectable; very low gene detection rate.\n\n**Fix:** Read column 4. Verify with `infer_experiment.py`.\n\n### Salmon NumReads summed naively -> length-by-condition bias\n\n**Trigger:** Salmon output read without tximport; per-transcript NumReads summed to gene level via groupby.\n\n**Mechanism:** NumReads is normalized against per-transcript EffectiveLength. Summing ignores that. If treatment shifts isoform usage from short to long, the gene appears upregulated even with constant total mRNA.\n\n**Symptom:** DE genes overlap with known isoform-switching genes (e.g., during development); fold changes don't replicate at the protein level.\n\n**Fix:** Use `tximport` (or `catchSalmon` in edgeR) to carry the length matrix as a DE offset.\n\n### RSEM zero-length transcript breaks tximport\n\n**Trigger:** `tximport(files, type='rsem')` errors out with \"all(lengths > 0) is not TRUE\".\n\n**Mechanism:** RSEM reports `effective_length = 0` for certain very-short or filtered transcripts.\n\n**Symptom:** Pipeline halts at import.\n\n**Fix:** `txi$length[txi$length == 0] \u003C- 1` after `tximport`; or filter `tx2gene` to exclude affected transcripts.\n\n### `_PAR_Y` stripped, chrY duplicates lost\n\n**Trigger:** GENCODE v40 quantification; `rownames(counts) \u003C- sub('\\\\..*', '', rownames(counts))`; duplicate row indices.\n\n**Mechanism:** Default regex strips `_PAR_Y` along with the version suffix; chrY PAR copies collapse onto chrX gene IDs.\n\n**Symptom:** Duplicate row warnings; sample-specific counts double for affected genes; downstream `aggregate(... ~ rownames)` adds them.\n\n**Fix:** Use the regex that preserves `_PAR_Y`: `sub('\\\\.[0-9]+(_PAR_Y)?$', '\\\\1', x)`. Or upgrade to GENCODE 44+ where the issue is gone.\n\n### HTSeq mode mismatched to library\n\n**Trigger:** Mouse RNA-seq with `--mode intersection-strict`; many overlapping-gene reads dropped; library size 30% lower than featureCounts on the same data.\n\n**Mechanism:** `intersection-strict` requires every base to overlap the same feature; reads crossing exon-intron boundaries or overlapping gene boundaries get dropped.\n\n**Symptom:** Lower count totals than other quantifiers; `__no_feature` and `__ambiguous` high.\n\n**Fix:** `--mode union` (default) or `--mode intersection-nonempty` for richer recovery.\n\n## Common errors\n\n| Error \u002F symptom | Cause | Fix |\n|-----------------|-------|-----|\n| `Error: all(lengths > 0) is not TRUE` | RSEM zero-length transcripts | `txi$length[txi$length == 0] \u003C- 1` |\n| Duplicate row indices after version strip | `_PAR_Y` collapsed | Use the preserving regex |\n| `__no_feature` >30% | Wrong reference; wrong strandedness; chrN naming mismatch | Verify reference + strandedness; check chr vs N naming |\n| `counts matrix should be integers` | Salmon\u002FRSEM raw counts to DESeq2 | Use `DESeqDataSetFromTximport`; or round (loses length correction) |\n| Library sizes ~2x expected for paired-end | featureCounts v2.0.2 `-p` without `--countReadPairs` | Add `--countReadPairs` |\n| TPM column has Inf values | Zero-length features | Filter `Length > 0` before TPM computation |\n| Salmon TPM doesn't sum to 1e6 | Pre-summed across samples or filtered subset | Recompute TPM after filter; or use original abundance column |\n\n## References\n\n- Liao Y, Smyth GK, Shi W. 2014. featureCounts: an efficient general purpose program for assigning sequence reads to genomic features. *Bioinformatics* 30(7):923-930. doi:10.1093\u002Fbioinformatics\u002Fbtt656\n- Anders S, Pyl PT, Huber W. 2015. HTSeq -- a Python framework to work with high-throughput sequencing data. *Bioinformatics* 31(2):166-169. doi:10.1093\u002Fbioinformatics\u002Fbtu638\n- Dobin A et al. 2013. STAR: ultrafast universal RNA-seq aligner. *Bioinformatics* 29(1):15-21. doi:10.1093\u002Fbioinformatics\u002Fbts635\n- Patro R, Duggal G, Love MI, Irizarry RA, Kingsford C. 2017. Salmon provides fast and bias-aware quantification of transcript expression. *Nat Methods* 14(4):417-419. doi:10.1038\u002Fnmeth.4197\n- Srivastava A, Malik L, Sarkar H, Zakeri M, Almodaresi F, Soneson C, Love MI, Kingsford C, Patro R. 2020. Alignment and mapping methodology influence transcript abundance estimation. *Genome Biol* 21:239. doi:10.1186\u002Fs13059-020-02151-8\n- Bray NL, Pimentel H, Melsted P, Pachter L. 2016. Near-optimal probabilistic RNA-seq quantification. *Nat Biotechnol* 34(5):525-527. doi:10.1038\u002Fnbt.3519\n- Pimentel H, Bray NL, Puente S, Melsted P, Pachter L. 2017. Differential analysis of RNA-seq incorporating quantification uncertainty. *Nat Methods* 14(7):687-690. doi:10.1038\u002Fnmeth.4324\n- Soneson C, Love MI, Robinson MD. 2015. Differential analyses for RNA-seq: transcript-level estimates improve gene-level inferences. *F1000Res* 4:1521. doi:10.12688\u002Ff1000research.7563.2\n- Love MI, Soneson C, Hickey PF, Johnson LK, Pierce NT, Shepherd L, Morgan M, Patro R. 2020. Tximeta: Reference sequence checksums for provenance identification in RNA-seq. *PLoS Comput Biol* 16(2):e1007664. doi:10.1371\u002Fjournal.pcbi.1007664\n- Frankish A et al. 2021. GENCODE 2021. *Nucleic Acids Res* 49(D1):D916-D923. doi:10.1093\u002Fnar\u002Fgkaa1087\n\n## Related Skills\n\n- gene-id-mapping - Building tx2gene; Ensembl version stripping with PAR_Y preservation\n- normalization - Composition bias, TMM\u002FRLE, the \"biotype filter before normalize\" rationale\n- metadata-joins - Sample name reconciliation across counts and metadata\n- sparse-handling - Single-cell sparse matrix formats\n- differential-expression\u002Fdeseq2-basics - tximport -> DESeqDataSetFromTximport workflow\n- differential-expression\u002Fedger-basics - catchSalmon for transcript-level DTE\n- differential-expression\u002Fbatch-correction - Batch covariate vs subtraction\n- rna-quantification\u002Ffeaturecounts-counting - Detailed featureCounts run patterns\n- rna-quantification\u002Falignment-free-quant - Salmon\u002Fkallisto invocation details\n- rna-quantification\u002Ftximport-workflow - Detailed tximport workflow\n- read-alignment\u002Fstar-alignment - STAR upstream of GeneCounts\n- read-qc\u002Fquality-reports - RIN, DV200 inputs to consider\n","## Version Compatibility\n\nReference examples tested with: pandas 2.2+, numpy 1.26+, scanpy 1.10+, anndata 0.10+, tximport 1.30+, tximeta 1.20+, GenomicFeatures 1.54+, Subread\u002FfeatureCounts 2.0.6+ (post-v2.0.2 API), STAR 2.7.10+, Salmon 1.10+, kallisto 0.48+, RSEM 1.3.3+, HTSeq 2.0+\n\nBefore using code patterns, verify installed versions match. If versions differ:\n- Python: `pip show \u003Cpackage>` then `help(module.function)` to check signatures\n- R: `packageVersion('\u003Cpkg>')` then `?function_name` to verify parameters\n- CLI: `\u003Ctool> --version` then `\u003Ctool> --help` to confirm flags\n\nIf code throws ImportError, AttributeError, or TypeError, introspect the installed\npackage and adapt the example to match the actual API rather than retrying.\n\n# Count Matrix Ingestion\n\n**\"Load my featureCounts \u002F Salmon \u002F STAR output into a count matrix\"** -> Parse the per-sample quantification, strip metadata, choose the correct strandedness or NumReads column, optionally apply length-bias correction via tximport, and return a gene-by-sample matrix appropriate for the downstream DE tool.\n\n## The Single Most Important Modern Insight -- `lengthScaledTPM` is NOT TPM; the naming has fooled many\n\n`tximport(..., countsFromAbundance='lengthScaledTPM')` returns a **count-scale matrix** (sum to library size) with the gene-length bias removed, intended as input to DE tools that cannot accept offsets (limma-voom). The word \"TPM\" in the option name has misled users into reporting these values as normalized abundance -- they are not.\n\nThe decision tree for `countsFromAbundance` (Soneson, Love, Robinson 2015 *F1000Res* 4:1521):\n\n| Option | Returns | Use with |\n|--------|---------|----------|\n| `'no'` (default) | Raw NumReads; length matrix passed as DESeq2\u002FedgeR offset | DESeq2 via `DESeqDataSetFromTximport`, edgeR via `DGEList` -- the correct path for DGE |\n| `'scaledTPM'` | TPM scaled to library size | DGE when downstream tool cannot accept offsets (rare) |\n| `'lengthScaledTPM'` | TPM scaled by avg transcript length AND library size | limma-voom for DGE; recommended modern default when offsets unavailable |\n| `'dtuScaledTPM'` | Per-transcript scaling by median isoform length | Differential Transcript Usage (DRIMSeq, DEXSeq) ONLY; requires `txOut=TRUE` |\n\nTwo adjacent traps with the same flavor of silent miscount:\n\n1. **featureCounts `-p` since v2.0.2 (March 2021) needs `--countReadPairs`**. Pre-v2.0.2, `-p` flagged paired-end AND counted pairs as fragments. Post-v2.0.2, `-p` only flags paired-end -- pairs are NOT counted as one fragment unless `--countReadPairs` is added. Old scripts run on new installs produce ~2x the expected counts (one count per mate). No warning. Always pass `-p --countReadPairs` together for paired-end.\n\n2. **STAR `ReadsPerGene.out.tab` column choice depends on library protocol**. Column 2 is unstranded; column 3 is forward-stranded; column 4 is reverse-stranded. Illumina TruSeq Stranded (the dominant kit, dUTP-based) is reverse-stranded -- column 4. Reading column 3 for TruSeq throws away ~95% of reads. Reading column 2 conflates antisense expression. Verify with RSeQC `infer_experiment.py` on a few BAMs before assembling the matrix.\n\n## Algorithmic Taxonomy\n\n| Source | Output structure | Read into | Caveats |\n|--------|------------------|-----------|---------|\n| featureCounts (Liao 2014) | TSV with 6 metadata cols + N sample cols | `read.delim` (R), `pd.read_csv(comment='#')` (Python) | -p\u002F--countReadPairs v2.0.2 break; -O double-counts; -s strandedness |\n| HTSeq-count (Anders 2015) | Per-sample 2-col TSV with `__` summary lines | Per-sample read, drop `__` rows, concat | `--mode` matters (union vs intersection-strict vs intersection-nonempty); `-s` strandedness |\n| STAR `--quantMode GeneCounts` | Per-sample 4-col `ReadsPerGene.out.tab` | Skip first 4 summary rows; pick column by strandedness | Column 4 = TruSeq Stranded |\n| Salmon `quant.sf` | Per-sample 5-col TSV (Name, Length, EffectiveLength, TPM, NumReads) | tximport (recommended) or manual NumReads sum (length-biased) | Selective alignment is the default since Salmon 1.0.0 (Srivastava 2020); the `--validateMappings` flag is now a no-op |\n| kallisto `abundance.tsv` \u002F `.h5` | Per-sample 5-col TSV; bootstraps in `.h5` | tximport (gene-level) or sleuth (transcript-level with bootstrap variance) | `kallisto quant -b 100` for sleuth |\n| RSEM `*.genes.results` \u002F `*.isoforms.results` | Per-sample TSV; expected_count is non-integer | tximport (`type='rsem'`) with `round()` via DESeq2; or manual | Zero-length transcripts cause `lengths > 0 is not TRUE` |\n| 10X Genomics CellRanger | filtered_feature_bc_matrix\u002F MTX dir OR `.h5` | scanpy `read_10x_mtx` \u002F `read_10x_h5`; Seurat `Read10X` | Single-cell convention: cells in rows |\n| AnnData `.h5ad` | scverse single-file binary | scanpy `read_h5ad`; in R via zellkonverter | `.X` vs `.layers['counts']` vs `.raw.X` semantics |\n| RDS (from R) | R-serialized object | `pyreadr` (Python); base R `readRDS` | For Seurat objects, convert first |\n\n## Decision Tree by Scenario\n\n| Scenario | Recommended approach |\n|----------|---------------------|\n| Bulk RNA-seq with featureCounts paired-end | `featureCounts -p --countReadPairs -s 2 -t exon -g gene_id` (TruSeq Stranded) |\n| Bulk RNA-seq with STAR | `--quantMode GeneCounts`; read column 4 for TruSeq Stranded |\n| Salmon\u002Fkallisto -> gene-level DGE with DESeq2 | `tximport(type='salmon', tx2gene)` + `DESeqDataSetFromTximport()` -- offsets handled |\n| Salmon -> gene-level DGE with limma-voom | `tximport(..., countsFromAbundance='lengthScaledTPM')` -- no offset |\n| Salmon\u002Fkallisto -> DTE (differential transcript expression) | `tximport(..., txOut=TRUE, countsFromAbundance='dtuScaledTPM')` for DRIMSeq\u002FDEXSeq; OR `edgeR::catchSalmon()` for the Baldoni 2024 framework |\n| kallisto with bootstraps -> sleuth (uncertainty-aware DE) | `sleuth_prep()` directly; do not go through tximport |\n| RSEM -> DESeq2 | `tximport(files, type='rsem', txIn=FALSE)` + `DESeqDataSetFromTximport()` |\n| Want automatic annotation provenance | `tximeta` (Love 2020 *PLoS Comp Biol* 16:e1007664) |\n| 3'-tagged library (10x bulk, QuantSeq) | `countsFromAbundance='no'` WITHOUT length offset -- length bias negligible |\n| 10X single-cell | `sc.read_10x_h5()` or `Read10X_h5()` |\n| Strandedness unknown | RSeQC `infer_experiment.py` on 1-2 BAMs BEFORE re-running quantification |\n\n## featureCounts -- The `-p` and `-O` Traps\n\n**Goal:** Run featureCounts correctly on paired-end stranded RNA-seq, then read the output without inheriting the metadata columns.\n\n**Approach:** CLI invocation with `-p --countReadPairs -s \u003Cstrandedness>`, plus optional `-O --fraction` for overlapping-gene handling; parse with pandas\u002Fread.delim stripping the 6 metadata columns.\n\n```bash\nfeatureCounts -T 8 -p --countReadPairs -s 2 \\\n    -t exon -g gene_id \\\n    -a annotation.gtf \\\n    -o featurecounts.txt \\\n    sample1.bam sample2.bam sample3.bam\n```\n\n```python\nimport pandas as pd\n\nfc = pd.read_csv('featurecounts.txt', sep='\\t', comment='#')\ncounts = fc.set_index('Geneid').iloc[:, 5:]\ncounts.columns = [c.replace('.bam', '').split('\u002F')[-1] for c in counts.columns]\n```\n\n```r\nfc \u003C- read.delim('featurecounts.txt', comment.char = '#', row.names = 1)\ncounts \u003C- fc[, 6:ncol(fc)]\ncolnames(counts) \u003C- gsub('.*\u002F|\\\\.bam$', '', colnames(counts))\n```\n\n| Flag | Meaning | Default | When to flip |\n|------|---------|---------|--------------|\n| `-p` | Input is paired-end | off | Always for paired-end -- AND add `--countReadPairs` |\n| `--countReadPairs` | Count fragment (pair) as one | off in v2.0.2+ | Always for paired-end (post-v2.0.2 API change) |\n| `-s 0|1|2` | Strandedness | 0 (unstranded) | `-s 2` for TruSeq Stranded; `-s 1` for forward kits |\n| `-O` | Allow multi-overlap | off | Off for typical DGE; on creates double-counts in overlapping genes |\n| `-M` | Count multi-mappers | off | Off for DGE; on with `--fraction` for fractional counting |\n| `-t` | Feature type | `exon` | Almost always `exon` |\n| `-g` | Group attribute | `gene_id` | `gene_id` for gene-level; `transcript_id` is wrong (use Salmon for transcript-level) |\n\n## STAR `--quantMode GeneCounts`\n\n**Goal:** Build a count matrix from STAR's per-sample 4-column output, choosing the correct strandedness column.\n\n**Approach:** Skip the first 4 summary rows; index column 0 (gene ID); pick column for strandedness; concat across samples.\n\n```python\nimport pandas as pd\nfrom pathlib import Path\n\ndef load_star_genecounts(filepaths, strandedness='reverse'):\n    '''Load STAR ReadsPerGene.out.tab files.\n    File columns (1-indexed): 1=gene_id, 2=unstranded, 3=forward, 4=reverse.\n    After read_csv with index_col=0, the three remaining columns are 0=unstranded, 1=forward, 2=reverse.\n    Illumina TruSeq Stranded is 'reverse'.\n    '''\n    col_map = {'unstranded': 0, 'forward': 1, 'reverse': 2}\n    col_idx = col_map[strandedness]\n    dfs = {}\n    for fp in filepaths:\n        sample = Path(fp).name.replace('_ReadsPerGene.out.tab', '')\n        df = pd.read_csv(fp, sep='\\t', header=None, index_col=0)\n        dfs[sample] = df.iloc[4:, col_idx]\n    return pd.DataFrame(dfs)\n```\n\nStrandedness verification: for a stranded library, the \"wrong\" strand column total should be \u003C5% of the \"right\" strand column total. If comparable, the library was unstranded or there was a kit \u002F config mix-up.\n\n```bash\ninfer_experiment.py -r annotation.bed -i sample.bam\n```\n\n## Salmon \u002F kallisto via tximport\n\n**Goal:** Import transcript-level Salmon or kallisto quantifications to gene-level counts with length-bias correction.\n\n**Approach:** Build a `tx2gene` mapping; call `tximport()` with the right `type` and `countsFromAbundance`; hand the result to DESeq2 or limma-voom.\n\n```r\nlibrary(tximport)\nlibrary(DESeq2)\n\ntx2gene \u003C- read.csv('tx2gene.csv')\n\nfiles \u003C- file.path('salmon_out', samples$id, 'quant.sf')\nnames(files) \u003C- samples$id\n\ntxi \u003C- tximport(files, type = 'salmon', tx2gene = tx2gene)\n\ndds \u003C- DESeqDataSetFromTximport(txi, colData = samples, design = ~ condition)\n```\n\nThe naive alternative -- summing NumReads to gene level -- is wrong when isoform usage varies across samples. NumReads is normalized against EffectiveLength per transcript; sum-then-DE introduces a length-by-condition bias indistinguishable from differential expression. tximport handles this by carrying the per-sample average transcript length matrix as a DESeq2\u002FedgeR offset.\n\nFor limma-voom (no offset mechanism):\n\n```r\ntxi \u003C- tximport(files, type = 'salmon', tx2gene = tx2gene,\n                countsFromAbundance = 'lengthScaledTPM')\ny \u003C- DGEList(counts = txi$counts)\nv \u003C- voom(y, design, plot = TRUE)\n```\n\nFor DTU (DRIMSeq, DEXSeq), use `dtuScaledTPM` with `txOut=TRUE`.\n\nFor 3'-tagged libraries (10x Chromium 3', QuantSeq), length bias is negligible; use `countsFromAbundance='no'` WITHOUT the length offset (`tximeta`'s argument or manual disable).\n\n## tximeta -- Automatic Provenance\n\n**Goal:** Import Salmon\u002Fkallisto with automatic linkage to the exact annotation release used in the index.\n\n**Approach:** `tximeta` (Love, Soneson, Hickey et al. 2020 *PLoS Comp Biol* 16:e1007664) inspects the Salmon index hash and pulls matching annotation and metadata; the resulting `SummarizedExperiment` carries the provenance.\n\n```r\nlibrary(tximeta)\n\ncoldata \u003C- data.frame(\n    names = samples$id,\n    files = file.path('salmon_out', samples$id, 'quant.sf'),\n    condition = samples$condition\n)\n\nse \u003C- tximeta(coldata)\ngse \u003C- summarizeToGene(se)\n\nlibrary(DESeq2)\ndds \u003C- DESeqDataSet(gse, design = ~ condition)\n```\n\nFor new projects, `tximeta` is the modern preference over hand-managed `tx2gene` -- it eliminates a class of \"wrong annotation\" bugs.\n\n## HTSeq-count\n\n```python\nimport pandas as pd\nfrom pathlib import Path\n\ndef load_htseq_counts(filepaths):\n    '''Load HTSeq count files, dropping the __no_feature etc. summary rows.'''\n    dfs = {}\n    for fp in filepaths:\n        sample = Path(fp).stem.replace('_counts', '')\n        df = pd.read_csv(fp, sep='\\t', header=None, index_col=0,\n                         names=['gene', 'count'])\n        df = df[~df.index.str.startswith('__')]\n        dfs[sample] = df['count']\n    return pd.DataFrame(dfs)\n```\n\nHTSeq overlap modes (Anders, Pyl, Huber 2015 *Bioinformatics* 31:166):\n\n| Mode | Behavior |\n|------|----------|\n| `union` (default) | Read assigned to union of overlapping features; >1 gene -> `__ambiguous` |\n| `intersection-strict` | Every base of read must overlap the same single feature |\n| `intersection-nonempty` | Intersection across positions; nonempty -> that gene wins |\n\n`-s yes|no|reverse`: TruSeq Stranded is `reverse`. The `--mode` and `-s` flags must match the library; mismatches are silent miscounts.\n\nThe `__no_feature` and `__ambiguous` totals are QC signals. If `__no_feature` > 30%: annotation incomplete, wrong reference, or strandedness misspecified. If `__ambiguous` > 10%: many overlapping gene annotations (common with comprehensive GTFs); consider `intersection-nonempty`.\n\n## RSEM expected_count\n\n```r\nlibrary(tximport)\nlibrary(DESeq2)\n\nfiles \u003C- file.path('rsem_out', paste0(samples$id, '.genes.results'))\nnames(files) \u003C- samples$id\ntxi \u003C- tximport(files, type = 'rsem', txIn = FALSE, txOut = FALSE)\n\ntxi$length[txi$length == 0] \u003C- 1\n\ndds \u003C- DESeqDataSetFromTximport(txi, colData = samples, design = ~ condition)\n```\n\nRSEM `expected_count` is non-integer (EM-derived). DESeq2 requires integers, but `DESeqDataSetFromTximport` rounds appropriately. The `txi$length == 0` substitution handles the \"Error: all(lengths > 0) is not TRUE\" panic from rRNA-filtered or zero-length transcripts.\n\nAvoid `round()` + `DESeqDataSetFromMatrix` -- that path loses the length correction.\n\n## kallisto + sleuth (Uncertainty-Aware DE)\n\n```bash\nkallisto quant -i index -o sample1_out -b 100 -t 8 read1_1.fq.gz read1_2.fq.gz\n```\n\n```r\nlibrary(sleuth)\n\ns2c \u003C- data.frame(sample = samples$id,\n                  condition = samples$condition,\n                  path = file.path('kallisto_out', samples$id))\n\nso \u003C- sleuth_prep(s2c, ~ condition,\n                  target_mapping = tx2gene,\n                  aggregation_column = 'gene_id',\n                  gene_mode = TRUE)\nso \u003C- sleuth_fit(so, ~ condition, 'full')\nso \u003C- sleuth_fit(so, ~ 1, 'reduced')\nso \u003C- sleuth_lrt(so, 'reduced', 'full')\nresults \u003C- sleuth_results(so, 'reduced:full')\n```\n\nSleuth (Pimentel et al. 2017 *Nat Methods* 14:687) uses the 100 bootstrap replicates from kallisto to decouple BIOLOGICAL variance (between-replicate) from INFERENTIAL variance (within-replicate, from EM uncertainty). For transcript-level analyses where quantification uncertainty matters (similar isoforms, low coverage), sleuth's response error linear model is more conservative than DESeq2 on plain counts. For well-quantified gene-level DGE, sleuth and DESeq2-via-tximport converge.\n\n## Salmon Selective Alignment\n\nSelective alignment (Srivastava 2020 *Genome Biol* 21:239) -- which combines fast pseudo-mapping with traditional alignment of seed extensions to improve quantification accuracy for transcripts with sequence similarity -- has been the **default** since Salmon 1.0.0. The historical `--validateMappings` flag that explicitly requested it is now a deprecated no-op; passing it does nothing.\n\n```bash\nsalmon quant -i index -l A \\\n    -1 read_1.fq.gz -2 read_2.fq.gz \\\n    --gcBias --seqBias \\\n    -p 8 -o sample_out\n```\n\n`--gcBias` corrects fragment GC bias; `--seqBias` corrects random hexamer priming bias. Both are recommended for any Salmon run from RNA-seq with biological condition variation. They affect EffectiveLength estimates and propagate through tximport.\n\n## 10X Genomics\n\n```python\nimport scanpy as sc\n\nadata = sc.read_10x_mtx('filtered_feature_bc_matrix\u002F')\nadata = sc.read_10x_h5('filtered_feature_bc_matrix.h5')\n```\n\n```r\nlibrary(Seurat)\nmat \u003C- Read10X(data.dir = 'filtered_feature_bc_matrix\u002F')\nmat \u003C- Read10X_h5('filtered_feature_bc_matrix.h5')\n```\n\n10X convention: cells in rows (AnnData) or cells in columns (Seurat after Read10X). The R\u002FPython convention differs by transpose.\n\n## Annotation Pre-Filtering -- rRNA, Mt-rRNA, Pseudogenes\n\n**Goal:** Drop biotypes that distort downstream normalization or are irrelevant to the question.\n\n**Approach:** Filter on `gene_biotype` from the GTF (Ensembl) or `gene_type` (GENCODE).\n\n```r\nlibrary(rtracklayer)\ngtf \u003C- import('Homo_sapiens.GRCh38.110.gtf.gz')\ngene_info \u003C- as.data.frame(gtf[gtf$type == 'gene',\n                                c('gene_id', 'gene_name', 'gene_biotype')])\n\nkeep_biotypes \u003C- c('protein_coding', 'lncRNA',\n                   'IG_C_gene', 'IG_D_gene', 'IG_J_gene', 'IG_V_gene',\n                   'TR_C_gene', 'TR_D_gene', 'TR_J_gene', 'TR_V_gene')\nkeep_genes \u003C- gene_info$gene_id[gene_info$gene_biotype %in% keep_biotypes]\ncounts_filt \u003C- counts[rownames(counts) %in% keep_genes, ]\n```\n\nMt-encoded protein-coding genes (MT-CO1, MT-ND1, ...) are a judgment call: include for tissue-specific work (heart, muscle); drop when mitochondrial fraction varies with cell stress and could confound normalization.\n\nIn single-cell, mitochondrial percentage is a STANDARD QC metric (cells with high %mito are stressed\u002Fdying); see `single-cell\u002Fpreprocessing`.\n\n## GENCODE vs Ensembl -- What Differs at the File Level\n\n| Difference | Ensembl | GENCODE |\n|------------|---------|---------|\n| chromosome naming | `1`, `2`, ..., `MT` | `chr1`, `chr2`, ..., `chrM` |\n| PAR gene encoding (releases 25-43) | Once on chrX | Both chrX and chrY with `_PAR_Y` suffix on chrY copies |\n| PAR gene encoding (releases 44+ \u002F Ensembl 110+) | Once on chrX | chrY copies get their own ENSG accessions; `_PAR_Y` retired |\n| Subset releases | Single release | `basic` (high-confidence subset) and `comprehensive` (everything) |\n\nCRITICAL: code that strips Ensembl version suffixes via `sub('\\\\..*', '', x)` ALSO strips the `_PAR_Y` tag in GENCODE 25-43, collapsing the chrY duplicate onto the chrX gene -- silently introducing duplicate row indices. Use `sub('\\\\.[0-9]+(_PAR_Y)?$', '\\\\1', x)` to preserve.\n\nBAM files aligned to GENCODE (`chr1`) cannot be quantified against Ensembl GTF (`1`) without rename. Mismatched naming causes `__no_feature` to dominate.\n\n## Filter Low-Count Genes\n\n```r\nlibrary(edgeR)\n\ny \u003C- DGEList(counts = counts, group = group)\nkeep \u003C- filterByExpr(y, design = model.matrix(~ condition, coldata))\ny \u003C- y[keep, , keep.lib.sizes = FALSE]\n```\n\n```python\nmin_counts, min_samples = 10, 3\nexpressed = (counts >= min_counts).sum(axis=1) >= min_samples\ncounts_filt = counts.loc[expressed]\n```\n\nSee `differential-expression\u002Fedger-basics` for `filterByExpr` semantics. DESeq2 has automatic independent filtering at `results()` time; manual pre-filter is speed-only.\n\n## Per-Method Failure Modes\n\n### featureCounts paired-end double-counted\n\n**Trigger:** Pipeline written against featureCounts pre-v2.0.2 used `-p` only; rerun on Subread 2.0.6 produces counts ~2x expected.\n\n**Mechanism:** Post-v2.0.2 `-p` flags paired-end but does NOT count pairs as fragments. Each mate is counted separately.\n\n**Symptom:** Library sizes ~2x what RNA-seq QC reports; downstream CPM compressed; \"more reads than expected\" panic.\n\n**Fix:** Add `--countReadPairs`. Re-run featureCounts.\n\n### STAR wrong strandedness column\n\n**Trigger:** TruSeq Stranded library quantified via STAR; user reads column 3 (forward); ~95% of reads dropped.\n\n**Mechanism:** TruSeq Stranded is reverse-stranded (column 4). Column 3 is forward-stranded; for a reverse library, almost no reads align in the forward orientation.\n\n**Symptom:** Library sizes ~5% of expected; almost no DE detectable; very low gene detection rate.\n\n**Fix:** Read column 4. Verify with `infer_experiment.py`.\n\n### Salmon NumReads summed naively -> length-by-condition bias\n\n**Trigger:** Salmon output read without tximport; per-transcript NumReads summed to gene level via groupby.\n\n**Mechanism:** NumReads is normalized against per-transcript EffectiveLength. Summing ignores that. If treatment shifts isoform usage from short to long, the gene appears upregulated even with constant total mRNA.\n\n**Symptom:** DE genes overlap with known isoform-switching genes (e.g., during development); fold changes don't replicate at the protein level.\n\n**Fix:** Use `tximport` (or `catchSalmon` in edgeR) to carry the length matrix as a DE offset.\n\n### RSEM zero-length transcript breaks tximport\n\n**Trigger:** `tximport(files, type='rsem')` errors out with \"all(lengths > 0) is not TRUE\".\n\n**Mechanism:** RSEM reports `effective_length = 0` for certain very-short or filtered transcripts.\n\n**Symptom:** Pipeline halts at import.\n\n**Fix:** `txi$length[txi$length == 0] \u003C- 1` after `tximport`; or filter `tx2gene` to exclude affected transcripts.\n\n### `_PAR_Y` stripped, chrY duplicates lost\n\n**Trigger:** GENCODE v40 quantification; `rownames(counts) \u003C- sub('\\\\..*', '', rownames(counts))`; duplicate row indices.\n\n**Mechanism:** Default regex strips `_PAR_Y` along with the version suffix; chrY PAR copies collapse onto chrX gene IDs.\n\n**Symptom:** Duplicate row warnings; sample-specific counts double for affected genes; downstream `aggregate(... ~ rownames)` adds them.\n\n**Fix:** Use the regex that preserves `_PAR_Y`: `sub('\\\\.[0-9]+(_PAR_Y)?$', '\\\\1', x)`. Or upgrade to GENCODE 44+ where the issue is gone.\n\n### HTSeq mode mismatched to library\n\n**Trigger:** Mouse RNA-seq with `--mode intersection-strict`; many overlapping-gene reads dropped; library size 30% lower than featureCounts on the same data.\n\n**Mechanism:** `intersection-strict` requires every base to overlap the same feature; reads crossing exon-intron boundaries or overlapping gene boundaries get dropped.\n\n**Symptom:** Lower count totals than other quantifiers; `__no_feature` and `__ambiguous` high.\n\n**Fix:** `--mode union` (default) or `--mode intersection-nonempty` for richer recovery.\n\n## Common errors\n\n| Error \u002F symptom | Cause | Fix |\n|-----------------|-------|-----|\n| `Error: all(lengths > 0) is not TRUE` | RSEM zero-length transcripts | `txi$length[txi$length == 0] \u003C- 1` |\n| Duplicate row indices after version strip | `_PAR_Y` collapsed | Use the preserving regex |\n| `__no_feature` >30% | Wrong reference; wrong strandedness; chrN naming mismatch | Verify reference + strandedness; check chr vs N naming |\n| `counts matrix should be integers` | Salmon\u002FRSEM raw counts to DESeq2 | Use `DESeqDataSetFromTximport`; or round (loses length correction) |\n| Library sizes ~2x expected for paired-end | featureCounts v2.0.2 `-p` without `--countReadPairs` | Add `--countReadPairs` |\n| TPM column has Inf values | Zero-length features | Filter `Length > 0` before TPM computation |\n| Salmon TPM doesn't sum to 1e6 | Pre-summed across samples or filtered subset | Recompute TPM after filter; or use original abundance column |\n\n## References\n\n- Liao Y, Smyth GK, Shi W. 2014. featureCounts: an efficient general purpose program for assigning sequence reads to genomic features. *Bioinformatics* 30(7):923-930. doi:10.1093\u002Fbioinformatics\u002Fbtt656\n- Anders S, Pyl PT, Huber W. 2015. HTSeq -- a Python framework to work with high-throughput sequencing data. *Bioinformatics* 31(2):166-169. doi:10.1093\u002Fbioinformatics\u002Fbtu638\n- Dobin A et al. 2013. STAR: ultrafast universal RNA-seq aligner. *Bioinformatics* 29(1):15-21. doi:10.1093\u002Fbioinformatics\u002Fbts635\n- Patro R, Duggal G, Love MI, Irizarry RA, Kingsford C. 2017. Salmon provides fast and bias-aware quantification of transcript expression. *Nat Methods* 14(4):417-419. doi:10.1038\u002Fnmeth.4197\n- Srivastava A, Malik L, Sarkar H, Zakeri M, Almodaresi F, Soneson C, Love MI, Kingsford C, Patro R. 2020. Alignment and mapping methodology influence transcript abundance estimation. *Genome Biol* 21:239. doi:10.1186\u002Fs13059-020-02151-8\n- Bray NL, Pimentel H, Melsted P, Pachter L. 2016. Near-optimal probabilistic RNA-seq quantification. *Nat Biotechnol* 34(5):525-527. doi:10.1038\u002Fnbt.3519\n- Pimentel H, Bray NL, Puente S, Melsted P, Pachter L. 2017. Differential analysis of RNA-seq incorporating quantification uncertainty. *Nat Methods* 14(7):687-690. doi:10.1038\u002Fnmeth.4324\n- Soneson C, Love MI, Robinson MD. 2015. Differential analyses for RNA-seq: transcript-level estimates improve gene-level inferences. *F1000Res* 4:1521. doi:10.12688\u002Ff1000research.7563.2\n- Love MI, Soneson C, Hickey PF, Johnson LK, Pierce NT, Shepherd L, Morgan M, Patro R. 2020. Tximeta: Reference sequence checksums for provenance identification in RNA-seq. *PLoS Comput Biol* 16(2):e1007664. doi:10.1371\u002Fjournal.pcbi.1007664\n- Frankish A et al. 2021. GENCODE 2021. *Nucleic Acids Res* 49(D1):D916-D923. doi:10.1093\u002Fnar\u002Fgkaa1087\n\n## Related Skills\n\n- gene-id-mapping - Building tx2gene; Ensembl version stripping with PAR_Y preservation\n- normalization - Composition bias, TMM\u002FRLE, the \"biotype filter before normalize\" rationale\n- metadata-joins - Sample name reconciliation across counts and metadata\n- sparse-handling - Single-cell sparse matrix formats\n- differential-expression\u002Fdeseq2-basics - tximport -> DESeqDataSetFromTximport workflow\n- differential-expression\u002Fedger-basics - catchSalmon for transcript-level DTE\n- differential-expression\u002Fbatch-correction - Batch covariate vs subtraction\n- rna-quantification\u002Ffeaturecounts-counting - Detailed featureCounts run patterns\n- rna-quantification\u002Falignment-free-quant - Salmon\u002Fkallisto invocation details\n- rna-quantification\u002Ftximport-workflow - Detailed tximport workflow\n- read-alignment\u002Fstar-alignment - STAR upstream of GeneCounts\n- read-qc\u002Fquality-reports - RIN, DV200 inputs to consider\n",{"name":5,"source":24,"license":15,"tool_type":25,"description":7,"imported_at":26,"source_path":27,"primary_tool":28,"source_repository":29},"https:\u002F\u002Fgithub.com\u002FGPTomics\u002FbioSkills","mixed","2026-09-20","expression-matrix\u002Fcounts-ingest\u002FSKILL.md","tximport","GPTomics\u002FbioSkills","main","2d1750717747097b602acf30bd09a7cbd4b4c6ea","awesome_skills\u002Fbiology\u002Fbio-expression-matrix-counts-ingest","https:\u002F\u002Fgithub.com\u002FMr-Nobody-Anonymous\u002FAll-skills","https:\u002F\u002Fgithub.com\u002FMr-Nobody-Anonymous","2026-11-09 11:45:36",{"source":8,"skillName":5,"commands":37,"prompt":47,"zipAvailable":48},[38,41,44],{"tool":39,"command":40},"npx","npx skills add Mr-Nobody-Anonymous\u002FAll-skills --skill bio-expression-matrix-counts-ingest",{"tool":42,"command":43},"bunx","bunx skills add Mr-Nobody-Anonymous\u002FAll-skills --skill bio-expression-matrix-counts-ingest",{"tool":45,"command":46},"pnpm","pnpm dlx skills add Mr-Nobody-Anonymous\u002FAll-skills --skill bio-expression-matrix-counts-ingest","安装技能，技能地址：https:\u002F\u002Fgithub.com\u002FMr-Nobody-Anonymous\u002FAll-skills\u002Fblob\u002Fmain\u002Fawesome_skills\u002Fbiology\u002Fbio-expression-matrix-counts-ingest\u002FSKILL.md",true,[50,56],{"path":51,"resourceType":52,"resourceTypeDesc":53,"sizeBytes":54,"sourceHtmlUrl":55},"examples\u002Fload_counts.py","SCRIPT","脚本",1796,"https:\u002F\u002Fgithub.com\u002FMr-Nobody-Anonymous\u002FAll-skills\u002Fblob\u002Fmain\u002Fawesome_skills\u002Fbiology\u002Fbio-expression-matrix-counts-ingest\u002Fexamples\u002Fload_counts.py",{"path":57,"resourceType":58,"resourceTypeDesc":59,"sizeBytes":60,"sourceHtmlUrl":61},"usage-guide.md","REFERENCE","参考文档",7671,"https:\u002F\u002Fgithub.com\u002FMr-Nobody-Anonymous\u002FAll-skills\u002Fblob\u002Fmain\u002Fawesome_skills\u002Fbiology\u002Fbio-expression-matrix-counts-ingest\u002Fusage-guide.md",[63,71,78,85,92,99],{"skillNo":64,"name":65,"nameZh":6,"description":66,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":67,"categories":68,"keywords":69,"publishedAt":70},"01M7FFFZPC2025JQQHPHDX154J","dei-assessment","Assess diversity, equity, and inclusion with metrics. TRIGGERS - Use when user needs help with dei-assessment related tasks.","https:\u002F\u002Fgithub.com\u002FMr-Nobody-Anonymous\u002FAll-skills\u002Fblob\u002Fmain\u002Fawesome_skills\u002Fhr-people\u002Fdei-assessment\u002FSKILL.md",[],[],"2026-11-16 10:05:34",{"skillNo":72,"name":73,"nameZh":6,"description":74,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":75,"categories":76,"keywords":77,"publishedAt":70},"01M7FFFWN8H4WA7A6Y398QKB6J","dei-action-plan","Create DEI action plans with goals and initiatives. TRIGGERS - Use when user needs help with dei-action-plan related tasks.","https:\u002F\u002Fgithub.com\u002FMr-Nobody-Anonymous\u002FAll-skills\u002Fblob\u002Fmain\u002Fawesome_skills\u002Fhr-people\u002Fdei-action-plan\u002FSKILL.md",[],[],{"skillNo":79,"name":80,"nameZh":6,"description":81,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":82,"categories":83,"keywords":84,"publishedAt":70},"01M7FFFSHTM0QZRVZ2NRJTJ0PY","culture-fit-assessment","Design culture add assessment processes. TRIGGERS - Use when user needs help with culture-fit-assessment related tasks.","https:\u002F\u002Fgithub.com\u002FMr-Nobody-Anonymous\u002FAll-skills\u002Fblob\u002Fmain\u002Fawesome_skills\u002Fhr-people\u002Fculture-fit-assessment\u002FSKILL.md",[],[],{"skillNo":86,"name":87,"nameZh":6,"description":88,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":89,"categories":90,"keywords":91,"publishedAt":70},"01M7FFFMA12RKEQA38KHJSXXPS","cowork-to-code-bridge","Use an already-installed, independently verified cowork-to-code bridge to run narrowly approved actions on the user's own macOS, Linux, or WSL2 machine through a local file queue.","https:\u002F\u002Fgithub.com\u002FMr-Nobody-Anonymous\u002FAll-skills\u002Fblob\u002Fmain\u002Fawesome_skills\u002Fhr-people\u002Fcowork-to-code-bridge\u002FSKILL.md",[],[],{"skillNo":93,"name":94,"nameZh":6,"description":95,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":96,"categories":97,"keywords":98,"publishedAt":70},"01M7FFFH1F6M8YRGW5V487GH1K","counter-offer-response","Draft counter-offer responses for retention. TRIGGERS - Use when user needs help with counter-offer-response related tasks.","https:\u002F\u002Fgithub.com\u002FMr-Nobody-Anonymous\u002FAll-skills\u002Fblob\u002Fmain\u002Fawesome_skills\u002Fhr-people\u002Fcounter-offer-response\u002FSKILL.md",[],[],{"skillNo":100,"name":101,"nameZh":6,"description":102,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":103,"categories":104,"keywords":105,"publishedAt":70},"01M7FFFDGEHV8Y7J9N40HZXW52","continuous-feedback-system","Design continuous feedback systems. TRIGGERS - Use when user needs help with continuous-feedback-system related tasks.","https:\u002F\u002Fgithub.com\u002FMr-Nobody-Anonymous\u002FAll-skills\u002Fblob\u002Fmain\u002Fawesome_skills\u002Fhr-people\u002Fcontinuous-feedback-system\u002FSKILL.md",[],[]]