batch-effect-correction
Use when correcting batch effects in merged bulk expression matrices with sample-level batch metadata while preserving biological group structure and generating before-and-after QC plots. NOT for: single-cell integration, raw FASTQ processing, differential expression without batch labels, or datasets without biological groups.
- License 许可
- MIT
- 最近更新
- 2026-10-12
- Stars
- ★ 2
- Forks
- 0
01 前言 Front Matter
- name
- batch-effect-correction
- license
- MIT
- description
- Use when correcting batch effects in merged bulk expression matrices with sample-level batch metadata while preserving biological group structure and generating before-and-after QC plots. NOT for: single-cell integration, raw FASTQ processing, differential expression without batch labels, or datasets without biological groups.
- skill-author
- AIPOCH
02 资源文件 16
- 参考文档references/algorithm.md3.7 KB
- 参考文档references/cli-guide.md6.6 KB
- 参考文档references/troubleshooting.md5.2 KB
- 脚本scripts/functions.R1.2 KB
- 脚本scripts/input_functions.R4.7 KB
- 脚本scripts/main.R4.3 KB
- 脚本scripts/output_utils.R1.8 KB
- 脚本scripts/plotting.R4.3 KB
- 脚本scripts/run_analysis.R1.2 KB
- 脚本scripts/utils.R3.8 KB
- 资产文件tests/data/expression_matrix_merged.csv104.6 KB
- 资产文件tests/data/sample_info.csv2.4 KB
- 其他tests/run_tests.R894 B
- 其他tests/testthat/helper-load-scripts.R459 B
- 其他tests/testthat/setup.R247 B
- 其他tests/testthat/test-batch-correction.R6.4 KB
03 SKILL.md 正文
Batch Effect Correction
Prerequisites
Run the following before the first analysis to install all required R packages:
Rscript -e "if (!require('BiocManager', quietly=TRUE)) install.packages('BiocManager'); BiocManager::install(c('sva', 'limma')); install.packages('ggplot2', repos='https://cloud.r-project.org')"
Note:
svaandlimmaare Bioconductor packages and requireBiocManagerfor installation.ggplot2is a standard CRAN package.
The skill cannot run until these packages are installed. In new or bare R environments, always run the prerequisite step first.
When to Read External Files
| Situation | File to Read | Purpose |
|---|---|---|
| Need algorithm details | references/algorithm.md |
ComBat workflow, assumptions, and QC logic |
| Need to run analysis | scripts/main.R |
Execute: Rscript scripts/main.R --input_file ... --group_file ... |
| Encounter errors | references/troubleshooting.md |
Common errors and solutions |
| Need CLI examples | references/cli-guide.md |
Detailed CLI usage examples and baseline run record |
| Need test data | tests/data/ |
Sample input files for testing |
Usage
Rscript scripts/main.R \
--input_file ./expression_matrix.csv \
--group_file ./sample_info.csv \
--output_dir ./output/ \
--batch_column batch \
--group_column group \
--sample_column sample \
--log_transform auto \
--timeout_seconds 600 \
--seed 42
Arguments
| Short | Long | Type | Default | Description |
|---|---|---|---|---|
-i |
--input_file |
character | required | Expression matrix file (genes as rows, samples as columns) |
-g |
--group_file |
character | required | Sample metadata file (sample ID, group, and batch columns) |
-o |
--output_dir |
character | ./output/ |
Output directory |
-b |
--batch_column |
character | batch |
Batch column name in metadata |
-c |
--group_column |
character | group |
Biological group column name in metadata |
-n |
--sample_column |
character | sample |
Sample ID column name in metadata |
-l |
--log_transform |
character | auto |
Log transform mode: auto, yes, no |
-t |
--timeout_seconds |
integer | 600 |
Elapsed time limit in seconds; use 0 to disable |
-s |
--seed |
integer | 42 |
Random seed for reproducibility |
Input Format
Expression Matrix (input_file)
Genes as rows, samples as columns, CSV format with gene ID in the first column.
"","Sample01","Sample02","Sample03"
"GeneA",5.12,4.87,6.03
"GeneB",8.44,8.11,7.95
Requirements:
- Gene IDs must be unique and non-empty
- Sample column names must be unique and non-empty
- Expression values must be numeric and finite
- Extra expression-matrix sample columns not present in metadata are allowed and will be ignored with a warning
Sample Metadata (group_file)
CSV with sample ID, biological group, and batch columns.
"sample","group","batch"
"Sample01","Control","Batch1"
"Sample02","Case","Batch1"
"Sample03","Case","Batch2"
Requirements:
- Sample IDs must be unique and non-empty
- At least 2 biological groups are required
- At least 2 batches are required
- Each group and each batch must contain at least 2 samples
- Metadata may describe a subset of expression-matrix samples; the analysis will keep only metadata-matched samples and warn about ignored expression columns
Output Files
| File | Description |
|---|---|
corrected_expression_matrix.csv |
Batch-corrected expression matrix |
matched_sample_info.csv |
Standardized metadata used in the analysis |
batch_before_boxplot.pdf |
Sample distribution boxplot before correction |
batch_after_boxplot.pdf |
Sample distribution boxplot after correction |
batch_before_pca.pdf |
PCA scatter plot before correction with batch-colored points |
batch_after_pca.pdf |
PCA scatter plot after correction with batch-colored points |
batch_before_clustering.pdf |
Hierarchical clustering before correction |
batch_after_clustering.pdf |
Hierarchical clustering after correction |
session_info.txt |
R session and package version info |
Workflow
Step 1: Validate Input
- Check file existence and non-empty input files
- Validate metadata column presence
- Verify expression values are numeric and finite
- Confirm at least 2 groups, 2 batches, and at least 2 samples per group/batch
Step 2: Align and Prepare Matrix
- Reorder expression columns to match metadata sample order
- Keep only metadata-matched samples; warn if the expression matrix contains extra samples absent from metadata
- Decide whether log transformation is needed (
auto,yes, orno) - Apply
log2(x + 1)only when required
Step 3: Run Batch Correction
- Build the design matrix with biological group information
- Run
sva::ComBat()to remove batch-driven variation - Preserve modeled biological group structure during correction
Step 4: Normalize and Export Results
- Apply
limma::normalizeBetweenArrays()after ComBat - Write the corrected matrix and matched metadata
- Save before/after QC plots and session information
Methods
ComBat
Empirical Bayes batch-effect correction using sva::ComBat(). Recommended when merged bulk expression datasets contain known batch labels and at least two biological groups.
Log Transformation
Supports auto, yes, and no. The auto mode applies log2(x + 1) only when the matrix appears to be on a raw-like scale.
normalizeBetweenArrays
Post-correction normalization with limma::normalizeBetweenArrays() to reduce remaining cross-sample distribution differences.
QC Visualization
Generates paired boxplots, PCA scatter plots with conditional batch ellipses, and hierarchical clustering plots before and after correction to assess whether batch-driven structure is reduced.
Agent Response Contract
After a successful run, report:
- Sample count retained after metadata matching and any subset filtering
- Batch count and group count used in the ComBat design matrix
- Log transformation applied (auto-detected, forced yes, or skipped)
- QC assessment: describe whether before/after PCA plots show reduced batch clustering
- Artifact paths:
corrected_expression_matrix.csv,batch_after_pca.pdf,batch_after_clustering.pdf
Examples
Basic Usage
Rscript scripts/main.R \
-i expression_matrix.csv \
-g sample_info.csv \
-o ./output
With Custom Metadata Columns
Rscript scripts/main.R \
-i expression_matrix.csv \
-g metadata.csv \
-o ./output \
-n sample_id \
-c condition \
-b platform_batch
Disable Log Transform and Timeout
Rscript scripts/main.R \
-i expression_matrix.csv \
-g sample_info.csv \
-o ./output \
-l no \
-t 0 \
-s 42
Error Handling
Common Errors
| Error | Cause | Solution |
|---|---|---|
SKILL_FILE_NOT_FOUND |
Input file does not exist | Check file path |
SKILL_EMPTY_FILE |
Input file exists but contains no data | Recreate or re-export the file |
SKILL_MISSING_COLUMNS |
Metadata file is missing sample, group, or batch columns | Check header names or pass custom column names |
SKILL_SAMPLE_MISMATCH |
Metadata sample IDs do not match expression matrix columns | Verify sample names between files |
SKILL_INVALID_DATA |
Dataset fails minimum design checks (< 2 batches, < 2 groups, < 2 samples per batch/group) | Review group counts, batch counts, and ID validity |
SKILL_INVALID_TYPE |
Expression values are non-numeric or non-finite | Clean matrix values before running |
SKILL_TIMEOUT |
Run exceeded the configured time limit | Increase --timeout_seconds or set it to 0 |
SKILL_DEPENDENCY_MISSING |
Required R package is not installed | Install with: Rscript -e "BiocManager::install(c('sva','limma')); install.packages('ggplot2')" |
SKILL_RUNTIME_ERROR |
Runtime I/O or filesystem error occurred | Check read/write permissions and environment |
IF error persists, READ: references/troubleshooting.md
Troubleshooting note: In environments where packages are not yet installed, SKILL_DEPENDENCY_MISSING will fire before file-validation or --help. Install dependencies first, then re-run to expose file-related errors or access --help.
Input Validation
This skill accepts:
- A bulk RNA-seq or microarray expression matrix (CSV, genes as rows, samples as columns)
- A sample metadata file (CSV) with sample ID, biological group, and batch columns; at least 2 batches and 2 biological groups are required
If the user's request does not involve batch effect correction on merged bulk expression matrices — for example, asking to integrate single-cell RNA-seq data, process raw FASTQ files, run differential expression without batch labels, or analyze datasets with only one batch — do not proceed with the workflow. Instead respond:
"Batch Effect Correction is designed to remove batch-driven variation from merged bulk expression matrices using ComBat, while preserving biological group structure. Your request appears to be outside this scope. Please provide a multi-batch expression matrix with sample-level batch metadata, or use a more appropriate tool for single-cell integration, differential expression, or raw sequencing processing."
Testing
Test with Sample Data
# Check help (requires packages installed)
Rscript scripts/main.R --help
# Run with bundled test data
Rscript scripts/main.R \
-i tests/data/expression_matrix_merged.csv \
-g tests/data/sample_info.csv \
-o tests/output/
Validation Commands
# Check corrected matrix exists
ls -la tests/output/corrected_expression_matrix.csv
# Check matched metadata exists
ls -la tests/output/matched_sample_info.csv
# Check PCA output exists
ls -la tests/output/batch_after_pca.pdf
Implementation Checklist
- [x] CLI parsing with
optparse - [x]
set.seed()for reproducibility - [x]
requireNamespace()dependency checks - [x] Session info recording
- [x] Time-limit support through
setTimeLimit() - [x] File reading instructions in SKILL.md
- [x] Modular script structure in
scripts/ - [x] Test data provided
- [x] Error handling with
SKILL_*codes - [x] QC plots generated before and after correction
- [x] References in
references/directory
Last updated: 2026-04-27 | Version: 1.1.0
04 同仓库更多 Skills
bio-workflows-metabolomics-pipeline
Orchestrates the untargeted LC-MS metabolomics pipeline end-to-end (xcms 4.x feature extraction, QC/drift/normalization, confidence-stratified annotation, permutation-validated statistics, background-aware pathway mapping), naming what each stage decides and where it silently fails. Use when running a full LC-MS metabolomics study from raw mzML to enriched pathways and needing the honest handoffs between stages. Each stage defers to its component skill for parameters and traps; for stable-isotope flux (a separate branch, not this untargeted flow) see metabolomics/isotope-tracing.
bio-metabolomics-xcms-preprocessing
Programmatic untargeted LC-MS feature extraction in R with the modern xcms 4.x MsExperiment/XcmsExperiment API, taking raw mzML to a feature table via CentWave peak detection, retention-time alignment, peak-density correspondence, gap-filling, CAMERA redundancy collapse, and built-in QC feature filtering. Use when converting centroided LC-MS runs into a features-by-samples matrix and deciding centWave/grouping/alignment parameters. For drift correction and QC/CV filtering execution see metabolomics/normalization-qc; for metabolite identification see metabolomics/metabolite-annotation; for the MS-DIAL GUI alternative with MS2Dec deconvolution see metabolomics/msdial-preprocessing; for downstream statistics see metabolomics/statistical-analysis.
bio-metabolomics-statistical-analysis
Decision-grade statistical analysis for metabolomics intensity tables. Covers transformation and scaling (Pareto vs unit-variance as a hidden hypothesis), unsupervised structure (PCA/HCA for QC), permutation-validated PLS-DA/OPLS-DA (R2 vs Q2, double CV, VIP as heuristic), univariate testing (Welch/Mann-Whitney/ANOVA/LMM with covariate adjustment), and dependence-aware multiple testing. Use when testing which metabolites differ, building or validating a discriminant model, choosing a scaling, or correcting many correlated tests. For sample-wise normalization/drift correction see metabolomics/normalization-qc; for ML classifiers and selection-inside-CV leakage see machine-learning/biomarker-discovery and machine-learning/model-validation; for pathway interpretation see metabolomics/pathway-mapping; for design/power/multiplicity regime see experimental-design/multiple-testing.
bio-metabolomics-pathway-mapping
Maps metabolomics results to biological pathways via over-representation (ORA), metabolite-set enrichment (MSEA/QEA), mummichog/PSEA on raw m/z peaks, and network-diffusion enrichment (FELLA), with correct background-set construction and honest interpretive ceilings. Use when interpreting differential metabolites or an untargeted LC-MS feature table in pathway context, choosing ORA vs MSEA vs mummichog vs topology, or setting the reference/background set. For annotation confidence levels feeding ORA see metabolomics/metabolite-annotation; for gene-set concepts see pathway-analysis/go-enrichment and pathway-analysis/gsea; for joint gene+metabolite pathways see multi-omics-integration/mofa-integration.
bio-metabolomics-normalization-qc
Designs QC, corrects signal drift, removes batch effects, filters features, normalizes samples, and imputes missing values for untargeted LC-MS/GC-MS metabolomics, framing each step as a measurement model that can create or erase biological signal. Use when processing a peak/feature table before statistical analysis, choosing a drift-correction or sample-normalization method, deciding QC RSD vs D-ratio filtering, or handling left-censored missing values. The feature table is produced by metabolomics/xcms-preprocessing or metabolomics/msdial-preprocessing; transformation/scaling for modeling defers to metabolomics/statistical-analysis; cross-study design issues link to experimental-design/batch-design.
bio-metabolomics-msdial-preprocessing
Runs the MS-DIAL preprocessing workflow (peak picking, MS2Dec spectral deconvolution, alignment, gap-filling) and imports the alignment-result table into R or Python with honest filtering. Use when preprocessing LC-MS DDA/DIA (SWATH) or GC-MS raw data with MS-DIAL, deciding MS-DIAL vs XCMS, configuring the MSDIALCUI console run, or parsing an MS-DIAL export into a clean feature matrix. For programmatic R peak detection and the feature-table-as-artifact framing see metabolomics/xcms-preprocessing; for lipid annotation mode see metabolomics/lipidomics; for MSI-level confidence honesty see metabolomics/metabolite-annotation; for drift correction and QC see metabolomics/normalization-qc.