roc-diagnostic-performance
Use when evaluating diagnostic biomarker performance from case-control expression data with logistic regression and ROC curves, exporting coefficient and AUC tables together with a ROC PDF. NOT for: survival analysis, time-to-event outcomes, multiclass classification, calibration curves, decision-curve analysis, or nomogram construction.
- License 许可
- Apache-2.0
- 最近更新
- 2026-10-12
- Stars
- ★ 2
- Forks
- 0
01 前言 Front Matter
- name
- roc-diagnostic-performance
- description
- Use when evaluating diagnostic biomarker performance from case-control expression data with logistic regression and ROC curves, exporting coefficient and AUC tables together with a ROC PDF. NOT for: survival analysis, time-to-event outcomes, multiclass classification, calibration curves, decision-curve analysis, or nomogram construction.
02 资源文件 20
- 参考文档references/algorithm.md2.8 KB
- 参考文档references/cli-guide.md2.7 KB
- 参考文档references/troubleshooting.md2.8 KB
- 脚本scripts/cli.R3.7 KB
- 脚本scripts/functions.R5.9 KB
- 脚本scripts/io.R1.3 KB
- 脚本scripts/main.R1.8 KB
- 脚本scripts/plotting.R2.4 KB
- 脚本scripts/run_analysis.R2.8 KB
- 脚本scripts/utils.R2.5 KB
- 脚本scripts/validation.R2.3 KB
- 资产文件tests/data/sample_expression_matrix.csv7.6 KB
- 资产文件tests/data/sample_group_info.csv2.2 KB
- 其他tests/expected_output/data/analysis_data.rds2.8 KB
- 其他tests/expected_output/data/roc_model.rds17.0 KB
- 参考文档tests/expected_output/session_info.txt1.8 KB
- 资产文件tests/expected_output/table/model_coefficients.csv568 B
- 资产文件tests/expected_output/table/roc_auc_summary.csv76 B
- 其他tests/run_smoke_test.R2.1 KB
- 脚本tests/run_smoke_test.sh127 B
03 SKILL.md 正文
ROC Diagnostic Performance
When to Use
Use this skill when you need to:
- evaluate one or more diagnostic marker genes in a case-control cohort;
- build a multivariable logistic regression diagnostic model from marker expression values;
- compare the ROC performance of the full model against individual markers.
Typical user requests:
- "Use these genes to build a diagnostic ROC model for case vs control samples."
- "Evaluate the AUC of FOXP3, CD45, and CD3E and plot all ROC curves together."
- "Run logistic regression on biomarker expression and export ROC results."
When Not to Use
Do not use this skill for:
- survival or prognostic analysis with time-to-event outcomes;
- multiclass classification tasks;
- calibration plots, nomograms, or decision-curve analysis;
- non-expression diagnostic inputs such as imaging, clinical scores, or mutation-only tables.
When to Read External Files
| Situation | File to Read | Purpose |
|---|---|---|
| Need algorithm details | references/algorithm.md |
Logistic regression, ROC, AUC, and modeling assumptions |
| Need to run analysis | scripts/main.R |
Execute Rscript scripts/main.R --expression_file ... --group_file ... |
| Encounter errors | references/troubleshooting.md |
Common SKILL_* errors and solutions |
| Need CLI examples | references/cli-guide.md |
Detailed command-line examples |
| Need test data | tests/data/ |
Example expression matrix and group file |
Usage
Rscript scripts/main.R \
--expression_file ./expression_matrix.csv \
--group_file ./group_info.csv \
--marker_genes FOXP3,CD45,CD3E \
--case_group Disease \
--output_dir ./output/ \
--seed 42
Arguments
| Short | Long | Type | Default | Description |
|---|---|---|---|---|
-e |
--expression_file |
character | required | Expression matrix file in CSV/TSV format |
-g |
--group_file |
character | required | Group file with sample IDs and labels |
-m |
--marker_genes |
character | required | Comma-separated marker genes |
-c |
--case_group |
character | required | Case group label in the group file |
--group_col |
character | NULL |
Optional group column name; auto-detected if omitted | |
-o |
--output_dir |
character | ./output/ |
Output directory |
--overwrite |
flag | FALSE |
Allow writing into a non-empty output directory | |
-s |
--seed |
integer | 42 |
Random seed for reproducibility |
-T |
--timeout_seconds |
integer | 0 |
Elapsed time limit in seconds; 0 disables timeout |
--plot_width |
double | 6 |
ROC plot width in inches | |
--plot_height |
double | 6 |
ROC plot height in inches | |
--font_family |
character | sans |
PDF font family | |
--line_colors |
character | #E64B35,#4DBBD5,#00A087,#3C5488,#F39B7F |
Comma-separated ROC line colors | |
--line_width |
double | 1.2 |
ROC curve line width | |
--show_diagonal |
character | true |
Show diagonal reference line: true or false |
|
--diagonal_color |
character | #7F7F7F |
Diagonal line color | |
--diagonal_lty |
integer | 2 |
Diagonal line type | |
--plot_title |
character | ROC Diagnostic Performance |
ROC plot title | |
--x_label |
character | 1 - Specificity |
X-axis label | |
--y_label |
character | Sensitivity |
Y-axis label | |
--base_cex |
double | 0.9 |
Base text-size multiplier | |
--legend_position |
character | bottomright |
Legend position | |
--legend_cex |
double | 0.8 |
Legend text size |
Input Format
Expression Matrix (expression_file)
CSV or TSV file with genes as rows and samples as columns. The first column must store unique gene identifiers.
gene,Sample1,Sample2,Sample3
FOXP3,8.4,7.1,3.8
CD45,2.1,1.9,5.4
CD3E,5.8,6.2,4.0
Requirements
- File extension must be
.csv,.tsv, or.txt. - The first column must contain non-missing, unique gene identifiers.
- Remaining columns must be sample IDs.
- Selected marker genes must have numeric finite expression values across matched samples.
Group File (group_file)
CSV or TSV file with sample IDs in the first column and at least one group-label column.
sample,group
Sample1,Disease
Sample2,Disease
Sample3,Control
Requirements
- File extension must be
.csv,.tsv, or.txt. - The first column must contain non-missing, unique sample IDs.
- At least one group column must be present.
- The
case_groupvalue must appear in the selected group column. - At least 10 matched samples, 2 case samples, and 2 control samples are required.
Output Files
| File | Description |
|---|---|
data/analysis_data.rds |
Matched sample-level analysis dataset used for model fitting |
data/roc_model.rds |
Saved logistic regression model bundle with data and selected genes |
table/model_coefficients.csv |
Logistic regression coefficients, z statistics, p-values, and odds ratios |
table/roc_auc_summary.csv |
AUC values for the full model and each marker |
plot/roc_curve.pdf |
ROC curves for the full model and individual markers |
session_info.txt |
Session information and run parameters |
model_coefficients.csv
| Column | Description |
|---|---|
term |
Model term name |
estimate |
Logistic regression coefficient |
std_error |
Standard error of the coefficient |
z_value |
Wald z statistic |
p_value |
Wald test p-value |
odds_ratio |
Exponentiated coefficient |
odds_ratio_95_ci |
Odds ratio with 95% confidence interval |
roc_auc_summary.csv
| Column | Description |
|---|---|
model |
Full model or marker name |
auc |
Area under the ROC curve |
Workflow
Step 1: Validate Input
- Check that the expression matrix and group file exist and have supported formats.
- Validate unique gene identifiers and sample IDs.
- Match samples shared by both files.
Step 2: Prepare Analysis Dataset
- Keep only the requested marker genes that exist in the expression matrix.
- Merge matched expression values with group labels.
- Convert the selected case group to binary outcome labels.
Step 3: Fit Logistic Regression
- Fit a multivariable logistic regression model using the selected markers.
- Extract coefficient estimates, standard errors, p-values, and odds ratios.
Step 4: Compute ROC Performance
- Generate the ROC curve of the full logistic model.
- Generate ROC curves for each individual marker.
- Calculate AUC values for the full model and each marker.
Step 5: Save Outputs
- Save the matched analysis dataset and model bundle as
.rdsfiles. - Save coefficient and AUC summary tables as
.csvfiles. - Save the combined ROC plot as a PDF.
Examples
Basic Usage
Rscript scripts/main.R \
-e expression_matrix.csv \
-g group_info.csv \
-m FOXP3,CD45,CD3E \
-c Disease \
-o ./output/
With Explicit Group Column and Custom Plot
Rscript scripts/main.R \
-e expression_matrix.csv \
-g group_info.csv \
-m FOXP3,CD45,CD3E \
-c Disease \
--group_col diagnosis \
--plot_width 8 \
--plot_height 6 \
--plot_title "Biomarker ROC Comparison" \
--legend_position topright \
-o ./output/
With Test Data
Rscript scripts/main.R \
-e tests/data/sample_expression_matrix.csv \
-g tests/data/sample_group_info.csv \
-m FOXP3,CD45,CD3E \
-c Disease \
-o tests/expected_output/ \
--overwrite
Error Handling
| Error | Cause | Solution |
|---|---|---|
SKILL_INVALID_PARAMETER |
Missing required argument, invalid option value, invalid matrix/group structure, invalid case label, insufficient case-control counts, or logistic fitting failure | Check argument names, input content, class balance, and model stability |
SKILL_FILE_NOT_FOUND |
Input file does not exist | Verify the file path |
SKILL_EMPTY_DATA |
Input file contains no usable rows, or no requested markers remain after filtering | Check file content, delimiter, and marker names |
SKILL_MISSING_COLUMNS |
Requested group column is absent | Verify --group_col and the group file header |
SKILL_SAMPLE_MISMATCH |
Expression matrix and group file do not share sample IDs | Verify that sample IDs match exactly between files |
SKILL_PACKAGE_NOT_FOUND |
Required R package is not installed | Install the missing CRAN package |
IF error persists, READ: references/troubleshooting.md
Testing
Smoke Test With Included Data
Rscript scripts/main.R --help
Rscript scripts/main.R \
-e tests/data/sample_expression_matrix.csv \
-g tests/data/sample_group_info.csv \
-m FOXP3,CD45,CD3E \
-c Disease \
-o tests/expected_output/ \
--overwrite
Automated Smoke Test Script
Rscript tests/run_smoke_test.R
Optional shell wrapper:
bash tests/run_smoke_test.sh
Expected Output
tests/expected_output/
|-- data/analysis_data.rds
|-- data/roc_model.rds
|-- plot/roc_curve.pdf
|-- session_info.txt
|-- table/model_coefficients.csv
`-- table/roc_auc_summary.csv
References
- Hosmer DW, Lemeshow S, Sturdivant RX (2013). Applied Logistic Regression.
- Fawcett T (2006). An Introduction to ROC Analysis. Pattern Recognition Letters.
- Robin X et al. (2011). pROC: an open-source package for R and S+ to analyze and compare ROC curves. BMC Bioinformatics.
For detailed algorithm, READ: references/algorithm.md
Implementation Checklist
- [x] CLI parsing with
optparse - [x]
set.seed()for reproducibility - [x]
requireNamespace()dependency checks - [x] Session info recording
- [x] Timeout parameter exposed as CLI option
- [x] File reading instructions in
SKILL.md - [x] Modular script structure in
scripts/ - [x] Test data provided in
tests/data/ - [x] Error handling with
SKILL_*codes - [x] References documented in
references/
Last updated: 2026-04-17 | Version: 2.1.0
04 同仓库更多 Skills
bio-workflows-metabolomics-pipeline
Orchestrates the untargeted LC-MS metabolomics pipeline end-to-end (xcms 4.x feature extraction, QC/drift/normalization, confidence-stratified annotation, permutation-validated statistics, background-aware pathway mapping), naming what each stage decides and where it silently fails. Use when running a full LC-MS metabolomics study from raw mzML to enriched pathways and needing the honest handoffs between stages. Each stage defers to its component skill for parameters and traps; for stable-isotope flux (a separate branch, not this untargeted flow) see metabolomics/isotope-tracing.
bio-metabolomics-xcms-preprocessing
Programmatic untargeted LC-MS feature extraction in R with the modern xcms 4.x MsExperiment/XcmsExperiment API, taking raw mzML to a feature table via CentWave peak detection, retention-time alignment, peak-density correspondence, gap-filling, CAMERA redundancy collapse, and built-in QC feature filtering. Use when converting centroided LC-MS runs into a features-by-samples matrix and deciding centWave/grouping/alignment parameters. For drift correction and QC/CV filtering execution see metabolomics/normalization-qc; for metabolite identification see metabolomics/metabolite-annotation; for the MS-DIAL GUI alternative with MS2Dec deconvolution see metabolomics/msdial-preprocessing; for downstream statistics see metabolomics/statistical-analysis.
bio-metabolomics-statistical-analysis
Decision-grade statistical analysis for metabolomics intensity tables. Covers transformation and scaling (Pareto vs unit-variance as a hidden hypothesis), unsupervised structure (PCA/HCA for QC), permutation-validated PLS-DA/OPLS-DA (R2 vs Q2, double CV, VIP as heuristic), univariate testing (Welch/Mann-Whitney/ANOVA/LMM with covariate adjustment), and dependence-aware multiple testing. Use when testing which metabolites differ, building or validating a discriminant model, choosing a scaling, or correcting many correlated tests. For sample-wise normalization/drift correction see metabolomics/normalization-qc; for ML classifiers and selection-inside-CV leakage see machine-learning/biomarker-discovery and machine-learning/model-validation; for pathway interpretation see metabolomics/pathway-mapping; for design/power/multiplicity regime see experimental-design/multiple-testing.
bio-metabolomics-pathway-mapping
Maps metabolomics results to biological pathways via over-representation (ORA), metabolite-set enrichment (MSEA/QEA), mummichog/PSEA on raw m/z peaks, and network-diffusion enrichment (FELLA), with correct background-set construction and honest interpretive ceilings. Use when interpreting differential metabolites or an untargeted LC-MS feature table in pathway context, choosing ORA vs MSEA vs mummichog vs topology, or setting the reference/background set. For annotation confidence levels feeding ORA see metabolomics/metabolite-annotation; for gene-set concepts see pathway-analysis/go-enrichment and pathway-analysis/gsea; for joint gene+metabolite pathways see multi-omics-integration/mofa-integration.
bio-metabolomics-normalization-qc
Designs QC, corrects signal drift, removes batch effects, filters features, normalizes samples, and imputes missing values for untargeted LC-MS/GC-MS metabolomics, framing each step as a measurement model that can create or erase biological signal. Use when processing a peak/feature table before statistical analysis, choosing a drift-correction or sample-normalization method, deciding QC RSD vs D-ratio filtering, or handling left-censored missing values. The feature table is produced by metabolomics/xcms-preprocessing or metabolomics/msdial-preprocessing; transformation/scaling for modeling defers to metabolomics/statistical-analysis; cross-study design issues link to experimental-design/batch-design.
bio-metabolomics-msdial-preprocessing
Runs the MS-DIAL preprocessing workflow (peak picking, MS2Dec spectral deconvolution, alignment, gap-filling) and imports the alignment-result table into R or Python with honest filtering. Use when preprocessing LC-MS DDA/DIA (SWATH) or GC-MS raw data with MS-DIAL, deciding MS-DIAL vs XCMS, configuring the MSDIALCUI console run, or parsing an MS-DIAL export into a clean feature matrix. For programmatic R peak detection and the feature-table-as-artifact framing see metabolomics/xcms-preprocessing; for lipid annotation mode see metabolomics/lipidomics; for MSI-level confidence honesty see metabolomics/metabolite-annotation; for drift correction and QC see metabolomics/normalization-qc.