[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"skill:01M75F3XAENBQFDAS3BQAXG24M":3,"skill-same:01M75F3XAENBQFDAS3BQAXG24M":49},{"skillNo":4,"name":5,"nameZh":6,"description":7,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":16,"categories":17,"keywords":18,"publishedAt":19,"linkOnly":20,"markdownContent":21,"markdownBody":22,"frontMatter":23,"refName":27,"commitSha":28,"skillDirectory":29,"forksCount":30,"repositoryHtmlUrl":31,"repositoryHomepageUrl":31,"ownerHtmlUrl":32,"ownerBlogUrl":6,"ownerDescription":33,"updatedAt":34,"install":35,"resources":48},"01M75F3XAENBQFDAS3BQAXG24M","ngs-pipeline-management",null,"Next-generation sequencing pipeline management with Snakemake and Nextflow. Use for building, running, and debugging reproducible NGS workflows for RNA-seq, WGS, ChIP-seq, ATAC-seq, and amplicon sequencing. Covers pipeline design patterns, HPC\u002Fcloud execution, containerization, and workflow optimization.","Lord1Egypt\u002Fai-skillforge","Lord1Egypt","https:\u002F\u002Favatars.githubusercontent.com\u002Fu\u002F99746359?v=4","01M7592YDNT0XK9WJ0T2MQVPDN","Mohamed Mounir Ali AL Keshita","01M7597M9CBBVAPX1CYE1KFTSM",2,"MIT","https:\u002F\u002Fgithub.com\u002FLord1Egypt\u002Fai-skillforge\u002Fblob\u002Fmain\u002Fskills\u002Fgemini\u002Fngs-pipeline-management\u002FSKILL.md",[],[],"2026-11-12 13:55:43",false,"---\nname: ngs-pipeline-management\ndescription: Next-generation sequencing pipeline management with Snakemake and Nextflow. Use for building, running, and debugging reproducible NGS workflows for RNA-seq, WGS, ChIP-seq, ATAC-seq, and amplicon sequencing. Covers pipeline design patterns, HPC\u002Fcloud execution, containerization, and workflow optimization.\nallowed-tools: Read Write Edit Bash\nlicense: MIT license\nmetadata:\n    skill-author: Lord1Egypt\n---\n\n# NGS Pipeline Management\n\n## Overview\n\nNGS pipeline management involves building reproducible, scalable workflows for processing next-generation sequencing data. This skill covers Snakemake and Nextflow for workflow definition, execution on HPC clusters and cloud platforms, containerization with Docker\u002FSingularity, and best practices for reproducible bioinformatics.\n\n## When to Use This Skill\n\n- Building RNA-seq, WGS, ChIP-seq, ATAC-seq, or amplicon pipelines\n- Running bioinformatics workflows on SLURM\u002FPBS\u002FSGE clusters\n- Scaling pipelines to AWS, GCP, or Azure\n- Containerizing workflows with Docker or Singularity\n- Debugging failed pipeline steps and log inspection\n- Managing conda environments within workflows\n- Parallelizing sample processing and step execution\n- Generating reproducible workflow reports\n\n## Quick Start\n\n### Snakemake RNA-seq Pipeline\n\n```python\n# Snakefile for bulk RNA-seq analysis\nSAMPLES = [\"sample1\", \"sample2\", \"sample3\", \"sample4\"]\nGENOME = \"GRCh38\"\n\nrule all:\n    input:\n        expand(\"results\u002Fcounts\u002F{sample}.counts.txt\", sample=SAMPLES),\n        \"results\u002Fmultiqc_report.html\",\n\nrule trim_reads:\n    input:\n        r1=\"data\u002Fraw\u002F{sample}_R1.fastq.gz\",\n        r2=\"data\u002Fraw\u002F{sample}_R2.fastq.gz\",\n    output:\n        r1=\"data\u002Ftrimmed\u002F{sample}_R1_trimmed.fastq.gz\",\n        r2=\"data\u002Ftrimmed\u002F{sample}_R2_trimmed.fastq.gz\",\n        json=\"qc\u002F{sample}_fastp.json\",\n        html=\"qc\u002F{sample}_fastp.html\",\n    threads: 8\n    shell:\n        \"\"\"\n        fastp -i {input.r1} -I {input.r2} \\\n              -o {output.r1} -O {output.r2} \\\n              -j {output.json} -h {output.html} \\\n              --thread {threads} --detect_adapter_for_pe\n        \"\"\"\n\nrule align_star:\n    input:\n        r1=\"data\u002Ftrimmed\u002F{sample}_R1_trimmed.fastq.gz\",\n        r2=\"data\u002Ftrimmed\u002F{sample}_R2_trimmed.fastq.gz\",\n        index=\"reference\u002Fstar_index\u002F\",\n    output:\n        bam=\"results\u002Fbam\u002F{sample}.Aligned.sortedByCoord.out.bam\",\n        log=\"results\u002Fbam\u002F{sample}.Log.final.out\",\n    threads: 16\n    shell:\n        \"\"\"\n        STAR --runThreadN {threads} \\\n             --genomeDir {input.index} \\\n             --readFilesIn {input.r1} {input.r2} \\\n             --readFilesCommand zcat \\\n             --outSAMtype BAM SortedByCoordinate \\\n             --outSAMattributes NH HI AS NM \\\n             --outFileNamePrefix results\u002Fbam\u002F{wildcards.sample}. \\\n             --quantMode GeneCounts\n        samtools index {output.bam}\n        \"\"\"\n\nrule feature_counts:\n    input:\n        bam=\"results\u002Fbam\u002F{sample}.Aligned.sortedByCoord.out.bam\",\n        gtf=\"reference\u002Fannotation.gtf\",\n    output:\n        counts=\"results\u002Fcounts\u002F{sample}.counts.txt\",\n    threads: 4\n    shell:\n        \"\"\"\n        featureCounts -T {threads} \\\n                      -a {input.gtf} \\\n                      -o {output.counts} \\\n                      -p -B -C \\\n                      {input.bam}\n        \"\"\"\n\nrule multiqc:\n    input:\n        expand(\"qc\u002F{sample}_fastp.json\", sample=SAMPLES),\n        expand(\"results\u002Fbam\u002F{sample}.Log.final.out\", sample=SAMPLES),\n        expand(\"results\u002Fcounts\u002F{sample}.counts.txt\", sample=SAMPLES),\n    output:\n        \"results\u002Fmultiqc_report.html\",\n    shell:\n        \"multiqc qc\u002F results\u002F -o results\u002F -n multiqc_report\"\n```\n\n### Running Snakemake\n\n```bash\n# Dry run (check workflow without executing)\nsnakemake -n --snakefile Snakefile\n\n# Local execution with 8 cores\nsnakemake --cores 8 --use-conda\n\n# SLURM cluster execution\nsnakemake --cluster \"sbatch -J {rule} -c {threads} --mem {resources.mem_mb}M -t 4:00:00\" \\\n          --jobs 50 --use-conda --latency-wait 60\n\n# With Singularity containers\nsnakemake --cores 8 --use-singularity --singularity-args \"--bind \u002Fdata:\u002Fdata\"\n\n# Generate workflow DAG\nsnakemake --dag | dot -Tpdf > workflow_dag.pdf\n```\n\n### Nextflow RNA-seq Pipeline\n\n```groovy\n\u002F\u002F main.nf — Nextflow DSL2 RNA-seq pipeline\nnextflow.enable.dsl=2\n\nparams.reads = \"data\u002Fraw\u002F*_{R1,R2}.fastq.gz\"\nparams.genome_index = \"reference\u002Fstar_index\"\nparams.gtf = \"reference\u002Fannotation.gtf\"\nparams.outdir = \"results\"\n\nprocess TRIM_READS {\n    tag \"$sample_id\"\n    publishDir \"${params.outdir}\u002Ftrimmed\", mode: \"copy\"\n    cpus 8\n\n    input:\n    tuple val(sample_id), path(reads)\n\n    output:\n    tuple val(sample_id), path(\"*_trimmed.fastq.gz\"), emit: trimmed\n    path \"*.json\", emit: json\n\n    script:\n    \"\"\"\n    fastp -i ${reads[0]} -I ${reads[1]} \\\n          -o ${sample_id}_R1_trimmed.fastq.gz \\\n          -O ${sample_id}_R2_trimmed.fastq.gz \\\n          -j ${sample_id}_fastp.json \\\n          --thread ${task.cpus} --detect_adapter_for_pe\n    \"\"\"\n}\n\nprocess ALIGN_STAR {\n    tag \"$sample_id\"\n    publishDir \"${params.outdir}\u002Fbam\", mode: \"copy\"\n    cpus 16\n    memory \"48 GB\"\n\n    input:\n    tuple val(sample_id), path(reads)\n    path index\n\n    output:\n    tuple val(sample_id), path(\"*.bam\"), emit: bam\n    path \"*.Log.final.out\", emit: log\n\n    script:\n    \"\"\"\n    STAR --runThreadN ${task.cpus} \\\n         --genomeDir $index \\\n         --readFilesIn ${reads[0]} ${reads[1]} \\\n         --readFilesCommand zcat \\\n         --outSAMtype BAM SortedByCoordinate \\\n         --outFileNamePrefix ${sample_id}.\n    samtools index ${sample_id}.Aligned.sortedByCoord.out.bam\n    \"\"\"\n}\n\nworkflow {\n    read_pairs = Channel.fromFilePairs(params.reads)\n    genome_index = file(params.genome_index)\n\n    trimmed = TRIM_READS(read_pairs)\n    ALIGN_STAR(trimmed.trimmed, genome_index)\n}\n```\n\n```bash\n# Run Nextflow pipeline\nnextflow run main.nf -profile docker --reads \"data\u002Fraw\u002F*_{R1,R2}.fastq.gz\"\n\n# Resume from checkpoint\nnextflow run main.nf -resume\n\n# AWS execution\nnextflow run main.nf -profile awsbatch \\\n    --outdir s3:\u002F\u002Fmy-bucket\u002Fresults \\\n    -work-dir s3:\u002F\u002Fmy-bucket\u002Fwork\n```\n\n### Snakemake Config and Resources\n\n```yaml\n# config.yaml\nsamples:\n  sample1:\n    r1: \"data\u002Fraw\u002Fsample1_R1.fastq.gz\"\n    r2: \"data\u002Fraw\u002Fsample1_R2.fastq.gz\"\n  sample2:\n    r1: \"data\u002Fraw\u002Fsample2_R1.fastq.gz\"\n    r2: \"data\u002Fraw\u002Fsample2_R2.fastq.gz\"\n\ngenome:\n  name: \"GRCh38\"\n  star_index: \"reference\u002Fstar_index\u002F\"\n  gtf: \"reference\u002FHomo_sapiens.GRCh38.110.gtf\"\n  fasta: \"reference\u002FGRCh38.fa\"\n\nparams:\n  fastp:\n    min_length: 20\n    quality: 20\n  star:\n    min_intron: 20\n    max_intron: 1000000\n```\n\n```python\n# Access config in Snakefile\nconfigfile: \"config.yaml\"\nSAMPLES = list(config[\"samples\"].keys())\nGTF = config[\"genome\"][\"gtf\"]\nSTAR_INDEX = config[\"genome\"][\"star_index\"]\n```\n\n### Pipeline Quality Monitoring\n\n```python\nimport subprocess\nimport pandas as pd\nimport json\nfrom pathlib import Path\n\ndef check_pipeline_status(results_dir: str) -> pd.DataFrame:\n    \"\"\"Check completion status of pipeline outputs.\"\"\"\n    results = Path(results_dir)\n    status = []\n\n    for sample_dir in sorted(results.glob(\"bam\u002F*.bam\")):\n        sample = sample_dir.stem.split(\".\")[0]\n        bam_size = sample_dir.stat().st_size \u002F 1e9  # GB\n\n        # Check STAR mapping stats\n        log_file = results \u002F f\"bam\u002F{sample}.Log.final.out\"\n        mapping_rate = None\n        if log_file.exists():\n            for line in log_file.read_text().splitlines():\n                if \"Uniquely mapped reads %\" in line:\n                    mapping_rate = float(line.split(\"|\")[1].strip().rstrip(\"%\"))\n\n        status.append({\n            \"sample\": sample,\n            \"bam_size_gb\": round(bam_size, 2),\n            \"mapping_rate_%\": mapping_rate,\n            \"status\": \"OK\" if mapping_rate and mapping_rate > 70 else \"WARN\",\n        })\n\n    return pd.DataFrame(status)\n\nstatus_df = check_pipeline_status(\"results\u002F\")\nprint(status_df.to_string(index=False))\nprint(f\"\\nAll passed: {(status_df['status'] == 'OK').all()}\")\n```\n\n### Docker\u002FSingularity for Reproducibility\n\n```dockerfile\n# Dockerfile for NGS pipeline tools\nFROM ubuntu:22.04\n\nRUN apt-get update && apt-get install -y \\\n    wget curl git samtools bcftools bedtools \\\n    && rm -rf \u002Fvar\u002Flib\u002Fapt\u002Flists\u002F*\n\n# Install STAR\nRUN wget https:\u002F\u002Fgithub.com\u002Falexdobin\u002FSTAR\u002Freleases\u002Fdownload\u002F2.7.11a\u002FSTAR_2.7.11a.zip \\\n    && unzip STAR_2.7.11a.zip && mv STAR_2.7.11a\u002FLinux_x86_64\u002FSTAR \u002Fusr\u002Flocal\u002Fbin\u002F\n\n# Install conda environments\nCOPY envs\u002Frnaseq.yaml .\nRUN conda env create -f rnaseq.yaml\n\nWORKDIR \u002Fdata\n```\n\n```bash\n# Build and use container\ndocker build -t ngs-pipeline:latest .\nsingularity pull ngs-pipeline.sif docker:\u002F\u002Fngs-pipeline:latest\n\n# Run in Snakemake\nsnakemake --use-singularity \\\n    --singularity-args \"--bind \u002Fdata:\u002Fdata,\u002Freference:\u002Freference\"\n```\n\n## Pipeline Design Best Practices\n\n| Principle | Implementation |\n|-----------|---------------|\n| Reproducibility | Pin tool versions in conda\u002FDocker |\n| Checkpointing | Use `-resume` (Nextflow) or `--rerun-incomplete` (Snakemake) |\n| Logging | Redirect stderr to log files per rule |\n| Resource management | Specify threads\u002Fmemory per rule |\n| Validation | `snakemake --lint` before production runs |\n| Portability | Use config files, avoid hardcoded paths |\n\n## Dependencies\n\n```bash\n# Snakemake\npip install snakemake pulp\n\n# Nextflow\ncurl -s https:\u002F\u002Fget.nextflow.io | bash\n\n# Common NGS tools via conda\nconda install -c bioconda star samtools featurecounts fastp multiqc\n```\n","# NGS Pipeline Management\n\n## Overview\n\nNGS pipeline management involves building reproducible, scalable workflows for processing next-generation sequencing data. This skill covers Snakemake and Nextflow for workflow definition, execution on HPC clusters and cloud platforms, containerization with Docker\u002FSingularity, and best practices for reproducible bioinformatics.\n\n## When to Use This Skill\n\n- Building RNA-seq, WGS, ChIP-seq, ATAC-seq, or amplicon pipelines\n- Running bioinformatics workflows on SLURM\u002FPBS\u002FSGE clusters\n- Scaling pipelines to AWS, GCP, or Azure\n- Containerizing workflows with Docker or Singularity\n- Debugging failed pipeline steps and log inspection\n- Managing conda environments within workflows\n- Parallelizing sample processing and step execution\n- Generating reproducible workflow reports\n\n## Quick Start\n\n### Snakemake RNA-seq Pipeline\n\n```python\n# Snakefile for bulk RNA-seq analysis\nSAMPLES = [\"sample1\", \"sample2\", \"sample3\", \"sample4\"]\nGENOME = \"GRCh38\"\n\nrule all:\n    input:\n        expand(\"results\u002Fcounts\u002F{sample}.counts.txt\", sample=SAMPLES),\n        \"results\u002Fmultiqc_report.html\",\n\nrule trim_reads:\n    input:\n        r1=\"data\u002Fraw\u002F{sample}_R1.fastq.gz\",\n        r2=\"data\u002Fraw\u002F{sample}_R2.fastq.gz\",\n    output:\n        r1=\"data\u002Ftrimmed\u002F{sample}_R1_trimmed.fastq.gz\",\n        r2=\"data\u002Ftrimmed\u002F{sample}_R2_trimmed.fastq.gz\",\n        json=\"qc\u002F{sample}_fastp.json\",\n        html=\"qc\u002F{sample}_fastp.html\",\n    threads: 8\n    shell:\n        \"\"\"\n        fastp -i {input.r1} -I {input.r2} \\\n              -o {output.r1} -O {output.r2} \\\n              -j {output.json} -h {output.html} \\\n              --thread {threads} --detect_adapter_for_pe\n        \"\"\"\n\nrule align_star:\n    input:\n        r1=\"data\u002Ftrimmed\u002F{sample}_R1_trimmed.fastq.gz\",\n        r2=\"data\u002Ftrimmed\u002F{sample}_R2_trimmed.fastq.gz\",\n        index=\"reference\u002Fstar_index\u002F\",\n    output:\n        bam=\"results\u002Fbam\u002F{sample}.Aligned.sortedByCoord.out.bam\",\n        log=\"results\u002Fbam\u002F{sample}.Log.final.out\",\n    threads: 16\n    shell:\n        \"\"\"\n        STAR --runThreadN {threads} \\\n             --genomeDir {input.index} \\\n             --readFilesIn {input.r1} {input.r2} \\\n             --readFilesCommand zcat \\\n             --outSAMtype BAM SortedByCoordinate \\\n             --outSAMattributes NH HI AS NM \\\n             --outFileNamePrefix results\u002Fbam\u002F{wildcards.sample}. \\\n             --quantMode GeneCounts\n        samtools index {output.bam}\n        \"\"\"\n\nrule feature_counts:\n    input:\n        bam=\"results\u002Fbam\u002F{sample}.Aligned.sortedByCoord.out.bam\",\n        gtf=\"reference\u002Fannotation.gtf\",\n    output:\n        counts=\"results\u002Fcounts\u002F{sample}.counts.txt\",\n    threads: 4\n    shell:\n        \"\"\"\n        featureCounts -T {threads} \\\n                      -a {input.gtf} \\\n                      -o {output.counts} \\\n                      -p -B -C \\\n                      {input.bam}\n        \"\"\"\n\nrule multiqc:\n    input:\n        expand(\"qc\u002F{sample}_fastp.json\", sample=SAMPLES),\n        expand(\"results\u002Fbam\u002F{sample}.Log.final.out\", sample=SAMPLES),\n        expand(\"results\u002Fcounts\u002F{sample}.counts.txt\", sample=SAMPLES),\n    output:\n        \"results\u002Fmultiqc_report.html\",\n    shell:\n        \"multiqc qc\u002F results\u002F -o results\u002F -n multiqc_report\"\n```\n\n### Running Snakemake\n\n```bash\n# Dry run (check workflow without executing)\nsnakemake -n --snakefile Snakefile\n\n# Local execution with 8 cores\nsnakemake --cores 8 --use-conda\n\n# SLURM cluster execution\nsnakemake --cluster \"sbatch -J {rule} -c {threads} --mem {resources.mem_mb}M -t 4:00:00\" \\\n          --jobs 50 --use-conda --latency-wait 60\n\n# With Singularity containers\nsnakemake --cores 8 --use-singularity --singularity-args \"--bind \u002Fdata:\u002Fdata\"\n\n# Generate workflow DAG\nsnakemake --dag | dot -Tpdf > workflow_dag.pdf\n```\n\n### Nextflow RNA-seq Pipeline\n\n```groovy\n\u002F\u002F main.nf — Nextflow DSL2 RNA-seq pipeline\nnextflow.enable.dsl=2\n\nparams.reads = \"data\u002Fraw\u002F*_{R1,R2}.fastq.gz\"\nparams.genome_index = \"reference\u002Fstar_index\"\nparams.gtf = \"reference\u002Fannotation.gtf\"\nparams.outdir = \"results\"\n\nprocess TRIM_READS {\n    tag \"$sample_id\"\n    publishDir \"${params.outdir}\u002Ftrimmed\", mode: \"copy\"\n    cpus 8\n\n    input:\n    tuple val(sample_id), path(reads)\n\n    output:\n    tuple val(sample_id), path(\"*_trimmed.fastq.gz\"), emit: trimmed\n    path \"*.json\", emit: json\n\n    script:\n    \"\"\"\n    fastp -i ${reads[0]} -I ${reads[1]} \\\n          -o ${sample_id}_R1_trimmed.fastq.gz \\\n          -O ${sample_id}_R2_trimmed.fastq.gz \\\n          -j ${sample_id}_fastp.json \\\n          --thread ${task.cpus} --detect_adapter_for_pe\n    \"\"\"\n}\n\nprocess ALIGN_STAR {\n    tag \"$sample_id\"\n    publishDir \"${params.outdir}\u002Fbam\", mode: \"copy\"\n    cpus 16\n    memory \"48 GB\"\n\n    input:\n    tuple val(sample_id), path(reads)\n    path index\n\n    output:\n    tuple val(sample_id), path(\"*.bam\"), emit: bam\n    path \"*.Log.final.out\", emit: log\n\n    script:\n    \"\"\"\n    STAR --runThreadN ${task.cpus} \\\n         --genomeDir $index \\\n         --readFilesIn ${reads[0]} ${reads[1]} \\\n         --readFilesCommand zcat \\\n         --outSAMtype BAM SortedByCoordinate \\\n         --outFileNamePrefix ${sample_id}.\n    samtools index ${sample_id}.Aligned.sortedByCoord.out.bam\n    \"\"\"\n}\n\nworkflow {\n    read_pairs = Channel.fromFilePairs(params.reads)\n    genome_index = file(params.genome_index)\n\n    trimmed = TRIM_READS(read_pairs)\n    ALIGN_STAR(trimmed.trimmed, genome_index)\n}\n```\n\n```bash\n# Run Nextflow pipeline\nnextflow run main.nf -profile docker --reads \"data\u002Fraw\u002F*_{R1,R2}.fastq.gz\"\n\n# Resume from checkpoint\nnextflow run main.nf -resume\n\n# AWS execution\nnextflow run main.nf -profile awsbatch \\\n    --outdir s3:\u002F\u002Fmy-bucket\u002Fresults \\\n    -work-dir s3:\u002F\u002Fmy-bucket\u002Fwork\n```\n\n### Snakemake Config and Resources\n\n```yaml\n# config.yaml\nsamples:\n  sample1:\n    r1: \"data\u002Fraw\u002Fsample1_R1.fastq.gz\"\n    r2: \"data\u002Fraw\u002Fsample1_R2.fastq.gz\"\n  sample2:\n    r1: \"data\u002Fraw\u002Fsample2_R1.fastq.gz\"\n    r2: \"data\u002Fraw\u002Fsample2_R2.fastq.gz\"\n\ngenome:\n  name: \"GRCh38\"\n  star_index: \"reference\u002Fstar_index\u002F\"\n  gtf: \"reference\u002FHomo_sapiens.GRCh38.110.gtf\"\n  fasta: \"reference\u002FGRCh38.fa\"\n\nparams:\n  fastp:\n    min_length: 20\n    quality: 20\n  star:\n    min_intron: 20\n    max_intron: 1000000\n```\n\n```python\n# Access config in Snakefile\nconfigfile: \"config.yaml\"\nSAMPLES = list(config[\"samples\"].keys())\nGTF = config[\"genome\"][\"gtf\"]\nSTAR_INDEX = config[\"genome\"][\"star_index\"]\n```\n\n### Pipeline Quality Monitoring\n\n```python\nimport subprocess\nimport pandas as pd\nimport json\nfrom pathlib import Path\n\ndef check_pipeline_status(results_dir: str) -> pd.DataFrame:\n    \"\"\"Check completion status of pipeline outputs.\"\"\"\n    results = Path(results_dir)\n    status = []\n\n    for sample_dir in sorted(results.glob(\"bam\u002F*.bam\")):\n        sample = sample_dir.stem.split(\".\")[0]\n        bam_size = sample_dir.stat().st_size \u002F 1e9  # GB\n\n        # Check STAR mapping stats\n        log_file = results \u002F f\"bam\u002F{sample}.Log.final.out\"\n        mapping_rate = None\n        if log_file.exists():\n            for line in log_file.read_text().splitlines():\n                if \"Uniquely mapped reads %\" in line:\n                    mapping_rate = float(line.split(\"|\")[1].strip().rstrip(\"%\"))\n\n        status.append({\n            \"sample\": sample,\n            \"bam_size_gb\": round(bam_size, 2),\n            \"mapping_rate_%\": mapping_rate,\n            \"status\": \"OK\" if mapping_rate and mapping_rate > 70 else \"WARN\",\n        })\n\n    return pd.DataFrame(status)\n\nstatus_df = check_pipeline_status(\"results\u002F\")\nprint(status_df.to_string(index=False))\nprint(f\"\\nAll passed: {(status_df['status'] == 'OK').all()}\")\n```\n\n### Docker\u002FSingularity for Reproducibility\n\n```dockerfile\n# Dockerfile for NGS pipeline tools\nFROM ubuntu:22.04\n\nRUN apt-get update && apt-get install -y \\\n    wget curl git samtools bcftools bedtools \\\n    && rm -rf \u002Fvar\u002Flib\u002Fapt\u002Flists\u002F*\n\n# Install STAR\nRUN wget https:\u002F\u002Fgithub.com\u002Falexdobin\u002FSTAR\u002Freleases\u002Fdownload\u002F2.7.11a\u002FSTAR_2.7.11a.zip \\\n    && unzip STAR_2.7.11a.zip && mv STAR_2.7.11a\u002FLinux_x86_64\u002FSTAR \u002Fusr\u002Flocal\u002Fbin\u002F\n\n# Install conda environments\nCOPY envs\u002Frnaseq.yaml .\nRUN conda env create -f rnaseq.yaml\n\nWORKDIR \u002Fdata\n```\n\n```bash\n# Build and use container\ndocker build -t ngs-pipeline:latest .\nsingularity pull ngs-pipeline.sif docker:\u002F\u002Fngs-pipeline:latest\n\n# Run in Snakemake\nsnakemake --use-singularity \\\n    --singularity-args \"--bind \u002Fdata:\u002Fdata,\u002Freference:\u002Freference\"\n```\n\n## Pipeline Design Best Practices\n\n| Principle | Implementation |\n|-----------|---------------|\n| Reproducibility | Pin tool versions in conda\u002FDocker |\n| Checkpointing | Use `-resume` (Nextflow) or `--rerun-incomplete` (Snakemake) |\n| Logging | Redirect stderr to log files per rule |\n| Resource management | Specify threads\u002Fmemory per rule |\n| Validation | `snakemake --lint` before production runs |\n| Portability | Use config files, avoid hardcoded paths |\n\n## Dependencies\n\n```bash\n# Snakemake\npip install snakemake pulp\n\n# Nextflow\ncurl -s https:\u002F\u002Fget.nextflow.io | bash\n\n# Common NGS tools via conda\nconda install -c bioconda star samtools featurecounts fastp multiqc\n```\n",{"name":5,"license":24,"metadata":25,"description":7,"allowed-tools":26},"MIT license",{"skill-author":9},"Read Write Edit Bash","main","3f88e1b76af67701a6405069a542e21a997d2868","skills\u002Fgemini\u002Fngs-pipeline-management",0,"https:\u002F\u002Fgithub.com\u002FLord1Egypt\u002Fai-skillforge","https:\u002F\u002Fgithub.com\u002FLord1Egypt","Unbreakable. Unstoppable. Unmatched⚡.\r\n Speaker of machine tongues 🤖  | World-builder of decentralized dreams 🌐 | Defender of the digital realm🛡️.","2026-06-05 10:16:24",{"source":8,"skillName":5,"commands":36,"prompt":46,"zipAvailable":47},[37,40,43],{"tool":38,"command":39},"npx","npx skills add Lord1Egypt\u002Fai-skillforge --skill ngs-pipeline-management",{"tool":41,"command":42},"bunx","bunx skills add Lord1Egypt\u002Fai-skillforge --skill ngs-pipeline-management",{"tool":44,"command":45},"pnpm","pnpm dlx skills add Lord1Egypt\u002Fai-skillforge --skill ngs-pipeline-management","安装技能，技能地址：https:\u002F\u002Fgithub.com\u002FLord1Egypt\u002Fai-skillforge\u002Fblob\u002Fmain\u002Fskills\u002Fgemini\u002Fngs-pipeline-management\u002FSKILL.md",true,[],[50,57,64,71,78,85],{"skillNo":51,"name":52,"nameZh":6,"description":53,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":54,"categories":55,"keywords":56,"publishedAt":19},"01M75GWTHWA52GYQKS347FF6CD","zarr-python","Chunked N-D arrays for cloud storage. Compressed arrays, parallel I\u002FO, S3\u002FGCS integration, NumPy\u002FDask\u002FXarray compatible, for large-scale scientific computing pipelines.","https:\u002F\u002Fgithub.com\u002FLord1Egypt\u002Fai-skillforge\u002Fblob\u002Fmain\u002Fskills\u002Fscientific\u002Fzarr-python\u002FSKILL.md",[],[],{"skillNo":58,"name":59,"nameZh":6,"description":60,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":61,"categories":62,"keywords":63,"publishedAt":19},"01M75GW9FBP9NAFBSNVJ23RHSX","what-if-oracle","Run structured What-If scenario analysis with multi-branch possibility exploration. Use this skill when the user asks speculative questions like \"what if...\", \"what would happen if...\", \"what are the possibilities\", \"explore scenarios\", \"scenario analysis\", \"possibility space\", \"what could go wrong\", \"best case \u002F worst case\", \"risk analysis\", \"contingency planning\", \"strategic options\", or any question about uncertain futures. Also trigger when the user faces a fork-in-the-road decision, wants to stress-test an idea, or needs to think through consequences before committing.","https:\u002F\u002Fgithub.com\u002FLord1Egypt\u002Fai-skillforge\u002Fblob\u002Fmain\u002Fskills\u002Fscientific\u002Fwhat-if-oracle\u002FSKILL.md",[],[],{"skillNo":65,"name":66,"nameZh":6,"description":67,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":68,"categories":69,"keywords":70,"publishedAt":19},"01M75GW351MZZQ1V7207PDCX5D","venue-templates","Access comprehensive LaTeX templates, formatting requirements, and submission guidelines for major scientific publication venues (Nature, Science, PLOS, IEEE, ACM), academic conferences (NeurIPS, ICML, CVPR, CHI), research posters, and grant proposals (NSF, NIH, DOE, DARPA). This skill should be used when preparing manuscripts for journal submission, conference papers, research posters, or grant proposals and need venue-specific formatting requirements and templates.","https:\u002F\u002Fgithub.com\u002FLord1Egypt\u002Fai-skillforge\u002Fblob\u002Fmain\u002Fskills\u002Fscientific\u002Fvenue-templates\u002FSKILL.md",[],[],{"skillNo":72,"name":73,"nameZh":6,"description":74,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":75,"categories":76,"keywords":77,"publishedAt":19},"01M75GVYGMAZDV2NHWSV6PP6TK","vaex","Use this skill for processing and analyzing large tabular datasets (billions of rows) that exceed available RAM. Vaex excels at out-of-core DataFrame operations, lazy evaluation, fast aggregations, efficient visualization of big data, and machine learning on large datasets. Apply when users need to work with large CSV\u002FHDF5\u002FArrow\u002FParquet files, perform fast statistics on massive datasets, create visualizations of big data, or build ML pipelines that do not fit in memory.","https:\u002F\u002Fgithub.com\u002FLord1Egypt\u002Fai-skillforge\u002Fblob\u002Fmain\u002Fskills\u002Fscientific\u002Fvaex\u002FSKILL.md",[],[],{"skillNo":79,"name":80,"nameZh":6,"description":81,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":82,"categories":83,"keywords":84,"publishedAt":19},"01M75GVS0ADH3DZ3QB564EVYC9","usfiscaldata","Query the U.S. Treasury Fiscal Data API for federal financial data including national debt, government spending, revenue, interest rates, exchange rates, and savings bonds. Access 54 datasets and 182 data tables with no API key required. Use when working with U.S. federal fiscal data, national debt tracking (Debt to the Penny), Daily Treasury Statements, Monthly Treasury Statements, Treasury securities auctions, interest rates on Treasury securities, foreign exchange rates, savings bonds, or any U.S. government financial statistics.","https:\u002F\u002Fgithub.com\u002FLord1Egypt\u002Fai-skillforge\u002Fblob\u002Fmain\u002Fskills\u002Fscientific\u002Fusfiscaldata\u002FSKILL.md",[],[],{"skillNo":86,"name":87,"nameZh":6,"description":88,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":89,"categories":90,"keywords":91,"publishedAt":19},"01M75GVDTBSXRQ4Q11RVJQC56M","treatment-plans","Generate concise (3-4 page), focused medical treatment plans in LaTeX\u002FPDF format for all clinical specialties. Supports general medical treatment, rehabilitation therapy, mental health care, chronic disease management, perioperative care, and pain management. Includes SMART goal frameworks, evidence-based interventions with minimal text citations, regulatory compliance (HIPAA), and professional formatting. Prioritizes brevity and clinical actionability.","https:\u002F\u002Fgithub.com\u002FLord1Egypt\u002Fai-skillforge\u002Fblob\u002Fmain\u002Fskills\u002Fscientific\u002Ftreatment-plans\u002FSKILL.md",[],[]]