playbooks

home / skills / starlitnightly / omicverse / tcga-preprocessing

tcga-preprocessing skill

/.claude/skills/tcga-preprocessing

This skill guides you through loading TCGA data, initializing metadata, and exporting annotated AnnData while enabling survival analyses.

npx playbooks add skill starlitnightly/omicverse --skill tcga-preprocessing

Review the files below or copy the command above to add this skill to your agents.

Files (2)
SKILL.md
3.3 KB
---
name: tcga-bulk-data-preprocessing-with-omicverse
title: TCGA bulk data preprocessing with omicverse
description: Guide Claude through ingesting TCGA sample sheets, expression archives, and clinical carts into omicverse, initialising survival metadata, and exporting annotated AnnData files.
---

# TCGA bulk data preprocessing with omicverse

## Overview
Follow this skill to recreate the preprocessing routine from [`t_tcga.ipynb`](../../omicverse_guide/docs/Tutorials-bulk/t_tcga.ipynb). It automates loading TCGA downloads, generating raw/normalised matrices, initialising metadata, and running survival analyses through `ov.bulk.pyTCGA`.

## Instructions
1. **Gather required downloads**
   - Confirm the user has:
     - `gdc_sample_sheet.<date>.tsv` from the TCGA Sample Sheet export.
     - The decompressed `gdc_download_xxxxx` directory containing expression archives.
     - The `clinical.cart.<date>` directory with clinical XML/JSON files.
   - Mention that sample data are available under [`omicverse_guide/docs/Tutorials-bulk/data/TCGA_OV/`](../../omicverse_guide/docs/Tutorials-bulk/data/TCGA_OV/).
2. **Initialise the TCGA helper**
   - Import `omicverse as ov` (and `scanpy as sc` if plotting) then call `ov.plot_set()`.
   - Instantiate `aml_tcga = ov.bulk.pyTCGA(sample_sheet_path, download_dir, clinical_dir)`.
   - Run `aml_tcga.adata_init()` to build the AnnData object with raw counts, FPKM, and TPM layers.
3. **Persist the dataset**
   - Encourage saving the initial AnnData: `aml_tcga.adata.write_h5ad('data/TCGA_OV/ov_tcga_raw.h5ad', compression='gzip')`.
   - When reloading, reconstruct the class with the same paths and call `aml_tcga.adata_read(<path>)`.
4. **Initialise metadata and clinical information**
   - Populate sample metadata using `aml_tcga.adata_meta_init()` to convert gene IDs to symbols and attach patient info.
   - Add survival attributes via `aml_tcga.survial_init()` (note the intentional spelling in the API).
5. **Perform survival analyses**
   - Plot gene-level survival curves with `aml_tcga.survival_analysis('GENE', layer='deseq_normalize', plot=True)`.
   - To process all genes, call `aml_tcga.survial_analysis_all()`; warn that it may take time.
6. **Export results**
   - Save enriched metadata to a new AnnData file (`aml_tcga.adata.write_h5ad('.../ov_tcga_survial_all.h5ad', compression='gzip')`).
   - Suggest exporting summary tables (e.g., survival statistics) if users need to share outputs outside Python.
7. **Troubleshooting tips**
   - Ensure TCGA archives are fully extracted; missing XML/TSV files trigger parsing errors.
   - The helper expects matching case IDs between the sample sheet and expression files—direct users to re-download if IDs do not
 align.
   - Survival plots require clinical dates; if absent, instruct users to check the `clinical_cart` contents.

## Examples
- "Read my TCGA OV download, initialise metadata, and plot MYC survival curves using DESeq-normalised counts."
- "Reload a saved AnnData file, attach survival annotations, and export the updated `.h5ad`."
- "Run survival analysis for all genes and store the enriched dataset."

## References
- Tutorial notebook: [`t_tcga.ipynb`](../../omicverse_guide/docs/Tutorials-bulk/t_tcga.ipynb)
- Sample dataset: [`data/TCGA_OV/`](../../omicverse_guide/docs/Tutorials-bulk/data/TCGA_OV/)
- Quick copy/paste commands: [`reference.md`](reference.md)

Overview

This skill guides Claude through ingesting TCGA sample sheets, expression archives, and clinical carts into omicverse to produce annotated AnnData files ready for downstream analysis. It automates building raw and normalized matrices, initializing sample and survival metadata, and exporting enriched .h5ad files. The workflow mirrors a reproducible Jupyter notebook routine for TCGA bulk RNA-seq preprocessing with omicverse.

How this skill works

The skill instructs loading three inputs: the TCGA sample sheet TSV, the decompressed expression download directory, and the clinical cart directory. It shows how to instantiate ov.bulk.pyTCGA, run adata_init() to assemble raw counts, FPKM and TPM layers, initialize metadata and survival attributes (noting the API method name survial_init()), and perform gene-level survival analyses. Final steps cover saving the AnnData object and exporting summary tables for sharing.

When to use it

  • Preparing TCGA bulk RNA-seq downloads for omicverse analysis and visualization.
  • Standardizing raw and normalized expression matrices into a single AnnData file.
  • Annotating samples with clinical metadata and survival attributes for downstream modeling.
  • Running gene-level survival plots or full-cohort survival scans before downstream statistics.
  • Recreating a published preprocessing pipeline or sharing processed TCGA datasets.

Best practices

  • Ensure the sample sheet, extracted expression archives, and clinical cart are complete and match by case IDs.
  • Save the initial assembled AnnData after adata_init() to avoid reprocessing large downloads.
  • Verify clinical XML/JSON contain date fields required for survival; otherwise survival will be incomplete.
  • Use consistent file paths when reconstructing the pyTCGA helper; call adata_read() to reload saved .h5ad.
  • Be aware that survial_analysis_all() processes many genes and can be time-consuming; run on a cluster if available.

Example use cases

  • Read a TCGA OV download set, build the AnnData with raw and DESeq-normalised layers, then plot MYC survival curves.
  • Reload a previously saved ov_tcga_raw.h5ad, run metadata and survival initialization, and export ov_tcga_survial_all.h5ad.
  • Run survial_analysis_all() to compute survival statistics across all genes and save summary tables for publication.
  • Troubleshoot ID mismatches between sample_sheet and expression files and re-download specific archives when needed.

FAQ

What inputs do I need to run this workflow?

You need the TCGA sample sheet TSV, the decompressed gdc_download directory with expression archives, and the clinical.cart directory containing clinical XML/JSON files.

Why is the method named survial_init() instead of survival_init()?

The omicverse API uses the intentionally spelled survial_init() method name; call it exactly as written to initialize survival attributes.