Scalability of anndata x alphapepttools#

Here, we demonstrate the scalability of anndata based on an example dataset by Albrecht et al, 2025

Albrecht, V., Müller-Reif, J. B., Brennsteiner, V. & Mann, M. A Simplified Perchloric Acid Workflow With Neutralization (PCA N) for Democratizing Deep Plasma Proteomics at Population Scale. Molecular & Cellular Proteomics 24, 101071 (2025).

In this study, Albrecht et al processed more than 2000 pooled plasma samples to benchmark the reproducibility of the PCA-N workflow, as part of a larger clinical cohort study.

import alphapepttools as apt
import anndata as ad
import pandas as pd
import numpy as np

1. Data loading#

alphapepttools features a data loader to quickly obtain example datasets.

# Careful! It's a 9GB report
report_path = apt.data.get_data("albrecht_pcan", output_dir=".")

./albrecht_pcan_report.tsv already exists (9664.169023513794 MB)

2. Parsing search engine results to AnnData: Precursors#

We showcase that anndata is capable of handling this comparatively large study and that we can extract the individual feature layers from the report.

The metadata from the PSM report is retained in the .var attribute of the precursor anndata object

%%time
precursors: ad.AnnData = apt.io.read_psm_table(
    # path to a long-format PSM report such as DIA-NN's report.tsv (in newer versions report.parquet)
    report_path,
    search_engine="diann",
    # The columns of the resulting AnnData object will be precursors, i.e. peptide sequences + modifications + charge. Note that we don't have to know the search engine's actual name for the precursor-level entity, as the underlying AlphaBase standardization allows us to access the general term 'precursor' here.
    level="precursors",
    # Intensity column entries become the numeric values of the resulting AnnData object.
    intensity_column="Precursor.Normalised",
    # Feature and sample dentification columns need to be selected and are automatically parsed to standardized names by the loader.
    feature_id_column="Precursor.Id",
    sample_id_column="Run",
    # Additional columns to keep around in the AnnData vars
    var_columns=[
        "Stripped.Sequence",
        "Precursor.Charge",
        "RT",
        "RT.Start",
        "RT.Stop",
        "IM",
        "Protein.Group",
        "Protein.Ids",
        "Genes",
        "MS2.Scan",
        "CScore",
        "Q.Value",
        "Precursor.Id",
        "Global.Q.Value",
        "Global.PG.Q.Value",
        "Lib.Q.Value",
        "Lib.PG.Q.Value",
    ],
)

CPU times: user 3min 15s, sys: 26.7 s, total: 3min 42s
Wall time: 3min 42s

Reading and pivoting the original data (9GB) takes a few minutes

3. Basic preprocessing#

Here, we subset the data to a all precursors whose retention time is below 5 minutes

precursors[:, precursors.var_names[(precursors.var["RT"] < 5)]]  # noqa: PLR2004

View of AnnData object with n_obs × n_vars = 1801 × 3948
    var: 'Stripped.Sequence', 'Precursor.Charge', 'RT', 'RT.Start', 'RT.Stop', 'IM', 'Protein.Group', 'Protein.Ids', 'Genes', 'MS2.Scan', 'CScore', 'Q.Value', 'Global.Q.Value', 'Global.PG.Q.Value', 'Lib.Q.Value', 'Lib.PG.Q.Value'

4. Data writing to disk#

Setting datatypes for saving the AnnData object to disk.

precursors.obs_names = pd.Index(precursors.obs.index.to_numpy())
precursors.var_names = pd.Index(precursors.var.index.to_numpy())

# For freely chosen additional columns, the loader does not make assumptions about datatypes. In order to save the AnnData object in the efficient h5ad format, we need to adjust the datatypes accordingly
for col in [
    "Precursor.Charge",
    "RT",
    "RT.Start",
    "RT.Stop",
    "IM",
    "MS2.Scan",
    "Q.Value",
    "Global.Q.Value",
    "Global.PG.Q.Value",
    "Lib.Q.Value",
    "Lib.PG.Q.Value",
]:
    precursors.var[col] = precursors.var[col].astype(float)

for col in ["Stripped.Sequence", "Protein.Group"]:
    precursors.var[col] = precursors.var[col].astype(str)


# Datatypes to float
precursors.X = np.where(pd.isna(precursors.X), np.nan, precursors.X).astype(float)

%%time
precursors.write_h5ad("./albrecht.precursors.h5ad")

CPU times: user 71.3 ms, sys: 54.2 ms, total: 125 ms
Wall time: 124 ms

%%time
precursors = ad.read_h5ad("./albrecht.precursors.h5ad")

CPU times: user 25.1 ms, sys: 49.2 ms, total: 74.3 ms
Wall time: 73.1 ms

Reading the same data from anndata takes less than a second.

2. Parsing search engine results to AnnData: Protein Groups#

%%time
proteins: ad.AnnData = apt.io.read_psm_table(
    report_path,
    search_engine="diann",
    level="proteins",  # NOTE: The underlying AlphaBase standardization allows us to access the  general term 'protein' here, irrespective of the actual search engine and its internal designation of the protein-level entity.
)
proteins

CPU times: user 3min 13s, sys: 27.7 s, total: 3min 41s
Wall time: 3min 41s

AnnData object with n_obs × n_vars = 1801 × 2161

3. Parsing search engine results to AnnData: Genes#

%%time
genes: ad.AnnData = apt.io.read_psm_table(
    report_path,
    search_engine="diann",  # NOTE: The underlying AlphaBase standardization allows us to access the  general term 'gene' here, irrespective of the actual search engine and its internal designation of the gene-level entity.
    level="genes",
)
genes

CPU times: user 3min 49s, sys: 27.8 s, total: 4min 17s
Wall time: 4min 18s

AnnData object with n_obs × n_vars = 1801 × 2105

Scalability of anndata x alphapepttools

Contents