Benchmarking Single-Cell Annotation Methods for T Cell Subsets: A Comprehensive Guide for Immunologists and Bioinformaticians

Abstract

This comprehensive review examines the current landscape of computational methods for annotating T cell subsets in single-cell RNA sequencing data. As T cells exhibit remarkable heterogeneity with continuously varying states rather than discrete subsets, accurate annotation remains challenging yet crucial for understanding immune responses in cancer, autoimmunity, and infectious diseases. We synthesize findings from recent benchmarking studies and methodological advances, covering foundational concepts, practical implementation of tools like TCAT, STCAT, SingleR, and Azimuth, troubleshooting common challenges, and validation strategies. The article provides researchers and drug development professionals with evidence-based guidance for selecting, implementing, and validating annotation approaches across diverse experimental contexts, with particular emphasis on emerging methods incorporating gene expression programs, multimodal data, and foundation models.

Understanding T Cell Heterogeneity and Annotation Challenges in Single-Cell Genomics

T cells are a cornerstone of the adaptive immune system, traditionally categorized into discrete, mutually exclusive subsets such as CD4+ helper T cells, CD8+ cytotoxic T cells, and various helper lineages (Th1, Th2, Th17) based on their surface markers, transcription factors, and cytokine profiles [1] [2]. This classical model has provided a valuable framework for understanding immune function in host defense, autoimmune diseases, and cancer [1]. However, the advent of high-resolution single-cell technologies has revealed a far more complex picture. Emerging evidence now conflicts with this canonical model, indicating that T cell states vary continuously, combine additively within a single cell, and exhibit significant plasticity in response to stimuli [3] [4]. This continuum of states challenges the traditional discrete classification and necessitates new analytical frameworks for characterizing T cell biology, with profound implications for both basic research and therapeutic development [3].

This guide objectively compares the predominant methodologies used to define T cell subsets and states, benchmarking their performance, underlying protocols, and applicability in research and drug development. We focus specifically on the contrast between traditional discrete clustering and emerging component-based models, providing the experimental data and quantitative comparisons essential for researchers to select appropriate tools for their work.

Methodological Comparison: Discrete Clustering vs. Component-Based Models

The analysis of T cell identity has been revolutionized by single-cell RNA sequencing (scRNA-seq). The predominant analytical approach has been clustering, which groups cells based on transcriptional similarity. However, newer component-based models like nonnegative matrix factorization (NMF) model a cell's transcriptome as an additive mixture of gene expression programs (GEPs) [3] [4]. The table below provides a direct comparison of these two methodologies.

Table 1: Benchmarking Discrete Clustering vs. Component-Based Models for T Cell Analysis

Feature	Discrete Clustering (e.g., Phenograph, Seurat)	Component-Based Models (e.g., cNMF, TCAT)
Underlying Model of Cell Identity	Assumes cells belong to discrete, mutually exclusive groups [3].	Models cell identity as a continuous mixture of overlapping gene expression programs (GEPs) [3] [4].
Handling of Co-Expressed Programs	Poor; proliferating cells from different lineages often cluster together, obscuring their origins [3].	Excellent; explicitly quantifies the simultaneous activity of multiple GEPs (e.g., lineage + activation) within a single cell [3].
Representation of Continuous States	Forces continuous transitions into arbitrary discrete groups, losing information [4].	Naturally captures continuous variation and plastic transitions between states [3] [4].
Cross-Dataset Reproducibility	Low; cluster labels and boundaries are often dataset-specific [3].	High; GEPs serve as a fixed coordinate system for comparing cells across different studies [3].
Identification of Rare Populations	Challenging; rare populations may be merged into larger clusters.	Effective; can quantify rarely used GEPs even in small query datasets [3].
Key Limitations	Obscures multilayered biology and continuous variation [3].	Requires a well-defined reference catalog of GEPs for optimal performance.

Quantitative Benchmarking of the TCAT Pipeline

The T-CellAnnoTator (TCAT) pipeline is a specific instantiation of the component-based model designed for scalable and reproducible T cell analysis. It was benchmarked on a massive dataset of 1.7 million T cells from 700 individuals across 38 tissues and five disease contexts (COVID-19, cancer, rheumatoid arthritis, osteoarthritis, and healthy) [3] [4]. The following performance data provides a quantitative basis for its evaluation.

Table 2: Performance Metrics of the TCAT (starCAT) Pipeline

Metric	Performance Result	Experimental Context
Reproducibility of GEPs	9 GEPs were identified across all 7 analyzed datasets (Mean Pearson R = 0.81) [3].	Analysis of 7 independent scRNA-seq datasets [3].
Catalog Size	46 consensus GEPs (cGEPs) identified [3].	27-36 more GEPs than prior analyses [3].
GEP Concordance	49 cGEPs were reproducible across 2 or more datasets (Mean R = 0.74) [3].	Significantly higher concordance than PCA-based methods [3].
Prediction Accuracy	Pearson R > 0.7 for inferring GEP usage in query datasets [3].	Benchmarking simulations with partially overlapping reference/query GEPs [3].
Performance on Small Datasets	Maintains constant performance; accurately quantifies rare GEPs in queries as small as 100 cells [3] [4].	Simulation of query datasets of 100 to 100,000 cells [3].
Discriminative Power	Achieved 86.2% accuracy, 85.7% sensitivity, and 80.9% specificity in distinguishing RA subtypes [5].	Analysis of 50 participants using FlowSOM clustering and SVM [5].

Experimental Protocols for Single-Cell T Cell Annotation

Workflow for Traditional Clustering-Based Analysis

The following diagram outlines the standard protocol for analyzing T cells via discrete clustering, commonly used with flow cytometry or scRNA-seq data.

Protocol Details:

• Sample Collection & Processing: Peripheral blood mononuclear cells (PBMCs) or tissues are collected and processed into single-cell suspensions. For tissue, this involves manual dissociation and enzymatic digestion (e.g., using human tumor dissociation enzymes and a gentleMACS dissociator) [6].
• Cell Staining & Acquisition: Cells are stained with fluorescently-labeled antibodies targeting surface markers (CD3, CD4, CD8, CD45RO, CCR7) and intracellular proteins (FOXP3, cytokines). Data is acquired via flow cytometers or, for higher dimensionality, mass cytometers (CyTOF) or spectral flow cytometers [5] [6].
• Computational Analysis: Dimensionality reduction is performed followed by application of clustering algorithms. Cells are grouped based on transcriptional or protein marker similarity.
• Cluster Annotation & Assignment: Resulting clusters are manually annotated based on canonical marker expression (e.g., CD4+FOXP3+ for Tregs), and each cell is assigned to a single, discrete subset [5].

Workflow for Component-Based GEP Analysis (TCAT/starCAT)

The TCAT/starCAT pipeline provides an alternative, component-based workflow for T cell annotation, as illustrated below.

Protocol Details:

• Reference Catalog Construction: Multiple large-scale scRNA-seq datasets are aggregated and batch-corrected using modified algorithms (e.g., Harmony) that produce non-negative, gene-level corrected data compatible with cNMF [3] [4].
• GEP Discovery with cNMF: Consensus Nonnegative Matrix Factorization (cNMF) is applied to learn GEP "spectra" (gene weights) and "usages" (their contribution to each cell's transcriptome). The algorithm is run multiple times to ensure robustness [3].
• Consensus GEP (cGEP) Definition: GEPs from different datasets are clustered based on correlation, and a final catalog of consensus GEPs (cGEPs) is defined as the average of each cluster. The published catalog contains 46 cGEPs [3].
• Projection onto Query Data: For a new query dataset, the pre-defined cGEPs are projected onto each cell using non-negative least squares (NNLS) regression via the starCAT algorithm. This quantifies the usage of each cGEP in every cell, resulting in a continuous, multi-faceted identity profile [3] [4].

The Scientist's Toolkit: Essential Research Reagents and Solutions

The following table details key reagents and computational tools essential for implementing the described experimental protocols.

Table 3: Key Research Reagent Solutions for T Cell Subset Analysis

Item Name	Function / Application	Specific Examples / Targets
Fluorescently-Conjugated Antibodies	Cell surface and intracellular protein detection for flow cytometry.	CD3, CD4, CD8, CD45RO, CCR7, CD25, FOXP3, CD161, HLA-DR, CD38 [5] [6].
Cell Isolation & Dissociation Kits	Gentle extraction of viable immune cells from tissue for analysis.	Human tumor dissociation enzyme kits (Miltenyi Biotec); gentleMACS dissociator [6].
Viability Stains	Discrimination of live/dead cells during flow cytometry analysis.	Live/Dead Fixable Blue (Invitrogen) or similar dyes [6].
CITE-seq Antibodies	Integration of surface protein expression with transcriptomic data in scRNA-seq.	Oligo-conjugated antibodies for markers like PD-1, CD4, CD8 [3].
Clustering Algorithms	Unsupervised identification of cell populations from high-dimensional data.	FlowSOM [5], Seurat clustering, Phenograph.
Component-Based Modeling Tools	Decomposing single-cell data into additive gene expression programs.	TCAT/starCAT [3], cNMF [3] [4], SPECTRA.
Zoldonrasib	Zoldonrasib, CAS:3034802-05-3, MF:C63H88F3N11O7, MW:1168.4 g/mol	Chemical Reagent
ITMN 4077	ITMN 4077, MF:C26H40N4O8S, MW:568.7 g/mol	Chemical Reagent

Implications for Therapeutic Development

The shift from a discrete to a continuous model of T cell biology has direct consequences for immunotherapies. Engineered T cell therapies, such as CAR-T and TCR-T cells, are "living drugs" with complex pharmacokinetics and pharmacodynamics [7] [8]. Their efficacy and toxicity are heavily influenced by the composition and differentiation state of the infused T cell product [8].

• Product Potency: The therapeutic success of CAR-T cells is linked to the presence of less-differentiated phenotypes like stem cell memory T (TSCM) and central memory T (TCM) cells, which have superior expansion potential and persistence in vivo compared to terminally differentiated effector T cells [8].
• Exposure-Response & Toxicity: Pharmacokinetic analysis of CAR-T cells reveals high inter-patient variability in exposure (Cmax and AUC), which correlates with both response and toxicity (e.g., cytokine release syndrome). A narrow therapeutic index is common, where exposure levels for efficacy overlap with those for toxicity [7].
• Overcoming Exhaustion: In cancer and chronic infections, T cells can adopt an "exhausted" state, characterized by upregulation of inhibitory receptors like PD-1, TIM-3, and LAG3, and impaired effector functions [1] [3]. New analytical frameworks can more precisely define this exhaustion state, separate from other activation programs, aiding the development of next-generation therapies that reverse or prevent T cell exhaustion [3].

The field of T cell biology is undergoing a foundational paradigm shift. The classical model of discrete, static subsets is being superseded by a more nuanced understanding of continuous, plastic states that can coexist within a single cell. This transition is driven by advanced single-cell technologies and, crucially, by the development of new computational frameworks like component-based models. As this guide has benchmarked, these new methods offer significant advantages in reproducibility, resolution, and biological relevance. For researchers and drug developers, embracing this continuous model is no longer optional but essential for unraveling the complexity of immune responses and designing the next generation of precise and effective T cell immunotherapies.

Limitations of Traditional Clustering for T Cell Annotation

Single-cell RNA sequencing (scRNA-seq) has revolutionized immunology by enabling researchers to decipher the vast diversity and dynamic nature of T cells, which play critical roles in cancer, infection, and autoimmune diseases [3] [9]. A crucial step in analyzing scRNA-seq data involves annotating cell types—assigning biological identities to cells based on their gene expression profiles [9] [10]. For years, the predominant approach has relied on traditional clustering methods, which group cells based on similarity in their transcriptomes, followed by manual annotation of these clusters using known marker genes [3] [11] [12]. However, the increasing complexity of T cell biology has exposed significant limitations in this approach. This article examines these limitations within the broader context of benchmarking single-cell annotation methods, highlighting why new computational frameworks are necessary for accurate T cell subset identification in research and drug development.

Core Limitations of Traditional Clustering Methods

Traditional clustering methods, such as those implemented in popular tools like Seurat, discretize what is fundamentally a continuous biological reality [3]. The following table summarizes their primary shortcomings:

Limitation	Impact on T Cell Annotation
Forced Discretization of Continuous States [3]	Obscures the true continuum of T cell states (e.g., between TH1, TH2, TH17), leading to an oversimplified and inaccurate representation of T cell phenotypes.
Inability to Resolve Co-expressed Gene Programs [3]	A single cell can activate multiple gene programs (GEPs) simultaneously (e.g., proliferation and cytotoxicity). Clustering often assigns cells to a single, dominant cluster, masking this functional complexity.
Failure to Delineate Canonical Subsets [3] [12]	Clustering often fails to identify many well-established T cell subsets, even when surface protein data from CITE-seq is integrated, reducing its utility for detailed immunophenotyping.
Sensitivity to Technical Noise [10]	The inherent sparsity and high noise levels in scRNA-seq data can lead to unstable clusters that reflect technical artifacts rather than true biology.
Dependence on Expert Knowledge [9] [11]	Manual annotation of clusters is labor-intensive, subjective, and can lead to inconsistent results across studies, especially when marker genes are expressed in multiple cell types.

The diagram below illustrates the fundamental disconnect between the biological reality of T cell states and the output of traditional clustering methods.

Emerging Alternative Frameworks and Benchmarking Insights

Recognizing these limitations, the field has developed advanced computational strategies that move beyond simple clustering. These methods can be broadly categorized, and their performance has been evaluated in several benchmarking studies [9] [12].

Component-Based Models

Methods like consensus Nonnegative Matrix Factorization (cNMF) model a cell's transcriptome as an additive mixture of biologically interpretable gene expression programs (GEPs) [3]. This allows a single cell to be characterized by multiple functional states simultaneously, directly addressing the co-expression limitation of clustering. The recently developed T-CellAnnoTator (TCAT) pipeline and its generalized software package starCAT use this principle to provide a reproducible framework for T cell annotation [3].

Automated Cell Type Annotation Tools

These tools reduce manual effort and subjectivity. They can be classified into several strategic approaches [9]:

• Reference-Based/Label Transfer (e.g., Azimuth, SingleR): Maps query cells to a pre-annotated reference dataset.
• Gene Set-Based/Supervised Learning (e.g., CellTypist): Trains a classification model on well-annotated datasets.
• Marker Gene-Based/Semi-Supervised (e.g., Garnett, scGate): Uses hierarchical models of marker genes to classify cells.

Performance Comparison in Benchmarking Studies

Independent evaluations have compared the performance of these automated methods. One study on COVID-19 PBMC data found that cell-based methods (e.g., Azimuth, SingleR) could confidently annotate a much higher percentage of cells than cluster-based methods (e.g., SCSA, scCATCH) [12]. Another benchmark focused on 10x Xenium spatial transcriptomics data concluded that SingleR was a top-performing tool, being fast, accurate, and producing results that closely matched manual annotation [13].

A separate evaluation of ten R packages highlighted that while Seurat (which uses clustering) was effective for annotating major cell types, it had a major drawback: poor performance in predicting rare cell populations and differentiating between highly similar cell types [14].

Experimental Protocols for Benchmarking Annotation Methods

To objectively compare the performance of traditional clustering against newer annotation methods, researchers employ rigorous benchmarking protocols. The workflow below outlines a standard methodology for such an evaluation, drawing from published benchmark studies [13] [12].

Key Experimental Steps:

• Ground Truth Establishment: Benchmarking relies on a trusted reference for validation. This can be a:

  • Gold-standard reference atlas like FANTOM5 [11].
  • Paired single-nucleus or scRNA-seq dataset from the same sample, as used in Xenium platform benchmarks [13].
  • CITE-seq data where surface protein abundance provides orthogonal evidence for cell identity [3].

• Data Preprocessing: Consistent and rigorous quality control (QC) is applied to both reference and query datasets. This includes:

  • Filtering cells based on detected gene counts, total molecules, and mitochondrial gene percentage [10] [13].
  • Removing potential doublets using tools like scDblFinder [13].
  • Normalization and scaling of gene expression values [13].

• Method Application and Evaluation: Multiple annotation methods are run on the same processed query data. Performance is quantified by:

  • Accuracy: The percentage of cells correctly annotated compared to the ground truth.
  • Recall: The percentage of cells that receive a confident annotation [12].
  • Sensitivity to Rare Cells: The method's ability to correctly identify small cell populations [14].
  • Robustness: Performance consistency when challenged with down-sampled genes or increased noise [14].

The Scientist's Toolkit: Essential Research Reagents and Computational Solutions

Success in single-cell T cell research depends on a combination of wet-lab reagents and dry-lab computational tools. The following table details key resources for implementing advanced annotation workflows.

Item Name	Function / Application	Context of Use
10x Genomics Xenium [13]	Imaging-based spatial transcriptomics platform; profiles hundreds of genes at single-cell resolution.	Analyzing spatial context of T cells in tumor microenvironments or tissues.
CITE-seq [3]	Measures single-cell transcriptome and surface protein expression simultaneously.	Provides orthogonal protein-level validation for RNA-based T cell annotations (e.g., validating CD8+ T cells).
CellMarker/CancerSEA [10] [11]	Databases of curated cell-type-specific marker genes.	Used by marker-based tools (SCSA, scCATCH) and for manual validation of annotations.
starCAT/TCAT [3]	Software pipeline for reproducible annotation based on predefined gene expression programs (GEPs).	Quantifying complex T cell activation states (proliferation, cytotoxicity, exhaustion) across datasets.
SingleR [13] [14] [12]	Reference-based cell type annotation tool using correlation analysis.	Fast and accurate annotation of PBMC or tissue-derived T cells against existing atlases.
Azimuth [13] [12]	Reference-based annotation method integrated into the Seurat ecosystem.	Projecting and annotating query datasets onto a pre-built, annotated reference UMAP.
Harmony [3]	Batch effect correction algorithm.	Integrating multiple scRNA-seq datasets to learn robust, dataset-invariant GEPs for tools like TCAT.
UPCDC-30245	UPCDC-30245, CAS:1883351-01-6, MF:C28H38FN5, MW:463.6 g/mol	Chemical Reagent
Ansamitocin P-3	Ansamitocin P-3, MF:C32H43ClN2O9, MW:635.1 g/mol	Chemical Reagent

The evidence is clear: traditional clustering methods, while foundational, possess critical limitations for annotating the complex and continuous spectrum of T cell states. They force a discrete structure on continuous biology, fail to resolve co-expressed gene programs, and struggle with rare populations. Benchmarking studies consistently show that newer, specialized computational methods—including reference-based tools like SingleR and Azimuth, and component-based models like starCAT/TCAT—offer superior accuracy, robustness, and resolution [3] [13] [12].

The future of T cell annotation lies in the continued development and adoption of these sophisticated frameworks. Emerging trends include the use of large language models to improve annotation accuracy and scalability, and the integration of isoform-level transcriptomic data from long-read sequencing to achieve even higher resolution in cell type definition [15]. For researchers and drug developers, moving beyond traditional clustering is no longer an option but a necessity to fully unravel T cell heterogeneity and its role in health and disease.

Key Technological Advances Enabling High-Resolution T Cell Profiling

High-resolution profiling of T cells is revolutionizing our understanding of immune responses in cancer, autoimmune diseases, and therapeutic interventions. By moving beyond traditional, discrete classifications, new technologies are revealing a continuum of T cell states and enabling the precise engineering of next-generation therapies. This guide objectively compares the performance of key technological advances—spanning spatial transcriptomics, computational annotation, functional screening, and morphological profiling—that are setting new benchmarks in single-cell T cell research.

Spatial Transcriptomics: Mapping Cellular Neighborhoods

Spatial transcriptomics technologies have bridged a critical gap in single-cell analysis by preserving the spatial context of cells within a tissue, which is crucial for understanding mechanisms like immune surveillance and tumor evasion.

Visium HD from 10x Genomics represents a significant leap forward, offering whole-transcriptome analysis at a 2-µm resolution, which approaches single-cell scale. When benchmarked against its predecessor, Visium v2 (with 55-µm resolution), Visium HD demonstrates superior performance in mapping the tumor immune microenvironment. A study profiling human colorectal cancer (CRC) samples showed that Visium HD's increased resolution allowed for the precise identification and localization of transcriptomically distinct macrophage subpopulations and a clonally expanded T cell population within different tumor niches [16]. The technology's high spatial accuracy was quantified by localizing known glandular marker genes, with 98.3–99% of transcripts found in their expected morphological structures [16].

Table 1: Performance Comparison of Spatial Transcriptomic Technologies

Technology	Resolution	Tissue Compatibility	Key Application in T Cell Profiling	Performance Metric
Visium HD (10x Genomics)	2-µm bins (single-cell scale)	FFPE, Fresh Frozen	Mapping immune cell niches and clonally expanded T cells in colorectal cancer [16].	Spatial accuracy of 98.3-99% for expected transcript localization [16].
Xenium In Situ (10x Genomics)	Subcellular	FFPE	Independent validation of macrophage and T cell localizations identified by Visium HD [16].	High spatial accuracy for targeted gene panels.
Visium v2 (10x Genomics)	55-µm spots	FFPE, Fresh Frozen	Serves as a benchmark; lower resolution limits precise cellular mapping [16].	~5,000 capture areas per slide [16].

Experimental Protocol for Spatial Transcriptomics

The typical workflow for a Visium HD experiment on FFPE tissue sections, as described by [16], involves:

• Tissue Preparation: Sectioning FFPE tissue blocks to a specified thickness (e.g., 5 µm).
• Probe Ligation: Target probes are hybridized to the tissue's mRNA and subsequently ligated.
• Capture with CytAssist: The CytAssist instrument is used to transfer the ligated probes from the tissue onto the Visium HD slide, which contains a continuous lawn of capture oligonucleotides, thereby preserving spatial information.
• Library Construction & Sequencing: The captured probes are used to construct sequencing libraries.
• Data Analysis: The Space Ranger pipeline processes the raw data, outputting it at 2-µm, 8-µm, and 16-µm bin resolutions. Data is then typically deconvolved using a single-cell reference atlas to annotate cell types.

Workflow for Visium HD Spatial Profiling

Computational Annotation: Deciphering T Cell Continuous States

Single-cell RNA sequencing (scRNA-seq) has revealed that T cell states exist on a continuum, challenging traditional clustering-based analysis. Component-based computational models have emerged to address this complexity.

The T-CellAnnoTator (TCAT) / starCAT pipeline uses consensus nonnegative matrix factorization (cNMF) to define a fixed catalog of 46 reproducible gene expression programs (cGEPs) from 1.7 million T cells across 38 human tissues and five disease contexts [3] [4]. When benchmarked against de novo cNMF analysis, starCAT demonstrated superior performance, especially for small query datasets. In simulations, it accurately inferred the usage of GEPs overlapping with the reference (Pearson R > 0.7) and maintained consistent performance even when the query dataset contained as few as 100 cells, whereas the performance of de novo cNMF declined significantly [4].

Table 2: Comparison of Single-Cell Annotation Methods for T Cells

Method	Core Approach	Key Advantage	Performance in Benchmarking
TCAT/starCAT	Predefined catalog of 46 cGEPs using cNMF.	Models continuous, co-expressed states; enables cross-dataset comparison.	Pearson R > 0.7 for GEP usage inference; outperforms de novo cNMF on small queries [3] [4].
De novo cNMF	Discovers GEPs anew for each dataset.	Does not require a predefined reference.	Performance declines with smaller dataset size (<20,000 cells) [4].
Hard Clustering (e.g., Seurat)	Discretizes cells into distinct clusters.	Widely adopted and user-friendly.	Fails to resolve co-expressed GEPs and continuous state transitions [3].
scTissueID	Machine learning with cell quality filtering.	High accuracy in cell and tissue type identification.	Outperformed 8 other annotation pipelines and 6 ML algorithms in a comparative study [17].

Experimental Protocol for Computational Annotation with starCAT

The workflow for applying the starCAT pipeline, as detailed in [3] [4], involves two major phases:

• Reference Catalog Construction:
  • Data Collection: Aggregate multiple large-scale scRNA-seq datasets (e.g., from blood and various tissues).
  • Batch Correction: Apply modified Harmony integration to gene-level data to remove technical artifacts while preserving non-negativity.
  • GEP Discovery: Run cNMF on each batch-corrected dataset to identify gene expression programs.
  • Consensus GEP Definition: Cluster highly correlated GEPs from different datasets to create a robust, multi-dataset catalog of consensus GEPs (cGEPs).
• Query Dataset Annotation:
  • GEP Usage Inference: For a new query dataset, use non-negative least squares (NNLS) regression to score the activity (usage) of each predefined cGEP in every cell.
  • Cell State Prediction: Leverage the GEP usages to predict functional features like T cell activation and exhaustion.

Functional Profiling: From Genetics to Morphology

Understanding T cell function extends beyond transcriptomics to include genetic determinants of efficacy and real-time functional responses.

CRISPR Screening for Enhanced T Cell Therapies

The CELLFIE platform is a CRISPR screening platform designed to discover gene knockouts that enhance the fitness and efficacy of primary human CAR T cells. In a genome-wide knockout screen, CAR T cells were stimulated via their CAR (using CD19+ K562 cells) or their endogenous TCR, with readouts for proliferation, fratricide, and exhaustion markers [18]. The platform achieved high-quality data, with stronger depletion of essential genes than some prior T cell screens. Key discoveries included RHOG and FAS knockouts, which were validated as potent enhancers of CAR T cell anti-tumor activity in xenograft models, both individually and synergistically as a double knockout [18].

High-Content Imaging for Morphological Profiling

A High-Content Cell Imaging (HCI) pipeline was developed to predict clinical response to natalizumab in multiple sclerosis patients by profiling the in vitro sensitivity of T cells to the drug. The method involved [19]:

• Stimulation & Staining: PBMCs from patients were exposed to natalizumab or a control and seeded onto VCAM-1-coated plates. Cells were stained for CD4/CD8, F-actin, and phosphorylated SLP76 (pSLP76).
• Automated Imaging: Confocal imaging with a 40x objective was performed on the basal plane of adherent cells.
• Feature Extraction: Metrics like cell area, width-to-length ratio, and F-actin/pSLP76 intensity were extracted. Unsupervised clustering of these morphological features from CD8+ T cells partially discriminated non-responder patients. Furthermore, a random forest model trained on these features predicted treatment response with 92% accuracy in a discovery cohort and 88% in a validation cohort [19].

Research Reagent Solutions for T Cell Profiling

Table 3: Essential Reagents and Tools for Advanced T Cell Profiling

Research Tool	Function	Application Example
CROP-seq-CAR Vector	Co-delivers CAR and gRNA via single lentivirus.	Enables pooled CRISPR screens in primary CAR T cells [18].
MHC Class I Tetramers / Anti-CD137	Identifies antigen-specific T cells.	Used with spectral flow cytometry for metabolic profiling of virus/tumor-specific CD8+ T cells [20].
Visium HD Spatial Gene Expression Slide	Captures whole-transcriptome data with spatial context.	Mapping immune cell niches in colorectal cancer FFPE samples [16].
LEVA (Light-induced EVP adsorption)	Patterns extracellular vesicles and particles with UV light.	Studying neutrophil swarming behavior in response to patterned bacterial EVPs [21].

The benchmarking of these advanced technologies reveals a clear trajectory in T cell profiling: each excels in a specific dimension. Visium HD provides unparalleled spatial context, TCAT/starCAT offers a reproducible framework for deciphering continuous transcriptional states, CELLFIE enables the systematic functional discovery of genetic enhancers for therapy, and HCI bridges in vitro assays with clinical response predictions. Together, they provide a powerful, multi-faceted toolkit for researchers and drug developers to decode T cell biology with unprecedented resolution and rigor, paving the way for more effective and personalized immunotherapies.

In the field of immunology, particularly in the study of T cell subsets, precise cell annotation is a critical prerequisite for understanding cellular heterogeneity, activation states, and functional programs. Single-cell RNA sequencing (scRNA-seq) has revealed that T cells exist along a continuum of states rather than in clearly distinct subsets, necessitating advanced computational frameworks for accurate characterization [3]. Traditional clustering approaches often discretize this continuous variation, obscuring co-expressed gene expression programs (GEPs) and limiting our understanding of T cell biology. The computational methods developed to address these challenges fall into three major paradigms: supervised, unsupervised, and semi-supervised learning, each with distinct strengths for specific research scenarios.

This guide provides an objective comparison of these annotation approaches within the context of benchmarking studies for T cell research, offering experimental data and methodological details to help researchers and drug development professionals select appropriate tools for their specific applications.

Methodological Foundations and Definitions

Core Computational Paradigms

Supervised learning represents a machine learning approach where algorithms are trained on labeled data, meaning each input data point is paired with a corresponding output label [22]. In the context of single-cell annotation, researchers use a reference dataset with pre-annotated cell types to train a model that can then predict cell types in new, unlabeled query datasets [10]. This approach requires high-quality, well-annotated reference data but typically yields highly accurate and reproducible annotations when such references are available.

Unsupervised learning encompasses methods that identify patterns and structures in unlabeled data without pre-existing annotations or training [23]. These techniques are particularly valuable for exploratory analysis where reference datasets may be incomplete or when discovering novel cell states. Common unsupervised approaches include clustering algorithms that group cells based on similarity metrics and dimensionality reduction methods that help visualize high-dimensional single-cell data [24].

Semi-supervised learning occupies a middle ground, leveraging both labeled and unlabeled data to improve annotation accuracy, particularly when fully labeled reference datasets are limited or costly to produce [22]. This approach can enhance model generalization by using the underlying structure of unlabeled data to inform decisions about cellular identities, making it particularly useful for rare cell type identification or when working with emerging technologies where comprehensive references are not yet established.

Technical Implementation in Single-Cell Research

In practical single-cell research, these paradigms manifest as specific computational tools and pipelines. For T cell studies, the recently developed T-CellAnnoTator (TCAT) pipeline utilizes a component-based model that simultaneously quantifies predefined gene expression programs capturing activation states and cellular subsets [3]. This approach improves upon traditional clustering by modeling transcriptomes as weighted mixtures of GEPs, enabling more nuanced characterization of T cell states.

Reference-based annotation methods like SingleR, Azimuth, and scMap employ correlation-based matching between query cells and reference profiles, while data-driven methods train classification models on pre-labeled cell type datasets [10]. The emergence of deep learning approaches has further expanded the toolkit, with methods like STAMapper using heterogeneous graph neural networks to transfer cell-type labels from scRNA-seq data to spatial transcriptomics data [25].

Table 1: Core Characteristics of Major Annotation Approaches

Characteristic	Supervised Learning	Unsupervised Learning	Semi-Supervised Learning
Input Data	Labeled reference datasets	Unlabeled data only	Mix of labeled and unlabeled data
Human Intervention	Required for labeling	Limited to interpretation	Moderate for validation
Primary Use	Predicting known cell types	Discovering novel patterns	Improving learning with limited labels
Computational Complexity	Generally simpler	More complex	Variable, often moderate
Accuracy on Known Types	Higher	Lesser	Can approach supervised performance
Novel Cell Type Discovery	Limited	Excellent	Moderate with proper implementation

Benchmarking Performance in Single-Cell Research

Experimental Frameworks for Method Evaluation

Rigorous benchmarking studies have established standardized protocols for evaluating annotation methods. A comprehensive assessment of reference-based annotation tools for 10x Xenium spatial transcriptomics data utilized paired single-nucleus RNA sequencing (snRNA-seq) data as a reference to minimize variability between reference and query datasets [13]. This approach enabled direct comparison of method performance against manual annotation based on established marker genes.

In large-scale T cell studies, researchers have analyzed over 1.7 million T cells from 700 individuals across 38 tissues and five disease contexts to identify 46 reproducible gene expression programs reflecting core T cell functions [3]. The reproducibility of these GEPs was quantified by clustering them across datasets, with consensus GEPs (cGEPs) defined as averages of each cluster. This massive dataset provides a robust foundation for benchmarking annotation methods.

Performance metrics commonly used in these evaluations include:

• Accuracy: The proportion of correctly annotated cells overall
• Macro F1 Score: The unweighted mean of per-class F1 scores, important for rare cell types
• Weighted F1 Score: The mean of per-class F1 scores weighted by support
• Cross-dataset Concordance: Measurement of how well GEPs reproduce across different studies

Comparative Performance Data

Recent benchmarking studies provide quantitative comparisons of annotation methods. In spatial transcriptomics annotation, STAMapper demonstrated significantly higher accuracy compared to competing methods (scANVI, RCTD, and Tangram) across 81 single-cell spatial transcriptomics datasets [25]. The method maintained superior performance even under conditions of poor sequencing quality, particularly for datasets with fewer than 200 genes where it achieved median accuracy of 51.6% compared to 34.4% for the second-best method at a down-sampling rate of 0.2.

For 10x Xenium data, a systematic evaluation of five reference-based methods (SingleR, Azimuth, RCTD, scPred, and scmapCell) against manual annotation found that SingleR performed best, being fast, accurate, and easy to use, with results closely matching manual annotation [13].

Table 2: Benchmarking Performance of Annotation Methods Across Technologies

Method	Type	Reported Accuracy	Best Use Context	Limitations
TCAT	Unsupervised	Identified 46 reproducible GEPs	Large-scale T cell analysis across tissues	Requires large datasets for optimal performance
STAMapper	Supervised	Significantly outperformed competitors (p = 1.3e-27 to 2.2e-14)	Spatial transcriptomics with limited genes	Complex implementation for non-specialists
SingleR	Supervised	Closest match to manual annotation	10x Xenium data with snRNA-seq reference	Performance depends on reference quality
cNMF/starCAT	Unsupervised	High cross-dataset reproducibility (mean R = 0.74-0.81)	Identifying co-expressed gene programs	May miss rare populations in small datasets
Semi-supervised	Semi-supervised	Improved accuracy with limited labels	Medical imaging with few expert annotations	Less accurate than supervised with full labels

Experimental Protocols for Annotation Benchmarking

Reference-Based Annotation Workflow

A standardized protocol for benchmarking reference-based annotation methods involves several key steps [13]:

• Reference Preparation: High-quality single-cell or single-nucleus RNA sequencing data is processed using standard pipelines (e.g., Seurat). Quality control includes removing cells without validated annotations, normalizing data, selecting highly variable genes, and performing dimensionality reduction.

• Query Data Processing: Spatial transcriptomics data undergoes similar quality control, with adjustments for technology-specific limitations (e.g., using all genes for Xenium data due to small panel size).

• Method Implementation: Each annotation tool is applied using the prepared reference:

  • SingleR: Uses correlation between reference and query datasets
  • Azimuth: Requires special reference preparation with SCTransform normalization
  • RCTD: Employs a regression framework accounting for platform effects
  • scPred: Trains a model on the reference before predicting query cell types
  • scmapCell: Creates a cell index before projection and cluster assignment

• Performance Validation: Predictions are compared against manual annotations based on marker genes, with metrics calculated for accuracy and cell type-specific performance.

Unsupervised GEP Discovery Pipeline

For unsupervised discovery of gene expression programs in T cells, the TCAT pipeline employs these key experimental steps [3]:

• Data Integration and Batch Correction: Multiple scRNA-seq datasets are integrated using modified Harmony algorithms to correct batch effects while maintaining non-negative values compatible with nonnegative matrix factorization.

• Consensus Nonnegative Matrix Factorization (cNMF):

  • Repeated NMF runs are performed to mitigate randomness
  • Outputs are combined into robust estimates of GEP spectra (gene weights)
  • Per-cell activities ("usages") are calculated reflecting relative GEP contributions

• Cross-Dataset GEP Alignment:

  • GEPs from different datasets are clustered based on similarity
  • Consensus GEPs (cGEPs) are defined as cluster averages
  • Reproducibility is quantified by measuring how many GEPs cluster across datasets

• Biological Validation:

  • cGEPs are annotated by examining top-weighted genes
  • Association with surface marker-based gating is assessed via multivariate logistic regression
  • Gene-set enrichment analysis provides functional annotations

Single-Cell Annotation Workflow Selection

Essential Research Reagents and Computational Tools

Table 3: Key Research Reagents and Computational Solutions for Single-Cell Annotation

Resource	Type	Function in Annotation	Applicable Context
10x Genomics Xenium	Platform	Provides imaging-based spatial transcriptomics data	Cellular-level spatial gene expression profiling
CITE-seq Data	Data Type	Enhances GEP interpretability with surface protein measurements	Multimodal cell state characterization
CellMarker/PanglaoDB	Database	Provides known marker genes for cell type identification	Manual annotation and method validation
Seurat	Software Toolkit	Standard pipeline for scRNA-seq data analysis	Data preprocessing, normalization, and clustering
Harmony	Algorithm	Corrects batch effects in integrated datasets	Multi-dataset analysis and reference construction
TCAT/starCAT	Pipeline	Quantifies predefined gene expression programs	T cell subset and activation state characterization
STAMapper	Tool	Transfers labels from scRNA-seq to spatial data	Single-cell spatial transcriptomics annotation
SingleR	Method	Fast correlation-based cell type prediction	Reference-based annotation with scRNA-seq data

The benchmarking data presented in this guide demonstrates that each major category of annotation approaches—supervised, unsupervised, and semi-supervised—has distinct advantages depending on the research context. For well-established cell types with available reference data, supervised methods like SingleR and STAMapper provide high accuracy and computational efficiency. For discovering novel cell states or when reference data is limited, unsupervised approaches like cNMF offer superior ability to identify biologically meaningful patterns without prior annotations.

In T cell research specifically, the continuum of activation states and the complex interplay of gene expression programs necessitates methods that can capture this complexity without imposing artificial discretization. The development of pipelines like TCAT and starCAT represents significant advances in this direction, enabling reproducible annotation across datasets and biological contexts.

As single-cell technologies continue to evolve, particularly in spatial transcriptomics and multi-omics integration, the field will likely see increased adoption of semi-supervised approaches that leverage limited expert knowledge while exploring the full complexity of cellular heterogeneity. Researchers should select annotation methods based on their specific biological questions, data characteristics, and the availability of validated reference data, using the benchmarking frameworks outlined here to validate their choices.

The Critical Role of Reference Databases and Atlas Projects

In the field of immunology, particularly in the study of T cells, single-cell RNA sequencing (scRNA-seq) has revealed a previously underestimated continuum of cellular states, moving beyond the traditional, discrete subsets like Th1, Th2, and Th17 [3] [4]. This complexity presents a major challenge: how can researchers consistently identify and compare T cell states across different studies, laboratories, and disease conditions? The answer lies in standardized reference databases and cell atlas projects. These resources provide a stable biological coordinate system, enabling reproducible annotation of scRNA-seq data and accelerating discoveries in cancer, autoimmunity, and infectious diseases [26] [27]. This guide benchmarks the performance of leading atlas-based annotation tools, providing experimental data and protocols to guide researchers in selecting the optimal method for their studies.

Comparative Performance of T Cell Annotation Tools

Table 1: Benchmarking of Major T Cell Reference Atlas and Annotation Tools

Tool Name	Core Methodology	Reference Scale & Context	Demonstrated Annotation Accuracy	Key Strengwarts	Identified Cell States/Programs
TCAT/starCAT [3] [4]	Consensus Non-negative Matrix Factorization (cNMF) & projection	1.7 million T cells; 700 individuals; 38 tissues; 5 diseases (COVID-19, cancer, RA) [3]	>0.7 Pearson R for GEP usage in simulations; outperforms de novo cNMF in small queries [4]	Quantifies co-expressed gene programs; captures continuous states; fast projection with NNLS [3]	46 reproducible consensus Gene Expression Programs (cGEPs) for subsets, exhaustion, cytotoxicity [3]
ProjecTILs [27] [28]	Reference atlas projection via batch-corrected integration	~16,800 murine T cells; 25 samples; melanoma & colon cancer models [27]	Accurate embedding and label transfer; characterizes states deviating from reference [27]	Preserves reference structure; allows discovery of "deviant" states; interactive atlases [27] [28]	9 broad functional clusters (e.g., Tpex, Tex, Th1-like, Tfh, Treg) [27]
STCAT [29]	Hierarchical models & marker correction	1.35 million human T cells; 35 conditions; 16 tissues [29]	28% higher accuracy vs. existing tools on 6 independent datasets [29]	Automated hierarchical annotation; integrated TCellAtlas database for browsing & analysis [29]	33 subtypes classified into 68 categories by subtype and state [29]

Experimental Protocols for Benchmarking Atlas Tools

The performance data presented in Table 1 are derived from rigorous experimental and computational benchmarks. Below are summaries of the key methodologies used to generate this validation data.

Simulation-Based Benchmarking of starCAT

• Objective: To evaluate the accuracy of transferring gene expression program (GEP) annotations from a large reference to a smaller query dataset, especially when their cellular compositions do not perfectly overlap [4].
• Protocol:
  • Data Simulation: Two large reference datasets (100,000 cells each) and a smaller query dataset (20,000 cells) were simulated. Each cell was programmed to express a combination of "subset-defining" and "non-subset" GEPs.
  • Introduction of Variance: The references were designed to either contain extra GEPs not present in the query or lack certain GEPs that were in the query. Only 90% of genes were shared between reference and query to mimic real-world conditions [4].
  • Analysis & Metric: GEPs were learned from the references using cNMF. starCAT was then used to project these reference GEPs onto the query dataset. Accuracy was quantified by the Pearson correlation (R) between the projected GEP usage values and the known ground-truth values from the simulation [4].
• Outcome: starCAT robustly inferred the usage of overlapping GEPs (R > 0.7) and correctly assigned low usage to non-overlapping GEPs. Notably, it outperformed running cNMF de novo on the smaller query dataset, demonstrating the advantage of leveraging a large, fixed reference [4].

Cross-Study Integration and Validation of ProjecTILs

• Objective: To construct a robust, batch-effect-corrected reference atlas of tumor-infiltrating T cell (TIL) states from multiple independent studies and validate its utility for projecting new data [27].
• Protocol:
  • Atlas Construction: scRNA-seq data from 21 murine tumors and 4 tumor-draining lymph nodes were collected from public sources and new experiments. The STACAS algorithm was used to integrate these datasets, correcting for technical batch effects while preserving biological variation [27].
  • Annotation: Unsupervised clustering and gene enrichment analysis, supported by a supervised classifier (TILPRED), were used to annotate functional clusters in the integrated map (e.g., Tpex, Tex, Tfh) [27].
  • Projection Validation: New query datasets were projected into the reference space. The method's accuracy was assessed by its ability to correctly place cells from known subtypes into their corresponding annotated areas of the reference atlas and to identify novel states not originally in the reference [27].
• Outcome: The resulting atlas provided a unified system of coordinates that summarized TIL diversity into distinct, biologically validated functional clusters. ProjecTILs successfully mapped new data into this stable reference, enabling direct cross-condition and cross-study comparisons [27].

Independent Dataset Validation of STCAT

• Objective: To assess the real-world annotation accuracy of STCAT against other tools on independently generated scRNA-seq datasets [29].
• Protocol:
  • Reference Building: A large human T cell reference (TCellAtlas) was constructed from over 1.3 million cells across diverse conditions and tissues.
  • Tool Comparison: STCAT and other existing annotation tools were applied to six independent validation datasets, including both cancer and healthy samples.
  • Accuracy Measurement: The tool's predictions were compared against manually curated, expert-based annotations considered to be the "ground truth." Accuracy was calculated as the percentage of correctly labeled cells across all major T cell subtypes [29].
• Outcome: STCAT achieved a 28% higher accuracy in annotating T cell subtypes compared to other tools on these independent datasets, validating its hierarchical model and marker correction approach [29].

Workflow Diagram: From Single-Cell Data to Annotated Atlas

The following diagram illustrates the general workflow for constructing a reference atlas and using it to annotate query datasets, integrating key steps from tools like TCAT and ProjecTILs.

Table 2: Key Databases and Computational Tools for T Cell Atlas Research

Resource Name	Type	Primary Function	Key Application in Research
TCellAtlas [29]	Transcriptomics Reference Database	Repository for T cell scRNA-seq profiles and annotations.	Serves as a standardized reference for automated annotation of human T cell subtypes and states using STCAT.
TCRdb [30]	TCR Sequence Database	Aggregates TCR sequences with metadata (antigen specificity, disease context).	Analyzing T cell repertoire diversity, clonal expansion, and antigen-specific responses across conditions.
VDJdb [30]	TCR Sequence Database	Curated repository of TCR sequences with known antigen specificity.	Identifying and studying antigen-specific T cells, crucial for vaccine and cancer immunotherapy development.
scGate [9]	Computational Tool	Marker-based automated cell purification using a hierarchical gating strategy.	Isolating pure populations of specific cell types from heterogeneous scRNA-seq data prior to deeper analysis.
CellTypist [9]	Computational Tool	Automated cell type annotation using a model trained on reference atlases.	Rapid, standardized annotation of immune cells in large-scale scRNA-seq datasets.

The benchmarking data clearly demonstrates that atlas-based annotation methods—TCAT/starCAT, ProjecTILs, and STCAT—provide a superior framework for the reproducible interpretation of T cell states compared to unsupervised analysis of individual datasets. Their ability to quantify complex, co-expressed gene programs [3], project new data into a stable reference space [27], and deliver higher annotation accuracy [29] makes them indispensable. The consistent finding of enriched Tregs in tumors and specific T helper states in disease stages across multiple studies and tools underscores the power of this approach. As these reference atlases and databases continue to expand in scale and diversity, they will form the foundational infrastructure for a new era of precision immunology, ultimately accelerating the development of novel diagnostics and immunotherapies.

Practical Implementation of T Cell Annotation Tools and Pipelines

Component-based models have emerged as powerful computational frameworks for analyzing single-cell RNA sequencing (scRNA-seq) data, addressing critical limitations of traditional clustering approaches. Unlike hard clustering methods that force cells into discrete, mutually exclusive groups, component-based models represent each cell's transcriptome as a weighted combination of gene expression programs (GEPs) [3]. This approach is particularly valuable for analyzing complex biological systems where cells exist along continuous phenotypic spectra and simultaneously execute multiple biological programs, as commonly observed in T cell biology [4].

These models mathematically decompose the high-dimensional gene expression matrix into two lower-dimensional matrices: a program matrix containing the defining genes for each GEP, and a usage matrix quantifying the activity of each program in every cell [31]. This formulation enables researchers to dissect the complex interplay between cell identity programs (defining cell types) and cell activity programs (reflecting dynamic processes like activation, exhaustion, or metabolic states) [31]. Within this landscape, methods based on non-negative matrix factorization (NMF) have gained prominence due to their ability to produce interpretable, parts-based representations that align well with biological intuition [32] [33].

This review comprehensively benchmarks two prominent approaches in this domain: traditional NMF-based methods and the recently developed TCAT/starCAT framework. We evaluate their performance in identifying and annotating T cell subsets, activation states, and functions, with particular emphasis on experimental validation, reproducibility, and applicability to translational research.

Methodological Frameworks and Algorithms

Non-Negative Matrix Factorization (NMF) Foundations

Non-negative matrix factorization is a family of algorithms that decompose a non-negative data matrix A (n genes × m cells) into two non-negative factor matrices: W (n genes × k programs) and H (k programs × m cells), such that A ≈ W × H [32]. The non-negativity constraint fosters interpretability as biological concepts like gene expression cannot be negative. Each column of W represents a metagene or GEP, defined by its constituent genes and their weights, while each column of H specifies how active these programs are in a given cell [32] [33].

Several NMF variants have been developed for biological applications. Consensus NMF (cNMF) addresses the instability of standard NMF by running multiple iterations with different initializations, filtering outlier components, and clustering results to produce robust consensus programs [31]. Nonnegative spatial factorization (NSF) extends NMF to spatial transcriptomics by incorporating Gaussian process priors that capture spatial correlation structure [33]. The NSF hybrid (NSFH) model further partitions variability into both spatial and nonspatial components, enabling quantification of spatial importance for genes and observations [33].

The TCAT/starCAT Framework

T-CellAnnoTator (TCAT) and its generalized counterpart starCAT represent a specialized pipeline for GEP-based annotation that builds upon but meaningfully extends traditional NMF approaches [3] [34]. The framework operates in two primary phases: GEP discovery and GEP annotation.

The discovery phase applies consensus NMF to multiple reference datasets while incorporating specific enhancements to improve cross-dataset reproducibility [3]. To address batch effects that typically confound cross-dataset analysis, TCAT adapts Harmony integration to provide batch-corrected nonnegative gene-level data, which is essential for matrix factorization [3]. For CITE-seq datasets, it further incorporates surface protein measurements into GEP spectra to enhance biological interpretability [3].

The annotation phase enables the transfer of learned GEPs to new query datasets using non-negative least squares (NNLS) regression to quantify the usage of predefined GEPs in each cell [3] [4]. This approach provides a consistent coordinate system for comparing cellular states across datasets, biological contexts, and experimental conditions, addressing a key limitation of de novo analysis methods.

Spectra: A Supervised Alternative

Spectra represents another recent advancement in component-based modeling that incorporates prior biological knowledge through gene-gene knowledge graphs [35]. Unlike purely unsupervised methods, Spectra uses existing gene sets and cell-type labels as input, explicitly models cell-type-specific factors, and represents input gene sets as a graph structure that guides the factorization [35]. This supervised approach allows Spectra to balance prior knowledge with data-driven discovery, detecting novel programs from residual unexplained variation while maintaining interpretability through graph-based regularization.

Experimental Benchmarking and Performance Comparison

Benchmarking Design and Metrics

Rigorous benchmarking of computational methods requires carefully designed experiments that evaluate performance across multiple dimensions. For component-based models, key evaluation criteria include: accuracy in recovering known biological signals, reproducibility across datasets and experimental conditions, interpretability of discovered programs, computational efficiency, and robustness to noise and batch effects [36] [3].

The single-cell integration benchmarking (scIB) framework provides quantitative metrics for assessing integration performance, focusing on both batch correction and biological conservation [36]. However, recent research has revealed limitations in these metrics for fully capturing intra-cell-type biological variation, leading to proposed enhancements in the scIB-E framework that better account for fine-grained biological structure preservation [36].

Performance Comparison Across Methods

Table 1: Comparative Performance of Component-Based Models in T Cell Applications

Method	GEP Reproducibility	Cross-Dataset Transfer	Runtime Efficiency	Biological Interpretability	Key Advantages
TCAT/starCAT	High (9 cGEPs across 7 datasets) [3]	Excellent (NNLS projection) [3]	Fast annotation phase [3]	High (46 curated cGEPs) [3]	Fixed GEP catalog enables consistent cross-dataset comparison
cNMF	Moderate (requires consensus approach) [31]	Limited (requires de novo analysis)	Moderate (requires multiple runs) [31]	Moderate to high [31]	Does not require pre-specified gene sets
Spectra	High (graph-guided factorization) [35]	Good (fixed factor representation)	Moderate (depends on graph size) [35]	High (incorporates prior knowledge) [35]	Balances prior knowledge with novel discovery
NMF (standard)	Low (high variability between runs) [31]	Poor	Fast (single run)	Variable (often mixed programs) [31]	Simple implementation

Table 2: Quantitative Benchmarking Results from Experimental Studies

Benchmark Scenario	TCAT/starCAT Performance	Alternative Method Performance	Experimental Context
GEP Transfer Accuracy	Pearson R > 0.7 for overlapping GEPs [3]	cNMF performance declines in small queries [3]	Simulation with partially overlapping GEPs between reference and query [3]
Factorization Interpretability	171/197 factors strongly constrained by biological graph (η ≥ 0.25) [35]	NMF and scHPF factors show poor agreement with annotated gene sets [35]	Application to breast cancer scRNA-seq data [35]
Cell-Type Specificity	Accurate restriction of CD8+ T cell programs to appropriate lineage [35]	ExpiMap and Slalom misassign TCR activity to myeloid/NK cells [35]	Analysis of tumor immune contexts [35]
Program Reproducibility	46 consensus GEPs from 1.7M cells across 38 tissues [3]	PCA components show substantially less concordance across datasets [3]	Integration of 7 scRNA-seq datasets [3]

Case Study: T Cell Atlas Integration

A landmark application of TCAT involved the integration of 1.7 million T cells from 700 individuals across 38 tissues and five disease contexts [3] [4]. This analysis identified 46 reproducible consensus GEPs (cGEPs) representing core T cell functions including proliferation, cytotoxicity, exhaustion, and effector states [3]. Notably, 9 cGEPs were consistently identified across all seven analyzed datasets, demonstrating exceptional reproducibility, while 49 cGEPs were found in two or more datasets [3].

When benchmarking cross-dataset transfer performance, TCAT significantly outperformed de novo cNMF analysis, particularly for small query datasets [3] [4]. This advantage stems from TCAT's ability to leverage large reference datasets for robust GEP discovery while efficiently projecting these programs onto query data using NNLS, unlike cNMF which must rediscover programs from limited data [3].

Experimental Protocols and Methodologies

TCAT/starCAT Workflow Implementation

The standard TCAT workflow consists of two sequential phases with distinct computational procedures:

Phase 1: Reference GEP Discovery

• Data Collection and Curation: Gather scRNA-seq datasets with appropriate cell-type annotations and experimental metadata [3].
• Batch Effect Correction: Apply modified Harmony integration to generate batch-corrected, nonnegative gene-level data compatible with NMF [3].
• Consensus NMF: Run cNMF with multiple initializations (typically 100-200 runs) and aggregate results to mitigate random initialization effects [3] [31].
• GEP Clustering and Consensus Building: Cluster highly correlated GEPs from different datasets and define cGEPs as cluster averages [3].
• Biological Annotation: Curate cGEPs through examination of top-weighted genes, gene-set enrichment analysis, and association with surface protein markers in CITE-seq data [3].

Phase 2: Query Dataset Annotation

• Data Preprocessing: Normalize query data using consistent methodology to reference datasets.
• GEP Usage Quantification: Apply non-negative least squares regression to estimate the activity of each reference cGEP in query cells [3].
• Cell State Prediction: Leverage GEP usages to predict additional cell features including lineage, TCR activation status, and cell cycle phase [3].
• Quality Control: Identify potential doublets or low-quality cells through aberrant GEP usage patterns [3].

GEP Discovery and Annotation Workflow in TCAT/starCAT

Validation Experiments and Statistical Analyses

Comprehensive validation is essential for establishing method reliability. Key experimental approaches include:

Simulation Studies: Benchmarking against ground truth data with known GEP structure provides quantitative performance assessment [3] [31]. Standard simulations involve generating synthetic scRNA-seq data where cells express specific combinations of predefined GEPs, then evaluating method accuracy in recovering these programs [3].

Cross-Validation: Splitting data into training and test sets, or using leave-one-dataset-out designs, assesses generalizability and robustness to batch effects [3].

Biological Validation: Experimental confirmation through complementary assays (CITE-seq, flow cytometry) or functional studies establishes biological relevance [3] [35]. For example, TCAT validation included association of cGEPs with surface protein markers in independent CITE-seq data [3].

Signaling Pathways and Biological Mechanisms

Key T Cell Signaling Pathways Identified Through Component-Based Models

Component-based models have elucidated complex signaling networks underlying T cell function and differentiation. The application of TCAT to 1.7 million T cells revealed 46 consensus GEPs encompassing diverse biological processes [3]:

T Cell Activation Programs: These include canonical TCR signaling, costimulatory pathways, and downstream effector programs. Spectra analysis successfully disentangled the highly correlated features of CD8+ T cell tumor reactivity and exhaustion, which are typically confounded in conventional analyses [35].

Cytokine and Helper T Cell Programs: TCAT identified specific GEPs for TH1, TH2, and TH17 responses, marked by characteristic transcription factors (TBX21, GATA3, RORC) and cytokines (IFNγ, IL-4/IL-5, IL-17/IL-26) [3]. Importantly, these programs were not mutually exclusive, with individual cells frequently co-expressing multiple helper program components.

Metabolic and Housekeeping Programs: Component-based models consistently identify metabolic pathways (oxidative phosphorylation, glycolysis) and cellular maintenance processes as shared activity programs across multiple cell types [35] [31].

Novel Activation States: Beyond canonical programs, these methods have discovered previously uncharacterized T cell states, including a T peripheral helper (TPH) GEP in rheumatoid arthritis characterized by PD-1, LAG3, and CXCL13 expression [3].

T Cell Signaling Pathways Identified by Component-Based Models

Research Reagent Solutions and Computational Tools

Table 3: Essential Research Resources for Implementing Component-Based Models

Resource Category	Specific Tools & Databases	Application in Workflow	Key Features
Computational Frameworks	starCAT [3], Spectra [35], cNMF [31], NSF [33]	GEP discovery and annotation	Specialized algorithms for biological data
Data Resources	Human Lung Cell Atlas [36], Mouse Cell Atlas [37], Tabula Muris Senis [37]	Reference datasets for benchmarking	Annotated single-cell datasets across tissues
Gene Set Collections	Immunology knowledge base (231 gene sets) [35], MSigDB, GO Biological Process	Prior knowledge incorporation	Curated gene programs for supervised analysis
Benchmarking Tools	scIB/scIB-E metrics [36], simulation frameworks	Method validation and comparison	Quantitative performance assessment
Visualization Platforms	Scanpy [35], UMAP [36]	Result interpretation and exploration	Dimensionality reduction and plotting

Discussion and Future Perspectives

Component-based models represent a significant advancement in single-cell computational biology, moving beyond discrete clustering to capture the continuous and multifaceted nature of cellular identity. The benchmarking data presented herein demonstrates that GEP-based approaches, particularly the TCAT/starCAT framework, offer substantial advantages for analyzing complex biological systems like T cell immunity.

The fixed coordinate system of GEPs enabled by TCAT/starCAT provides a reproducible foundation for comparing cellular states across experiments, donors, and disease conditions [3]. This addresses a critical challenge in single-cell biology, where batch effects and technical variability often obscure biological signals. The ability to project new datasets onto established GEP coordinates facilitates meta-analysis and integrative studies at unprecedented scale.

However, important challenges remain. The selection of appropriate factorization rank (number of components) continues to involve subjective elements, though methods like residual error analysis and consensus clustering provide guidance [32] [31]. Additionally, while nonnegativity enhances interpretability, it may limit the ability to model transcriptional repression, potentially necessitating complementary analyses for comprehensive regulatory inference.

Future methodological developments will likely focus on multimodal integration, simultaneously modeling gene expression, chromatin accessibility, and surface protein measurements within a unified factorization framework [3]. Temporal modeling approaches that capture dynamic program activation along developmental trajectories represent another promising direction. As single-cell technologies continue to evolve, component-based models will play an increasingly essential role in translating high-dimensional molecular measurements into biological insight and clinical applications.

For researchers selecting among these methods, TCAT/starCAT provides superior performance for cross-dataset analysis and annotation tasks, particularly when studying T cell biology, while Spectra offers advantages when incorporating specific prior knowledge through gene-gene graphs [35]. Traditional NMF approaches remain valuable for exploratory analysis when established references are unavailable or when studying systems where comprehensive reference catalogs have not yet been developed.

The accurate annotation of cell types, particularly within the complex and plastic T cell compartment, is a critical step in single-cell RNA sequencing (scRNA-seq) analysis. Reference-based annotation tools allow researchers to automatically classify cells from a new experiment (query dataset) by comparing them to expertly annotated reference atlases. This guide objectively compares three prominent tools—SingleR, Azimuth, and CellTypist—focusing on their performance, underlying methodologies, and application in benchmarking studies, with a specific emphasis on T cell subsets research.

Tool Comparison at a Glance

The following table summarizes the key characteristics and performance of SingleR, Azimuth, and CellTypist based on recent benchmarking studies.

Table 1: Comparison of Reference-Based Annotation Tools

Feature	SingleR	Azimuth	CellTypist
Primary Method	Correlation-based (Spearman)	Seurat integration & label transfer	Logistic regression models
Reference Handling	Pre-annotated reference dataset	Pre-processed and optimized reference map	Pre-trained or custom models
Benchmarked Performance (Xenium)	Best - Closest to manual annotation [38] [13]	Good	Not assessed in available studies
Benchmarked Performance (Lung Atlas)	Not assessed in top results	85.8% overall accuracy [39]	87.5% overall accuracy [39]
Strengths	Fast, accurate, easy to use [38]	Part of Seurat ecosystem; preserves reference structure [27]	Scalable to large references; supports online learning [39]
Considerations	Assigns a label to every cell (no "unknown" class by default) [12]	Requires reference building with Azimuth [38]	Performance can vary based on model and dataset [39]

Performance and Experimental Data

Benchmarking on Spatial Transcriptomics Data

A 2025 study specifically benchmarked cell type annotation methods for 10x Xenium spatial transcriptomics data, which profiles only several hundred genes. Using a human HER2+ breast cancer dataset and a paired single-nucleus RNA-seq (snRNA-seq) reference, the study compared five reference-based methods against manual annotation based on marker genes.

In this challenging context with a limited gene panel, SingleR was the top-performing tool, with its predictions most closely matching the manual annotations. The study noted it was also fast and easy to use [38] [13]. Azimuth, scPred, and scmapCell also showed reasonable performance, while RCTD was found to be less suitable for this data type without significant parameter adjustment [13].

Benchmarking for Atlas Integration

Another benchmarking study focused on integrating cell types from two lung atlas datasets—the Human Lung Cell Atlas (HLCA) and the LungMAP single-cell reference (CellRef). The study evaluated tools based on their accuracy in matching query cells to expert-annotated reference labels [39].

Table 2: Lung Atlas Cross-Matching Performance

Tool	Overall Accuracy	Macro F1 Score
CellTypist	87.5%	0.87
Azimuth	85.8%	0.85
scArches	83.8%	0.83
FR-Match	80.5%	0.80

CellTypist achieved the highest overall accuracy and F1 score, a metric that balances precision and recall. The study highlighted that while both Azimuth and CellTypist performed well, they exhibited complementary strengths, with variations in accuracy when annotating specific and rare cell types [39].

Experimental Protocols in Benchmarking Studies

The robustness of the tool comparisons relies on standardized evaluation workflows. The following diagram illustrates the typical protocol used in the cited benchmarking studies.

Detailed Methodological Steps

• Data Collection and Reference Preparation: Benchmarking studies use publicly available or newly generated datasets. For example, a breast cancer Xenium dataset with a paired snRNA-seq reference was used to ensure minimal variability between reference and query [38] [13]. The reference data is meticulously annotated, often using marker gene expression and copy number variation (inferCNV) analysis to identify tumor cells [13].
• Query Data Preprocessing: The query data undergoes standard scRNA-seq preprocessing (quality control, normalization) using pipelines like Seurat. For spatial data like Xenium, which has a small gene panel, the feature selection step is sometimes skipped, and all genes are used for scaling [13].
• Tool Execution and Label Transfer: Each tool is run according to its specific requirements:
  • SingleR: The SingleR() function is applied directly to the normalized query data and the reference dataset [13].
  • Azimuth: The reference is converted into an Azimuth-compatible object using AzimuthReference(). The query is then annotated using the RunAzimuth() function, which projects it into the reference space [38] [13].
  • CellTypist: A pre-trained model or a model built from the reference dataset is used to predict labels for the query cells [39].
• Performance Evaluation: Predictions from each tool are compared against the "ground truth" manual annotations. Metrics such as overall accuracy, F1 score, and the percentage of cells confidently annotated are calculated. The composition of predicted cell types is also compared to manual annotations to assess biological plausibility [38] [12] [39].

The Scientist's Toolkit

The table below lists key reagents and computational resources essential for performing reference-based cell type annotation as described in the experimental protocols.

Table 3: Essential Research Reagents and Resources

Item	Function / Description	Example Use Case
10x Xenium Gene Panel	A pre-designed panel of several hundred genes for imaging-based spatial transcriptomics.	Generating query data for benchmarking on spatial platforms [38].
Paired snRNA-seq Data	Single-nucleus RNA-seq data from the same sample as the spatial data.	Serves as an ideal, minimally variable reference dataset [13].
Seurat R Toolkit	A comprehensive R package for single-cell genomics data analysis.	Used for data preprocessing, normalization, integration, and running Azimuth [38].
Cell Type Marker Gene List	A curated list of genes known to be specifically expressed in particular cell types.	Used for manual annotation, which serves as the ground truth for benchmarking [12].
inferCNV Software	A computational tool to infer copy number variations from scRNA-seq data.	Used to identify and annotate tumor cells in the reference atlas [13].
Dideoxy-amanitin	Dideoxy-amanitin, MF:C39H54N10O12S, MW:887.0 g/mol	Chemical Reagent
TC-F2	TC-F2, MF:C26H25N5O2, MW:439.5 g/mol	Chemical Reagent

The choice of an optimal reference-based annotation tool depends on the specific biological context, data type, and research goals. For T cell research, where distinguishing between highly similar states (like exhausted and effector T cells) is crucial, the choice of a well-curated T-cell-specific reference is as important as the tool itself.

• SingleR excels in speed and accuracy, particularly on challenging data like imaging-based spatial transcriptomics, making it a robust and user-friendly choice [38] [13].
• Azimuth integrates seamlessly into the widely used Seurat workflow and is effective for projecting query data into a stable, curated reference atlas without altering its structure [27].
• CellTypist demonstrates high accuracy in large-scale atlas integration tasks and offers scalability for use with extensive reference collections [39].

Benchmarking studies consistently reveal that these tools have complementary strengths. Therefore, researchers working with T cell subsets may benefit from a consensus approach, using multiple tools to corroborate findings, especially when characterizing novel or rare cell states.

The ability to accurately characterize T cell states is a cornerstone of modern immunology, with critical implications for understanding cancer, autoimmune diseases, and infection responses. Single-cell RNA sequencing (scRNA-seq) has revealed an unprecedented degree of T cell diversity, moving beyond simple discrete classifications to reveal a continuum of cellular states. However, this complexity presents a substantial analytical challenge. Traditional clustering approaches often fail to capture the continuous nature of T cell differentiation and the co-expression of multiple gene programs within individual cells. In response to these limitations, specialized computational platforms have emerged to provide more standardized, reproducible, and biologically meaningful annotation of T cell states.

Two prominent platforms in this field are ProjecTILs, a well-established method for reference atlas projection, and STCAT (star-CellAnnoTator), a newer pipeline that quantifies predefined gene expression programs (GEPs). This guide provides an objective comparison of these platforms, detailing their methodologies, performance, and optimal use cases to help researchers and drug development professionals select the appropriate tool for their specific research context.

ProjecTILs: Reference Atlas Projection

ProjecTILs is a computational framework designed to project new scRNA-seq data into curated reference atlases without altering the reference structure. This approach enables the direct comparison of query datasets against a stable, annotated system of coordinates. The method addresses a key limitation of unsupervised analysis by providing a consistent framework for interpreting T cell states across different studies, conditions, and time points.

The platform incorporates specialized reference atlases for specific biological contexts, including a comprehensive atlas of tumor-infiltrating T cell (TIL) states built from integrated data across multiple murine melanoma and colon adenocarcinoma studies. This reference map captures key T cell subtypes such as naive-like, effector-memory (EM), precursor-exhausted (Tpex), terminally-exhausted (Tex), follicular-helper (Tfh), and regulatory T cells (Tregs), providing a foundation for consistent annotation [27].

STCAT (T-CellAnnoTator): Gene Expression Program Quantification

STCAT (within the broader starCAT framework) introduces a different paradigm for T cell characterization by simultaneously quantifying predefined gene expression programs that capture activation states, cellular subsets, and core functions. Rather than relying on discrete clustering, STCAT models transcriptomes as weighted mixtures of GEPs, reflecting the biological reality that individual cells can express multiple functional programs simultaneously.

The platform was developed through analysis of approximately 1.7 million T cells from 700 individuals across 38 tissues and five disease contexts, identifying 46 reproducible GEPs that reflect core T cell functions including proliferation, cytotoxicity, exhaustion, and effector states. These GEPs serve as a fixed coordinate system for comparing cellular activities across datasets, enabling the detection of subtle state changes that might be obscured by discrete clustering approaches [3].

Comparative Methodological Analysis

Technical Approaches and Underlying Algorithms

Table 1: Core Methodological Comparison of STCAT and ProjecTILs

Methodological Aspect	STCAT (star-CellAnnoTator)	ProjecTILs
Core Approach	Quantifies predefined gene expression programs (GEPs)	Projects cells into reference atlas space
Primary Algorithm	Consensus Nonnegative Matrix Factorization (cNMF) with nonnegative least squares for query projection	Reference-based projection using STACAS/Seurat integration and PCA transformation
Data Input	scRNA-seq data (UMI counts/TPMs); optionally incorporates CITE-seq protein measurements	scRNA-seq expression matrix (UMI counts or TPMs)
Reference Dependency	Uses a fixed catalog of 46 consensus GEPs derived from 1.7 million T cells	Requires a pre-constructed reference atlas with annotated cell states
Batch Correction	Adapted Harmony integration for nonnegative, gene-level batch correction	STACAS algorithm designed for integrating datasets with limited cell subtype overlap
Key Output	GEP usage scores representing contribution of each program to a cell's state	Projected coordinates in reference space and cell state predictions

Experimental Workflows and Implementation

The experimental workflows for both platforms follow structured pipelines from raw data to biological interpretation, though they differ significantly in their intermediate steps and analytical approaches.

Performance Benchmarking and Experimental Validation

Methodological Benchmarking Results

Independent benchmarking studies provide critical insights into the performance characteristics of reference-based annotation methods. While direct comparisons between STCAT and ProjecTILs are limited in the current literature, evaluations against common tasks and alternative methods highlight their relative strengths.

In benchmarking studies of reference mapping approaches, ProjecTILs has demonstrated robust performance for T cell-specific analysis. In one comprehensive evaluation that included Harmony, Seurat Anchored-rPCA, and other methods, ProjecTILs maintained accurate projection capabilities while preserving reference atlas structure [40]. The method has shown particular strength in maintaining biological consistency when projecting data from different conditions or timepoints onto a stable reference framework.

STCAT has been systematically validated through simulation studies where reference and query datasets contained partially overlapping GEPs. In these controlled experiments, STCAT accurately inferred the usage of overlapping GEPs (Pearson R > 0.7) while correctly predicting low usage of non-overlapping programs. The method outperformed direct application of cNMF to query datasets, particularly for smaller sample sizes where de novo GEP discovery becomes challenging [3].

Application-Based Performance Metrics

Table 2: Experimental Performance and Application Characteristics

Performance Metric	STCAT	ProjecTILs
Prediction Accuracy	High for overlapping GEPs (R > 0.7 in simulations)	Accurate label transfer in T-cell contexts
Novel State Detection	Identifies cells with GEP combinations not in reference	Characterizes states "deviating" from reference subtypes
Handling Rare Cell Types	Can quantify rare GEPs hard to identify de novo	Dependent on reference completeness
Cross-Platform Robustness	Maintains performance with smaller query datasets	Stable performance across sequencing technologies
Computational Efficiency	Reduced runtime versus de novo analysis	Efficient projection without reference retraining
Experimental Validation	Identified immunotherapy response predictors	Applied to perturbation effects in infection/cancer models

Practical Implementation Guide

Research Reagent Solutions and Computational Requirements

Table 3: Essential Research Reagents and Computational Resources

Resource Type	Specific Resource	Function in Analysis	Availability
Reference Atlases	Murine tumor-infiltrating T cell atlas	ProjecTILs reference for cancer immunology	https://doi.org/10.6084/m9.figshare.12489518
	Viral infection CD8+ T cell atlas	ProjecTILs reference for infection models	https://spica.unil.ch/refs/viral-CD8-T
GEP Catalogs	46 consensus T cell GEPs	STCAT reference for human T cell states	Supplementary Table 2 in [3]
Software Packages	ProjecTILs R package	Reference projection and visualization	GitHub: carmonalab/ProjecTILs [28]
	starCAT/TCAT pipeline	GEP usage quantification and annotation	Available upon publication [3]
Data Integration Tools	STACAS algorithm	Batch correction for heterogeneous datasets	Integrated in ProjecTILs [27]
	Harmony integration	Batch effect correction for cNMF	Adapted in STCAT pipeline [3]

Selection Guidelines for Different Research Contexts

The benchmarking data presented in this guide demonstrates that both STCAT and ProjecTILs offer robust solutions for T cell annotation, but with distinct methodological approaches and optimal application domains. ProjecTILs excels in scenarios where well-curated reference atlases exist and researchers need to map query data onto established T cell subtypes, particularly in murine models or human contexts with appropriate references. Its preservation of reference space structure enables direct comparison across experiments and conditions.

STCAT offers a complementary approach that addresses the continuous nature of T cell states through gene expression program quantification. Its fixed catalog of 46 consensus GEPs provides a stable framework for comparing activation states and subset distributions across diverse human tissue contexts, with particular strength in detecting mixed cellular states that might be obscured by discrete classification.

Future methodological developments will likely incorporate multi-omic integration, with both platforms already supporting CITE-seq protein measurements to enhance annotation accuracy. As single-cell technologies continue to evolve, the availability of comprehensive benchmarking data—such as that provided in this guide—will be essential for helping researchers select appropriate analytical tools for their specific immunological research questions.

This guide provides an objective comparison of two prominent marker-based, semi-automated cell type annotation tools—scGate and Garnett—within the broader context of single-cell RNA sequencing (scRNA-seq) analysis. Focusing on their application to T cell subset research, we compare their methodologies, performance metrics, and optimal use cases based on published benchmarking studies. Accurately identifying T cell subsets is crucial for research in immunology, cancer, and drug development, yet it remains challenging due to the high transcriptional similarity between closely related T cell states [41] [42].

The following table summarizes the core characteristics of both tools, highlighting their shared semi-supervised, marker-based approach and key differentiators.

Feature	scGate	Garnett
Core Methodology	Hierarchical, multi-layer gating similar to flow cytometry [42]	Hierarchical classification using a regularized elastic-net generalized linear model [43] [41] [42]
Primary Classification Strategy	Marker-based purification of "pure" cell populations [42]	Supervised machine learning from user-provided cell type definitions [41]
Underlying Algorithm	Rule-based filtering at each node of the hierarchy	Elastic-net multinomial regression [43] [41]
Handling of Unknown Cell Types	Yes (cells not meeting "pure" criteria are unlabeled) [42]	Yes [43]
Dependence on Reference Atlases	No (reference-free) [42]	No (reference-free) [41] [42]
Key Advantage	High interpretability; user has direct control via marker lists [42]	Can learn a classifier from the data for application to new datasets [41]

Experimental Performance and Benchmarking Data

Independent evaluations provide critical insights into how these tools perform in realistic research scenarios, particularly for the difficult task of annotating highly similar T cell subsets.

Performance in Classifying T Cell Subsets

A significant challenge in T cell research is distinguishing between closely related subsets, such as CD4+ versus CD8+ T cells, and further subdividing them into naive, central memory (TCM), effector memory (TEM), and terminally differentiated effector memory (TEMRA) populations [41] [42]. These subsets are well-defined by surface proteins but often exhibit overlapping transcriptional profiles.

Tools like Garnett and scGate are designed to address this challenge by incorporating prior knowledge. However, a benchmark study evaluating 22 classification methods on 27 public datasets found that incorporating prior knowledge in the form of marker genes did not consistently improve performance compared to general-purpose classifiers like Support Vector Machines (SVM) in intra-dataset prediction tasks [44]. This suggests that while marker-based methods are intuitive, their accuracy can be context-dependent.

Comparative Performance in Broader Cell Annotation Tasks

In a broader evaluation of ten cell annotation methods, Garnett was included among the tools assessed for their ability to perform intra-dataset and inter-dataset predictions [43]. The study assessed robustness to challenges like gene filtering and similarity among cell types. While methods like Seurat and SingleR performed well in annotating major cell types, they struggled with rare populations and highly similar cell types [43]. This context is important for T cell research, where distinguishing between highly similar T cell states is often the primary goal.

Detailed Experimental Protocols

To ensure reproducibility and provide a clear framework for researchers to implement these tools or design their own benchmarks, we outline the core workflows and key reagent solutions.

Core Workflow for scGate and Garnett

Both tools employ a hierarchical, semi-supervised approach but differ in their specific implementation after the initial manual step. The following diagram illustrates the shared initial stage and subsequent divergent paths in their classification workflows.

Key Research Reagent Solutions

The primary "reagents" for these computational tools are the data and gene markers used to define cell types. The table below lists essential components for implementing scGate and Garnett in T cell research.

Item	Function/Description	Relevance to T Cell Research
Marker Gene Panel	A curated list of genes known to define specific cell types.	Crucial for defining T cell subsets (e.g., CD3E for T cells; CD4, CD8A for major subsets; CCR7, SELL for memory states) [42].
Cell Type Hierarchy	A tree structure defining the relationship between cell types (e.g., Immune -> Lymphocyte -> T cell -> CD4+ T cell).	Provides the organizational backbone for both tools, reflecting T cell lineage relationships [41] [42].
scRNA-seq Dataset	The input gene expression matrix (cells x genes) to be annotated.	The primary data for analysis; quality and depth directly impact annotation accuracy of complex T cell states [42].
High-Quality Reference Data (Optional)	Well-annotated datasets (e.g., from sorted cells) used for training Garnett classifiers.	Can improve model generalizability for classifying T cell subsets across different studies [41].

The choice between scGate and Garnett depends heavily on the specific research goals, dataset characteristics, and the desired level of user control versus automation.

• Use scGate when you need maximum interpretability and direct control over the gating logic, similar to flow cytometry. It is particularly useful for rapid purification of specific T cell populations of interest from a heterogeneous sample using a well-established marker panel [42].
• Use Garnett when your goal is to train a reusable classifier on a well-annotated or sorted dataset that can be consistently applied to classify multiple new datasets, such as in a large-scale study profiling T cells across many patients [41] [42].

For the most accurate annotation of complex T cell subsets, a two-step process is strongly recommended [42]. This involves an initial automated annotation using a tool like scGate or Garnett, followed by expert manual validation through inspection of cluster-specific marker genes. This hybrid approach leverages the speed and reproducibility of automation while incorporating crucial biological expertise to catch misannotations, especially for transcriptionally similar T cell states like naive, memory, and exhausted T cells.

Emerging Foundation Models and Large Language Models for Single-Cell Data

The advent of single-cell RNA sequencing (scRNA-seq) has revolutionized biological research by enabling the characterization of cellular heterogeneity at unprecedented resolution. However, the analysis of scRNA-seq data presents significant challenges, including high dimensionality, technical noise, and batch effects. In response, the field has seen the emergence of foundation models (FMs) and large language models (LLMs) specifically designed for single-cell data. These models leverage self-supervised learning on massive-scale single-cell datasets to learn universal representations of cells and genes that can be adapted to various downstream tasks. For researchers focusing on complex cellular systems like T cell subsets, where traditional clustering approaches often fail to capture continuous states and mixed gene expression programs, these new computational approaches offer promising alternatives for precise cell state annotation and biological discovery.

Performance Benchmarking: How Models Compare on Key Tasks

Quantitative Performance Across Evaluation Metrics

Independent benchmark studies have evaluated single-cell foundation models (scFMs) against traditional methods across multiple tasks. One comprehensive study assessed six prominent scFMs (Geneformer, scGPT, UCE, scFoundation, LangCell, and scCello) alongside established baselines using 12 evaluation metrics spanning unsupervised, supervised, and knowledge-based approaches [45].

Table 1: Performance of Single-Cell Foundation Models on Cell-Level Tasks

Model	Batch Integration (ASW Batch↑)	Cell Type Annotation (Accuracy↑)	Biological Conservation (cLISI↑)	Computational Efficiency
scANVI	0.82	0.89	0.85	Medium
Scanorama	0.85	0.84	0.82	High
scVI	0.81	0.83	0.80	Medium
Harmony	0.79	0.81	0.78	High
Geneformer	0.76	0.79	0.81	Low
scGPT	0.78	0.82	0.83	Low
Traditional ML	0.75	0.80	0.76	Very High

The benchmarking revealed that no single scFM consistently outperformed all others across every task, emphasizing that optimal model selection depends on specific use cases, dataset size, and computational constraints [45]. While scFMs demonstrated robustness and versatility, simpler machine learning models sometimes showed better performance on specific datasets, particularly under resource constraints.

Large Language Models for Cell Type Annotation

For de novo cell type annotation using LLMs, benchmarking with the AnnDictionary package has revealed performance variations across model providers [46].

Table 2: Performance of Large Language Models on De Novo Cell Type Annotation

LLM Provider	Model	Agreement with Manual Annotation	Inter-LLM Agreement	Major Cell Type Accuracy
Anthropic	Claude 3.5 Sonnet	Highest	High	>85%
OpenAI	GPT-4	High	Medium	>80%
Google	PaLM 2	Medium	Medium	75-80%
Meta	Llama 2	Medium	Low	70-75%

Claude 3.5 Sonnet achieved the highest agreement with manual annotations, with most major LLMs demonstrating over 80-90% accuracy for annotating major cell types [46]. The performance variation highlights the importance of model selection for automated annotation pipelines.

Experimental Protocols in Model Benchmarking

Standardized Evaluation Frameworks

Comprehensive benchmarking studies have established rigorous methodologies for evaluating single-cell foundation models. The standard protocol involves:

• Model Selection and Training: Studies typically evaluate multiple scFMs (e.g., Geneformer, scGPT, UCE, scFoundation, LangCell, scCello) alongside traditional methods (Seurat, Harmony, scVI) under consistent conditions [45].

• Task Design: Evaluation encompasses both gene-level and cell-level tasks. Gene-level tasks include tissue specificity prediction and Gene Ontology term prediction. Cell-level tasks include batch integration, cell type annotation, cancer cell identification, and drug sensitivity prediction [45].

• Dataset Curation: Benchmarks use diverse datasets with high-quality labels that span multiple biological conditions, tissues, and species. For example, the Asian Immune Diversity Atlas (AIDA) v2 from CellxGene serves as an independent validation dataset to mitigate data leakage concerns [45].

• Evaluation Metrics: A comprehensive set of metrics (e.g., kBET, ASW, cLISI, iLISI, ARI, NMI) assesses both batch effect removal and biological conservation. Novel ontology-informed metrics like scGraph-OntoRWR and Lowest Common Ancestor Distance (LCAD) evaluate biological relevance [45].

T Cell-Specific Annotation Protocols

For T cell research, specialized annotation pipelines like T-CellAnnoTator (TCAT) and its generalized framework starCAT have been developed to address the unique challenges of T cell heterogeneity [3] [47]. The experimental workflow involves:

Figure 1: starCAT Workflow for T Cell Annotation

• Data Collection and Preprocessing: Analysis of 1.7 million T cells from 700 individuals across 38 tissues and five disease contexts [3].

• Consensus Nonnegative Matrix Factorization (cNMF): Application of cNMF to identify Gene Expression Programs (GEPs) with enhancements for cross-dataset reproducibility, including Harmony integration for batch correction and incorporation of surface protein data from CITE-seq [3].

• GEP Catalog Construction: Identification of 46 consensus GEPs (cGEPs) capturing T cell subsets, activation states, and functions through cross-dataset clustering of similar GEPs [3].

• Annotation with starCAT: Using nonnegative least squares to quantify the activity of predefined GEPs in new query datasets, enabling consistent cell state representation across studies [3].

Research Reagent Solutions for Single-Cell AI

Table 3: Essential Research Reagents and Computational Tools

Resource Name	Type	Primary Function	Relevance to T Cell Research
AnnDictionary	Software Package	LLM-provider-agnostic cell type annotation	Enables de novo T cell subset annotation with multiple LLM backends [46]
starCAT/TCAT	Annotation Pipeline	Quantifies predefined GEPs in single-cell data	Specifically designed for T cell states and activation programs [3]
Tabula Sapiens	Reference Atlas	Multi-tissue scRNA-seq reference	Provides benchmark for T cell annotation across tissues [46]
CZ CELLxGENE	Data Platform	Curated single-cell datasets	Includes Asian Immune Diversity Atlas (AIDA) for validation [45]
scGPT	Foundation Model	Generative pre-training for single-cell data	Transfer learning for T cell perturbation responses [48]
Geneformer	Foundation Model	Transformer model trained on single-cell data	Predicts T cell development trajectories [48]

Specialized Applications in T Cell Research

Addressing T Cell Heterogeneity

Traditional clustering approaches have limitations for T cell analysis because transcriptomes reflect the expression of multiple gene expression programs (GEPs) that vary continuously, combine additively within individual cells, and exhibit stimulus-dependent plasticity [3]. The starCAT framework addresses these challenges by simultaneously quantifying multiple predefined GEPs that capture T cell subsets, activation states, and functions, moving beyond discrete classification to continuous state assessment.

Clinically Relevant Discoveries

When applied to T cell analysis across multiple disease contexts, TCAT has identified 46 reproducible GEPs reflecting core T cell functions including proliferation, cytotoxicity, exhaustion, and effector states [3]. This approach has been used to characterize activation GEPs that predict immune checkpoint inhibitor response across multiple tumor types, demonstrating the clinical translational potential of these advanced annotation methods [47].

The benchmarking of emerging foundation models and large language models for single-cell data reveals a rapidly evolving landscape where no single solution dominates all applications. For T cell researchers, the choice between complex foundation models and simpler alternatives depends on multiple factors including dataset size, task complexity, need for biological interpretability, and computational resources. While scFMs demonstrate remarkable versatility and biological relevance, traditional methods remain competitive for specific tasks, particularly under resource constraints.

Future development in this field will likely focus on improved multimodal integration, better incorporation of biological prior knowledge, and enhanced scalability. For the T cell research community, specialized frameworks like starCAT that leverage predefined GEP catalogs offer a promising path toward more reproducible and biologically meaningful cell state annotation that can bridge across studies and accelerate therapeutic development.

Spatial transcriptomics has revolutionized biological research by enabling the mapping of gene expression within the intact architectural context of tissues. For the study of complex biological systems such as the tumor microenvironment and immune responses, accurate cell type annotation is a critical first step in the analysis pipeline. Among commercially available platforms, the 10x Genomics Xenium In Situ system has emerged as a prominent imaging-based technology capable of mapping hundreds to thousands of genes at subcellular resolution. This guide objectively compares the performance of various computational annotation methods applied specifically to Xenium data, with particular emphasis on applications in T cell biology and immunology research.

Performance Benchmarking of Annotation Methods

Comprehensive Method Comparison

Independent benchmarking studies have systematically evaluated the performance of various reference-based cell type annotation tools when applied to Xenium data. The performance metrics, including accuracy, computational efficiency, and ease of use, provide critical guidance for researchers selecting appropriate methods for their specific applications.

Table 1: Performance Benchmarking of Cell Type Annotation Methods for Xenium Data

Method	Overall Performance	Accuracy	Speed	Ease of Use	Key Strengths
SingleR	Best	High	Fast	Easy	Fast, accurate, easy to use with results closely matching manual annotation [13]
Azimuth	Good	High	Medium	Medium	Series of computational tools for reference-mapping of single cell data [49]
RCTD	Good	High	Medium	Medium	Comprehensive mapping of tissue cell architecture via integrated single cell and spatial transcriptomics [13] [49]
scPred	Moderate	Medium	Medium	Medium	Trains a model for reference and predicts cell types [13]
scmapCell	Moderate	Medium	Medium	Medium	Predicts cell types based on correlation between reference and query datasets [13]

Platform-Specific Technical Considerations

The performance of cell type annotation is intrinsically linked to the data quality generated by different spatial transcriptomics platforms. Xenium demonstrates specific technical characteristics that influence downstream annotation accuracy:

• Sensitivity and Specificity: In comparative analyses of commercial platforms, Xenium's detection efficiency matches other in situ hybridization-based technologies like MERSCOPE and Molecular Cartography, with sensitivity between 1.2 and 1.5 times higher than scRNA-seq (Chromium v2) [50]. Its specificity, while slightly lower than some platforms, remains consistently higher than CosMx [50].

• Cell Segmentation Accuracy: Xenium utilizes a multimodal fluorescence-based cell segmentation approach that maintains structural integrity of irregularly shaped cells, outperforming Visium HD's bin-based segmentation in complex tissue architectures like colorectal cancer [51]. This accurate cellular boundary identification is crucial for precise cell type assignment.

• Resolution and Panel Size: Xenium offers subcellular resolution with targeted gene detection. While early panels included several hundred genes, newer Xenium 5K panels now profile up to 5,000 genes [52] [51], significantly enhancing the ability to resolve subtle cell states, including T cell subsets.

Experimental Protocols for Benchmarking Studies

Reference-Based Annotation Workflow

The benchmarking protocol for evaluating annotation methods on Xenium data follows a standardized workflow to ensure fair comparison:

1. Reference Dataset Preparation: A high-quality single-cell or single-nucleus RNA sequencing (scRNA-seq/snRNA-seq) reference is essential. The protocol involves:

• Quality control to remove cells without proper annotation and potential doublets using tools like scDblFinder [13]
• Normalization using the Seurat standard pipeline with NormalizeData function [13]
• Cell type labeling using known marker genes and computational methods like inferCNV for identifying tumor cells [13]

2. Xenium Data Processing: The query Xenium data undergoes specific processing:

• Quality control filtering to remove cells annotated as "Unlabeled" or with fewer than 10 transcript counts [13] [50]
• Normalization without feature selection due to the limited gene panel size [13]
• Dimension reduction using PCA and UMAP for visualization [13]

3. Method-Specific Reference Preparation: Each annotation method requires tailored reference preparation:

• Azimuth: RunUMAP with return.model=TRUE and SCTransform normalization [13]
• RCTD: Reference function in spacexr package [13]
• scmap and SingleR: SingleCellExperiment object format [13]
• scPred: Seurat object format [13]

4. Cell Type Prediction Execution: Each method is run with platform-specific parameters:

• SingleR: SingleR function in SingleR package with default parameters [13]
• Azimuth: RunAzimuth function in Azimuth package [13]
• RCTD: create.RCTD and run.RCTD functions with adjusted parameters to retain all cells [13]
• scPred: trainModel and scPredict functions [13]
• scmapCell: indexCell, scmapCell, and scmapCell2Cluster functions [13]

5. Performance Evaluation: Method performance is assessed by comparing the composition of predicted cell types with manual annotation based on known marker genes [13].

Advanced Spatial Clustering for Xenium Data

Beyond reference-based annotation, spatial clustering methods are crucial for identifying novel tissue domains and microenvironments. For Xenium data, the Banksy algorithm has demonstrated superior performance in clustering spatially coherent regions compared to default graph-based methods [51] [49]. The Banksy workflow involves:

• Neighborhood Analysis: Augmenting the features of each cell with an average of the features of its spatial neighbors along with neighborhood feature gradients [49]
• Multi-scale Clustering: Identifying spatial domains at multiple resolutions to capture both fine-grained and broad tissue organizations
• Integration with Marker Expression: Validating clusters using known marker genes to ensure biological relevance

Figure 1: Experimental workflow for benchmarking cell type annotation methods on Xenium data, covering from data input to performance evaluation.

Application to T Cell Biology

Mapping T Cell Heterogeneity and TCR Repertoires

The application of Xenium to T cell research enables unprecedented insights into the spatial organization of T cell subsets and their clonal distribution within tissues. Specialized methods have been developed to address the unique challenges of T cell annotation:

• T Cell Receptor Profiling: Methods like Slide-TCR-seq enable simultaneous sequencing of whole transcriptomes and TCRs within intact tissues at 10-μm resolution [53]. This approach adapts rhTCRseq to spatial contexts, allowing specific multiplexed PCR amplification of TCR transcripts while preserving spatial information [53].

• Spatial Repertoire Analysis: In human lymphoid tissues, spatial TCR sequencing has revealed distinct repertoires in germinal centers compared to other tissue regions, demonstrating how T cell clonotypes are non-randomly distributed in tissues [53].

• T Cell State Identification: Xenium's resolution enables discrimination of T cell functional states (e.g., cytotoxic, exhausted, helper) through integrated analysis of marker genes and spatial context, providing insights into their functional specialization within microenvironments.

Table 2: Key Marker Genes for T Cell Subset Annotation in Xenium Panels

T Cell Subset	Key Marker Genes	Spatial Distribution Patterns
Cytotoxic CD8+ T cells	CD8A, CD8B, GZMB, PRF1	Tumor-invasive margin, tumor cores [53]
Helper CD4+ T cells	CD4, IL7R, CXCL13	Germinal centers, tertiary lymphoid structures [53]
T follicular helper (Tfh)	CXCR5, BCL6, PDCD1	Germinal centers of lymphoid tissues [53]
Regulatory T cells (Treg)	FOXP3, IL2RA, CTLA4	Tumor microenvironment, immune niches [54]
Exhausted T cells	LAG3, HAVCR2, TIGIT	Chronic infection sites, tumor microenvironments [51]

3D and Subcellular Analysis Capabilities

Xenium data provides three-dimensional coordinates for each transcript, enabling advanced analytical approaches beyond conventional 2D analysis:

• Subcellular Localization: Segmentation-free models like SSAM and Points2Regions can identify subcellular mRNA clusters classified as nuclear, cytoplasmic, or extracellular [50]. These patterns show associations with specific cell types and reveal subtle expression variations between nuclear and cytoplasmic compartments [50].

• 3D Tissue Reconstruction: The z-dimension information allows detection of potential mixed-source signals from cells overlapping in tissue depth, found in approximately 1.8% of total cells [50]. This capability is particularly valuable for understanding the complex spatial relationships between T cells and their targets in three-dimensional space.

Figure 2: Integrated workflow for spatial T cell receptor and transcriptome analysis, enabling identification of spatially distinct T cell niches.

The Scientist's Toolkit

Essential Computational Tools

Table 3: Essential Computational Tools for Xenium Data Analysis

Tool	Function	Application in Xenium Analysis
SingleR	Reference-based cell type annotation	Fast and accurate cell type prediction for Xenium data [13] [49]
Banksy	Spatial clustering algorithm	Identifies spatially coherent domains in Xenium data, superior to default clustering [51] [49]
Cellpose	Anatomical segmentation algorithm	Cell segmentation for spatial transcriptomics data [49]
RCTD	Cell type deconvolution	Comprehensive mapping of tissue cell architecture [13] [49]
SSAM	Segmentation-free spatial analysis	Identifies cell-type-specific clusters without cell segmentation [50]
Points2Regions	Subcellular pattern identification	Classifies mRNA clusters as nuclear, cytoplasmic, or extracellular [50]

Experimental Design Considerations

For researchers designing Xenium experiments focused on T cell biology, several factors critically impact annotation success:

• Panel Design: Custom gene panels should include not only canonical T cell markers (CD3D, CD4, CD8A) but also functional state markers (GZMB, FOXP3, CXCL13) and, if possible, TCR constant region sequences for clonotype mapping [53].

• Reference Quality: A high-quality matched scRNA-seq reference is invaluable for annotation accuracy. References should ideally include comprehensive T cell subsets from the same tissue type and disease context [13].

• Segmentation Strategy: For T cells, which often exhibit complex morphologies in tissues, Xenium's multimodal segmentation typically outperforms nuclei-based approaches, preserving crucial cytoplasmic transcripts that define functional states [51].

• Spatial Context Integration: Methods like Banksy that incorporate neighborhood information improve the identification of spatially coherent T cell niches and microenvironments [49].

Benchmarking studies consistently demonstrate that careful selection of computational methods significantly enhances the biological insights gained from Xenium spatial transcriptomics data. For cell type annotation, SingleR emerges as the top-performing method, balancing accuracy, speed, and ease of use. The integration of spatial clustering algorithms like Banksy further enables the discovery of biologically relevant tissue domains. For T cell research, specialized approaches that combine transcriptome profiling with TCR sequencing and account for spatial context provide unprecedented views of immune responses in situ. As Xenium panels continue to expand in gene capacity and analytical methods mature, the platform promises to deliver increasingly detailed maps of the spatial organization of immune responses in health and disease.

Overcoming Common Challenges in T Cell Annotation Workflows

Addressing Batch Effects and Technical Variability Across Datasets

In single-cell RNA sequencing (scRNA-seq) studies, batch effects refer to technical variations introduced when samples are processed in different groups or under varying conditions, such as using different reagent lots, personnel, equipment, or sequencing runs [55]. These non-biological factors can confound the ability to measure true biological variation between samples, potentially leading to misinterpreted results and reduced reproducibility [55]. The challenge is particularly pronounced in T cell research, where continuous phenotypic states and subtle differences in activation markers require highly sensitive analytical approaches [3].

Technical variability in scRNA-seq arises from multiple sources throughout the experimental workflow, including mRNA capture efficiency, reverse transcription efficiency, amplification bias, and sequencing depth [56] [57]. This variability is especially problematic in studies of cellular heterogeneity, as technical artifacts can be mistaken for novel biological discoveries [57]. For example, differences in cell-specific detection rates driven by batch effects have been shown to create artificial cell groups in unsupervised analyses [57]. Addressing these challenges requires both careful experimental design and specialized computational correction methods.

Computational Methods for Batch Effect Correction

Several computational methods have been developed to address batch effects in single-cell data, each employing different algorithmic strategies. These methods aim to remove technical variation while preserving biologically relevant signals. The selection of an appropriate method depends on factors such as dataset size, complexity, and the specific research question.

The table below summarizes key batch effect correction methods and their primary characteristics:

Method	Underlying Algorithm	Key Features	Applicability
Harmony	Iterative clustering and integration	Removes technical variation while preserving biological diversity; suitable for large datasets [55]	scRNA-seq, cross-dataset integration
Mutual Nearest Neighbors (MNN)	Nearest neighbor matching	Identifies mutual nearest neighbors across batches to correct expression values [55]	scRNA-seq, cross-platform data
LIGER	Integrative non-negative matrix factorization (NMF)	Jointly factorizes multiple datasets to identify shared and dataset-specific factors [55]	Multi-modal data integration
Seurat Integration	Canonical Correlation Analysis (CCA) and mutual nearest neighbors	Anchors identification across datasets for label transfer and integration [55] [58]	scRNA-seq, cross-modality annotation
Bridge Integration	Multimodal anchoring	Uses paired multi-omic data as a bridge to connect unimodal datasets without gene activity calculation [58]	scATAC-seq to scRNA-seq annotation

Performance Comparison of Batch Correction Methods

Benchmarking studies have evaluated these methods under various conditions to assess their effectiveness in real-world scenarios. In evaluations focusing on scATAC-seq data annotation, Bridge integration demonstrated robust performance across different data sizes, mislabeling rates, and sequencing depths, outperforming other methods in overall accuracy for complex human datasets [58]. scJoint showed strong performance for mouse tissues but tended to assign cells to similar cell types in datasets with deep annotations [58].

For general scRNA-seq annotation, a comprehensive benchmark of 22 classification methods revealed that Support Vector Machine (SVM) classifiers consistently achieved high performance across diverse datasets [44]. Methods with rejection options, such as SVMrejection, scmapcell, and scPred, can assign cells as "unlabeled" when classification confidence is low, potentially reducing misannotation at the cost of leaving some cells unclassified [44].

Experimental Design for Assessing Batch Effects

Quality Control and Preprocessing

Robust assessment of batch effects begins with stringent quality control (QC) and preprocessing steps. Essential QC metrics include:

• Number of detected genes per cell: Filters out low-quality cells [10]
• Total molecule count: Assesses sequencing depth [10]
• Mitochondrial gene percentage: Identifies stressed or dying cells [10]
• Proportion of spike-in reads: Detects technical artifacts [56]

Visual inspection of capture sites, as performed in Fluidigm C1 platform studies, combined with data-driven filtering based on the expression profiles of empty wells, significantly enhances quality assessment [56]. After QC, normalization accounts for technical variables like sequencing depth and library preparation efficiency.

Experimental Designs for Technical Variability Assessment

Well-designed experiments enable accurate estimation of technical variability:

• Technical replicates: Processing aliquots of the same biological sample through separate single-cell workflows [56]
• Balanced batch designs: Distributing biological conditions across processing batches to avoid confounding [55]
• Multiplexing libraries: Pooling libraries across flow cells to distribute technical variation [55]
• Control materials: Using external RNA controls (ERCC spike-ins) and unique molecular identifiers (UMIs) to quantify technical noise [56]

A well-executed experimental design for assessing technical variability in iPSC lines involved three independent C1 collections per individual, with both ERCC spike-in controls and UMIs incorporated into sample processing [56]. This design enabled researchers to distinguish technical variation from biological differences between individuals.

Benchmarking Frameworks for Annotation Methods

Evaluation Metrics and Performance Assessment

Standardized evaluation metrics are essential for comparing annotation methods across studies. Key metrics include:

• Overall accuracy: Proportion of correctly classified cells [58] [44]
• Weighted accuracy: Considers similarity between cell types in prediction probability vectors [58]
• F1-score (macro): Harmonic mean of precision and recall [58] [44]
• Percentage of unclassified cells: Indicates classifier confidence and rejection rate [44]

Performance evaluation should assess both intra-dataset (cross-validation within dataset) and inter-dataset (across datasets) prediction accuracy [44]. Intra-dataset evaluations provide ideal scenarios for assessing methodological aspects, while inter-dataset tests reflect realistic application conditions where technical variability between references and queries exists [44].

Special Considerations for T Cell Annotation

T cell annotation presents unique challenges due to the continuous nature of T cell states and the co-expression of multiple gene expression programs (GEPs) within individual cells [3]. Traditional clustering approaches often fail to delineate canonical T cell subsets because they discretize continuous states [3]. Component-based models like nonnegative matrix factorization (NMF) and the recently developed T-CellAnnoTator (TCAT) pipeline better capture this complexity by modeling GEPs as additive components within each cell [3].

The TCAT pipeline, applied to 1.7 million T cells from 700 individuals across 38 tissues, identified 46 reproducible GEPs reflecting core T cell functions including proliferation, cytotoxicity, exhaustion, and effector states [3]. This approach enables more precise characterization of T cell activation states that predict response to immune checkpoint inhibitors across multiple tumor types [3].

Figure 1: Workflow for addressing batch effects in single-cell annotation

Practical Application to T Cell Subset Research

Case Study: Annotating T Cell States Across Datasets

Applying batch correction methods to T cell data requires special considerations. A recent study analyzing 1.7 million T cells from multiple tissues and disease contexts developed a specialized approach to handle batch effects while preserving subtle T cell state differences [3]. The researchers adapted Harmony to provide batch-corrected nonnegative gene-level data compatible with component-based models like NMF, which are particularly suited for T cell states [3].

The resulting starCAT framework enables quantification of predefined GEP activities in new query datasets, maintaining consistent cell state representation across studies [3]. This approach successfully identified 46 consensus GEPs capturing T cell subsets, activation states, and functions, demonstrating better cross-dataset reproducibility than principal components [3].

Performance Across Sequencing Platforms

Different scRNA-seq platforms exhibit distinct technical characteristics that influence annotation accuracy. 10x Genomics droplet-based methods enable high-throughput profiling but produce sparser data, while Smart-seq2 full-length methods offer higher gene detection sensitivity but at lower throughput [10]. These technical differences significantly impact annotation performance, particularly for rare cell types where detection of key marker genes may be platform-dependent [10].

Benchmarking studies reveal that methods like SVM and SingleR maintain robust performance across platforms, while others show platform-specific biases [44] [13]. For spatial transcriptomics technologies like 10x Xenium, which profile only several hundred genes, SingleR has demonstrated superior performance compared to other annotation methods [13].

Research Reagent Solutions

Essential reagents and computational tools for managing batch effects:

Resource	Type	Function	Example Applications
ERCC Spike-in Controls	Synthetic RNA mixtures	Quantify technical variation and normalization [56]	scRNA-seq protocol optimization
Unique Molecular Identifiers (UMIs)	Molecular barcodes	Correct for amplification bias by counting molecules [56]	Accurate quantification of gene expression
CellMarker Database	Marker gene repository	Reference for cell type annotation and validation [10]	Annotation of known cell types
Harmony	Computational algorithm	Batch effect correction for large datasets [55]	Integrating samples across multiple batches
Seurat	R toolkit	Single-cell analysis including integration methods [55]	Standard scRNA-seq analysis workflow
VDJdb	TCR specificity database	Reference for T cell receptor annotation [59]	Identifying antigen-specific T cells
ePytope-TCR	TCR-epitope prediction framework	Predict binding between TCRs and epitopes [59]	TCR specificity profiling

Addressing batch effects and technical variability is essential for robust single-cell annotation, particularly in T cell research where states exist along a continuum. Based on current benchmarking evidence:

• For standard scRNA-seq annotation, SVM-based classifiers and SingleR provide consistently strong performance across diverse datasets [44] [13].
• For cross-modality annotation (e.g., scATAC-seq to scRNA-seq), Bridge integration leveraging multimodal data as a bridge outperforms methods requiring gene activity calculation [58].
• For T cell-specific applications, component-based models like TCAT that quantify gene program activities better capture biological complexity than discrete clustering [3].
• Experimental design remains crucial—technical replicates, balanced batch designs, and control molecules (UMIs, spike-ins) provide necessary data for effective batch correction [56] [55].

As single-cell technologies continue to evolve, maintaining standardized benchmarking frameworks will be essential for validating new computational methods against these established approaches.

Strategies for Handling Rare Cell Populations and Unconventional T Cells

Single-cell RNA sequencing (scRNA-seq) has revolutionized our understanding of cellular heterogeneity, enabling researchers to dissect the complex landscape of immune cells at unprecedented resolution. Within the adaptive immune system, T cells exhibit remarkable diversity, not only in their conventional αβ T cell subsets but also in the more enigmatic unconventional T cell populations. These unconventional T cells—including mucosal-associated invariant T (MAIT) cells, γδ T cells, invariant natural killer T (iNKT) cells, and double-negative T cells—possess unique antigen recognition mechanisms and function at the interface of innate and adaptive immunity. However, their accurate identification and characterization in scRNA-seq data present significant challenges due to their rarity, phenotypic plasticity, and overlapping marker expression with conventional T cells.

The accurate annotation of these rare and unconventional T cell populations is critical for advancing our understanding of immune responses in cancer, infectious diseases, and autoimmune disorders. Traditional clustering approaches often fail to resolve these populations, as they discretize continuous cellular states and struggle with cells that co-express multiple gene programs. This limitation has driven the development of specialized computational methods that can better capture the complexity of T cell phenotypes and functions. This review provides a comprehensive comparison of current computational strategies for annotating rare and unconventional T cells, evaluating their performance, experimental requirements, and applicability to different research scenarios.

Computational Methodologies for Cell Annotation

Classification of Annotation Approaches

Computational methods for single-cell annotation can be broadly categorized into four main classes based on their underlying principles and implementation. Specific gene expression-based methods utilize known marker gene information to manually label cells by identifying characteristic gene expression patterns of specific cell types [10]. Reference-based correlation methods categorize unknown cells into corresponding known cell types based on the similarity of gene expression patterns to those in a preconstructed reference library [10]. Data-driven reference methods predict cell types by training classification models on pre-labeled cell type datasets [10]. Large-scale pretraining-based methods use large-scale unsupervised learning to capture deep relationships between cell types by studying generic cell features and gene expression patterns [10].

Each approach presents distinct advantages and limitations for identifying rare and unconventional T cell populations. Marker-based methods offer simplicity and interpretability but struggle with novel cell states and populations that lack well-defined markers. Reference-based methods provide standardization and reproducibility but depend heavily on the quality and comprehensiveness of the reference atlas. Recently, component-based models like nonnegative matrix factorization (NMF) have emerged as powerful alternatives that model gene expression programs (GEPs) as gene expression vectors and transcriptomes as weighted mixtures of GEPs [3]. Unlike principal component analysis (PCA), NMF components correspond to biologically interpretable GEPs reflecting cell types and functional states that additively contribute to a transcriptome, making them particularly suitable for capturing the complex phenotypes of unconventional T cells.

Specialized Tools for Rare Cell Population Annotation

Table 1: Computational Methods for Annotating Rare and Unconventional T Cells

Method	Approach	Strengths	Limitations	Recommended Use Cases
TCAT/starCAT	Component-based (cNMF) with predefined GEP catalog	Quantifies multiple co-expressed programs; identifies 46 reproducible T cell cGEPs; cross-dataset compatibility	Complex implementation; requires large reference data	Comprehensive T cell state analysis across multiple datasets
SingleR	Reference-based correlation	Fast, accurate, easy to use; outperforms other methods in benchmarking	Limited to cell types in reference; struggles with novel populations	Rapid annotation with available high-quality reference
Azimuth	Reference-based integration	Leverages SCTransform normalization; robust to technical variance	Computationally intensive; requires reference building	Integrating new data with existing atlas frameworks
scPred	Supervised machine learning	Probabilistic classification; confidence scores	Requires extensive training data; performance depends on feature selection	When confident training set is available
RCTD	Spatial mapping	Designed for spatial transcriptomics; accounts for cellular mixtures	Optimized for sequencing-based spatial data	Mapping unconventional T cells in tissue contexts

The T-CellAnnoTator (TCAT) pipeline represents a specialized approach specifically designed for T cell characterization that simultaneously quantifies predefined gene expression programs (GEPs) capturing activation states and cellular subsets [3]. By analyzing 1.7 million T cells from 700 individuals across 38 tissues and five disease contexts, TCAT identified 46 reproducible GEPs reflecting core T cell functions including proliferation, cytotoxicity, exhaustion, and effector states. The method improves upon traditional clustering by modeling transcriptomes as additive combinations of GEPs, thereby capturing the continuous nature of T cell states. The broader software package starCAT generalizes this framework, enabling reproducible annotation in other cell types and tissues.

For spatial transcriptomics data, particularly imaging-based platforms like 10x Xenium with limited gene panels, reference-based methods face additional challenges. A recent benchmarking study demonstrated that SingleR outperformed other methods including Azimuth, RCTD, scPred, and scmapCell for Xenium data, with results closely matching manual annotation [13]. This performance advantage makes SingleR particularly valuable for identifying rare immune populations in spatial contexts where gene information is limited.

Experimental Design and Workflow Considerations

Sample Preparation and Data Generation

The accurate annotation of rare and unconventional T cell populations begins with appropriate experimental design and sample preparation. Tissue-specific distribution patterns significantly impact the detection likelihood of these populations—γδ T cells preferentially localize to epithelial-rich environments such as the intestines, respiratory mucosa, and reproductive tissues, representing only 1–5% of T cells in peripheral blood [60]. MAIT cells are predominantly found in mucosal tissues and liver, while iNKT cells are most abundant in the liver and adipose tissue. These distribution patterns should inform tissue selection when studying specific unconventional T cell subsets.

The choice of sequencing platform substantially impacts annotation outcomes. Droplet-based methods like 10x Genomics enable profiling of large cell numbers but yield sparser data, potentially missing critical marker genes for rare populations [10]. Full-transcriptome methods like Smart-seq2 provide greater sensitivity for detecting weakly expressed genes but at lower throughput and higher cost. For comprehensive atlas construction, researchers often employ multiple technologies to balance depth and breadth, though this introduces integration challenges. When studying unconventional T cells specifically, targeted approaches that enrich for these populations through cell sorting or antibody-based capture can significantly enhance detection resolution.

Quality control steps must be carefully implemented to preserve rare cell populations. Standard threshold-based filtering approaches risk eliminating genuine rare populations misclassified as doublets or low-quality cells. Adaptive QC models and data-driven thresholding provide more nuanced alternatives [26]. Doublet detection tools like Scrublet or scDblFinder should be used with caution, as their parameters may need adjustment to prevent excessive removal of true rare cell types [26] [13]. Additionally, the inclusion of protein markers through CITE-seq can significantly enhance GEP interpretability and annotation accuracy for unconventional T cells, as demonstrated in the TCAT framework [3].

Reference Atlas Construction and Integration

The construction of comprehensive reference atlases is fundamental for accurate annotation of rare immune populations. A well-constructed T cell atlas should capture cellular diversity across tissues, developmental stages, and disease conditions [26]. The scope must balance breadth and depth—too narrow a scope misses relevant cellular states, while too broad a scope introduces batch effects that obscure biological signals. The Human Cell Atlas, Immune Cell Atlas, and Human Cell Landscape provide valuable resources, but may lack sufficient representation of tissue-specific unconventional T cell states [10].

When building custom references, several key considerations optimize performance for rare population detection. Cross-dataset integration should incorporate harmony or similar algorithms to correct technical variation while preserving biological heterogeneity [61]. Stratified sampling ensures adequate representation of rare populations, potentially requiring oversampling of tissues where these cells are enriched. Metadata standardization enables effective query matching, particularly for activation states and tissue origins that influence unconventional T cell phenotypes. Multi-omics integration combining transcriptomic, epigenetic, and protein data enhances resolution for distinguishing closely related cell states.

Table 2: Experimental Protocols for Unconventional T Cell Analysis

Protocol Step	Key Considerations	Recommendations for Rare Populations
Tissue Processing	Dissociation method impacts viability and gene expression	Enzymatic combinations that preserve surface markers; minimize stress responses
Cell Enrichment	Selection strategies affect population representation	FACS sorting with multiple surface markers; minimal activation during processing
Library Preparation	Platform choice balances depth and throughput	Targeted approaches for known subsets; full transcriptome for discovery
Sequencing Depth	Coverage requirements for rare population detection	50,000-100,000 reads/cell for conventional populations; increased depth for rare subsets
Multiplexing	Sample indexing and pooling	Hashtag antibodies (CITE-seq) to track samples without separate libraries

For unconventional T cells specifically, references should include comprehensive representation of their diverse functional states. MAIT cells exhibit context-dependent polarization, with distinct transcriptional profiles in cancer versus infection [62] [63]. γδ T cells encompass functionally specialized subsets (Vδ1, Vδ2, Vδ3) with different tissue distributions and effector functions [60]. iNKT cells show functional heterogeneity with differential cytokine production capacities. References that capture this diversity enable more precise annotation and functional interpretation.

Analytical Framework for Unconventional T Cells

Methodological Strategies for Rare Population Identification

The identification and annotation of unconventional T cells requires specialized analytical approaches that address their unique characteristics. Gene expression program analysis using methods like TCAT has proven particularly valuable, as it identified 46 consensus GEPs in T cells including programs specific to unconventional subsets [3]. This approach enables the quantification of multiple co-expressed biological programs within individual cells, capturing the innate-like activation pathways and effector functions that characterize unconventional T cells.

Multi-tiered annotation strategies that combine automated methods with expert validation yield the most reliable results for rare populations. An effective workflow begins with broad classification using reference-based methods (e.g., SingleR) followed by subclustering of T cell populations and application of specialized tools like TCAT for fine-grained annotation. Marker-based validation then confirms population identities using established markers: γδ T cells (TRDC, TRGC1/2), MAIT cells (TRAV1-2, SLC4A10, KLRB1), and iNKT cells (TRAV10, TRAJ18) [60] [62]. This combined approach leverages the scalability of automated methods with the biological precision of marker-based verification.

Continuous monitoring of annotation confidence is critical when working with rare populations. Methods like scPred provide probability scores for cell type assignments, enabling researchers to set thresholds for high-confidence annotations and flag ambiguous cells for further investigation [13]. For unconventional T cells specifically, cross-referencing with T cell receptor (TCR) sequencing data can validate identities—for example, confirming MAIT cells through their canonical TRAV1-2-TRAJ33 TCR rearrangement [62]. This multi-modal verification is particularly important given the phenotypic plasticity and context-dependent gene expression of these populations.

Addressing the Long-Tail Distribution Challenge

The "long-tail" distribution problem—where rare cell types are underrepresented in reference datasets—presents a fundamental challenge for unconventional T cell annotation. Several strategies address this limitation. Transfer learning approaches fine-tune models pre-trained on large atlases to specific tissues or conditions where unconventional T cells may be enriched. Data augmentation techniques synthetically increase representation of rare populations by creating perturbed versions of existing cells, improving classifier performance. Few-shot learning methods specifically designed for low-abundance cell types can identify populations represented by only a handful of cells in reference data.

When encountering potentially novel unconventional T cell states, open-world recognition frameworks differentiate between known classes and genuinely novel populations [10]. Methods like SCTrans leverage transformer architectures with self-attention mechanisms to identify discriminative gene combinations that may represent previously uncharacterized cell states [10]. These approaches are particularly valuable for unconventional T cell biology, where new functional states and subsets continue to be discovered across different tissue environments and disease contexts.

Diagram 1: Analytical workflow for identifying rare and unconventional T cell populations, showing progression from general processing to specialized T cell analysis and rare population resolution.

Functional Characterization of Unconventional T Cells

Context-Dependent Functional States

Unconventional T cells exhibit remarkable functional plasticity that must be considered during annotation and interpretation. MAIT cells demonstrate dual roles in cancer immunity—displaying potent cytotoxicity against multiple myeloma cell lines in some contexts [63], while exhibiting exhausted phenotypes (PD-1^high^Tim-3^+^CD39^+^) in hepatocellular carcinoma associated with poor clinical outcomes [63]. Similarly, γδ T cells can switch between pro-inflammatory and regulatory functions based on local microenvironmental cues [60]. These functional states are reflected in their gene expression programs, which can be quantified using component-based approaches like TCAT.

The functional characterization of unconventional T cells benefits greatly from integrated analysis of transcriptomic data with TCR sequencing. For MAIT cells, confirmation of their canonical TCRα rearrangement (TRAV1-2-TRAJ33 in humans) validates identity while transcriptomics reveals functional polarization [62]. For γδ T cells, pairing Vδ chain usage (Vδ1, Vδ2, Vδ3) with transcriptional programs links repertoire to function—Vδ2 T cells typically respond to phosphoantigens while Vδ1 T cells often exhibit tissue-resident properties [60]. These multi-modal approaches move beyond simple classification to functional assessment of unconventional T cell states.

Spatial Localization and Cell-Cell Interactions

Spatial context profoundly influences unconventional T cell function, making spatial transcriptomics particularly valuable for their characterization. MAIT cells localize to mucosal barriers where they interact with commensal bacteria and epithelial cells [63]. γδ T cells are enriched in epithelial tissues where they function in tissue surveillance and repair [60]. iNKT cells accumulate in adipose tissue and liver where they modulate metabolic inflammation [62]. Understanding these spatial distributions informs both experimental design and analytical interpretation.

Cell-cell communication analysis tools like CellChat can reconstruct interaction networks between unconventional T cells and their microenvironment [61]. In cancer contexts, these analyses have revealed immunosuppressive interactions between exhausted MAIT cells and myeloid cells [63], as well as activating interactions between γδ T cells and dendritic cells [60]. For spatial transcriptomics data, methods like RCTD can map unconventional T cells to tissue locations, revealing their spatial relationships with other immune and stromal cells [13]. These approaches contextualize unconventional T cell functions within tissue microenvironments.

Research Reagent Solutions

Table 3: Essential Research Reagents for Unconventional T Cell Studies

Reagent Category	Specific Examples	Research Application	Technical Considerations
Surface Markers for FACS	TCRγδ, Vδ2, Vδ1 (γδ T cells); TRAV1-2 (MAIT cells); CD1d tetramers (iNKT cells)	Isolation and validation of unconventional T cell subsets	Multi-color panels required due to shared markers; activation-sensitive epitopes
Functional Assays	5-OP-RU-MR1 tetramers (MAIT cells); α-GalCer-loaded CD1d tetramers (iNKT cells); phosphoantigen stimulation (Vδ2 T cells)	Functional characterization and antigen specificity	Tetramer quality critical; appropriate positive and negative controls
Reference Databases	CellMarker 2.0; PanglaoDB; Immune Cell Atlas; TCAT cGEP catalog	Annotation and marker validation	Regular updates needed; platform-specific expression patterns
CITE-seq Antibodies	CD3, CD4, CD8α, CD161, TCRγδ, TCR Vα7.2 (MAIT), CD45, CD69, PD-1	Multi-modal validation of cell identities	Titration required to minimize background; isotype controls essential
Activation Stimuli	IL-12+IL-18 (MAIT cells); IPP/HMBPP (Vδ2 T cells); α-GalCer (iNKT cells)	Functional assessment and expansion	Dose optimization required; cytokine production measured after 4-6 hours

The selection of appropriate research reagents is critical for successful unconventional T cell studies. Validated antibody panels must account for shared surface markers—CD161 is expressed by both MAIT cells and some conventional T cells, requiring additional specificity markers for unambiguous identification [63]. Antigen-loaded tetramers provide the highest specificity for detecting unconventional T cells with defined antigen specificity, particularly for MAIT cells (MR1-5-OP-RU tetramers) and iNKT cells (CD1d-α-GalCer tetramers) [62]. Functional assay reagents should accommodate the unique activation requirements of different unconventional T cell subsets, including cytokine combinations (IL-12+IL-18 for MAIT cells) and metabolic antigens (phosphoantigens for Vδ2 T cells) [60] [63].

For computational annotation, comprehensive reference datasets must include appropriate representation of unconventional T cell states across tissues and conditions. The TCAT framework provides a catalog of 46 consensus GEPs derived from 1.7 million T cells across 38 tissues, offering a robust foundation for identifying unconventional T cell states [3]. Supplementing with tissue-specific references—particularly from mucosal sites, liver, and adipose tissue where unconventional T cells are enriched—improves annotation accuracy for these specialized populations.

The accurate annotation of rare and unconventional T cell populations requires specialized methodological approaches that address their unique characteristics, including innate-like activation pathways, tissue-specific distributions, and context-dependent functional plasticity. Component-based methods like TCAT that quantify gene expression programs offer significant advantages over traditional clustering for capturing the continuous and mixed states of unconventional T cells. Reference-based methods like SingleR provide robust annotation when comprehensive references are available, while spatial methods like RCTD enable contextualization within tissue microenvironments.

Future methodological developments will likely focus on several key areas. Multi-omics integration combining transcriptomic, epigenetic, and proteomic data will enhance resolution for distinguishing closely related unconventional T cell states. Dynamic modeling approaches will better capture the functional plasticity and state transitions that characterize these populations. Universal representation learning will address the long-tail distribution problem by enabling effective knowledge transfer across datasets and conditions. As single-cell technologies continue to evolve, so too will our ability to resolve and characterize the full diversity of unconventional T cells in health and disease.

The strategic selection of annotation methods should be guided by specific research questions, sample types, and available references. For discovery-focused studies of unconventional T cell biology, component-based approaches like TCAT offer the greatest insights into cellular states and functions. For clinical applications where standardization and reproducibility are prioritized, reference-based methods like SingleR provide more consistent performance. In all cases, multi-modal validation incorporating protein expression, TCR sequencing, and functional assays remains essential for confirming the identities and states of these enigmatic immune cells.

Dealing with Missing Markers and Gene Dropout in scRNA-seq Data

Single-cell RNA sequencing (scRNA-seq) has revolutionized our ability to dissect cellular heterogeneity within complex tissues, proving particularly valuable for characterizing the diverse and dynamic nature of T cell populations in health and disease [42]. However, a significant technical challenge persists: the prevalent issue of gene dropout, where transcripts are observed in some cells but not detected in others of the same type [57] [64]. This phenomenon results in excessively sparse data, with missing rates averaging 90% at the individual cell level [65]. For T cell research, this poses a critical problem for accurate cell type annotation, as dropouts can affect key marker genes used to distinguish between closely related T cell subsets such as TH1, TH2, TH17, and various cytotoxic and exhausted T cell populations [3] [42].

The implications of dropout events extend beyond mere missing data. High dropout rates can break the assumption that similar cells are close in expression space, thereby compromising the stability of clustering results and potentially obscuring biologically relevant T cell subpopulations [66]. This technical variability, compounded by batch effects and the inherent complexity of T cell phenotypes, necessitates specialized computational approaches to recover true biological signals and enable precise cellular annotation [57] [42].

Computational Strategy Comparison: Performance Benchmarking

Multiple computational strategies have been developed to address the dropout challenge, each with distinct methodological approaches and performance characteristics. The table below summarizes the key methods, their underlying algorithms, and their applicability to T cell research.

Table 1: Comparison of Computational Methods for Handling scRNA-seq Dropouts

Method	Core Algorithm	Primary Strategy	Key Advantages	T Cell Application Evidence
cnnImpute [67]	Convolutional Neural Network	Missing value imputation	High accuracy (Pearson R), preserves cell clusters	Not specifically validated on T cells
TCAT/starCAT [3]	Consensus NMF (cNMF)	Fixed GEP catalog & annotation	Quantifies 46 reproducible T cell GEPs; predicts immunotherapy response	Specifically designed for T cells; validated on 1.7M cells
Co-occurrence Clustering [64]	Binary pattern analysis	Utilizes dropout patterns as signal	Identifies major cell types without imputation; pathway-based	Demonstrated on PBMC data containing T cells
MAGIC [67]	Graph-based diffusion	Data smoothing & imputation	Preserves global expression patterns	General purpose, not T cell-specific
scImpute [67]	Mixture modeling	Targeted dropout imputation	Borrows information from similar cells	General purpose, not T cell-specific
DeepImpute [67]	Neural network	Missing value recovery	Fast, high accuracy	General purpose, not T cell-specific

Quantitative Performance Assessment

Benchmarking studies provide crucial insights into the practical performance of these methods. In comprehensive evaluations using Jurkat T cell data, methods demonstrated varying capabilities in recovering missing values:

Table 2: Quantitative Benchmarking of Imputation Methods on scRNA-seq Data

Method	Mean Square Error (MSE)	Pearson Correlation Coefficient (PCC)	Runtime Efficiency	Cluster Stability Improvement
cnnImpute	Lowest (P < 0.014)	Highest (P < 0.014)	Moderate	Not reported
DeepImpute	Low	High	Fast	Not reported
DCA	Low	High	Slow	Not reported
MAGIC	Moderate	Moderate	Fast	Limited with high dropouts [66]
scImpute	Moderate	Moderate	Moderate	Limited with high dropouts [66]
SAVER	Moderate	Low	Moderate	Not reported
Default Clustering	N/A	N/A	N/A	Poor with high dropouts [66]

For T cell-specific annotation, TCAT/starCAT represents a specialized approach that simultaneously quantifies predefined gene expression programs (GEPs) capturing T cell subsets, activation states, and functions [3]. This method identified 46 reproducible GEPs across 1.7 million T cells from 700 individuals, spanning 38 tissues and five disease contexts, demonstrating exceptional utility for deciphering complex T cell biology beyond what standard clustering approaches can achieve [3].

Experimental Protocols for Robust T Cell Annotation

TCAT/starCAT Pipeline for T Cell GEP Quantification

The TCAT (T-CellAnnoTator) pipeline employs a sophisticated workflow for reproducible T cell annotation:

• Reference GEP Catalog Construction: Apply consensus nonnegative matrix factorization (cNMF) to multiple scRNA-seq datasets to identify robust gene expression programs. The algorithm has been augmented with Harmony integration for batch correction while maintaining nonnegative values essential for biological interpretation [3].

• GEP Usage Quantification: Project query datasets onto the reference GEP catalog using nonnegative least squares to quantify program activities in new cells. This ensures consistent cell state representation across datasets and enables quantification of rare GEPs that might be missed in smaller datasets [3].

• Cell Feature Prediction: Leverage GEP usages to predict additional cell features including lineage, T cell antigen receptor (TCR) activation, and cell cycle phase [3].

• Experimental Validation: The method has been experimentally validated to demonstrate novel activation programs and applied to characterize activation GEPs that predict immune checkpoint inhibitor response across multiple tumor types [3].

Co-occurrence Clustering Using Dropout Patterns

This innovative approach treats dropouts as biological signals rather than noise:

• Data Binarization: Transform the scRNA-seq count matrix into binary representation (0 = dropout, 1 = detected) to capture the dropout pattern [64].

• Gene-Gene Co-occurrence Analysis: Compute statistical measures for co-occurrence between gene pairs, identifying genes that tend to be co-detected in common cell subsets [64].

• Pathway Signature Identification: Partition the gene-gene graph using community detection (e.g., Louvain algorithm) to identify gene clusters/pathways with high co-occurrence [64].

• Pathway Activity Representation: For each gene pathway, calculate the percentage of detected genes per cell to create a low-dimensional activity representation [64].

• Cell Clustering: Build a cell-cell graph based on pathway activity distances and apply community detection to identify cell clusters with distinct dropout patterns [64].

This method has successfully identified major cell types in PBMC datasets based solely on dropout patterns, performing comparably to methods using quantitative expression of highly variable genes [64].

Table 3: Key Research Reagent Solutions for scRNA-seq Dropout Mitigation

Resource Type	Specific Tool/Platform	Application Context	Performance Considerations
Reference Databases	TCAT GEP Catalog [3]	T cell subset annotation	46 reproducible GEPs across 38 tissues
Annotation Tools	SingleR [13]	General cell type annotation	Best performance in Xenium spatial data
Annotation Tools	Azimuth [13]	Reference-based mapping	Requires UMAP model integration
Annotation Tools	scGate [42]	Marker-based annotation	Flow cytometry-like gating strategy
Experimental Platforms	10x Genomics [65] [68]	High-throughput scRNA-seq	Higher throughput but increased sparsity
Experimental Platforms	Smart-seq2 [65]	Full-length transcriptome	Higher sensitivity but lower throughput
Quality Control Tools	VICE [65]	Data quality evaluation	Estimates true positive rate of DE results
Spatial Technologies	10x Xenium [13]	Spatial transcriptomics	Small gene panel (300-500 genes)

Integrated Workflow Recommendations for T Cell Researchers

Based on the benchmarking data and methodological comparisons, an optimal workflow for addressing dropouts in T cell scRNA-seq data should incorporate:

Method Selection Guidelines

• For Comprehensive T Cell Subset Discovery: Implement TCAT/starCAT as a specialized framework for identifying and quantifying T cell-specific gene expression programs, particularly when working with large-scale datasets across multiple conditions or tissues [3].

• For Standard Imputation Needs: Apply cnnImpute for general missing value recovery due to its superior accuracy in benchmarking studies, while being mindful of potential over-smoothing of biological heterogeneity [67].

• For Exploring Rare Populations: Consider co-occurrence clustering when analyzing complex T cell populations where traditional highly-variable-gene approaches may miss biologically important subsets [64].

• For Spatial Transcriptomics: Utilize SingleR for cell type annotation in imaging-based spatial data like Xenium, where small gene panels necessitate robust reference mapping [13].

Experimental Design Considerations

To minimize the impact of dropouts at the source, researchers should:

• Ensure adequate cell numbers, with at least 500 cells per cell type per individual recommended for reliable quantification [65]
• Account for platform-specific characteristics, as 10x Genomics data typically exhibits higher sparsity than Smart-seq2 data [65]
• Implement rigorous quality control procedures to remove low-quality cells while preserving biological heterogeneity [68]
• Apply appropriate normalization strategies such as SCTransform or regularized negative binomial regression to address technical variability [68]

The prevalence of gene dropout in scRNA-seq data presents both a challenge and an opportunity for T cell researchers. While traditional approaches often treat dropouts as technical noise to be eliminated or corrected, emerging strategies demonstrate the value of embracing dropout patterns as biologically informative signals [64]. The benchmarking data presented here reveals that method selection should be guided by specific research goals: TCAT/starCAT for comprehensive T cell program discovery, cnnImpute for general-purpose imputation with high accuracy, and co-occurrence clustering for detecting rare populations that might be overlooked by conventional approaches.

As single-cell technologies continue to evolve, integrating multiple complementary approaches—combined with careful experimental design and appropriate quality control—will provide the most robust solutions for unraveling the complexity of T cell populations in health and disease. Future directions will likely involve tighter integration of imputation methods with specialized annotation frameworks and the development of platforms specifically optimized for the challenging characteristics of adaptive immune cell transcriptomes.

Optimizing Parameters for Complex T Cell States and Activation Programs

Single-cell RNA sequencing (scRNA-seq) has revolutionized our understanding of T cell biology, revealing a continuum of cellular states that defy traditional classification into discrete subsets. The prevailing method of unsupervised clustering followed by manual annotation has proven inadequate for capturing the complex, co-expressed gene programs underlying T cell activation, differentiation, and function [3] [69]. This limitation is particularly problematic in therapeutic contexts like cancer immunotherapy and autoimmune disease, where precise identification of T cell states can predict treatment response and guide intervention strategies.

Within this benchmarking framework, we evaluate computational annotation methods that move beyond traditional clustering to address T cell heterogeneity. We compare the performance of established and emerging tools, focusing on their accuracy in identifying predefined T cell subsets and activation states, their reproducibility across datasets, and their utility in predicting clinically relevant T cell functions.

Methodological Comparison: Annotation Approaches for T Cell Biology

Beyond Clustering: New Frameworks for T Cell Annotation

Traditional clustering approaches discretize cells into artificial categories, obscuring the co-expressed gene programs that reflect distinct biological functions. This often fails to delineate canonical T cell subsets, with clusters frequently mixing CD4+ and CD8+ T cells despite their distinct biological roles [3] [69]. The factors driving standard clustering are often related to technical artifacts (TCR sequences, immunoglobulin transcripts) rather than biologically meaningful T cell phenotypes.

Component-based models like nonnegative matrix factorization (NMF) overcome these limitations by modeling gene expression programs (GEPs) as vectors and transcriptomes as weighted mixtures of these GEPs [3]. Unlike principal component analysis, NMF components correspond to biologically interpretable programs reflecting cell types and functional states that additively contribute to a cell's transcriptome.

Table 1: Comparison of T Cell Annotation Methodologies

Method Type	Representative Tools	Underlying Principle	Advantages	Limitations
Unsupervised Clustering	Seurat	Groups cells based on gene expression similarity	Widely adopted, no prior knowledge required	Obscures co-expressed programs, poor subset discrimination [69]
Component-Based Models	cNMF, SPECTRA	Decomposes expression matrices into interpretable programs	Identifies biologically meaningful GEPs, handles continuous states [3]	Computational intensity, parameter sensitivity
Reference-Based Annotation	SingleR, Azimuth, starCAT/TCAT	Projects query data onto reference datasets	Consistent cross-dataset comparison, rapid analysis [3] [13]	Reference quality dependency, may miss novel states
Multimodal Integration	CITE-seq enabled methods	Combines RNA with protein surface markers	Enhanced interpretability, validation with protein expression [3]	Increased cost, technical complexity

The starCAT/TCAT Framework for Reproducible Annotation

The T-CellAnnoTator (TCAT) pipeline introduces a specialized framework for T cell characterization that simultaneously quantifies predefined GEPs capturing activation states and cellular subsets. Its generalized counterpart, starCAT, extends this approach to other cell types and tissues [3]. The methodology involves:

• Reference Catalog Construction: Applying consensus NMF (cNMF) to large-scale collections of T cells (1.7 million cells from 700 individuals across 38 tissues and five diseases) to identify robust GEPs.
• Batch Effect Correction: Adapting Harmony integration to provide batch-corrected, nonnegative gene-level data compatible with cNMF requirements.
• Query Projection: Using starCAT to infer activities of predefined reference GEPs in new datasets via nonnegative least squares, enabling consistent cross-dataset comparisons.

This approach identifies 46 reproducible GEPs reflecting core T cell functions including proliferation, cytotoxicity, exhaustion, and effector states. Experimental validation has confirmed new activation programs and identified GEPs predictive of immune checkpoint inhibitor response across multiple tumor types [3].

Performance Benchmarking: Accuracy and Reproducibility

Cross-Dataset Reproducibility and State Detection

Benchmarking analyses demonstrate significant advantages of program-based approaches over traditional methods. In evaluations across seven datasets spanning 1.7 million T cells, GEPs showed high cross-dataset reproducibility, with nine consensus GEPs (cGEPs) supported by all seven datasets (mean Pearson R = 0.81) and 49 by two or more datasets [3]. This substantially exceeds the concordance of traditional gene expression principal components across datasets.

The starCAT framework maintains performance even when reference and query datasets have partially overlapping GEPs, accurately inferring usage of overlapping GEPs (Pearson R > 0.7) while correctly predicting low usage of non-overlapping programs [3]. This robustness is particularly valuable for analyzing small query datasets where de novo identification of rare GEPs would be challenging.

Table 2: Quantitative Performance Metrics of Annotation Methods

Performance Metric	Unsupervised Clustering	Component-Based Models	Reference-Based Projection
Cross-Dataset Reproducibility	Low (technical artifact-driven) [69]	High (mean R = 0.74-0.81) [3]	High (maintains consistent GEP usage) [3]
Subset Discrimination	Poor (mixes CD4+/CD8+ T cells) [69]	Excellent (identifies 46 reproducible cGEPs) [3]	Excellent (leverages predefined subset programs)
Rare Population Detection	Limited	Moderate (depending on dataset size)	Excellent (even in small query datasets) [3]
Run Time	Fast	Computational intensive	Rapid projection once reference established [3]
Clinical Predictive Value	Limited	Demonstrated for immunotherapy response [3]	High (fixed coordinate system for comparison)

Spatial Transcriptomics Applications

With the emergence of imaging-based spatial transcriptomics platforms like 10x Xenium, benchmarking of annotation methods has extended to spatial contexts. These technologies present unique challenges due to their small gene panels (several hundred genes), making manual annotation difficult. Performance comparisons of reference-based methods on Xenium data identified SingleR as the top-performing tool, being fast, accurate, and producing results closely matching manual annotation [13].

Other reference-based methods including Azimuth, RCTD, scPred, and scmapCell showed variable performance on spatial data, with accuracy highly dependent on reference quality and parameter optimization [13]. This highlights the importance of platform-specific benchmarking when selecting annotation approaches.

Experimental Protocols for Method Validation

Protocol 1: Establishing a Reference GEP Catalog with cNMF

Application: Creating a comprehensive catalog of T cell gene expression programs for use as a reference framework.

Workflow:

• Data Collection and Curation: Aggregate multiple scRNA-seq datasets encompassing diverse biological contexts (health, infection, autoimmunity, cancer). The TCAT reference incorporated 1.7 million T cells from 38 tissues and 5 disease contexts [3].
• Quality Control and Batch Correction: Perform rigorous QC using metrics including detected genes per cell, mitochondrial gene percentage, and potential doublet identification. Apply specialized batch correction (modified Harmony) that maintains nonnegative values for NMF compatibility [3].
• Consensus NMF Application: Run multiple NMF iterations with different initializations and combine outputs into robust GEP estimates (spectra) and per-cell activities (usages) [3] [70].
• GEP Clustering and Curation: Cluster similar GEPs across datasets to define consensus GEPs (cGEPs). Curate cGEPs by examining top-weighted genes, gene-set enrichment, and association with surface protein markers when CITE-seq data is available [3].
• Experimental Validation: Confirm biological relevance of newly identified GEPs through in vitro or in vivo models. TCAT validation included identifying GEPs predictive of immune checkpoint inhibitor response [3].

Protocol 2: Query Projection with starCAT/TCAT

Application: Annotating new T cell datasets using a predefined reference catalog.

Workflow:

• Reference Alignment: Map genes between reference and query datasets, handling non-overlapping gene sets.
• GEP Usage Inference: Apply nonnegative least squares to quantify the activity of each reference GEP in every query cell.
• Cell State Prediction: Leverage GEP usages to predict additional features including lineage, TCR activation status, and cell cycle phase [3].
• Cross-Platform Validation: For spatial transcriptomics data, use paired single-nucleus RNA sequencing as reference when possible to minimize technical variability [13].
• Performance Assessment: Compare results with orthogonal validation methods such as surface protein expression (CITE-seq) or functional assays.

The Scientist's Toolkit: Essential Research Reagents and Solutions

Table 3: Key Research Reagents for T Cell Activation and Annotation Studies

Reagent/Solution	Function	Application Context
Anti-CD3/CD28 Antibodies	TCR and costimulation signaling	In vitro T cell activation, expansion for therapeutic manufacturing [71]
Functionalized Microbeads	Controlled T cell activation	Bioreactor-based CAR-T cell production with modulated exhaustion [71]
OVA323-339 Peptide	MHC-II restricted antigen presentation	Antigen-specific CD4+ T cell activation in OT-II mouse models [72]
Recombinant Cytokines (IL-2, IL-12, etc.)	Polarization and survival signals	T cell differentiation to specific helper subsets (Th1, Th2, Th17) [73]
CD6-Targeting Reagents	Modulating costimulatory signals	Investigating dual immunomodulatory roles in T cell activation [72]
Optogenetic Receptor Systems	Precise control of receptor-ligand kinetics	Dissecting temporal binding requirements for T cell activation [74]

Biological Insights: From Annotation to Activation Mechanisms

Decoding T Cell Activation Through Program-Based Annotation

Program-based annotation has revealed that T cell stimulation strength controls the rate of individual cell responses within a population rather than fundamentally altering response programs [74]. Single-cell measurements have identified both digital ("on/off") and analog (graded) activation behaviors, with some markers like IRF4 showing purely analog responses while others exhibit hybrid behaviors [74].

The application of TCAT to T cell activation has identified distinct antigen-specific activation (ASA) states and GEPs associated with response to immune checkpoint inhibitors [3]. This provides a molecular framework for understanding how T cells integrate signals from TCR engagement, costimulatory molecules, and cytokines to enact appropriate functional responses.

Clinical Translation and Therapeutic Optimization

In cancer immunotherapy, proper annotation of T cell states has direct clinical relevance. Exhaustion GEPs identified through programs like TCAT show predictive value for immune checkpoint inhibitor response [3]. In CAR-T cell manufacturing, controlled activation systems using anti-CD3/CD28-functionalized microbeads in stirred-tank bioreactors yield up to 10-fold expansion with reduced exhaustion markers compared to static cultures [71].

The emerging understanding of T cell activation as a feedback-controlled program reveals inherent trade-offs between pathogen clearance and immunopathology [75]. Optimization of T cell-based therapies requires careful balancing of T cell affinity for target antigens ("quality") and the abundance of infused cells ("quantity") [75].

Benchmarking single-cell annotation methods reveals that program-based approaches like starCAT/TCAT outperform traditional clustering for identifying complex T cell states and activation programs. The optimal parameter set for T cell annotation includes: (1) using large, diverse reference catalogs of gene expression programs; (2) implementing batch correction compatible with nonnegative matrix factorization; (3) leveraging protein surface markers when available to enhance interpretation; and (4) validating identified states against functional outcomes.

For spatial transcriptomics data, reference-based methods like SingleR show superior performance, particularly when using paired single-cell references. As T cell therapies advance, controlled activation parameters—including stimulation strength, duration, and costimulatory context—emerge as critical factors determining functional outcomes and therapeutic efficacy. The integration of computational annotation with mechanistic studies provides a powerful framework for advancing both basic T cell biology and clinical applications in immunotherapy.

In single-cell RNA sequencing (scRNA-seq) research, particularly in the complex domain of T cell immunology, cell type annotation is a foundational step. The inherent limitations of both fully automated algorithms and purely manual expert annotation have necessitated a more robust hybrid approach. The Two-Step Annotation Protocol, which involves primary annotations by automated algorithms followed by expert-based manual interrogation, has emerged as a current gold-standard in the field [9]. This methodology is especially critical for T cell studies due to the exceptional heterogeneity of T cells, the continuous nature of their transcriptional states, and the highly polymorphic nature of their receptors [9] [3]. This guide objectively compares the performance of leading tools and frameworks that enable or utilize this two-step philosophy, providing researchers with experimental data to inform their analytical choices.

Comparing Automated Annotation Methods for the First Step

The initial automated step in the protocol leverages computational tools to assign preliminary labels to cells. These methods can be broadly categorized by their underlying learning approach and operational strategy. The following table summarizes the core characteristics of several prominent tools.

Table 1: Comparison of Automated Cell Type Annotation Methods

Method	Learning Approach	Core Strategy	Key Advantages	Reported Limitations
Supervised Methods (e.g., SingleR, CellTypist) [9]	Supervised	Trains a model on pre-annotated reference datasets to predict labels for a new query dataset.	Robust to missing marker genes; uses entire gene expression profile for classification [9].	Performance depends on similarity between reference and query data; fails with high heterogeneity [9].
Semi-Supervised Methods (e.g., SCINA, scGate) [9]	Semi-Supervised	Uses a predefined set of marker genes in a hierarchical or consensus model to annotate cells.	Highly interpretable; user-can tailor marker lists; good for novel datasets dissimilar to references [9] [3].	Requires substantial prior knowledge to define new marker models; can be subjective [9].
Reference-Based Label Transfer (e.g., Azimuth, Symphony) [9]	Varies	Maps query data to an existing annotated reference and transfers labels based on a joint embedding.	Reuses high-quality annotations from previous experiments; efficient [9].	Cannot identify new cell types; requires strong similarity between query and reference [9].
Ensemble Methods (e.g., popV) [76]	Ensemble	Combines predictions from multiple algorithms (e.g., RF, SVM, scANVI) into a consensus label.	Provides built-in uncertainty estimation; reduces reliance on any single method; more accurate and robust [76].	Computationally more intensive than single methods; complex setup [76].
Component-Based Models (e.g., TCAT/starCAT) [3]	Unsupervised	Uses models like NMF to learn gene expression programs (GEPs); cells are annotated based on GEP activities.	Captures co-expressed programs and continuous cell states; generalizes well across datasets [3].	Less transparent than marker-based methods; complex biological interpretation [3].
LLM-Assisted Tools (e.g., scExtract) [77]	Supervised / LLM	Leverages Large Language Models to extract annotation guidelines from research articles to automate processing.	Integrates prior knowledge from literature; can process datasets without a separate reference [77].	Potential for LLM "hallucinations"; performance depends on quality and clarity of the source article [77].

Performance Benchmarking Data

Independent benchmarking studies provide critical quantitative data for comparing these tools. The following table consolidates key performance metrics from evaluations on real single-cell datasets.

Table 2: Benchmarking Performance of Annotation Tools Across Tissues

Method	Reported Overall Accuracy (Human BMMC) [58]	Reported Performance on Xenium Spatial Data [13]	Key Strengths	Key Weaknesses
SingleR	Moderate	Best performer: Fast, accurate, and easy to use, closely matching manual annotation [13].	Fast, user-friendly, robust performance across modalities [13].	Can be outperformed by more complex methods in specific niches [58].
Bridge Integration	Highest (for human BMMC & PBMC) [58]	Not assessed in cited study.	Robust to data size, mislabeling, and sequencing depth; does not require gene activity calculation [58].	Requires multimodal data as a "bridge" [58].
scJoint	High for tissues with major labels [58]	Not assessed in cited study.	Efficient for cross-modality annotation [58].	Tends to assign cells to similar types; poorer performance on complex, deeply annotated datasets [58].
Azimuth	Information Missing	Performance evaluated but not top-ranked [13].	Part of a widely used and interoperable ecosystem (Seurat) [13].	Accuracy can be lower than other methods like SingleR in some contexts [13].
Conos	Low [58]	Not assessed in cited study.	Most time and memory efficient [58].	Worst performer in terms of prediction accuracy [58].
popV	High (on Lung Cell Atlas) [76]	Not assessed in cited study.	Accurately annotates majority of cells and highlights challenging populations via uncertainty scores [76].	Not the fastest method; retrain mode can take an hour for 100k cells [76].

The Essential Second Step: Expert Validation and Manual Curation

The second step of the protocol is the manual inspection and validation of automated annotations by a domain expert. This process is crucial for several reasons:

• Addressing Uncertainty: Automated methods, especially those like popV that provide uncertainty scores, can highlight cell populations that are challenging to classify. Experts can then focus their manual efforts on these ambiguous cells [76].
• Biological Plausibility Check: Experts verify that the annotated cell types make sense in the biological context of the sample (e.g., tissue type, disease state) [9].
• Identifying Novelty: Manual interrogation is often the primary way to discover novel cell states or subtypes that were not present in the reference data used for automated annotation [9].

The manual process typically involves inspecting the expression of known marker genes across the clusters generated by the automated tool to verify their identity [9].

Detailed Experimental Protocols for Featured Methods

The popV (Popular Vote) Consensus Workflow

popV implements the two-step protocol by design, providing both automated consensus labels and flags for manual inspection.

Detailed Protocol [76]:

• Input: An unannotated query dataset and an annotated reference dataset (both as raw count matrices).
• Algorithm Execution: Run eight different annotation methods (Random Forest, SVM, scANVI, OnClass, Celltypist, and kNN after batch correction with scVI, BBKNN, and Scanorama).
• Consensus Aggregation:
  • Perform a majority vote across all methods. OnClass is given multiple votes across the Cell Ontology hierarchy to account for "out-of-sample" cell types.
  • Designate a single consensus annotation for each cell.
• Uncertainty Quantification:
  • Calculate an "algorithm-extrinsic" consensus score (number of methods agreeing, from 1 to 8).
  • Output the "algorithm-intrinsic" uncertainty score from each of the eight individual methods.
• Expert Validation: The final report includes confusion matrices and visualizations. Researchers are expected to manually inspect cells with low consensus scores.

The TCAT/starCAT Gene Expression Program Workflow

T-CellAnnoTator (TCAT) and its generalized version starCAT offer a different approach based on pre-defined Gene Expression Programs (GEPs).

Detailed Protocol [3]:

• GEP Catalog Construction (Reference): Apply consensus Nonnegative Matrix Factorization (cNMF) to large, batch-corrected collections of T cell scRNA-seq datasets to derive a fixed catalog of reproducible GEPs.
• Annotation of Query Data:
  • Use the starCAT algorithm to quantify the activity ("usage") of each predefined GEP in every cell of a new query dataset using nonnegative least squares.
  • The resulting GEP usage matrix represents the cell's state.
• Cell State Interpretation: Annotate cells by associating the dominant GEPs with known T cell subsets (e.g., a GEP with high FOXP3 indicates Treg cells) or activation states (e.g., cytotoxicity, exhaustion).
• Expert Validation: Researchers manually validate the assignments by examining the top-weighted genes in the active GEPs and cross-referencing them with known biology.

The scExtract LLM-Assisted Workflow

scExtract leverages Large Language Models (LLMs) to automate the initial annotation by mimicking a human researcher reading a publication.

Detailed Protocol [77]:

• Input: Provide the raw expression matrix and the text of the associated research article.
• LLM-Powered Processing:
  • The LLM agent extracts processing parameters (e.g., mitochondrial gene filter thresholds) and the number of clusters from the "Methods" section of the article.
  • The tool executes these steps using the scanpy pipeline.
• LLM-Powered Annotation:
  • For each cluster, the LLM generates a list of marker genes.
  • Incorporating background knowledge from the article, the LLM assigns a cell type to each cluster.
  • An optimization step queries the expression of characteristic marker genes to refine annotations and mitigate LLM hallucinations.
• Output and Validation: The output is an automatically annotated dataset. As with all methods, expert validation is recommended to ensure biological accuracy.

Visualizing the Two-Step Annotation Workflow

The following diagram illustrates the logical flow and key decision points in the Two-Step Annotation Protocol.

The Scientist's Toolkit: Essential Research Reagents and Solutions

Successful implementation of the two-step annotation protocol relies on both computational tools and biological resources. The following table details key reagents and datasets essential for this research.

Table 3: Key Research Reagent Solutions for T Cell scRNA-seq Annotation

Item Name	Type	Primary Function in Annotation	Examples / Notes
Curated Reference Atlas	Dataset	Serves as a high-quality ground truth for supervised and reference-based annotation methods.	Tabula Sapiens [76], Human Cell Atlas (HCA) [77], Human Lung Cell Atlas [76].
Cell Ontology (CL)	Ontology	Provides a standardized, hierarchical vocabulary for cell types, enabling consistent labeling and harmonization across datasets and tools [76].	Used by popV and OnClass for consensus prediction and "out-of-sample" annotation [76].
Multimodal Bridge Data	Dataset (e.g., CITE-seq)	Enables methods like Bridge Integration by simultaneously measuring RNA and surface proteins, improving annotation accuracy without gene activity inference [58].	Critical for benchmarking and for tools that integrate protein expression to enhance GEP interpretation [3] [58].
Marker Gene Database	Knowledge Base	Used by semi-supervised tools and for expert validation during manual curation to confirm cell identity [9] [77].	Can be general (e.g., CellMarker) or study-specific, extracted from literature by tools like scExtract [77].
Batch Correction Tool	Software	Harmonizes multiple datasets to remove technical variation, a prerequisite for accurate label transfer and integration [9] [77].	Scanorama [76], Harmony [3], scVI [76], and their prior-informed variants (e.g., scanorama-prior) [77].

Selecting Appropriate Methods for Specific Tissue Contexts and Disease States

Accurately annotating T cell subsets in single-cell RNA sequencing (scRNA-seq) data remains a critical challenge for researchers studying immune responses in cancer, autoimmunity, and infectious diseases. Traditional clustering-based approaches often fail to capture the continuous spectrum of T cell states, leading to inaccurate biological interpretations [69]. This guide provides an objective comparison of current computational methods for T cell annotation, evaluating their performance across diverse tissue contexts and disease states to empower researchers in selecting optimal tools for their specific applications.

Performance Benchmarking: Quantitative Comparison of Annotation Methods

Comprehensive benchmarking studies reveal significant variability in the performance of cell type annotation methods. The following table summarizes the quantitative performance metrics of leading tools across multiple evaluation studies:

Table 1: Performance Comparison of T Cell Annotation Methods

Method	Approach	Reported Accuracy	Strengths	Limitations
STAMapper	Heterogeneous graph neural network	Highest accuracy on 75/81 datasets; significantly outperforms competitors (p = 2.2e-14 to 1.3e-36) [78]	Excellent for spatial transcriptomics; robust with limited genes	Requires more computational resources
TCAT/starCAT	Consensus nonnegative matrix factorization (cNMF)	Identifies 46 reproducible GEPs; high cross-dataset concordance (mean R = 0.81) [3]	Comprehensive T cell states; predicts immunotherapy response	Complex setup for novel users
STCAT	Hierarchical models with marker correction	28% higher accuracy than existing tools across 6 independent datasets [29]	Automated hierarchical annotation; handles tissue context	Limited to T cell annotation
SingleR	Reference-based correlation	Best performance for Xenium platform; fast and easy to use [13]	User-friendly; fast processing; good with matched reference	Performance declines with reference mismatch
scANVI	Variational autoencoder	Second-best performance after STAMapper [78]	Handles complex integration; good with multiple datasets	Requires significant computational power
CellTypist	Logistic regression classifier	65.4% match to manual annotations in AIDA dataset [79]	Pre-trained models available; no clustering needed	Infrequent model updates

Performance in Challenging Conditions

Method performance varies significantly under suboptimal conditions such as limited gene input or poor sequencing quality. STAMapper demonstrates remarkable robustness, maintaining superior accuracy even with fewer than 200 genes (median 51.6% vs. 34.4% for scANVI at 0.2 down-sampling rate) [78]. RCTD shows better performance on spatial datasets with more than 200 genes, while scANVI tends to outperform it on datasets with fewer than 200 genes [78].

For spatial transcriptomics data, particularly with imaging-based technologies like Xenium, SingleR emerges as the optimal choice due to its balance of accuracy, speed, and ease of use [13]. Its performance closely matches manual annotation while dramatically reducing analysis time.

Experimental Protocols and Methodologies

TCAT/starCAT Framework for Reproducible T Cell Annotation

The TCAT (T-CellAnnoTator) pipeline employs a sophisticated workflow for comprehensive T cell state characterization:

• Dataset Integration: Analyzes 1.7 million T cells from 700 individuals across 38 tissues and 5 disease contexts [3]
• Consensus NMF: Applies batch-corrected nonnegative matrix factorization to identify gene expression programs (GEPs)
• GEP Catalog Creation: Derives 46 consensus GEPs capturing T cell subsets, activation states, and functions
• Query Projection: Uses starCAT to project predefined GEPs onto new datasets via nonnegative least squares

The experimental validation included association with surface marker-based gating of canonical T cell subsets in a COVID-19 PBMC CITE-seq reference, with multivariate logistic regression revealing strong associations between specific cGEPs and T cell subsets (P value < 1 × 10⁻²⁰⁰) [3].

TCAT Analysis Workflow: From raw data to T cell annotation

STCAT Hierarchical Annotation Protocol

STCAT employs a structured approach for automated T cell annotation:

• Reference Construction: Builds comprehensive reference from 1,348,268 T cells across 35 conditions and 16 tissues [29]
• Hierarchical Classification: Classifies T cells into 33 subtypes followed by 68 state-based categories
• Marker Correction: Implements automated correction to refine annotations based on marker expression
• Validation: Cross-validation across independent datasets including cancer and healthy samples

This method successfully identified CD4+ Th17 cell enrichment in late-stage lung cancer patients and MAIT cell prevalence in milder-stage COVID-19 patients across multiple datasets [29].

STAMapper for Spatial Transcriptomics

STAMapper utilizes a heterogeneous graph neural network approach for spatial transcriptomics annotation:

• Graph Construction: Models cells and genes as distinct node types connected based on expression patterns [78]
• Message Passing: Updates latent embeddings through neighborhood information aggregation
• Attention Mechanism: Employs graph attention classifier with varying weights to connected genes
• Cross-entropy Optimization: Uses modified loss function to quantify prediction discrepancies

The method was validated on 81 scST datasets comprising 344 slices from 8 technologies and 5 tissues, demonstrating superior performance in cross-technology applications [78].

Method Selection Framework for Different Research Contexts

Decision Framework for Method Selection

Method Selection Guide: Choosing the right tool for your research context

Tissue and Disease-Specific Recommendations

Table 2: Optimal Methods for Specific Research Contexts

Research Context	Recommended Method	Key Supporting Evidence
Tumor Microenvironments	TCAT/starCAT	Identified activation GEPs predictive of immune checkpoint inhibitor response across multiple tumor types [3]
Autoimmune Diseases	STCAT	Consistently identified Th17 enrichment in inflammatory contexts; validated in rheumatoid arthritis [29]
Infectious Diseases	TCAT/starCAT	Discovered COVID-19-specific T cell activation programs across multiple datasets [3]
Spatial Transcriptomics	STAMapper	Highest accuracy on 75/81 datasets across 8 technologies and 5 tissues [78]
Xenium Platform	SingleR	Best performance for imaging-based spatial data with limited gene panels [13]
Rare Cell Type Detection	STAMapper	Superior identification of rare cell types crucial for comprehensive immune profiling [78]

Essential Research Reagent Solutions

Table 3: Key Research Reagents and Computational Resources for T Cell Annotation

Resource	Type	Function	Application Context
TCAT/starCAT Package	Software Pipeline	Quantifies predefined gene expression programs for T cell states	Cancer immunology, therapeutic response prediction [3]
STCAT Tool	Automated Annotation	Hierarchical T cell classification with marker correction	Cross-condition comparison, clinical biomarker discovery [29]
STAMapper	Graph Neural Network	Transfers labels from scRNA-seq to spatial transcriptomics	Spatial T cell distribution, tissue microenvironment studies [78]
CellTypist Models	Pre-trained Classifiers	Automated cell type prediction using logistic regression	Rapid annotation of immune cells in standard tissues [79]
SingleR Package	Reference-based Tool	Fast correlation-based cell type identification	General-purpose annotation with good reference available [13]
TCellAtlas Database	Reference Database	Comprehensive T cell reference with 33 subtypes and 68 states	Querying T cell expression profiles across conditions [29]

The benchmarking data presented in this guide demonstrates that method selection for T cell annotation must be guided by specific research contexts, technology platforms, and biological questions. While STAMapper excels in spatial transcriptomics applications, TCAT/starCAT provides the most comprehensive characterization of T cell states in complex disease contexts, and STCAT offers superior performance for clinical applications with its hierarchical approach.

Future developments in T cell annotation will likely address current limitations in unsupervised clustering [69] through improved integration of multi-omic measurements, including simultaneous analysis of TCR sequences and surface protein expression. As single-cell technologies continue to evolve, the development of context-specific benchmarks and standardized evaluation frameworks will be essential for advancing reproducible research in immunology and therapeutic development.

Benchmarking Annotation Accuracy and Method Performance

Benchmarking studies are fundamental to the advancement of single-cell genomics, providing critical assessments of analytical methods and technological platforms. In the specialized field of T cell research, standardized evaluations enable researchers to select optimal methodologies for dissecting T cell heterogeneity, activation states, and functional programs. The complex continuum of T cell states revealed by single-cell RNA sequencing (scRNA-seq) necessitates robust analytical frameworks that move beyond traditional clustering approaches, which often fail to resolve co-expressed gene programs [3].

Comprehensive benchmarking requires unified evaluation metrics, standardized datasets, and reproducible experimental protocols. These components allow for direct comparison of computational tools across diverse biological contexts, from fundamental immunology to clinical applications in cancer immunotherapy and autoimmune disease. This guide synthesizes key metrics and frameworks from recent benchmarking studies to empower researchers in evaluating single-cell annotation methods for T cell research.

Key Evaluation Frameworks and Metrics

Computational Framework Evaluation

Table 1: Key Evaluation Metrics for Single-Cell Annotation Methods

Metric Category	Specific Metrics	Interpretation	Relevance to T Cell Research
Accuracy Metrics	Prediction accuracy, Cell-type F1 score, Balanced accuracy	Proportion of correctly annotated cells	Measures ability to distinguish T cell subsets (e.g., Treg vs. cytotoxic T cells)
Technical Performance	Running time, Memory usage, Computational scalability	Practical computational requirements	Critical for large datasets (>100,000 cells) common in T cell studies
Sensitivity Analysis	Rare cell detection sensitivity, Marker gene detection rate	Ability to identify rare populations and key genes	Essential for detecting rare antigen-specific T cells or transitional states
Reproducibility	Cross-dataset consistency, Batch effect resistance	Stability across different datasets/conditions	Ensures findings generalize across donors, tissues, and diseases
Spatial Concordance	Transcript-protein alignment, Spatial clustering accuracy	Agreement with protein markers and tissue architecture	Validates spatial localization of T cells in tumor microenvironments

The evaluation of computational frameworks for T cell annotation extends beyond simple accuracy measurements. For methods like T-CellAnnoTator (TCAT) and its generalized counterpart starCAT, reproducibility across datasets is paramount. These frameworks employ consensus nonnegative matrix factorization (cNMF) to quantify predefined gene expression programs (GEPs) simultaneously, enabling the identification of 46 reproducible GEPs reflecting core T cell functions including proliferation, cytotoxicity, exhaustion, and effector states [3]. Benchmarking studies must assess how well these methods perform across diverse biological contexts, from blood to tissues and across different disease states.

The ePytope-TCR framework provides specialized evaluation for TCR-epitope prediction methods, addressing a critical need in T cell immunology. This framework integrates 21 different TCR-epitope prediction models and offers standardized interfaces for interoperability with common TCR repertoire data formats. Benchmarking within this framework has revealed significant biases in prediction scores between different epitope classes and limited generalization for less frequently observed epitopes, highlighting important considerations for researchers studying antigen-specific T cell responses [59].

Experimental Platform Benchmarking

Table 2: Spatial Transcriptomics Platform Performance Metrics

Platform	Technology Type	Genes Captured	Resolution	Sensitivity	Specificity	Cell Segmentation Accuracy
Xenium 5K	Imaging-based	5,001	Subcellular	High	High	High with nuclear markers
CosMx 6K	Imaging-based	6,175	Subcellular	Moderate	High	Moderate
Visium HD FFPE	Sequencing-based	18,085	2 μm	High	High	Requires computational inference
Stereo-seq v1.3	Sequencing-based	Whole transcriptome	0.5 μm	Variable	High	Challenging without staining

Recent systematic benchmarking of high-throughput spatial transcriptomics platforms across human tumors reveals critical performance differences that directly impact T cell research. These evaluations assess platforms across multiple metrics including capture sensitivity, specificity, diffusion control, cell segmentation accuracy, and concordance with protein expression from adjacent tissue sections [80]. For T cell studies, platform selection depends on the specific research questions—whether prioritizing whole transcriptome coverage or single-cell resolution with targeted gene panels.

For imaging-based spatial data like 10x Xenium, specialized benchmarking studies have evaluated reference-based cell type annotation methods. These studies demonstrate that SingleR outperforms other methods (Azimuth, RCTD, scPred, and scmapCell) in accuracy and speed for the Xenium platform, with results closely matching manual annotation based on marker genes [13]. This is particularly relevant for T cell microenvironment studies where accurate annotation of T cell subsets within tissue architecture is essential.

Experimental Protocols and Methodologies

Benchmarking Computational Methods

A standardized computational benchmarking workflow ensures fair method comparison.

The benchmarking protocol for computational methods begins with comprehensive dataset curation, incorporating both synthetic and real-world data spanning multiple biological contexts. For T cell-specific benchmarking, this includes datasets from blood and tissues across healthy individuals and those with conditions like COVID-19, cancer, rheumatoid arthritis, or osteoarthritis [3]. The ground truth datasets should encompass manual annotations based on marker genes and, where possible, paired TCR sequence information to validate functional subsets.

Method application follows a standardized pipeline where each algorithm processes the same curated datasets using consistent preprocessing steps. For spatial transcriptomics benchmarking, this includes uniform quality control measures—filtering cells based on detected gene counts, total molecule counts, and mitochondrial gene expression percentages [13]. Potential doublets should be identified and removed using tools like scDblFinder to ensure reference data quality.

Performance assessment employs the metrics outlined in Table 1, with particular emphasis on metrics most relevant to T cell biology. For spatial methods, additional validation against protein expression data from technologies like CODEX provides crucial ground truth for T cell localization and subset identification [80]. The benchmarking of TCR-epitope predictors must include evaluation on challenging datasets that test generalization to unknown epitopes and detection of cross-reactivity toward epitope mutations [59].

Experimental Platform Comparison

Systematic platform evaluation identifies optimal technologies for specific applications.

Benchmarking experimental platforms requires uniform sample processing across technologies. This involves collecting serial sections from the same biological samples—typically human tumors like colon adenocarcinoma, hepatocellular carcinoma, and ovarian cancer—to enable direct comparison [80]. Samples are processed according to each platform's specific requirements, with formalin-fixed paraffin-embedded (FFPE) blocks for some platforms and fresh-frozen OCT-embedded blocks for others.

Ground truth establishment is critical for rigorous platform evaluation. This includes profiling proteins using CODEX on tissue sections adjacent to those used for each spatial transcriptomics platform, providing protein-level validation of transcriptomic findings [80]. Additionally, scRNA-seq performed on matched samples offers a complementary reference for evaluating transcript capture efficiency. For T cell-specific evaluations, flow cytometry or CITE-seq data from dissociated portions of the same tissue can provide immune subset validation.

Cross-platform performance assessment examines multiple dimensions including molecular capture efficiency for marker genes, sensitivity across entire gene panels, diffusion control, cell segmentation accuracy, and concordance with orthogonal validation data. For T cell research, particular attention should be paid to the detection of key immune markers (CD3, CD4, CD8, PD-1, etc.) and the ability to resolve T cell subsets within the spatial context of tissues [80].

Table 3: Essential Research Resources for T Cell Benchmarking Studies

Resource Category	Specific Resources	Application in T Cell Research	Key Features
Reference Databases	PanglaoDB, CellMarker, Immune Cell Atlas	Marker gene identification for cell annotation	Curated cell-type signatures, immune-specific markers
Data Repositories	HCA, MCA, GEO, GTEx, SPATCH	Source of benchmarking datasets	Multi-tissue, multi-disease, standardized processing
Software Tools	SingleR, Azimuth, scPred, RCTD	Automated cell type annotation	Reference-based prediction, spatial mapping
Experimental Platforms	10x Xenium, CosMx, Visium HD	Spatial transcriptomics profiling	Subcellular resolution, targeted immune panels
TCR Resources	IEDB, VDJdb, McPAS-TCR	TCR specificity analysis	Curated TCR-epitope pairs, disease associations

The reference databases form the foundation for accurate cell type annotation. PanglaoDB and CellMarker provide comprehensive marker gene information that enables initial cell type identification, while immune-specific databases like the Immune Cell Atlas offer detailed immune subset signatures crucial for resolving T cell heterogeneity [10]. These resources must be dynamically updated to incorporate newly discovered cell states and markers, particularly for activated T cell subsets that may express non-canonical gene combinations.

Software tools for T cell research span multiple methodologies, each with strengths for specific applications. SingleR demonstrates superior performance for reference-based annotation of spatial transcriptomics data, while specialized tools like TCAT employ component-based models to resolve overlapping gene expression programs within individual T cells [3] [13]. For TCR repertoire analysis, ePytope-TCR provides a unified framework for applying and comparing multiple TCR-epitope prediction models [59].

Experimental platforms continue to evolve, with each technology offering distinct advantages for T cell research. The benchmarking results indicate that Xenium 5K provides superior sensitivity for marker gene detection, while Visium HD FFPE offers whole transcriptome coverage at high resolution [80]. Platform selection should align with research priorities—whether emphasizing discovery (whole transcriptome) or targeted analysis with high sensitivity (imaging-based).

Comprehensive benchmarking studies provide the critical foundation for methodological advancement in single-cell T cell research. The frameworks, metrics, and protocols outlined here enable rigorous evaluation of both computational and experimental approaches, guiding researchers toward optimal methods for their specific applications. As single-cell technologies continue to evolve, maintaining standardized benchmarking practices will ensure that new methods are properly validated against established standards while demonstrating meaningful improvements for resolving T cell biology.

The integration of multimodal data—combining transcriptomics, TCR sequencing, spatial context, and protein expression—represents the future of T cell characterization. Benchmarking frameworks must accordingly expand to evaluate how well methods integrate these complementary data types to provide a unified understanding of T cell identity, function, and spatial organization. Through continued methodological development and rigorous evaluation, the field will advance toward increasingly accurate and comprehensive characterization of T cells in health and disease.

Performance Comparison Across Annotation Tools and Algorithms

Single-cell RNA sequencing (scRNA-seq) has revolutionized our understanding of cellular heterogeneity, particularly in complex immune systems like T cell biology. Accurate cell type annotation is a critical bottleneck in this process, traditionally relying on manual expert knowledge which is time-consuming and irreproducible [81]. The exponential growth in both the number of cells profiled and the complexity of datasets has driven the development of numerous automated computational methods. This comparison guide provides an objective performance evaluation of single-cell annotation tools and algorithms, specifically contextualized within T cell subsets research.

Benchmarking studies are essential for validating bioinformatics methodologies in single-cell oncology and immunology [82]. For T cells specifically, traditional clustering approaches face significant limitations because T cell transcriptomes reflect multiple co-expressed gene expression programs (GEPs) that vary continuously and combine additively within individual cells [3]. This complexity necessitates specialized annotation pipelines that can accurately decipher T cell subsets, activation states, and functions across diverse biological contexts.

Performance Metrics and Evaluation Frameworks

Key Performance Indicators

Robust benchmarking of annotation tools requires multiple complementary metrics. The Adjusted Rand Index (ARI) quantifies clustering quality by comparing predicted and ground truth labels, with values from -1 to 1 (closer to 1 indicating better performance) [83]. Normalized Mutual Information (NMI) measures the mutual information between clustering and ground truth, normalized to [0, 1] [83]. F1-score (both macro and weighted) balances precision and recall, with macro F1 being particularly important for imbalanced cell-type distributions [78]. Cohen's kappa (κ) assesses agreement with manual annotation while accounting for chance [46]. Additional metrics include Clustering Accuracy (CA) and Purity [83], with computational efficiency measured through peak memory usage and running time [83].

Experimental Setups for Evaluation

Performance evaluation typically employs two experimental setups: within-dataset predictions (intra-dataset) and across-dataset predictions (inter-dataset) [81]. Intra-dataset evaluation assesses performance when reference and query data come from the same dataset, typically using cross-validation. Inter-dataset evaluation tests generalizability across different datasets, platforms, and biological conditions—a more challenging but clinically relevant scenario. For spatial transcriptomics, additional challenges include lower sequencing quality and fewer genes, requiring specialized benchmarking approaches [78].

Table 1: Key Performance Metrics for Annotation Tool Evaluation

Metric	Calculation Range	Ideal Value	Primary Application
Adjusted Rand Index (ARI)	-1 to 1	Closer to 1	General clustering quality
Normalized Mutual Information (NMI)	0 to 1	Closer to 1	Label agreement assessment
F1-score (Macro)	0 to 1	Closer to 1	Imbalanced class performance
F1-score (Weighted)	0 to 1	Closer to 1	Balanced class performance
Cohen's Kappa	-1 to 1	Closer to 1	Agreement with manual annotation
Accuracy	0 to 1	Closer to 1	Overall correctness
Computational Time	Seconds to hours	Lower	Practical efficiency
Peak Memory Usage	MB to GB	Lower	Scalability assessment

Comprehensive Performance Comparison

Reference-Based Annotation Tools

Reference-based annotation methods transfer cell-type labels from well-annotated reference datasets to query data. A recent benchmark evaluating five reference-based methods on 10x Xenium breast cancer data identified SingleR as the best performing tool, being "fast, accurate and easy to use, with results closely matching those of manual annotation" [38]. The performance evaluation involved preparing a high-quality single-cell RNA reference from paired 10x Flex single-nucleus RNA sequencing data, then applying the annotation tools to Xenium spatial data.

For spatial transcriptomics specifically, STAMapper—a heterogeneous graph neural network—demonstrated superior performance across 81 single-cell spatial transcriptomics datasets from eight technologies and five tissues [78]. It achieved significantly higher accuracy compared to competing methods (scANVI, RCTD, and Tangram), particularly excelling with datasets containing fewer than 200 genes where it maintained a median accuracy of 51.6% even at low down-sampling rates (0.2), compared to 34.4% for the second-best method [78].

Table 2: Performance Comparison of Reference-Based Annotation Methods

Tool	Algorithm Type	Best For	Accuracy Range	Key Strength	Limitation
SingleR [38]	Correlation-based	Xenium data	High (matches manual)	Speed, ease of use	Not specified
STAMapper [78]	Graph neural network	Low-gene spatial data	51.6% (challenging conditions)	Handles poor sequencing quality	Complex architecture
scANVI [78]	Variational autoencoder	General spatial	34.4% (challenging conditions)	Good overall performance	Lower accuracy with <200 genes
RCTD [78]	Regression framework	Spatial with >200 genes	Moderate	Effective with sufficient genes	Poor with limited genes
Tangram [78]	Pattern matching	General spatial	Lower than alternatives	Conceptual simplicity	Lower accuracy overall
Azimuth [38]	Reference mapping	Not specified	Moderate	Integration with Seurat	Not top performer
scMAP [38]	Projection-based	Not specified	Moderate	Computational efficiency	Not top performer
scPred [38]	Classification	Not specified	Moderate	Probabilistic outputs	Not top performer

Clustering Algorithms for Single-Cell Data

A comprehensive benchmark of 28 clustering algorithms across 10 paired single-cell transcriptomic and proteomic datasets revealed that scDCC, scAIDE, and FlowSOM consistently achieved top-tier performance for both transcriptomic and proteomic data [83]. Interestingly, these three methods maintained their superior performance across different omics modalities, though their ranking slightly changed: scAIDE ranked first for proteomic data, followed by scDCC and FlowSOM [83].

The benchmarking evaluated algorithms across three categories: classical machine learning-based methods (SC3, FFC, CIDR, etc.), community detection-based methods (PARC, Leiden, Louvain, etc.), and deep learning-based methods (DESC, scDCC, scGNN, etc.) [83]. For researchers prioritizing memory efficiency, scDCC and scDeepCluster are recommended, while TSCAN, SHARP, and MarkovHC excel in time efficiency [83]. Community detection-based methods generally offer a balanced trade-off between performance and computational demands [83].

Large Language Models for Cell Annotation

The emerging approach of using large language models (LLMs) for cell-type annotation has shown promising results. The AnnDictionary package, which supports all common LLM providers through a unified interface, enabled the first benchmarking study of major LLMs at de novo cell-type annotation [46]. Performance varied significantly with model size, with Claude 3.5 Sonnet achieving the highest agreement with manual annotation and recovering close matches of functional gene set annotations in over 80% of test sets [46].

LLM-based annotation of most major cell types exceeded 80-90% accuracy, demonstrating the potential of this approach [46]. AnnDictionary includes numerous optimizations for atlas-scale data and provides multiple annotation strategies: based on single marker gene lists, comparing several lists using chain-of-thought reasoning, deriving cell subtypes, and using expected cell types as context [46].

Specialized T Cell Annotation Methods

For T cell research specifically, T-CellAnnoTator (TCAT) and its generalized framework starCAT address the unique challenges of T cell annotation by simultaneously quantifying predefined gene expression programs (GEPs) that capture activation states and cellular subsets [3]. Analyzing 1.7 million T cells from 700 individuals across 38 tissues and five disease contexts, TCAT identified 46 reproducible GEPs reflecting core T cell functions including proliferation, cytotoxicity, exhaustion, and effector states [3].

A critical finding in T cell annotation research highlights the limitations of unsupervised clustering for T cell subsets. One study demonstrated that standard unsupervised clustering frequently fails to separate CD4+ and CD8+ T cells, with most clusters containing mixtures of both, implying that "many of the T cells would be incorrectly classified if using the standard cluster-based annotation approach" [69]. The factors driving clustering were primarily related to cellular functions (glucose metabolism), T-cell receptor (TCR), immunoglobulin and HLA transcripts—not typical phenotypic markers [69].

Detailed Methodologies of Key Experiments

AnnDictionary LLM Benchmarking Protocol

The AnnDictionary benchmarking study used the Tabula Sapiens v2 single-cell transcriptomic atlas following standardized pre-processing procedures [46]. For each tissue independently, researchers normalized, log-transformed, identified high-variance genes, scaled, performed PCA, calculated neighborhood graphs, clustered with the Leiden algorithm, and computed differentially expressed genes for each cluster [46]. LLMs then annotated each cluster based on top differentially expressed genes, with the same LLM reviewing labels to merge redundancies and fix spurious verbosity [46].

Agreement with manual annotation was assessed using multiple methods: direct string comparison, Cohen's kappa (κ), and two LLM-derived rating systems—one providing binary yes/no matches and another rating quality as perfect, partial, or not-matching [46]. To calculate Cohen's kappa across all annotation columns, researchers computed a unified set of categories using an LLM, basing all agreement metrics on these unified columns for consistency [46].

TCAT/starCAT Framework for T Cell Annotation

The TCAT methodology begins with augmented consensus nonnegative matrix factorization (cNMF) to enhance gene expression program discovery [3]. To improve cross-dataset GEP reproducibility, researchers adapted Harmony to provide batch-corrected nonnegative gene-level data, as standard batch-correction methods introduce negative values incompatible with cNMF [3]. The modified cNMF also incorporates surface protein measurements from CITE-seq data to enhance GEP interpretability [3].

starCAT infers the usage of GEPs learned in a reference dataset in new query datasets using nonnegative least squares, similarly to NMFproject [3]. This approach ensures consistent cell state representation for comparison across datasets, can quantify rarely used GEPs that are difficult to identify de novo in small query datasets, and markedly reduces runtime compared to de novo analysis [3]. Benchmarking through simulations demonstrated starCAT's accuracy in inferring usage of overlapping GEPs (Pearson R > 0.7) even when references contained extra or missing GEPs relative to queries [3].

Figure 1: TCAT/starCAT Workflow for T Cell Annotation. The framework processes reference datasets to create a catalog of gene expression programs (GEPs), then projects query datasets onto this reference space for consistent annotation across studies.

Spatial Transcriptomics Benchmarking Methodology

The spatial transcriptomics benchmarking study collected 81 single-cell spatial transcriptomics datasets comprising 344 slices and 16 paired scRNA-seq datasets from identical tissues [78]. These datasets originated from eight different spatial technologies (MERFISH, NanoString, STARmap, etc.) and five tissue types (brain, embryo, retina, kidney, liver) [78]. All datasets included manual annotations provided by the original authors, with cell-type labels in paired scRNA-seq and spatial datasets manually aligned for ground truth validation [78].

To evaluate performance under challenging conditions, researchers applied four different down-sampling rates (0.2, 0.4, 0.6, 0.8) to simulate poor sequencing quality [78]. Performance was assessed using accuracy, macro F1 score, and weighted F1 score, with statistical significance calculated using paired t-tests across all datasets [78]. This comprehensive approach allowed robust evaluation of each method's sensitivity to data quality and gene panel size.

Table 3: Key Research Reagent Solutions for Single-Cell Annotation Studies

Reagent/Resource	Function in Annotation Research	Example Applications
Tabula Sapiens v2 [46]	Reference atlas for benchmarking	LLM annotation validation
10x Xenium platform [38] [78]	Imaging-based spatial transcriptomics	Reference-based method testing
CITE-seq data [3] [69]	Simultaneous RNA and protein measurement	GEP validation with surface markers
HER2+ breast cancer data [38]	Paired snRNA-seq and spatial reference	Xenium method benchmarking
10x Flex single-nucleus RNA-seq [38]	High-quality reference data	Spatial annotation ground truth
COVID-19 PBMC datasets [3]	T cell activation context	GEP discovery in immune response
Lung cancer cell line panel [82]	Controlled heterogeneity benchmark	Algorithm validation in oncology
SPDB database [83]	Single-cell proteomic data resource	Cross-modal clustering evaluation

Figure 2: Decision Framework for Selecting Annotation Tools. This workflow guides researchers to appropriate annotation methods based on their data type, experimental goals, and computational resources.

Based on comprehensive benchmarking evidence, tool selection should be guided by specific research contexts. For spatial transcriptomics with limited genes (<200), STAMapper provides superior accuracy, particularly under challenging data quality conditions [78]. For Xenium data specifically, SingleR offers an optimal balance of speed, accuracy, and ease of use [38]. In T cell research, TCAT/starCAT enables reproducible annotation of activation states and functions across diverse biological contexts [3]. When computational resources permit, LLM-based approaches using AnnDictionary with Claude 3.5 Sonnet achieve impressive agreement with manual annotation [46]. For general clustering applications, scAIDE, scDCC, and FlowSOM deliver top-tier performance across both transcriptomic and proteomic data [83].

Future development in single-cell annotation should address current limitations in unsupervised clustering for complex cell populations like T cells [69], improve methods for cross-platform and cross-tissue generalization, and develop more efficient algorithms that maintain accuracy with increasing dataset scales. The integration of multiple modalities—RNA, protein, spatial context—through methods like STAMapper and starCAT represents the most promising direction for comprehensive cell identity resolution in complex biological systems.

Accuracy Assessment for Specific T Cell Subsets and Rare Populations

The accurate identification of T cell subsets and rare populations is a cornerstone of modern immunology, with critical implications for understanding immune responses in cancer, autoimmunity, and infectious diseases. Single-cell RNA sequencing (scRNA-seq) has revealed an unprecedented diversity of T cell states that exist along a continuum rather than as discrete subsets, presenting significant challenges for traditional clustering-based annotation methods [3]. The functional plasticity of T cell populations—including Th1, Th2, Th17, Tfh, and Treg subsets—necessitates sophisticated analytical frameworks that can capture their complex transcriptional programs and activation states [26]. This comparison guide provides a comprehensive benchmarking of current computational annotation methodologies, evaluating their performance across specific T cell subsets and rare populations to guide researchers in selecting appropriate tools for their experimental needs.

Comparative Performance of Annotation Methodologies

Quantitative Benchmarking Across Method Categories

Table 1: Performance Comparison of Major Annotation Method Categories for T Cell Subsets

Method Category	Representative Tools	Overall Accuracy (ARI)	Rare Cell Detection	Similar Subset Resolution	Reference Basis	Key Limitations
Program-Based Annotation	TCAT/starCAT, cNMF	0.81-0.95 (cGEP reproducibility) [3]	Excellent (identifies rare activation programs)	High (46 consensus GEPs resolved) [3]	Predefined gene expression programs	Requires extensive reference data
LLM-Based Annotation	AnnDictionary, scExtract	80-90% (major types) [46]	Moderate (improves with article context) [77]	Variable (depends on model size) [46]	Marker genes + literature context	Cost, API dependencies
Clustering Algorithms	scDCC, scAIDE, FlowSOM	0.72-0.85 (transcriptomics) [83]	Poor (tend to favor major types) [77]	Moderate to High	Unsupervised clustering	Discretizes continuous states
Traditional Reference-Based	Seurat, SingleR, CP, RPC	0.75-0.90 (intra-dataset) [43]	Poor (Seurat struggles with rare populations) [43]	Moderate (SingleR/RPC better for similar types) [43]	Well-annotated reference datasets	Limited novel type discovery

Specialized T Cell Annotation Tools

The T-CellAnnoTator (TCAT) pipeline represents a significant advancement for T cell-specific annotation by simultaneously quantifying predefined gene expression programs (GEPs) that capture activation states, cellular subsets, and core functions. Through analysis of 1.7 million T cells from 700 individuals across 38 tissues and 5 disease contexts, TCAT identified 46 reproducible GEPs reflecting core T cell functions including proliferation, cytotoxicity, exhaustion, and effector states [3]. This program-based approach demonstrates particular strength in resolving closely related T cell subsets that traditional clustering methods often conflate.

Component-based models like nonnegative matrix factorization (NMF) overcome key limitations of clustering by modeling GEPs as gene expression vectors and transcriptomes as weighted mixtures of these GEPs. Unlike principal component analysis, NMF components correspond to biologically interpretable GEPs reflecting cell types and functional states that additively contribute to a transcriptome, preserving the continuous nature of T cell states [3].

Experimental Protocols for Benchmarking Studies

Consensus Nonnegative Matrix Factorization (cNMF) Protocol for T Cell GEPs

Reference Dataset Curation: The TCAT benchmarking compiled seven scRNA-seq datasets spanning blood and tissues from healthy individuals and those with COVID-19, cancer, rheumatoid arthritis, or osteoarthritis. After quality control, 1.7 million cells remained from 905 samples from 695 individuals [3].

Batch Effect Correction: Standard batch-correction methods are incompatible with cNMF as they introduce negative values or modify low-dimensional embeddings rather than gene-level data. The protocol adapted Harmony to provide batch-corrected nonnegative gene-level data while preserving biological variability [3].

GEP Discovery and Consensus Building: Applied cNMF to each batch-corrected dataset independently, then clustered GEPs found across datasets. A consensus GEP (cGEP) was defined as the average of each cluster. The reproducibility was quantified with nine cGEPs supported by all seven datasets (mean Pearson R = 0.81, P < 1 × 10⁻⁵⁰ for all pairs) [3].

Validation with Surface Protein Markers: For CITE-seq datasets, the protocol incorporated surface protein measurements into GEP spectra to enhance interpretability. Multivariate logistic regression revealed strong associations between specific cGEPs and canonical T cell subsets defined by surface markers (P value < 1 × 10⁻²⁰⁰, coefficient > 0.35) [3].

Large Language Model Benchmarking Protocol

Dataset Preparation and Pre-processing: The AnnDictionary benchmarking utilized the Tabula Sapiens v2 single-cell transcriptomic atlas. For each tissue independently, researchers normalized, log-transformed, identified high-variance genes, scaled, performed PCA, calculated neighborhood graphs, clustered with the Leiden algorithm, and computed differentially expressed genes for each cluster [46].

LLM Annotation and Evaluation: Fifteen different LLMs annotated each cluster with a cell type label based on its top differentially expressed genes. The same LLM then reviewed its labels to merge redundancies and fix spurious verbosity. Agreement with manual annotation was assessed using direct string comparison, Cohen's kappa (κ), and LLM-derived quality ratings [46].

Cross-Model Consensus Building: To calculate Cohen's kappa between LLMs and with manual annotations, researchers computed a unified set of labels using an LLM to harmonize terminology across all annotation columns [46].

Clustering Algorithm Benchmarking Protocol

Dataset Selection and Processing: The clustering evaluation employed 10 paired single-cell transcriptomic and proteomic datasets from SPDB and Seurat v3, encompassing over 50 cell types and more than 300,000 cells across 5 tissue types. These paired multi-omics datasets were obtained using CITE-seq, ECCITE-seq, and Abseq technologies [83].

Performance Metrics and Ranking: Evaluated 28 clustering algorithms using Adjusted Rand Index (ARI), Normalized Mutual Information (NMI), Clustering Accuracy (CA), Purity, Peak Memory, and Running Time. Methods were ranked based on a comprehensive strategy that considered all metrics, with ARI and NMI serving as primary metrics [83].

Robustness Assessment: Investigated impact of highly variable genes (HVGs) and cell type granularity on clustering performance. Utilized 30 simulated datasets to assess how varying noise levels and dataset sizes influence clustering outcomes [83].

Workflow Visualization of Benchmarking Approaches

T Cell Annotation Benchmarking Workflow illustrates the comprehensive pipeline for evaluating annotation methods, from dataset preprocessing through multiple methodological approaches to multi-faceted performance assessment.

Table 2: Key Research Reagent Solutions for T Cell Annotation Studies

Category	Specific Resource	Application in T Cell Annotation	Performance Considerations
Single-Cell Technologies	10X Genomics Chromium	High-throughput scRNA-seq of T cells	Enables multimodal profiling with feature barcoding [84]
	CITE-seq	Simultaneous mRNA and surface protein measurement	Enhances GEP interpretability with protein validation [3]
	Smart-Seq2	Full-length transcriptome sequencing	Higher sensitivity for rare transcripts [43]
Reference Datasets	Tabula Sapiens v2	Reference atlas for cross-study benchmarking	Contains manually annotated T cell subsets [46]
	Human Cell Atlas	Multi-tissue reference for rare population identification	Systematic coverage across tissues [85]
	Cellxgene	Curated collection of annotated datasets	1458 datasets as of 2024 for validation [77]
Computational Frameworks	Scanpy	Standard Python framework for scRNA-seq analysis	Foundation for custom annotation pipelines [77]
	AnnData	Primary data structure for single-cell analysis	Enables efficient storage and manipulation [46]
	LangChain	LLM integration backbone	Supports multiple model providers in AnnDictionary [46]

Advanced Considerations for Rare T Cell Population Annotation

Multimodal Integration for Enhanced Resolution

The combination of transcriptomic and TCR repertoire analysis through single-cell technologies provides novel insights into the functional character of T cell immunity. Recent computational developments enable integrative analysis of gene expression and TCR profiles, allowing researchers to connect clonotype information with functional states [84]. This approach is particularly valuable for rare antigen-specific populations that may be missed by transcriptomic analysis alone.

Methods like scExtract address the critical challenge of batch effect correction while preserving biological diversity, especially for rare populations. By incorporating prior annotation information through modified versions of scanorama and cellhint, these approaches demonstrate enhanced batch correction results while maintaining rare population integrity [77]. The scExtract framework specifically implements scanorama-prior, which considers prior differences between cell types when constructing mutual nearest neighbors, adjusting weighted distances between cells across datasets for more accurate integration.

TCR Repertoire Analysis Considerations

While RNA-seq data can be used for TCR repertoire extraction, benchmarking studies reveal significant discrepancies between TCR sequences extracted from RNA-seq data compared to dedicated TCR-seq methods. The lack of significant improvement with longer read lengths, combined with the absence of correlation to T cell abundance, emphasizes the necessity of using dedicated T cell receptor sequencing methodologies for repertoire-focused studies [86].

Based on comprehensive benchmarking studies, method selection for T cell annotation should be guided by specific research goals:

• For T cell-specific program identification: TCAT/starCAT provides the highest resolution for activation states and functional programs across diverse immunological contexts [3].

• For rapid annotation with literature integration: LLM-based tools like AnnDictionary and scExtract offer compelling performance (80-90% accuracy for major types) with decreasing manual effort [46] [77].

• For large-scale atlas construction: Clustering algorithms like scAIDE, scDCC, and FlowSOM provide robust performance for transcriptomic data, with scAIDE and scDCC also excelling for proteomic data [83].

• For reference-based annotation with comprehensive atlases: Traditional methods like Seurat and SingleR maintain strong performance, though with limitations for rare populations and highly similar subsets [43].

The field continues to evolve rapidly with emerging capabilities in multimodal integration, LLM-based annotation, and specialized frameworks for immune cells. Researchers should validate selected methods using their specific tissue contexts and T cell populations of interest, particularly when studying rare populations in disease settings where accurate annotation is critical for both biological insight and therapeutic development.

Cross-Dataset Validation and Generalizability Testing

In single-cell RNA sequencing (scRNA-seq) research, cross-dataset validation serves as a critical methodology for assessing the generalizability and robustness of cell type annotation tools. This process involves training algorithms on one dataset (reference) and evaluating their performance on entirely separate datasets (query) generated from different studies, platforms, or biological contexts. For T cell subset research specifically, where cellular states exist along a complex continuum rather than in discrete categories, establishing reproducible annotation methods remains particularly challenging [3]. The validation framework ensures that computational methods can identify biologically meaningful T cell programs—such as cytotoxicity, exhaustion, and effector states—across diverse patient populations, tissue types, and disease contexts, thereby confirming that findings reflect true biology rather than dataset-specific artifacts.

The fundamental challenge in single-cell method validation stems from technical variability introduced by different experimental protocols, sequencing platforms, and laboratory conditions, coupled with biological variability across donors, tissues, and disease states. Cross-dataset validation directly addresses these challenges by testing whether gene expression programs (GEPs) identified in one context can be reliably detected in others. This approach has revealed that methods performing well in intra-dataset evaluations often fail when applied across datasets, highlighting the importance of rigorous generalizability testing [3] [87]. For research and drug development professionals, these validation approaches provide critical quality assurance that biological discoveries—particularly those identifying potential therapeutic targets—maintain consistency across diverse human populations and experimental conditions.

Comparative Performance of Single-Cell Annotation Methods

Quantitative Benchmarking Results

Comprehensive benchmarking studies provide crucial empirical evidence for method selection in T cell research. The following table summarizes the cross-dataset performance of major annotation tools:

Table 1: Cross-dataset performance of single-cell annotation methods

Method	Underlying Approach	Reported Cross-Dataset Accuracy	Key Strengths	Limitations
TCAT/starCAT [3]	Consensus Nonnegative Matrix Factorization (cNMF)	Identified 46 reproducible GEPs across 7 datasets (1.7M cells)	High reproducibility (68.4-96.8% GEPs shared across datasets)	Requires batch correction for cross-dataset application
PCLDA [87]	PCA + Linear Discriminant Analysis	Top-tier accuracy in 35/35 evaluation scenarios; stable across platforms	Interpretable, computationally efficient, robust to technical variance	Simpler model may miss extremely rare populations
scExtract [77]	LLM-based + prior-informed integration	Outperformed reference transfer methods in benchmarks	Automated processing leveraging article context	Potential sensitivity to annotation errors
LICT [88]	Multi-LLM integration + credibility evaluation	69.4% full match in gastric cancer; 48.5% in embryo data	Objective reliability assessment without reference data	Performance drops in low-heterogeneity datasets
scID [87]	Modified LDA for single-cell data	Consistently outperformed by vanilla LDA in experiments	Designed specifically for single-cell data	Underperforms simpler alternatives in validation

Beyond accuracy metrics, computational efficiency and interpretability represent practical considerations for research implementation. Methods like PCLDA emphasize simplicity and transparency, providing clear mechanistic insights into classification decisions through linear combinations of gene expressions [87]. In contrast, more complex approaches may offer marginally better performance in specific contexts but suffer from longer computation times and reduced interpretability, which can hinder biological discovery and clinical translation.

Specialized Tools for T Cell Research

T cell-specific annotation tools have emerged to address the unique challenges of characterizing T cell states, functions, and antigen specificities:

Table 2: Specialized tools for T cell receptor and antigen specificity analysis

Tool	Specialized Application	Data Input	Key Output
ITRAP [89]	TCR-pMHC pairing confidence	Single-cell sequencing with DNA-barcoded pMHC multimers	High-confidence TCR-antigen pairs with reduced artifacts
TCRseq Methods [90]	TCR repertoire profiling	Genomic DNA or RNA from T cells	Comprehensive TRB and TRA diversity assessment
TCAT [3]	T cell activation states and functions	scRNA-seq from multiple tissues/diseases	46 reproducible GEPs for subset and activation annotation

These specialized tools address critical gaps in conventional annotation approaches, particularly for T cell receptor repertoire analysis and antigen specificity mapping. However, cross-validation studies have revealed substantial methodological biases in TCR sequencing approaches, with marked differences in accuracy and reproducibility for TRA versus TRB chains across nine commercial and academic methods [90]. Similarly, ITRAP addresses the critical need for data-driven filtering of sequencing artifacts in single-cell TCR-pMHC pairing data, significantly improving specificity and sensitivity for identifying true T cell antigen interactions [89].

Experimental Protocols for Cross-Dataset Validation

Reference-Based Validation Framework

The reference-based validation framework tests a method's ability to consistently annotate cell types when applied to new datasets. The starCAT pipeline exemplifies this approach [3]:

• Reference Catalog Construction: Apply consensus nonnegative matrix factorization (cNMF) to multiple large-scale T cell datasets (e.g., 1.7 million cells from 700 individuals) to identify robust gene expression programs (GEPs). Batch correction methods like Harmony are adapted to provide batch-corrected nonnegative gene-level data compatible with cNMF requirements.

• Query Dataset Processing: For new datasets, use nonnegative least squares to quantify the activity of predefined reference GEPs within each cell, enabling direct comparison without re-deriving programs.

• Performance Assessment: Evaluate annotation accuracy by comparing with manual annotations or ground truth labels when available. Assess reproducibility by measuring concordance of GEP activities across datasets from different biological contexts.

This framework's effectiveness was demonstrated through systematic benchmarking where starCAT accurately inferred usage of GEPs overlapping between reference and query datasets (Pearson R > 0.7) while predicting low usage of non-overlapping GEPs, outperforming direct application of cNMF to query datasets, particularly for smaller query datasets [3].

Figure 1: Workflow for reference-based cross-dataset validation

Cross-Platform and Cross-Tissue Validation

Robust validation requires testing annotation methods across diverse technical and biological conditions. The PCLDA benchmarking protocol exemplifies this comprehensive approach [87]:

• Dataset Selection: Curate multiple scRNA-seq datasets generated using different sequencing platforms (e.g., 10X Genomics, Smart-seq2) and from diverse tissue sources (e.g., blood, tumor, lymphoid tissues).

• Experimental Design: Implement both intra-dataset (cross-validation within dataset) and inter-dataset (cross-platform) validation scenarios. For T cell-specific validation, ensure representation of diverse T cell states across healthy and disease contexts.

• Evaluation Metrics: Assess accuracy using standard classification metrics (e.g., F1-score, precision, recall) and biological coherence through enrichment analysis of known T cell marker genes.

This protocol applied to 22 public scRNA-seq datasets across 35 distinct evaluation scenarios demonstrated that simpler methods like PCLDA often match or outperform more complex models in cross-platform conditions, highlighting how model complexity can sometimes reduce generalizability [87].

Novel LLM-Based Validation Approaches

Recent advances incorporate large language models (LLMs) for automated annotation validation. The LICT framework introduces innovative strategies for reliability assessment [88]:

• Multi-Model Integration: Leverage complementary strengths of multiple LLMs (GPT-4, LLaMA-3, Claude 3, Gemini, ERNIE) to reduce uncertainty and increase annotation reliability, particularly for low-heterogeneity datasets where individual models struggle.

• "Talk-to-Machine" Iteration: Implement human-computer interaction where the LLM is queried to provide marker genes for predicted cell types, followed by expression validation and structured feedback to refine annotations.

• Objective Credibility Evaluation: Assess annotation reliability based on marker gene expression patterns within the input dataset, providing reference-free validation. An annotation is deemed reliable if >4 marker genes are expressed in ≥80% of cells within the cluster.

This approach demonstrated particular strength in identifying reliably annotated cell types, with LICT-generated annotations outperforming manual annotations in credibility assessments for PBMC and low-heterogeneity datasets [88].

Research Reagent Solutions for T Cell Studies

Table 3: Key reagents and resources for single-cell T cell studies

Reagent/Resource	Application	Function in Experimental Design
DNA-barcoded pMHC multimers [89]	TCR specificity profiling	Simultaneously identify pMHC specificity and TCR sequence of individual cells
Cell hashing antibodies [89]	Sample multiplexing	Enable pooling of samples from multiple donors/conditions while maintaining sample origin information
Commercial TCRseq kits [90]	TCR repertoire profiling	Amplify and sequence TRA and TRB chains using multiplex-PCR or RACE-PCR approaches
Reference atlas datasets [3]	Method benchmarking	Provide standardized data for comparing annotation method performance across laboratories
CITE-seq antibodies [3]	Multimodal validation	Incorporate surface protein measurements to enhance annotation interpretability

Beyond wet-lab reagents, computational resources form an essential component of modern validation pipelines:

Reference Catalogs: Curated collections of gene expression programs, such as the 46 T cell cGEPs identified from 1.7 million cells [3], provide standardized frameworks for method comparison. Benchmarking Datasets: Publicly available datasets with ground truth annotations, like those in cellxgene [77], enable standardized performance assessment. Containerized Pipelines: Software tools like scExtract that automate processing from raw data to annotation facilitate reproducible comparisons across computational environments [77].

Best Practices and Implementation Guidelines

Recommendations for Robust Validation

Based on comprehensive benchmarking studies, several key practices emerge for ensuring reliable cross-dataset validation:

• Prioritize Diverse Dataset Selection: Include datasets spanning multiple tissues, disease states, and sequencing platforms to assess method performance across biologically and technically diverse contexts. Studies limited to peripheral blood mononuclear cells (PBMCs) may not generalize to tissue-resident T cell populations [3] [88].

• Implement Appropriate Batch Correction: Adapt standard batch correction methods like Harmony to provide batch-corrected nonnegative gene-level data compatible with decomposition algorithms like cNMF, as negative values can invalidate downstream analyses [3].

• Validate with Multiple Modalities: Incorporate protein expression data from CITE-seq when available to confirm RNA-based annotations, as surface markers often provide more definitive identification of canonical T cell subsets [3].

• Apply Objective Reliability Metrics: Implement credibility assessments based on marker gene expression patterns rather than relying solely on agreement with manual annotations, which may contain biases and inconsistencies [88].

• Balance Complexity and Interpretability: Consider simpler, interpretable models like PCLDA that demonstrate robust performance across validation scenarios, as their transparency facilitates biological interpretation and troubleshooting [87].

Figure 2: Comprehensive validation workflow integrating multiple assessment dimensions

Emerging Trends and Future Directions

The field of cross-dataset validation continues to evolve with several promising developments:

LLM Integration: Tools like scExtract demonstrate how large language models can automate annotation pipelines by extracting processing parameters from research articles, potentially reducing manual curation time and improving reproducibility [77]. Prior-Informed Integration: New algorithms like scanorama-prior and cellhint-prior incorporate preliminary annotation information to enhance batch correction while preserving biological diversity, addressing a key limitation of conventional integration methods [77]. Multi-Model Consensus Approaches: Frameworks like LICT that leverage multiple LLMs show promise for improving reliability, particularly for challenging low-heterogeneity cell populations where individual models exhibit performance limitations [88].

For research and drug development professionals, these advances offer increasingly robust frameworks for validating T cell annotations across diverse human populations, ultimately strengthening the foundation for translational discoveries and therapeutic development.

Spatial Transcriptomics Annotation Benchmarking Results

Spatial transcriptomics (ST) technologies have revolutionized biological research by enabling comprehensive gene expression profiling within intact tissue architecture, preserving the spatial context that is lost in single-cell RNA sequencing (scRNA-seq) [80] [91]. These technologies bridge a critical gap in understanding cellular heterogeneity, tissue organization, and cell-cell interactions in development, disease, and normal physiology [80]. A fundamental step in analyzing ST data is cell-type annotation—the process of identifying and labeling distinct cell populations based on their gene expression profiles within their spatial context [13] [78]. Accurate annotation is crucial for downstream analyses, including characterizing tumor microenvironments, mapping neural circuits, understanding developmental processes, and identifying novel cellular states [80] [3].

The field encompasses two primary technological approaches: imaging-based spatial transcriptomics (iST), which utilizes sequential hybridization and imaging of fluorescently labeled probes to profile targeted genes at single-molecule resolution, and sequencing-based spatial transcriptomics (sST), which captures polyadenylated RNA on spatially barcoded arrays for unbiased whole-transcriptome analysis [80] [91]. As these technologies rapidly advance, achieving subcellular resolution and expanding gene panels, rigorous benchmarking becomes essential to guide researchers in selecting appropriate platforms and computational methods for their specific biological questions [80] [91] [92].

This review synthesizes recent benchmarking studies evaluating ST platforms and annotation methodologies, with particular emphasis on applications in T cell biology. We provide comprehensive performance comparisons, detailed experimental protocols, and practical recommendations to assist researchers in navigating this complex landscape.

Benchmarking Spatial Transcriptomics Platforms

Performance Comparison of Major Platforms

Recent systematic evaluations have compared high-throughput ST platforms with subcellular resolution using uniformly processed clinical samples from various human tumors, including colon adenocarcinoma, hepatocellular carcinoma, and ovarian cancer [80]. These studies established ground truth datasets using complementary modalities: CODEX for protein profiling on adjacent tissue sections and scRNA-seq on the same samples [80]. The platforms benchmarked include Stereo-seq v1.3, Visium HD FFPE, CosMx 6K, and Xenium 5K, selected for their high-throughput capabilities (>5,000 genes) and subcellular resolution (≤2 μm) [80].

Table 1: Performance Metrics of High-Throughput Spatial Transcriptomics Platforms

Platform	Technology Type	Spatial Resolution	Gene Panel Size	Sensitivity (Marker Genes)	Specificity	Cell Segmentation Accuracy	Concordance with CODEX
Xenium 5K	Imaging-based (iST)	Subcellular	5001 genes	High, especially for lineage markers	High	Superior, precise for small cells	High spatial concordance
CosMx 6K	Imaging-based (iST)	Subcellular	6175 genes	Moderate	High	Good	Moderate
Visium HD FFPE	Sequencing-based (sST)	2 μm	18,085 genes	High	High	Moderate (can divide cells across spots)	High
Stereo-seq v1.3	Sequencing-based (sST)	0.5 μm	Transcriptome-wide	Variable	High	Limited by diffusion	Moderate

The benchmarking revealed that Xenium 5K demonstrated superior sensitivity for multiple marker genes and excelled in precise cell segmentation, particularly for identifying small cell types like lymphocytes [80] [93]. Its single-molecule, imaging-based approach enabled accurate mapping of transcripts within individual cells and showed higher annotation reliability and spatial concordance with protein data, especially in complex tumor microenvironments [80].

Visium HD FFPE delivered robust, transcriptome-wide spatial gene expression maps with enhanced diffusion control and minimal spatial artifacts [80] [93]. While its single-cell segmentation was less precise than imaging-based platforms (sometimes dividing individual cells across multiple spots), its ability to capture a broad range of transcripts makes it ideal for discovery-focused studies requiring comprehensive spatial profiling [80] [93].

For sequencing-based platforms, a separate systematic comparison of 11 sST methods across reference tissues (mouse embryonic eyes, hippocampal regions, and olfactory bulbs) found that sensitivity varied significantly based on sequencing depth and molecular diffusion characteristics [91]. Stereo-seq demonstrated the highest capturing capability when using all available reads, though no platforms reached saturation even at high sequencing depths, suggesting potential for increased sensitivity with further optimization [91].

Platform Selection for T Cell Research

For T cell-specific investigations, platform selection depends on the research question. Imaging-based platforms (Xenium, CosMx) offer superior resolution for mapping immune cell neighborhoods and identifying rare T cell states within tissue contexts [80] [13]. Their precise cell segmentation enables accurate characterization of T cell interactions with tumor cells and other immune populations.

Sequencing-based platforms (Visium HD, Stereo-seq) provide the unbiased transcriptome coverage necessary for discovering novel T cell states and activation programs without prior knowledge of relevant genes [80] [91]. This makes them particularly valuable for exploratory studies in unconventional tissue sites or disease states with poorly characterized T cell responses.

Diagram: Decision framework for selecting spatial transcriptomics platforms in T cell research. The pathway begins with defining the specific T cell biology question, which informs requirements for resolution, gene panel size, and the balance between discovery and targeted approaches, ultimately leading to platform selection and subsequent analysis.

Benchmarking Cell Type Annotation Methods

Performance Comparison of Annotation Algorithms

Accurate cell type annotation is particularly challenging for ST data due to technical limitations, including sparse gene expression profiles, platform-specific artifacts, and the integration of spatial information [13] [78]. For imaging-based ST data with limited gene panels, reference-based annotation methods that transfer labels from well-annotated scRNA-seq datasets have shown considerable promise [13].

A comprehensive benchmarking study evaluated five reference-based annotation methods (SingleR, Azimuth, RCTD, scPred, and scmapCell) on Xenium data from human HER2+ breast cancer, using manual annotation based on marker genes as ground truth [13]. The study utilized paired single-nucleus RNA sequencing (snRNA-seq) data from the same sample as reference to minimize technical variability.

Table 2: Performance Comparison of Reference-Based Cell Type Annotation Methods

Method	Underlying Algorithm	Accuracy	Advantages	Limitations	Runtime
SingleR	Correlation-based	High	Fast, easy to use, results closely match manual annotation	Requires high-quality reference	Fast
Azimuth	Integration-based	Moderate	User-friendly, integrated with Seurat	Limited customization	Moderate
RCTD	Regression-based	Moderate	Accounts for platform effects	Parameter sensitivity	Moderate
scPred	Machine learning	Moderate	Probabilistic predictions	Requires training	Fast
scmapCell	Projection-based	Lower	Simple implementation	Lower accuracy	Fast

The benchmarking revealed that SingleR was the best-performing method for the Xenium platform, being fast, accurate, and easy to use, with results closely matching manual annotation [13]. Its correlation-based approach effectively handled the limited gene panels characteristic of imaging-based spatial technologies.

For single-cell resolution ST data across diverse technologies and tissues, a separate evaluation of 81 datasets compared STAMapper—a heterogeneous graph neural network that transfers cell-type labels from scRNA-seq to single-cell spatial transcriptomics (scST) data—against competing methods (scANVI, RCTD, and Tangram) [78]. STAMapper demonstrated significantly higher accuracy in annotating cells across multiple metrics (accuracy, macro F1 score, and weighted F1 score) and maintained superior performance even under poor sequencing quality with various down-sampling rates [78].

Specialized Annotation for T Cell States

Traditional clustering approaches often fail to capture the continuous nature of T cell states and the co-expression of multiple gene programs within individual cells [3]. To address this limitation, T-CellAnnoTator (TCAT) was developed as a specialized framework that simultaneously quantifies predefined gene expression programs (GEPs) capturing T cell activation states, subsets, and functions [3].

By analyzing 1.7 million T cells from 700 individuals across 38 tissues and five disease contexts, TCAT identified 46 reproducible GEPs reflecting core T cell functions, including proliferation, cytotoxicity, exhaustion, and effector states [3]. This component-based model represents transcriptomes as weighted mixtures of GEPs, preserving the continuous and combinatorial nature of T cell states that is obscured by discrete clustering approaches.

The starCAT software package generalizes this framework to other cell types and tissues, enabling reproducible annotation based on fixed, multi-dataset catalogs of GEPs [3]. This approach provides a consistent coordinate system for comparing cell states across datasets, identifies rarely used GEPs that might be missed in de novo analyses, and significantly reduces computational runtime compared to analyzing each dataset independently.

Experimental Protocols for Benchmarking

Sample Preparation and Multi-Omics Profiling

Comprehensive benchmarking requires carefully controlled experimental designs with appropriate ground truth datasets. The following protocol outlines the standardized approach used in recent benchmarking studies [80]:

• Sample Collection and Processing: Collect treatment-naïve tumor samples (e.g., colon adenocarcinoma, hepatocellular carcinoma, ovarian cancer) and divide them into multiple portions for parallel processing into formalin-fixed paraffin-embedded (FFPE) blocks, fresh-frozen (FF) blocks embedded in optimal cutting temperature (OCT) compound, or single-cell suspensions [80].

• Serial Sectioning: Generate serial tissue sections of uniform thickness (typically 5-10 μm) for parallel profiling across multiple ST platforms and complementary modalities. Maintain consistent orientation and registration between sections [80].

• Multi-Platform ST Profiling: Process adjacent sections across selected high-throughput ST platforms (e.g., Stereo-seq v1.3, Visium HD FFPE, CosMx 6K, Xenium 5K) following manufacturer protocols with minimal modifications [80].

• Ground Truth Establishment:

  • Perform CODEX (Co-Detection by Indexing) multiplexed protein profiling on tissue sections adjacent to those used for ST platforms to establish protein-based cellular reference maps [80].
  • Conduct single-cell RNA sequencing on matched dissociated tumor samples to generate transcriptomic references [80].
  • Manual annotation of cell types for both scRNA-seq and CODEX data, along with nuclear boundaries in H&E and DAPI-stained images [80].

• Data Integration: Implement computational pipelines to align multi-omics datasets in a common coordinate system, enabling cross-platform and cross-modality comparisons [80].

Diagram: Experimental workflow for benchmarking spatial transcriptomics platforms. The process begins with sample collection and processing, followed by serial sectioning for multi-platform profiling and ground truth establishment, culminating in data integration and benchmarking.

Evaluation Metrics and Analysis Framework

Standardized evaluation metrics are essential for objective platform and method comparisons. Recent benchmarking studies have employed the following comprehensive assessment framework [80] [92]:

• Molecular Capture Efficiency:

  • Sensitivity: Assess detection of diverse cell marker genes across platforms, calculating total transcript count per gene and correlating with matched scRNA-seq profiles [80].
  • Specificity: Evaluate background signals and off-target capture using negative control regions and synthetic RNA spikes [80].
  • Dynamic Range: Quantify linearity of transcript detection across expression levels using serial dilutions or spike-in controls [91].

• Spatial Fidelity:

  • Diffusion Control: Measure transcript dispersion from source using distance-decay analysis of highly localized markers [80] [91].
  • Spot Swapping: Quantify RNA bleeding to surrounding spots using specialized algorithms like SpotClean [94].
  • Spatial Resolution: Determine minimum distinguishable feature size using point spread function estimation or line-pair analysis [80].

• Cell Segmentation Performance:

  • Nuclear Segmentation Accuracy: Compare automated segmentation against manual nuclear boundaries annotated from DAPI/H&E images [80].
  • Cellular Boundary Delineation: Evaluate whole-cell segmentation using membrane markers or computational boundary prediction [13].

• Cell Annotation Accuracy:

  • Concordance with Protein Reference: Calculate overlap between transcriptomically-defined clusters and protein-based cell types from CODEX [80].
  • Cluster Purity: Assess separation of known cell types using supervised classification metrics (ARI, NMI) [92] [95].
  • Rare Cell Detection: Evaluate sensitivity for identifying low-abundance populations (<5% of total cells) [78].

• Spatial Analysis Capabilities:

  • Spatially Variable Genes: Identify genes with non-random spatial patterns using specialized algorithms (SpatialDE, SPARK) [95].
  • Domain Identification: Assess detection of spatially coherent regions using clustering metrics with manual annotations as ground truth [92] [95].
  • Architecture Preservation: Evaluate maintenance of tissue organization using spatial autocorrelation metrics [92].

Essential Research Reagents and Tools

Experimental Reagents

Table 3: Key Research Reagent Solutions for Spatial Transcriptomics

Reagent/Tool	Function	Application Notes
CODEX Multiplexed Panels	Simultaneous protein detection for ground truth	Validates ST annotations, 30-60 protein markers recommended
Visium HD FFPE Gene Panel	Whole transcriptome capture	18,085 genes, requires tissue optimization
Xenium 5K Gene Panel	Targeted transcript detection	5001 genes, optimized for cell typing
CosMx 6K Gene Panel	Targeted transcript detection	6175 genes, includes morphology markers
Nuclear Segmentation Dyes	Cell boundary identification	DAPI, Hoechst for nuclear segmentation
Membrane Staining Markers	Cellular boundary delineation	Wheat Germ Agglutinin (WGA), CellMask
Tissue Preservation Reagents	Sample integrity maintenance	OCT for fresh frozen, formalin for FFPE

Computational Tools

A meta-review of spatial transcriptomics software benchmarking provides recommendations across four key analytical areas [95]:

• Tissue Architecture Identification: BASS and BayesSpace consistently rank among top performers for accuracy across multiple benchmarks, with SpaGCN and Seurat offering good performance with faster runtimes [95].

• Spatially Variable Gene Detection: SPARK and SpatialDE show robust performance across diverse tissue types and technologies, effectively controlling false discovery rates while maintaining sensitivity [95].

• Cell-Cell Communication Analysis: COMMOT and DeepTalk integrate spatial constraints with ligand-receptor databases to predict interaction probabilities, though performance varies significantly based on data quality and resolution [95].

• Deconvolution: Cell2location and RCTD accurately estimate cell-type proportions in spot-based data, with Cell2location particularly effective for resolving rare populations [95].

Spatial transcriptomics annotation has progressed dramatically, with current platforms achieving subcellular resolution and computational methods enabling accurate cell type mapping. Based on comprehensive benchmarking studies:

For T cell research, imaging-based platforms (Xenium, CosMx) provide superior resolution for mapping immune cell interactions and rare states, while sequencing-based platforms (Visium HD, Stereo-seq) offer discovery potential for novel T cell programs [80] [93].

For annotation methodology, SingleR performs excellently for standard cell typing, while specialized frameworks like TCAT/starCAT better capture the continuous nature of T cell states and activation programs [13] [3].

Experimental design should incorporate multi-omics ground truth (CODEX, scRNA-seq) and standardized evaluation metrics to ensure robust benchmarking and biological validity [80] [92]. As spatial technologies continue evolving, ongoing benchmarking will remain essential for maximizing biological insights from these powerful tools.

Emerging Standards and Best Practices for Validation

The characterization of T cell subsets, activation states, and functions represents a critical frontier in immunology research with profound implications for understanding cancer, autoimmune diseases, and infectious diseases. Traditional T cell classification based on discrete subsets (Th1, Th2, Th17, etc.) has been fundamentally challenged by single-cell RNA sequencing (scRNA-seq) technologies, which reveal a continuum of T cell states without clearly distinct clusters [96] [3]. This paradigm shift has created an urgent need for new analytical frameworks and standardized validation approaches that can reliably capture the complexity of T cell biology.

The emerging consensus recognizes that a cell's transcriptome reflects the expression of multiple gene expression programs (GEPs)—co-regulated gene modules reflecting distinct biological functions such as cell type, activation states, life cycle processes, or external stimuli responses [3]. However, the predominant analysis approach of clustering has significant limitations for interpreting T cell profiles, as it forces cells into discrete groups that cannot easily reflect the multiplicity of GEPs they express [3]. This methodological challenge underscores the critical importance of robust validation frameworks for single-cell annotation methods.

Within this context, benchmarking studies have become essential for establishing emerging standards and best practices for validation. This comparison guide objectively evaluates the performance of leading computational methods for T cell annotation, with a specific focus on their experimental validation, reproducibility across datasets, and applicability to diverse research contexts.

Methodological Framework: From Component-Based Models to Standardized Annotation

The Shift from Clustering to Component-Based Models

Component-based models such as nonnegative matrix factorization (NMF), hierarchical Poisson factorization, and SPECTRA address key limitations of clustering approaches by modeling GEPs as gene expression vectors and transcriptomes as weighted mixtures of GEPs [3]. Unlike principal component analysis (PCA), NMF components correspond to biologically interpretable GEPs reflecting cell types and functional states that additively contribute to a transcriptome [3]. This fundamental methodological advancement enables:

• Simultaneous quantification of multiple biological programs within individual cells
• Continuous representation of cell states rather than arbitrary discretization
• Cross-dataset comparison using GEP vectors as a fixed coordinate system

The TCAT/stARcat Computational Framework

The T-CellAnnoTator (TCAT) pipeline represents a sophisticated implementation of component-based modeling specifically designed for T cell annotation [96] [3]. TCAT simultaneously quantifies predefined GEPs capturing activation states and cellular subsets, advancing beyond traditional clustering-based approaches. The broader starCAT framework (with "star" as a wildcard placeholder) generalizes this approach across tissues and cell types [3].

The TCAT workflow incorporates two critical computational innovations:

• Augmented consensus NMF (cNMF): An enhanced version of the published cNMF algorithm that incorporates batch correction compatible with nonnegative matrix factorization and integrates surface protein measurements for CITE-seq datasets to improve GEP interpretability [3].

• Reference-based projection: The starCAT algorithm enables GEPs learned in reference datasets to be transferred to new query datasets using nonnegative least squares regression, providing a consistent representation of cell states across biological contexts [3].

The following diagram illustrates the integrated TCAT/stARcat workflow for reproducible cell state annotation:

Figure 1: TCAT/stARcat Workflow for Reproducible Cell State Annotation. The framework establishes a fixed catalog of gene expression programs (GEPs) from a multi-dataset T cell atlas, which can then be projected onto new query datasets for consistent annotation.

Experimental Benchmarking: Performance Comparison of Annotation Methods

Benchmarking Design and Validation Metrics

Comprehensive benchmarking of single-cell annotation methods requires careful experimental design incorporating multiple validation approaches. The emerging standard for validation in this field integrates:

• Simulation studies where ground truth is known, enabling quantitative accuracy assessment
• Cross-dataset reproducibility analysis measuring concordance of GEPs across biological contexts
• Method comparison against established benchmarks and manual expert annotation
• Biological validation through association with independent cellular measurements

For imaging-based spatial transcriptomics data, a recent benchmark established a practical workflow for preparing high-quality single-cell RNA references and evaluating accuracy across multiple annotation tools [13]. This approach emphasizes the importance of using paired single-nucleus RNA sequencing (snRNA-seq) profiles as references to minimize variability between reference and query datasets.

Performance Comparison of Reference-Based Annotation Methods

A systematic benchmarking study evaluated five reference-based cell type annotation methods (SingleR, Azimuth, RCTD, scPred, and scmapCell) against manual annotation using marker genes on 10x Xenium data of human breast cancer [13]. The study utilized a paired 10x Flex single-nucleus RNA sequencing (snRNA-seq) profile as a reference to minimize variability, with careful quality control and normalization procedures.

Table 1: Performance Comparison of Reference-Based Cell Annotation Methods

Method	Accuracy vs. Manual Annotation	Running Time	Ease of Use	Key Strengths	Key Limitations
SingleR	Highest concordance	Fast	Easy to use	High accuracy, speed, simple implementation	Limited customization options
Azimuth	High concordance	Moderate	Intermediate	Pre-built references, automated workflow	Requires reference preparation
RCTD	Moderate concordance	Slow	Complex	Designed for spatial data, accounts for mixture	Computationally intensive, complex parameter tuning
scPred	Moderate concordance	Moderate	Intermediate	Machine learning approach, probabilistic assignments	Requires model training
scmapCell	Lower concordance	Fast	Easy to use	Simple projection method	Lower accuracy for rare cell types
TCAT	High cross-dataset reproducibility	Varies by query size	Programmatic or web interface	Captures continuous states, 46 reproducible GEPs	Requires large reference atlas

The benchmarking results demonstrated that SingleR performed best for the Xenium platform, being fast, accurate, and easy to use, with results closely matching manual annotation [13]. TCAT has shown exceptional performance for capturing continuous T cell states across diverse biological contexts, identifying 46 reproducible GEPs from 1.7 million T cells across 38 tissues and five disease contexts [3].

TCAT Validation Through Simulation and Biological Concordance

The TCAT framework was rigorously validated through comprehensive simulation studies where reference and query datasets had only partially overlapping GEPs [3]. In these simulations, TCAT accurately inferred the usage of GEPs overlapping between reference and query (Pearson R > 0.7) and predicted low usage of extra GEPs in the reference that were not in the query [3].

Strikingly, TCAT obtained better concordance with simulated ground truth GEP usages than direct application of cNMF to the query, despite the reference GEPs having extra or missing GEPs relative to the query [3]. This performance advantage was particularly pronounced for smaller query datasets, demonstrating that TCAT maintains performance across dataset sizes while de novo cNMF performance declines with smaller sample sizes [3].

Biological validation revealed that GEPs identified by TCAT were highly reproducible across datasets, with 9 cGEPs supported by all seven datasets analyzed (mean Pearson R = 0.81) and 49 by two or more datasets (mean R = 0.74) [3]. This cross-dataset reproducibility substantially exceeded that of gene expression principal components, highlighting the robustness of the identified biological programs [3].

Experimental Protocols for Method Validation

Reference Dataset Preparation Protocol

Establishing high-quality reference datasets is foundational for reliable cell annotation. The emerging best practices include:

• Multi-dataset integration: Combine data from multiple studies to capture biological diversity while accounting for batch effects. The TCAT reference incorporated 1.7 million T cells from 700 individuals across 38 tissues and five disease contexts [3].

• Rigorous quality control: Remove low-quality cells, doublets, and potential artifacts. The benchmarking protocol for Xenium data removed cells annotated as "Unlabeled" and predicted potential doublets using scDblFinder [13].

• Appropriate normalization: Apply method-specific normalization procedures. For Azimuth, the protocol used SCTransform function in Seurat for reference normalization, while for other methods, standard LogNormalize was sufficient [13].

• Batch effect correction: Implement compatible correction methods that maintain nonnegative values required for NMF-based approaches. TCAT adapted Harmony to provide batch-corrected nonnegative gene-level data [3].

Method-Specific Parameter Configuration

Optimal performance of annotation methods requires careful parameter configuration:

• SingleR: Use default parameters with fine-tuning of quantile settings for improved resolution of similar cell types [13].

• Azimuth: Generate specialized references using AzimuthReference function and apply RunAzimuth for query projection with return.model = TRUE to enable UMAP projection [13].

• RCTD: Adjust critical parameters including UMImin, countsMIN, genecutoff, fccutoff, fccutoffreg (set to 0); UMIminsigma (set to 1); and CELLMININSTANCE (set to 10) to retain all cells in spatial data [13].

• TCAT: Apply cNMF to each batch-corrected dataset independently with multiple replicates (k-means consensus) followed by cross-dataset GEP clustering to establish consensus GEPs [3].

Validation Metrics and Statistical Assessment

Comprehensive method validation should incorporate multiple statistical measures:

• Concordance with manual annotation: Calculate proportion agreement for cell type assignments compared to expert curation using marker genes [13].

• Cross-dataset reproducibility: Quantify Pearson correlation of GEP spectra across independent datasets [3].

• Simulation accuracy: Compare inferred usages to known ground truth in simulated data using Pearson R [3].

• Biological coherence: Assess enrichment of known marker genes and pathways in annotated cell populations [3].

Table 2: Essential Research Reagents and Computational Resources for T Cell Annotation

Category	Specific Resource	Function/Application	Key Features
Experimental Platforms	10x Xenium	Imaging-based spatial transcriptomics	Single-cell resolution, 100s of genes
	10x Visium	Sequencing-based spatial transcriptomics	Whole transcriptome, tissue architecture
	CITE-seq	Cellular indexing of transcriptomes and epitopes	Integrated RNA and protein measurement
Reference Data	Human Cell Atlas	Comprehensive reference cell states	Multi-tissue, multi-donor integration
	TCAT T cell catalog	46 consensus GEPs for T cells	Disease-specific activation states
Computational Tools	SingleR	Reference-based cell type annotation	Fast, accurate, easy implementation
	TCAT/starCAT	GEP-based annotation framework	Continuous states, cross-dataset reproducibility
	Azimuth	Automated reference mapping	Pre-built references, Seurat integration
	RCTD	Cell type decomposition for spatial data	Accounts for mixture models, spatial context
Software Environments	Seurat	Single-cell analysis toolkit	Comprehensive workflow, spatial support
	Bioconductor	Genomic analysis packages	Interoperability, standardized objects
	Scanpy	Python-based single-cell analysis	Scalability, Python ecosystem integration

Emerging Standards and Implementation Recommendations

Standards for T Cell Nomenclature and Annotation

The evolving understanding of T cell biology has prompted the development of new nomenclature guidelines advocating for:

• Explicit operational definitions: Research publications should clearly define the experimental basis for subset designations in the methods section, providing transparency on the rationale for subset designations [97].

• Modular nomenclature: This emerging paradigm eschews conceptualization of antigen-experienced T cells as belonging to a few idealized subsets, instead simply indicating individual biological properties present in a T cell population with brief descriptors [97].

• Standardized marker definitions: Establish consistent marker combinations for major T cell differentiation states (naive, memory, effector) with species-specific considerations [97].

Best Practices for Validation and Reporting

Based on the consensus from multiple benchmarking studies, the following best practices are recommended:

• Implement multi-level validation: Combine simulation studies, cross-dataset reproducibility assessment, and biological validation using independent methodologies.

• Report comprehensive metrics: Include accuracy measures, computational performance, and ease-of-use assessments to provide complete method characterization.

• Utilize appropriate references: Employ paired references when possible (e.g., snRNA-seq from same tissue) to minimize technical variability.

• Address method-specific limitations: Select methods based on specific research contexts—SingleR for standard annotation tasks, RCTD for complex spatial mixtures, and TCAT for capturing continuous T cell activation states.

• Ensure reproducibility: Document all parameters, software versions, and reference sources to enable method replication and comparison across studies.

The following diagram illustrates the integrated validation framework recommended for benchmarking single-cell annotation methods:

Figure 2: Multi-dimensional Framework for Annotation Method Validation. Comprehensive benchmarking integrates simulation studies, cross-dataset reproducibility, comparison to manual annotation, and biological validation.

The emerging standards for validation of single-cell annotation methods emphasize reproducible, biologically meaningful characterization of cell states rather than mere technical performance metrics. The field is transitioning from discrete clustering approaches toward component-based models that capture the continuous nature of cellular identity, particularly evident in T cell biology.

TCAT represents a significant advancement for T cell annotation, providing a reproducible framework that identifies 46 consensus gene expression programs across diverse biological contexts [3]. For standard annotation tasks, particularly with spatial transcriptomics data, SingleR demonstrates excellent performance with balanced accuracy, speed, and usability [13].

Critical to advancing the field is the adoption of standardized validation practices that integrate multiple evidence sources, clear reporting standards, and modular nomenclature systems that reflect the continuous nature of cellular identity. As single-cell technologies continue to evolve, these validation frameworks will ensure that biological insights remain robust, reproducible, and meaningful across diverse research contexts.

Conclusion

The field of T cell annotation in single-cell genomics has progressed significantly, with multiple robust methods now available for researchers. Benchmarking studies consistently show that while general-purpose classifiers like SVM perform well, specialized tools such as TCAT, STCAT, and SingleR offer advantages for capturing the complex continuum of T cell states. The emerging consensus emphasizes a two-step approach combining automated annotation with expert validation, particularly for challenging subsets like unconventional T cells and rare populations. Future directions will likely involve greater integration of multimodal data, including paired TCR sequences and protein measurements, more sophisticated foundation models pretrained on massive cell atlases, and improved methods for spatial context analysis. These advances will further enhance our ability to decipher T cell biology in health and disease, ultimately accelerating therapeutic development in immunology and oncology. As the field moves toward standardized benchmarking practices and more comprehensive reference atlases, researchers should prioritize method selection based on their specific biological questions, tissue contexts, and available computational resources.

References

• 1. T Cell - an overview | ScienceDirect Topics [https://www.sciencedirect.com/topics/biochemistry-genetics-and-molecular-biology/t-cell]
• 2. T cell - Wikipedia [https://en.wikipedia.org/wiki/T_cell]
• 3. Reproducible single-cell annotation of programs ... [https://www.nature.com/articles/s41592-025-02793-1]
• 4. Reproducible single cell annotation of programs underlying T ... [https://pmc.ncbi.nlm.nih.gov/articles/PMC11092745/]
• 5. Comprehensive and advanced T cell cluster analysis for ... [https://www.frontiersin.org/journals/immunology/articles/10.3389/fimmu.2025.1491041/full]
• 6. T cell subsets of urine-derived lymphocytes (UDLs) serve ... [https://pmc.ncbi.nlm.nih.gov/articles/PMC12496083/]
• 7. Making drugs from T cells: The quantitative pharmacology ... [https://www.nature.com/articles/s41540-024-00355-3]
• 8. Model‐informed drug development of autologous CAR‐T cell ... [https://pmc.ncbi.nlm.nih.gov/articles/PMC10681459/]
• 9. Current annotation strategies for T cell phenotyping of single ... [https://pmc.ncbi.nlm.nih.gov/articles/PMC10768068/]
• 10. An overview of computational methods in single-cell ... [https://pmc.ncbi.nlm.nih.gov/articles/PMC12065632/]
• 11. A Cell Type Annotation Tool for Single-Cell RNA-seq Data [https://www.frontiersin.org/journals/genetics/articles/10.3389/fgene.2020.00490/full]
• 12. Comparison of cell type annotation algorithms for revealing ... [https://www.frontiersin.org/journals/systems-biology/articles/10.3389/fsysb.2022.1026686/full]
• 13. Benchmarking cell type annotation methods for 10x Xenium ... [https://bmcbioinformatics.biomedcentral.com/articles/10.1186/s12859-025-06044-0]
• 14. Evaluation of Cell Type Annotation R Packages on Single- ... [https://www.sciencedirect.com/science/article/pii/S1672022920301443]
• 15. Advancing Automated Cell Type Annotation with Large ... [https://www.sciencedirect.com/science/article/pii/S2001037025004775]
• 16. High-definition spatial transcriptomic profiling of immune ... [https://www.nature.com/articles/s41588-025-02193-3]
• 17. High resolution tissue and cell type identification via single ... [https://journals.plos.org/plosone/article?id=10.1371/journal.pone.0318151]
• 18. Systematic discovery of CRISPR-boosted CAR T cell ... [https://www.nature.com/articles/s41586-025-09507-9]
• 19. In vitro morphological profiling of T cells predicts clinical ... [https://www.nature.com/articles/s41467-025-60224-3]
• 20. Metabolic profiling of antigen-specific CD8+ T cells by ... [https://www.sciencedirect.com/science/article/pii/S2667237525002218]
• 21. New light-based technology arranges cellular messengers ... [https://www.news-medical.net/news/20251118/New-light-based-technology-arranges-cellular-messengers-with-unprecedented-precision.aspx]
• 22. Supervised vs. Unsupervised Learning: What's the ... [https://www.ibm.com/think/topics/supervised-vs-unsupervised-learning]
• 23. Supervised vs Unsupervised Learning - Difference ... [https://aws.amazon.com/compare/the-difference-between-machine-learning-supervised-and-unsupervised/]
• 24. Comparison of Supervised, Unsupervised, Semi-Supervised ... [https://ifoadatascienceresearch.github.io/tutorial/comparison/]
• 25. High-precision cell-type mapping and annotation of single-cell ... [https://genomebiology.biomedcentral.com/articles/10.1186/s13059-025-03773-6]
• 26. Sketching T cell atlases in the single‐cell era: challenges and ... [https://pmc.ncbi.nlm.nih.gov/articles/PMC12392707/]
• 27. Interpretation of T cell states from single ... [https://www.nature.com/articles/s41467-021-23324-4]
• 28. carmonalab/ProjecTILs: Interpretation of cell states using ... [https://github.com/carmonalab/ProjecTILs]
• 29. An automatic annotation tool and reference database for T ... [https://www.sciencedirect.com/science/article/pii/S2095927325002889]
• 30. T Cell Receptor Sequence Database [https://www.cd-genomics.com/resource-t-cell-receptor-sequence-database.html]
• 31. Identifying gene expression programs of cell-type identity ... [https://elifesciences.org/articles/43803]
• 32. Non-Negative Matrix Factorization for the Analysis of Complex ... [https://pmc.ncbi.nlm.nih.gov/articles/PMC2623306/]
• 33. Nonnegative spatial factorization applied to spatial genomics [https://www.nature.com/articles/s41592-022-01687-w]
• 34. T-CellAnnoTator – reproducible single-cell annotation of ... [https://www.rna-seqblog.com/t-cellannotator-reproducible-single-cell-annotation-of-programs-underlying-t-cell-subsets-activation-states-and-functions/]
• 35. Supervised discovery of interpretable gene programs from ... [https://www.nature.com/articles/s41587-023-01940-3]
• 36. Benchmarking deep learning methods for ... - Genome Biology [https://genomebiology.biomedcentral.com/articles/10.1186/s13059-025-03869-z]
• 37. SingleCellGGM enables gene expression program ... [https://pmc.ncbi.nlm.nih.gov/articles/PMC11294836/]
• 38. Benchmarking cell type annotation methods for 10x Xenium ... [https://pmc.ncbi.nlm.nih.gov/articles/PMC11744978/]
• 39. Benchmarking single cell transcriptome matching methods ... [https://pubmed.ncbi.nlm.nih.gov/40568082/]
• 40. ProjectSVR: mapping single-cell RNA-seq data to ... [https://pmc.ncbi.nlm.nih.gov/articles/PMC12599310/]
• 41. Multimodal hierarchical classification of CITE-seq data ... [https://pmc.ncbi.nlm.nih.gov/articles/PMC11840950/]
• 42. Current annotation strategies for T cell phenotyping of ... [https://www.frontiersin.org/journals/immunology/articles/10.3389/fimmu.2023.1306169/full]
• 43. Evaluation of Cell Type Annotation R Packages on Single ... [https://pmc.ncbi.nlm.nih.gov/articles/PMC8602772/]
• 44. A comparison of automatic cell identification methods for ... [https://genomebiology.biomedcentral.com/articles/10.1186/s13059-019-1795-z]
• 45. Biology-driven insights into the power of single-cell foundation ... [https://genomebiology.biomedcentral.com/articles/10.1186/s13059-025-03781-6]
• 46. Benchmarking cell type and gene set annotation by large ... [https://www.nature.com/articles/s41467-025-64511-x]
• 47. Reproducible single-cell annotation of programs underlying T ... [https://pubmed.ncbi.nlm.nih.gov/40903640/]
• 48. LLM4Cell: A Survey of Large Language and Agentic ... [https://arxiv.org/html/2510.07793v2]
• 49. Analysis Approaches for Spatial Gene Expression Data [https://www.10xgenomics.com/analysis-guides/spatial-gex-lit-review]
• 50. Optimizing Xenium In Situ data utility by quality assessment ... [https://www.nature.com/articles/s41592-025-02617-2]
• 51. Comparing Xenium 5K and Visium HD data from identical ... [https://pmc.ncbi.nlm.nih.gov/articles/PMC12298044/]
• 52. Systematic benchmarking of high-throughput subcellular ... [https://pmc.ncbi.nlm.nih.gov/articles/PMC12534522/]
• 53. Spatially mapping T cell receptors and transcriptomes ... [https://pmc.ncbi.nlm.nih.gov/articles/PMC9745674/]
• 54. Comparison of imaging based single-cell resolution spatial ... [https://www.nature.com/articles/s41467-025-63414-1]
• 55. Batch Effect Correction [https://www.10xgenomics.com/analysis-guides/introduction-batch-effect-correction]
• 56. Batch effects and the effective design of single-cell gene ... [https://pmc.ncbi.nlm.nih.gov/articles/PMC5206706/]
• 57. Missing data and technical variability in single-cell RNA ... [https://pmc.ncbi.nlm.nih.gov/articles/PMC6215955/]
• 58. Benchmarking automated cell type annotation tools for ... [https://pmc.ncbi.nlm.nih.gov/articles/PMC9792779/]
• 59. Benchmarking of T cell receptor-epitope predictors with ... [https://pmc.ncbi.nlm.nih.gov/articles/PMC12366652/]
• 60. The Emerging Roles and Therapeutic Potential of ... [https://pmc.ncbi.nlm.nih.gov/articles/PMC12459626/]
• 61. Integrated bulk RNA and single-cell analysis with ... [https://journals.plos.org/plosone/article?id=10.1371/journal.pone.0322326]
• 62. Unconventional T cells in anti-cancer immunity [https://www.frontiersin.org/journals/immunology/articles/10.3389/fimmu.2025.1618393/full]
• 63. Unconventional T Cells' Role in Cancer: Unlocking Their ... [https://www.mdpi.com/2073-4409/14/10/720]
• 64. Embracing the dropouts in single-cell RNA-seq analysis [https://www.nature.com/articles/s41467-020-14976-9]
• 65. Precision and Accuracy in Quantitative Measurement of Gene ... [https://pmc.ncbi.nlm.nih.gov/articles/PMC11042208/]
• 66. The impact of dropouts in scRNAseq dense neighborhood ... [https://pmc.ncbi.nlm.nih.gov/articles/PMC11992407/]
• 67. cnnImpute: missing value recovery for single cell RNA ... [https://www.nature.com/articles/s41598-024-53998-x]
• 68. Challenges in Single-Cell RNA Seq Data Analysis & ... [https://www.elucidata.io/blog/challenges-and-solutions-in-single-cell-rna-seq-data-analysis]
• 69. The misuse of unsupervised clustering for T-cell annotation [https://www.sciencedirect.com/science/article/pii/S2667119025000163]
• 70. Reproducible annotation of T cell subsets and activation ... [https://pubmed.ncbi.nlm.nih.gov/40903642/]
• 71. Controlled activation modulates T-cell expansion and ... [https://www.sciencedirect.com/science/article/pii/S146532492500060X]
• 72. The OT-II model reveals dual in vitro and in vivo ... [https://www.frontiersin.org/journals/immunology/articles/10.3389/fimmu.2025.1571590/full]
• 73. Identification of potential new T cell activation molecules [https://www.nature.com/articles/s41598-024-73003-9]
• 74. Staggered starts in the race to T cell activation - PMC [https://pmc.ncbi.nlm.nih.gov/articles/PMC7612485/]
• 75. Design principles of the cytotoxic CD8 + T-cell response [https://arxiv.org/html/2509.22997v1]
• 76. Consensus prediction of cell type labels in single-cell data ... [https://www.nature.com/articles/s41588-024-01993-3]
• 77. scExtract: leveraging large language models for fully ... [https://genomebiology.biomedcentral.com/articles/10.1186/s13059-025-03639-x]
• 78. High-precision cell-type mapping and annotation of single ... [https://pmc.ncbi.nlm.nih.gov/articles/PMC12502291/]
• 79. Get, set, annotate - an overview of cell type prediction ... [https://www.cellkb.com/blog/cell_type_annotation_review]
• 80. Systematic benchmarking of high-throughput subcellular ... [https://www.nature.com/articles/s41467-025-64292-3]
• 81. A comparison of automatic cell identification methods for ... [https://www.rna-seqblog.com/a-comparison-of-automatic-cell-identification-methods-for-single-cell-rna-sequencing-data/]
• 82. A single cell RNAseq benchmark experiment embedding “ ... [https://www.nature.com/articles/s41597-024-03002-y]
• 83. Comparative benchmarking of single-cell clustering ... [https://genomebiology.biomedcentral.com/articles/10.1186/s13059-025-03719-y]
• 84. Recent advances in T-cell receptor repertoire analysis [https://www.sciencedirect.com/science/article/pii/S2667119022000015]
• 85. Comparison, calibration, and benchmarking of high ... [https://explore.data.humancellatlas.org/projects/a9c022b4-c771-4468-b769-cabcf9738de3]
• 86. RNA-seq based T cell repertoire extraction compared with ... [https://pubmed.ncbi.nlm.nih.gov/40191834/]
• 87. PCLDA: An interpretable cell annotation tool for single- ... [https://pmc.ncbi.nlm.nih.gov/articles/PMC12329077/]
• 88. Evaluation of cell type annotation reliability using a large ... [https://www.nature.com/articles/s42003-025-08745-x]
• 89. Improved T receptor antigen pairing through cell -driven filtering of... data [https://elifesciences.org/articles/81810]
• 90. of Benchmarking receptor repertoire... | Nature Biotechnology T cell [https://www.nature.com/articles/s41587-020-0656-3?error=cookies_not_supported&code=f8311462-4aaf-4cf4-b37f-be84fc377b79]
• 91. Systematic comparison of sequencing-based spatial ... [https://www.nature.com/articles/s41592-024-02325-3]
• 92. Benchmarking clustering, alignment, and integration methods ... [https://genomebiology.biomedcentral.com/articles/10.1186/s13059-024-03361-0]
• 93. Driving Innovation in Spatial Transcriptomics [https://bonsailab.com/driving-innovation-in-spatial-transcriptomics-benchmarking-high-throughput-platforms/]
• 94. Benchmarking spatial transcriptomics technologies with the ... [https://pmc.ncbi.nlm.nih.gov/articles/PMC11954323/]
• 95. A Meta-Review of Spatial Transcriptomics Analysis Software [https://pmc.ncbi.nlm.nih.gov/articles/PMC12293887/]
• 96. Reproducible single cell annotation of programs underlying T-cell subsets, activation states, and functions - PubMed [https://pubmed.ncbi.nlm.nih.gov/38746317/]
• 97. Guidelines for T cell nomenclature [https://www.nature.com/articles/s41577-025-01238-2]