Chatomics Field GuideWhat They Don't Teach You

Sanity check · Spatial Transcriptomics

How to Choose a Normalization Method in Spatial Transcriptomics

Library-size normalization removes the exact tissue-architecture signal you're trying to map, and the scRNA-seq default you already know is often the wrong call here.

By Ming "Tommy" Tang, Director of Bioinformatics in Big Pharma · Reviewed September 2026 · 5 min read

You just brought your Visium or Xenium data into Seurat or Scanpy and ran the same normalization step you'd use for a dissociated scRNA-seq object: SCTransform, or normalize_total followed by log1p. Clusters come out clean, the UMAP looks reasonable, and you move on to labeling domains. Then someone points out that your cluster boundaries line up almost exactly with total UMI counts per spot, or that a region you know from the H&E image is necrotic and RNA-poor and it just vanished into the background of your normalized data.

That's not a coincidence, and it's not a bug in your code. In dissociated single-cell data, library size is overwhelmingly a technical artifact: same cell type, more reads, bigger number, safe to normalize away. In spatial data, total counts per spot also track cell density and tissue architecture, which is biology. Normalize it away with the wrong method and you erase the signal you set out to map, then publish a clean, confident, wrong story about tumor stroma or immune infiltration.

This page walks through which normalization method answers which question, when SCTransform helps and when it actively hurts domain identification, why CPM and TPM were never built for a targeted imaging panel, and how to check in the next hour whether your clusters are tracking biology or just read depth.

What it looks like when it's happening

  • Louvain or Leiden clusters correlate strongly with total UMI counts per spot on the PCA or UMAP.
  • The spatial feature plot of total counts (nCount_Spatial) looks visually identical to your cluster/domain assignment plot on the tissue image.
  • A tissue region you can identify on the H&E image (necrotic core, dense tumor nest, sparse stroma) doesn't show up as its own cluster after SCTransform, but does show up with raw or log-normalized counts.
  • Top differentially expressed genes between Visium clusters are dominated by highly-expressed housekeeping genes with no cell-type specificity.
  • Comparing Xenium or CosMx panel counts to a whole-transcriptome bulk or scRNA-seq reference, every gene looks 'downregulated' simply because the panel has a few hundred to a few thousand genes instead of twenty thousand.
  • After CPM-normalizing a targeted imaging panel, marker gene levels differ across regions you know have similar cell-type composition, but track cell or segmentation area instead.
  • Two serial sections from the same condition, normalized independently, show a systematic shift in cluster proportions that has nothing to do with the biological variable you're testing.

Why it happens

Library-size normalization methods share one core assumption: most genes don't change between the units you're comparing, so any difference in total counts must be technical. That assumption is reasonable when comparing replicate scRNA-seq libraries of the same cell type. It breaks the moment total counts vary because of real biology, and in spatial data that's exactly what happens: a 55-micron Visium spot can contain anywhere from one to ten cells, so total counts per spot track local cell density, which tracks tissue architecture. A dense tumor nest and a sparse stromal region will genuinely differ in RNA content per spot, not because of a sequencing artifact but because they contain a different number and type of cells. Normalizing that away doesn't clean your data, it deletes the thing you're studying.

SCTransform makes this concrete. It fits a regularized negative binomial regression that explicitly models and removes the relationship between gene counts and library size. In dissociated scRNA-seq, that's the correct move: library size there really is mostly technical. In spatial data, library size is confounded with cell-type composition and tissue architecture, so SCTransform strips out both the technical effect and the biological signal riding on the same axis. This is why Genome Biology's 2024 benchmarking found that no normalization at all outperformed SCTransform for spatial domain identification, and why a community discussion on Visium normalization practices found that clustering accuracy was largely insensitive to normalization choice, with SCTransform as the one consistent underperformer. Seurat's own spatial vignette recommends SCTransform but explicitly flags that best-practice normalization for spatial data is still an open, developing question, not a settled answer.

Imaging-based platforms add a second, separate mechanism. On Xenium, MERFISH, and CosMx, you're measuring a targeted panel of a few hundred to a few thousand genes, and total counts per cell reflect probe hybridization efficiency and panel design, not sequencing depth. DESeq2, TMM, CPM, and TPM were all built around the assumption of whole-transcriptome sequencing where depth genuinely varies for technical reasons. Apply that logic to a fixed gene panel and you launder panel-composition effects into your normalized values, distorting signal in region- or cell-type-specific ways rather than removing noise.

A related but distinct error compounds this: comparing panel-based counts directly to a whole-transcriptome reference (bulk RNA-seq, an scRNA-seq atlas) without first restricting to the shared gene set. Every gene in the panel dataset will look artificially depleted relative to the reference, because the comparison never accounted for the fact that one dataset measures 300 genes and the other measures 20,000.

The checks

Run them in order. Each one tells you what healthy looks like and what the problem looks like.

0/8 checked · saved in this browser

  1. Before touching a normalization function, note whether you have Visium/Slide-seq (55-micron spots pooling 1-10 cells, whole-transcriptome coverage) or Xenium/MERFISH/CosMx (single-cell resolution, a panel of a few hundred to a few thousand genes). This single fact eliminates half the candidate methods before you run anything.

    Healthy
    You can state in one sentence which platform you have and whether total counts represent sequencing depth or probe hybridization success.
    Red flag
    You're running the same normalization code you copy-pasted from a Visium tutorial on a Xenium object, or vice versa, without checking whether the assumptions transfer.
  2. Plot total counts (nCount_Spatial in Seurat, total_counts in Scanpy) as a spatial feature plot on the tissue image, and put it side by side with your cluster or domain assignment plot.

    r
    SpatialFeaturePlot(brain, features = "nCount_Spatial")
    SpatialDimPlot(brain, group.by = "seurat_clusters")
    Healthy
    Some texture overlap is normal, dense regions genuinely have more RNA, but the two plots are not interchangeable.
    Red flag
    The total-count heatmap and the cluster map are visually indistinguishable: your clusters are just a discretized library-size gradient.
  3. Compute the correlation between total counts per spot and PC1 (or whichever PC drives your clustering). Do this before you commit to a normalization method and again after, so you know what the normalization actually changed.

    r
    cor(Embeddings(obj, "pca")[, 1], obj$nCount_Spatial)
    Healthy
    A weak-to-moderate correlation that you can explain biologically (e.g., stroma has fewer cells per spot than tumor nests).
    Red flag
    Correlation above roughly 0.7-0.8 with no biological explanation, or a correlation that flips sign after 'fixing' normalization, suggesting you've swapped one artifact for another.
  4. Pull cell area or segmentation volume from your Xenium/MERFISH/CosMx metadata and correlate it against total counts per cell.

    Healthy
    Some positive correlation, since larger cells physically hold more transcript, but it shouldn't fully explain your expression differences across regions.
    Red flag
    Total counts track segmentation area almost perfectly, meaning your 'expression' differences across tissue regions are really a segmentation artifact, not biology.
  5. If you're comparing Xenium/CosMx counts to bulk RNA-seq or a scRNA-seq reference, first intersect the gene sets and confirm what fraction of the panel is shared with the reference.

    Healthy
    You explicitly restrict comparisons to the shared gene set and normalize both datasets to that shared set, not to their full respective gene universes.
    Red flag
    Every gene looks 'downregulated' in the panel dataset relative to the reference, a signature of comparing a few-hundred-gene panel to a 20,000-gene whole-transcriptome library without adjusting for panel size.
  6. Cluster the same Visium object three ways: raw log1p(counts) with no size-factor correction, the standard normalize_total + log1p pipeline, and SCTransform. Compare each cluster map against known tissue regions from the H&E image or pathologist annotation.

    python
    sc.pp.normalize_total(adata, inplace=True)
    sc.pp.log1p(adata)
    # compare against: sc.pp.log1p(adata_raw) with no normalize_total step
    Healthy
    Domain boundaries that line up with histology stay roughly consistent across raw and log-normalized versions; SCTransform is the one most likely to blur or shift them.
    Red flag
    SCTransform clusters lose a domain boundary that's visibly obvious on the H&E image and present in both the raw and log-normalized versions, matching the reported failure mode for SCTransform on spatial domain identification.
  7. If the question is differential expression between conditions (not domain identification within a section), check that counts were aggregated to pseudobulk per sample and passed through DESeq2's size-factor estimation, rather than compared as CPM or TPM values directly.

    r
    dds <- DESeq2::DESeqDataSetFromMatrix(countData, colData, design = ~condition)
    dds <- DESeq2::estimateSizeFactors(dds)
    normalized_counts <- DESeq2::counts(dds, normalized = TRUE)
    Healthy
    A documented size-factor estimation step in the DE pipeline, distinct from whatever normalization was used for clustering.
    Red flag
    Someone ran a t-test or Wilcoxon test directly on CPM or TPM values and called it differential expression between conditions.
  8. If you have multiple tissue sections, cluster each section independently first, then check whether replicate sections from the same condition group together before you make any cross-sample comparison.

    Healthy
    Sections from the same condition show overlapping cluster composition; between-condition differences are visible on top of that baseline agreement.
    Red flag
    PC1 or the top clustering axis separates sections rather than conditions, meaning section-to-section technical variation, not normalization choice, is the dominant confound.

What to do about it

Domain identification and clustering within a Visium section: skip normalization or use plain log1p

When: You're identifying spatial domains or clusters within a single tissue section and total counts correlate with regions you can independently confirm from histology.

Use raw counts or log1p(counts) without a size-factor step. Avoid SCTransform for this task; recent benchmarking found no normalization outperformed SCTransform for domain identification because SCTransform removes the library-size signal that is confounded with real tissue architecture.

Caveat: This only holds within a single section. The moment you compare across sections or samples, unnormalized counts reintroduce technical batch variation you do need to correct.

Cross-sample or cross-condition differential expression: pseudobulk + DESeq2 median-of-ratios

When: The question is condition-level, e.g., tumor vs normal across multiple patients or sections, not within-section domain mapping.

Aggregate spot- or cell-level counts to pseudobulk per sample, then run DESeq2's median-of-ratios normalization, the same approach used for bulk RNA-seq DE.

Caveat: Pseudobulking discards spatial resolution. Only use this when the biological question genuinely lives at the sample level, not within-tissue.

Imaging-based panels (Xenium, MERFISH, CosMx): normalize by cell area instead of library size

When: You're working with a targeted panel where total counts reflect probe hybridization success and panel design rather than sequencing depth.

Use area-based normalization (dividing by cell or segmentation area) instead of library-size methods like CPM, TMM, or SCTransform, which were built around whole-transcriptome sequencing-depth assumptions that don't hold for a targeted panel.

Caveat: This depends entirely on segmentation quality. Bad cell boundaries produce bad area estimates, and the normalization inherits that error directly.

Spatially-aware normalization when you need variance stabilization without losing domain signal

When: SCTransform's tendency to strip biology along with technical noise is a real blocker, but you still want a model-based normalization rather than raw counts.

Evaluate SpaNorm, which uses spatial coordinates alongside expression to decompose spatially-smoothed variation into technical and biological components, retaining more signal than SCTransform, Giotto, scran, or RUV-III-NB on domain-strength benchmarks.

Caveat: This is a newer method with less field-wide validation than SCTransform or DESeq2. Treat results as provisional and check them against a no-normalization baseline before trusting them for a paper.

Comparing panel-based counts to a whole-transcriptome reference: restrict to the shared gene set first

When: You need to compare Xenium or CosMx expression levels to bulk RNA-seq or a scRNA-seq atlas.

Subset both datasets to the intersecting gene panel before normalizing either one, then normalize within that shared set. Don't normalize each dataset independently on its full gene universe and compare the outputs.

Caveat: You lose any genes not in the panel, which can mean losing the exact marker you wanted to validate. State this limitation explicitly when reporting results.

General-purpose exploratory pass: Scanpy's normalize_total + log1p

When: You need a fast, platform-agnostic first look and aren't yet committed to a specific downstream question.

Run sc.pp.normalize_total(adata) followed by sc.pp.log1p(adata) as a first pass, matching the standard Scanpy spatial tutorial workflow.

Caveat: This still assumes most genes don't change between spots, which is the exact assumption that fails when cell-type composition is what varies spatially. Re-run check 2 and 3 above before trusting clusters built on this normalization.

When not to "fix" it

If total counts correlate with your clusters because those clusters are true tissue regions with genuinely different cell densities, e.g., a necrotic core versus viable tumor, or dense epithelium versus sparse stroma, that correlation is the expected biology, not a confound to correct. Forcing SCTransform or aggressive size-factor correction onto that data to "fix" the correlation will flatten a real signal. Before you touch the normalization, check the correlated regions against an independent source: the H&E image, a pathologist's annotation, or known marker genes for the cell types you expect there. If those line up, leave the correlation alone and report it as biology, not artifact.

Five things experienced analysts do here

  1. Check platform type before picking a method. Sequencing-based (Visium) and imaging-based (Xenium, MERFISH, CosMx) platforms need different normalization logic, and that one check eliminates half the candidate methods up front.
  2. Never mix normalization methods within the same comparison. Normalizing one batch with SCTransform and another with CPM and then comparing them directly will produce a difference that's purely methodological.
  3. Keep raw counts in the object as the ground truth layer. Treat every normalized version as a recomputable downstream layer, never the thing you discard the raw data in favor of.
  4. When in doubt, run the analysis with and without normalization and compare the domain structure, since the field itself, per Seurat's own documentation, hasn't settled on a single best method for spatial data yet.
  5. Loop in a pathologist or the H&E image before deciding a library-size-correlated cluster is noise. What looks like a technical artifact from the count matrix alone is often a real tissue region from the slide.

Questions people ask

Should I use SCTransform for spatial transcriptomics?

Use it cautiously. Seurat's own spatial vignette recommends it but flags that spatial normalization best practices are still developing, and a 2024 Genome Biology benchmark found SCTransform underperformed no normalization at all for spatial domain identification, because it removes the library-size signal that's confounded with real tissue architecture. It's a reasonable default for scRNA-seq-style downstream tasks on spatial data, but check domain identification results against histology before trusting it for that specific task.

CPM vs TPM for spatial transcriptomics: which one should I use?

Neither is designed for spatial cross-condition comparisons on its own. CPM adjusts only for sequencing depth; TPM adds a gene-length correction on top. Both are useful for eyeballing a single gene across spots or cells, but neither corrects for RNA composition changes, which is exactly what varies spatially. For differential expression, use DESeq2's median-of-ratios normalization instead, typically on pseudobulked data.

Which normalization should I use for differential expression in spatial transcriptomics?

If the comparison is between conditions or samples, aggregate counts to pseudobulk per sample and run DESeq2's median-of-ratios normalization, the same gold-standard approach used in bulk RNA-seq. Don't run a statistical test directly on CPM or TPM values, they lack the composition correction that a proper DE method needs.

Do Xenium and MERFISH need the same normalization as Visium?

No. Visium's total counts reflect sequencing depth and cell density within a spot; Xenium, MERFISH, and CosMx total counts reflect probe hybridization success against a fixed gene panel, not sequencing depth. Library-size methods built for whole-transcriptome data, including DESeq2, TMM, and SCTransform, can distort signal on a targeted panel. Area-based normalization is a better fit for imaging-based platforms.

What normalization should I use before making a heatmap of spatial RNA-seq data?

It depends on what the heatmap is meant to show. For visualizing domains within one Visium section, log1p on raw or lightly normalized counts is safer than SCTransform, which can blur domain boundaries. For a heatmap comparing conditions across samples, use pseudobulk counts normalized with DESeq2's median-of-ratios so the color scale reflects real expression differences, not depth differences.

Related pages

Related reading on the blog

Sources

  1. Analysis, visualization, and integration of spatial datasets with Seurat — Source for the SCTransform command and Seurat's own caveat that spatial normalization best practices are unsettled.
  2. Library size confounds biology in spatial transcriptomics data — Source for the finding that no normalization outperforms SCTransform for spatial domain identification.
  3. Normalization, Orchestrating Spatial Transcriptomics Analysis with Bioconductor — Source for area-based normalization and the imaging-panel confound with sequencing-based methods.
  4. Analysis and visualization of spatial transcriptomics data — Source for the normalize_total + log1p Scanpy workflow.
  5. Visium normalization best practices — Source for the finding that clustering accuracy is largely insensitive to normalization choice except for SCTransform underperforming.

Part of the Normalization choice series.