Chatomics Field GuideWhat They Don't Teach You

Sanity check · Spatial Transcriptomics

How to Choose Cell QC Thresholds in Spatial Transcriptomics

Copying the 10% mito cutoff from a PBMC tutorial into your tumor Visium slide will quietly delete every high-mitochondrial tumor spot you came to study.

By Ming "Tommy" Tang, Director of Bioinformatics in Big Pharma · Reviewed September 2026 · 4 min read

You ran the Visium or Xenium vignette, pasted in percent.mt < 5% and nFeature > 200 from a PBMC tutorial, and moved on to clustering. Everything downstream looks fine: UMAP has clean islands, marker genes make sense for the clusters you kept. Nothing in the pipeline warns you that the cutoff was wrong.

The cost shows up later, when a pathologist asks where the tumor cells went, or when a reviewer points out that your "immune infiltrate" cluster is defined by low counts, not by markers. In solid tissue, high mitochondrial content and low RNA content are often real cell biology, not damage. A fixed threshold copied from dissociated PBMCs cannot tell the difference, and it deletes the cells you were trying to study.

This page gives you a concrete order of operations: what to plot before setting any threshold, how to compute a per-sample cutoff instead of a borrowed one, and how to check what you actually removed against the tissue image. Budget under an hour to run these checks on one sample before you lock a threshold for the rest of the experiment.

What it looks like when it's happening

  • A violin plot of percent.mt shows a long right tail that overlaps almost entirely with one cluster in your UMAP.
  • After filtering at percent.mt < 5%, an entire expected cell type disappears from the object (cardiomyocytes, hepatocytes, tumor epithelium).
  • Spots flagged as low quality on nCount or nFeature form a visible ring or patch when plotted back onto the H&E image, instead of scattering randomly.
  • Applying the same fixed cutoff to two tissue sections from the same experiment removes 2% of spots in one section and 40% in the other.
  • Xenium or CosMx cells fail an nFeature > 200 filter carried over from scRNA-seq almost universally, because the panel only covers a few hundred genes total.
  • A cluster in your downstream analysis is defined mainly by high mito percentage and low counts, not by real marker genes.
  • Cells-per-spot metadata, when available, shows spots with more than 10 nuclei sitting right where the 'failed QC' spots cluster on the image.

Why it happens

Mitochondrial percentage is a proxy for cell damage in dissociated scRNA-seq: cells that burst during dissociation leak cytoplasmic RNA and retain mostly mitochondrial transcripts, so their reads look mito-heavy. In situ, that logic breaks down. Cardiomyocytes, hepatocytes, and many tumor cells run mitochondria-dense as part of normal metabolism, often in the 20 to 30 percent range. A single fixed cutoff borrowed from a PBMC tutorial cannot distinguish a dying cell from a metabolically active cell type. The same problem applies at the low end: plasma cells and neutrophils are naturally RNA-sparse, so a minimum count filter can remove them wholesale.

Visium adds a second layer: a 55 micron spot is not a cell. It pools RNA from roughly 1 to 10 cells, so the mito percentage and count you measure per spot is a mixture, not a single cell's readout. A spot sitting over mixed tumor and stroma looks different from a spot over pure stroma, and none of the QC logic built for dissociated single cells accounts for that averaging. Tissue-level artifacts, folds, bubbles, off-tissue background, necrotic zones, also produce QC failures that are spatially structured rather than randomly scattered, which is exactly what makes plotting excluded spots back onto the image so diagnostic.

Imaging-based platforms invert the problem. Xenium, MERFISH, and CosMx give true single-cell resolution but only measure a targeted panel of a few hundred to a few thousand genes. A gene-count floor like 200, trivial to clear with whole-transcriptome scRNA-seq, can fail most cells in a panel-based dataset for no biological reason. Total counts per cell there also scale with cell segmentation area and probe density, not purely RNA content, so importing scRNA-seq thresholds into an imaging-based analysis measures the wrong thing.

The checks

Run them in order. Each one tells you what healthy looks like and what the problem looks like.

0/7 checked · saved in this browser

  1. Compute percent.mt and plot nCount, nFeature, and percent.mt as violin or histogram plots split by sample or tissue section, not pooled together.

    r
    spatial_obj[["percent.mt"]] <- PercentageFeatureSet(spatial_obj, pattern = "^MT-")
    VlnPlot(spatial_obj, features = c("nCount_Spatial", "nFeature_Spatial", "percent.mt"),
            pt.size = 0, group.by = "orig.ident")
    Healthy
    Each sample shows a roughly unimodal distribution for counts and features, with mito percentage forming a visible bulk plus a smaller tail. Medians can differ between samples; that's expected.
    Red flag
    A bimodal distribution, a spike of spots at near-zero counts, or one section's median mito running two to three times higher than another section processed the same way, which points to a batch or tissue-quality effect, not a shared threshold.
  2. Flag outliers as more than 3 median absolute deviations from that sample's own median, on the log scale for count-based metrics, computed separately per sample or section.

    r
    library(scuttle)
    qc_mito <- isOutlier(spatial_obj$percent.mt, nmads = 3, type = "higher",
                          batch = spatial_obj$orig.ident)
    qc_lib  <- isOutlier(spatial_obj$nCount_Spatial, log = TRUE, nmads = 3, type = "lower",
                          batch = spatial_obj$orig.ident)
    Healthy
    The numeric cutoff differs across samples and lines up with where the tail visibly separates from the bulk of the histogram from check 1.
    Red flag
    The same threshold value comes out for every sample (a sign you forgot to set batch=, or its Python equivalent), or the cutoff falls in the middle of the distribution instead of in the tail.
  3. Color a spatial plot by QC pass/fail using SpatialFeaturePlot in Seurat or sq.pl.spatial_scatter in Squidpy, then look at where the failing spots sit relative to the H&E or DAPI image.

    python
    adata.obs["qc_fail"] = adata.obs["pct_counts_mt"] > mito_threshold
    sq.pl.spatial_scatter(adata, color="qc_fail", shape=None)
    Healthy
    Failing spots are scattered close to random, or concentrated at the physical edge of the tissue, over folds, bubbles, or annotated necrotic regions.
    Red flag
    Failing spots form a coherent patch overlapping a real tissue structure, a tumor nest, a gland, a cell-dense layer, which means the filter is deleting a real region, not noise.
  4. Sum the negative-control-probe and system-control columns per cell and look at the distribution, rather than assuming a fixed count-per-cell threshold covers QC on its own.

    python
    neg_cols = [c for c in adata.obs.columns if "NegControl" in c or "control" in c.lower()]
    adata.obs["neg_probe_count"] = adata.obs[neg_cols].sum(axis=1)
    (adata.obs["neg_probe_count"] > 0).mean()
    Healthy
    Most cells carry zero negative-probe counts; a modest tail gets flagged, roughly the 16 percent range reported for one public Xenium dataset in the OSTA book.
    Red flag
    A large fraction of cells carry nonzero negative-probe counts, which points to segmentation bleed-through between neighboring cells or a panel/probe problem, not routine background.
  5. If nuclei segmentation counts are available in the spot metadata, look at spots with unusually high cell counts (the OSTA DLPFC example flags spots above 10 cells) rather than relying on RNA counts alone.

    Healthy
    Cell counts per spot are consistent with known tissue density for that region, typically single digits for a 55 micron spot.
    Red flag
    Spots with high cells-per-spot values cluster at the same locations as spots already failing count or mito QC, usually tissue edges or folded regions, indicating a shared imaging or segmentation artifact rather than independent biology.
  6. After a first-pass clustering, plot percent.mt and nCount by cluster and check whether any cluster is defined mainly by QC metrics rather than marker genes.

    r
    VlnPlot(spatial_obj, features = c("percent.mt", "nCount_Spatial"),
            group.by = "seurat_clusters", pt.size = 0)
    Healthy
    Mito percentage varies by cluster in a way that tracks known biology, for example higher in a tumor or cardiomyocyte-annotated cluster.
    Red flag
    A cluster's top distinguishing features are nCount and percent.mt rather than real marker genes, which means that cluster is a QC artifact, not a cell type.
  7. Subset the cells or spots your current threshold removed and run a quick clustering and marker check on them alone.

    Healthy
    Excluded cells look like debris or ambient RNA: no coherent marker signature, low complexity across the board.
    Red flag
    Excluded cells form a cluster with a clear, known marker signature (for example immunoglobulin genes marking plasma cells), meaning the threshold deleted a real cell type, not noise.

What to do about it

Replace fixed cutoffs with per-sample MAD-based thresholds

When: Your vignette-copied percent.mt or nCount cutoff removes wildly different fractions of spots across samples in the same experiment.

Compute median absolute deviation per sample (3 MADs is the common default, log scale for count metrics) using scuttle::isOutlier with a batch argument in R, or the scipy median_abs_deviation equivalent in Python, and use that as your cutoff instead of a fixed percentage.

Caveat: MAD assumes a roughly unimodal distribution. A tissue with two very different regions (tumor plus adjacent normal) can produce a bimodal mito distribution that breaks a single global MAD call; compute it separately per annotated region if that happens.

Switch to spatially-aware outlier detection when failures cluster on the image

When: Check 3 shows QC failures forming a coherent patch over a real tissue structure instead of scattering randomly.

Use a local-neighbor method such as SpotSweeper's localOutliers(), which compares each spot to its k nearest spatial neighbors (k=36 in the published example) instead of the whole-slide median, so a genuinely low-count region doesn't get flagged just because it differs from the rest of the tissue.

Caveat: Requires spatial coordinates and an extra package, runs slower than a global cutoff, and you still need to eyeball the result on the image.

Cross-check high-mito spots against known biology before removing them

When: Percent.mt is high but concentrated in a region that plausibly corresponds to a high-metabolism cell type, such as tumor core or a cardiomyocyte layer.

Overlay percent.mt on the spatial plot next to the pathologist's ROI annotation or a marker gene known for that cell type, and only filter if the high-mito spots don't match a coherent, expected structure.

Caveat: This is a judgment call, not an automated cutoff, and ideally involves someone who can read the histology.

Use imaging-specific QC metrics instead of scRNA-seq count thresholds

When: You're working with Xenium, CosMx, or MERFISH and applying a gene-count floor like 200 carried over from whole-transcriptome scRNA-seq.

Drop cells with zero total counts first (subset(xenium.obj, subset = nCount_Xenium > 0) in Seurat, or sc.pp.filter_cells(adata, min_counts=10) in Scanpy), then filter on signal density (transcripts per cell area), cell area and aspect ratio, and exclude any cell with nonzero negative-probe or control counts.

Caveat: There's no universal published cutoff for panel-based platforms yet; these thresholds need to be set and inspected per panel and per tissue, not copied from a tutorial dataset.

Flag, don't blindly drop, high cells-per-spot Visium spots

When: Check 5 shows spots with unusually high nuclei counts, roughly above 10 in the DLPFC reference, clustering at tissue edges or folds.

Treat those spots as a segmentation or tissue-quality flag rather than running them through the standard count filter. Consider excluding them separately or routing them to a deconvolution method built for multi-cell spots instead of a per-cell QC cutoff.

Caveat: Needs a nuclei segmentation step on the image, which not every pipeline runs by default.

When not to "fix" it

If the spots or cells failing your mito or count filter sit inside a region a pathologist has already annotated as tumor, necrotic, or a known high-metabolism cell type, the "problem" is the biology you came to measure. Filtering it out to make the QC plots look clean throws away the signal. Necrotic or hypoxic tissue genuinely has degraded RNA, and that degradation pattern is often itself the phenotype worth mapping, not an artifact to discard before analysis starts.

Five things experienced analysts do here

  1. Compute QC metrics and thresholds per sample or per section, never pooled across an experiment. Tissue-section batch effects routinely outweigh the biological differences you're trying to detect.
  2. Never carry a percent.mt cutoff from a PBMC or blood tutorial into solid tissue without looking at the distribution first. Dissociated blood cells and intact tissue sections fail QC for different reasons.
  3. Plot excluded spots or cells back onto the tissue image before finalizing any threshold. A spatially coherent patch of 'failed' spots is the fastest way to catch a cutoff that's deleting a real structure.
  4. For imaging-based panels, judge cells by signal density, transcripts per unit area, not raw gene or transcript counts. Raw counts conflate panel size and cell size with RNA content.
  5. Keep the table of excluded cells after filtering. Periodically re-cluster it and check for marker genes; if a recognizable cell type keeps showing up in your 'removed' bucket, your threshold is systematically biased, not just noisy.

Questions people ask

What mitochondrial percentage cutoff should I use for spatial transcriptomics?

There is no single number that works across tissues and platforms. Compute a per-sample, MAD-based threshold instead of reusing a scRNA-seq default like 5 to 10 percent. The OSTA book's DLPFC brain example flags Visium spots above roughly 28 to 30 percent mito, but that number is specific to that tissue and chemistry, not a rule to import into a tumor or cardiac dataset.

How many genes per cell should I require in Xenium or MERFISH data?

Imaging panels only cover a few hundred to a few thousand targeted genes, so a whole-transcriptome nFeature cutoff like 200 doesn't transfer. Squidpy's Xenium tutorial instead filters on min_counts=10 total transcripts and min_cells=5 genes detected in at least 5 cells, paired with cell area, negative-probe counts, and signal density rather than a raw gene-count floor.

What is a MAD-based QC threshold and why use it instead of a fixed cutoff?

MAD stands for median absolute deviation. A cell or spot is flagged as an outlier when it falls more than a set number of MADs, commonly 3, from that sample's own median, which retains roughly 99 percent of values for a normal distribution. Because the cutoff is computed from each sample's own distribution, it adapts to real differences between sections instead of applying one borrowed number everywhere.

Should I filter Visium spots with a high cells-per-spot count?

A cells-per-spot value above roughly 10, the threshold used in the OSTA DLPFC example, usually signals a segmentation or tissue-density problem rather than genuine biology. Flag and inspect those spots on the tissue image before deciding whether to exclude them or route them to a deconvolution method built for multi-cell spots.

Can removing high-mitochondrial cells delete real biology in spatial data?

Yes. Tumor cells, cardiomyocytes, and other metabolically active cell types carry naturally high mitochondrial content, so a uniform mito filter can systematically remove exactly the population you're studying. Always check where the flagged spots or cells sit on the tissue image before applying the filter for good.

Related pages

Related reading on the blog

Sources

  1. Quality Control, Orchestrating Spatial Transcriptomics Analysis with Bioconductor — Four core QC metrics, DLPFC example cutoffs, and SpotSweeper's local-neighbor MAD approach for Visium.
  2. Quality Control for Image-based Platforms, Orchestrating Spatial Transcriptomics Analysis with Bioconductor — QC metrics specific to imaging-based platforms: cell area, negative probe counts, signal density.
  3. Seurat - Guided Clustering Tutorial (pbmc3k) — PercentageFeatureSet with the ^MT- pattern and the common 5 percent mito cutoff this page argues against copying blindly.
  4. Analysis of Image-based Spatial Data in Seurat — VlnPlot and subset() commands used to inspect and filter Xenium nCount/nFeature.
  5. Analyze Xenium data, Squidpy Tutorial — min_counts=10 / min_cells=5 filtering example for imaging-based panels.
  6. scanpy.pp.calculate_qc_metrics, Scanpy Documentation — QC metric computation used to build the per-sample distributions checked in this guide.
  7. SpotSweeper: spatially-aware quality control for spatial transcriptomics — Spatially-aware local outlier detection (k=36 neighbors) that avoids deleting real low-count cell types.
  8. Biology-inspired data-driven quality control for scientific discovery in single-cell transcriptomics — Rationale for the 3-MAD outlier threshold used throughout this guide.
  9. Visium HD Workflow, Orchestrating Spatial Transcriptomics Analysis with Bioconductor — Visium HD's 2 µm bins change the count distribution enough that Visium spot-level thresholds don't transfer directly.

Part of the QC thresholds series.