Sanity check · Single-Nucleus RNA-seq
How to Tell If You Overclustered in Single-Nucleus RNA-seq
Leiden will happily hand you fifty clusters from thirty cell types if the resolution lets it; the object itself can't tell you which cuts are real.
By Ming "Tommy" Tang, Director of Bioinformatics in Big Pharma · Reviewed September 2026 · 5 min read
You ran the standard pipeline on frozen-tissue nuclei: SCTransform or sc.pp.normalize_total, PCA, FindClusters at the default resolution or a Leiden sweep, and the UMAP came back with 20-plus islands. Now you have to decide which of those islands are cell types you can name in a figure legend and which are the algorithm doing exactly what modularity optimization does: cutting the graph as many times as the resolution parameter permits.
This matters more in snRNA-seq than in whole-cell data. Nuclei carry a different noise profile: sequencing depth swings harder because intronic reads dominate the count matrix, ambient RNA from lysed cytoplasm contaminates a larger share of each droplet, and the mitochondrial-percentage filter means something different than it does in scRNA-seq. Import a scRNA-seq mental model wholesale and you'll either merge real neuronal subtypes into mush or publish a cell type that's actually a library-size artifact.
This page gives you a sequence of checks, cheapest first, to run on a clustered object before you annotate anything. Budget an hour: most of it is plotting metadata you already have, not new analysis.
What it looks like when it's happening
- Two clusters differ by only a handful of genes and none of them is a marker you'd stake a figure caption on
- A cluster shows up reliably in one sample's UMAP and disappears once you integrate across samples
- Cluster boundaries reshuffle completely when you nudge resolution from 0.6 to 0.8
- What looks like two clusters is really one continuous gradient on the UMAP with no clear gap between them
- A cluster's top differentially expressed genes are ribosomal, mitochondrial, or immediate-early stress genes (FOS, JUN, HSPA1A) instead of cell-type markers
- Cluster identity tracks nCount_RNA or nFeature_RNA more tightly than it tracks any expression program
- A 'new' cluster is mostly cell-cycle genes (MKI67, TOP2A) peeling proliferating cells off an otherwise normal population
- A disproportionate fraction of one cluster gets flagged as doublets by DoubletFinder
Why it happens
Louvain and Leiden are modularity optimizers, not cell-type detectors. Raise the resolution parameter and they keep cutting the neighbor graph into smaller pieces, whether or not those pieces correspond to anything biological. Benchmarks on data simulated from a single true cell type found Seurat's default resolution of 0.8 produced false clusters in 100% of replicates, typically 4 to 7 spurious clusters per run. Resolution choice is really a Type I / Type II error tradeoff: push it high and you invent cell types that don't exist, push it low and you merge real ones together. There's no resolution value that avoids both errors for all datasets.
snRNA-seq adds mechanisms that scRNA-seq intuition doesn't prepare you for. Roughly half the signal in a nucleus is unspliced pre-mRNA, so if your reference annotation doesn't count intronic reads, you lose most of your depth and get sparser, noisier per-cell profiles than the same tissue would give you dissociated. That depth is not uniform across nuclei, and cells with more UMIs and more detected genes cluster together in PCA/UMAP by library size alone if you haven't normalized carefully, producing a split that tracks total counts, not cell state. Nuclei preps also carry more ambient RNA than whole-cell preps: cytoplasm lyses during dissociation and its transcripts end up in the surrounding solution, then in every droplet. Since nuclei should contain almost no mitochondrial transcripts, any measurable mito percentage in a "clean" nucleus is a contamination signal, not baseline noise the way it is in scRNA-seq. High ambient contamination and low unspliced fraction move together, and a cluster defined by that combination is a quality gradient, not a cell type.
Biological confounders compound the technical ones. Cell-cycle genes can split proliferating cells of one real type into their own island, and cell-cycle signal is also a documented source of false-positive doublet calls, meaning an overclustered object can simultaneously hide real doublets and manufacture fake cell-type-like divisions from mixed singlet/doublet populations.
None of this is specific to brain tissue, but brain tissue punishes the wrong response to it. Neuronal subtypes genuinely number in the dozens: interneuron subclasses alone routinely split into ten or more real, markerable populations. The generic scRNA-seq heuristic of "does this look like roughly ten to fifteen clusters" doesn't transfer. You have to replace count-based intuition with marker-based and cross-sample stability checks, because a large cluster count is expected biology here, not a red flag by itself.
The checks
Run them in order. Each one tells you what healthy looks like and what the problem looks like.
0/8 checked · saved in this browser
Plot UMAP colored by total UMI count and by number of detected genes per nucleus, before you look at cluster identity at all.
rFeaturePlot(seu, features = c("nCount_RNA", "nFeature_RNA"))- Healthy
- Depth gradients are diffuse across the whole plot, not aligned with a cluster boundary.
- Red flag
- A cluster boundary lines up with a sharp gradient in nCount_RNA or nFeature_RNA, that cluster is probably a library-size split, not a cell type.
Violin- or box-plot percent.mt (Seurat) or pct_counts_mt (Scanpy) grouped by cluster.
pythonsc.pl.violin(adata, ['pct_counts_mt'], groupby='leiden')- Healthy
- Mito percentage is near zero across essentially every cluster, since nuclei should carry almost no mitochondrial transcripts.
- Red flag
- One cluster has visibly elevated mito percentage relative to the rest, that's ambient contamination from lysed cytoplasm, not a distinct nuclear cell state.
Compute the fraction of unspliced reads per nucleus (from spliced/unspliced layers if you counted with an intron-aware reference) and compare it against percent.mt across clusters.
- Healthy
- Unspliced fraction is uniformly high across clusters and does not covary with mito percentage.
- Red flag
- A cluster with low unspliced fraction also has high mito percentage, that combination is the signature of ambient contamination, not a new nuclear population.
Cluster the same object at several resolutions and compare cluster membership across the sweep, either by eye on the UMAP or with a tool like clustree.
pythonfor res in [0.02, 0.5, 2.0]: sc.tl.leiden(adata, key_added=f'leiden_res_{res:4.2f}', resolution=res)- Healthy
- A core set of cluster boundaries is stable across a broad resolution range; only fine substructure appears or disappears as resolution rises.
- Red flag
- Cluster identities reshuffle wholesale between adjacent resolution values, or a cluster only exists in a narrow resolution window, that cluster's existence depends on a parameter choice, not on the data.
Run differential expression of each cluster against the rest and read the top genes.
pythonsc.tl.rank_genes_groups(adata, groupby='leiden', method='wilcoxon')- Healthy
- Top genes are known, cell-type-specific markers you recognize, and you can state the cluster's identity in one sentence.
- Red flag
- Top genes are ribosomal, mitochondrial, stress-response (FOS, JUN), or generic housekeeping genes with no cell-type marker in the top ranks.
Compare your data-derived top markers per cluster against a curated marker gene dictionary for your tissue.
pythonscanpy.tl.marker_gene_overlap(adata, marker_genes_dict, key='leiden')- Healthy
- Clusters that you'd call real cell types show meaningfully higher overlap (jaccard/overlap coefficient) with their expected marker set than with unrelated ones.
- Red flag
- A cluster's overlap scores are flat and low across every reference cell type in your marker dictionary, nothing in your reference explains it.
Cluster and plot the UMAP without running Harmony, scVI, or any batch-correction step, then cross-tabulate cluster membership by sample.
- Healthy
- A real, shared cell type shows up across all samples in roughly consistent proportions (allowing for known biological differences between conditions).
- Red flag
- A cluster is essentially 100% from one sample. That's either a real sample-specific or condition-specific state worth flagging, or a batch/technical artifact, either way, it is not evidence of a universal cell type, and integrating before this check will hide the pattern entirely.
Score cell-cycle genes per cluster and run doublet detection (e.g. DoubletFinder), then check whether a candidate cluster is disproportionately high on either score.
- Healthy
- Cell-cycle scores and doublet fractions are distributed similarly across clusters, or track a known proliferative population you already expect.
- Red flag
- A cluster is dominated by high cell-cycle scores or a high doublet-call rate, you're looking at a cell-cycle split of an existing type, or a doublet population masquerading as a new one.
What to do about it
Lower the resolution or merge clusters that fail the marker check
When: Two or more clusters share no distinguishing marker gene, and their split only appears in a narrow resolution window.
Re-run clustering at a lower resolution, or manually merge the clusters in your object and re-derive markers on the merged group to confirm the combined cluster now has a coherent marker set.
Caveat: Manual merging is a judgment call. Document which clusters you merged and why (which check failed), so the decision is reproducible and reviewable, not just "we picked res=0.6 because it looked right."
Re-normalize or regress out depth before re-clustering
When: A cluster boundary tracks nCount_RNA/nFeature_RNA rather than an expression program.
Confirm you counted with an intron-aware reference (snRNA-seq loses most reads without one), then re-normalize with a depth-aware method and re-cluster on the corrected embedding.
Caveat: Regressing out depth too aggressively can also remove real biological signal correlated with depth, such as larger, more transcriptionally active cell types. Check that known cell-type markers still separate cleanly after correction.
Filter on the ambient-contamination signature before clustering, not after
When: A cluster has elevated mito% together with low unspliced fraction, the ambient RNA signature.
Filter droplets on the combination of mitochondrial percentage, unspliced fraction, and nuclear-vs-cytoplasm marker expression (the approach QClus uses for snRNA-seq) before you cluster, rather than trying to explain the artifact away after the fact.
Caveat: Aggressive ambient filtering can also remove real nuclei that happen to be smaller or more fragile in a given cell type. Check that you aren't systematically depleting one known population before and after filtering.
Treat a sample-restricted cluster as a hypothesis, not an artifact to discard
When: A cluster appears in only one sample or condition and passes the marker check on its own.
Keep it separate, validate its markers specifically within that sample, and flag it explicitly as a condition-specific or donor-specific candidate state rather than merging it away or waiting for it to appear in other samples.
Caveat: A single-sample cluster needs replication before you build a claim on it. One sample is an observation, not a validated finding.
Regress cell-cycle score or remove doublets, then re-cluster
When: A candidate cluster is explained by high cell-cycle scores or a high doublet-call rate rather than a distinct expression program.
Run doublet detection (e.g. DoubletFinder) and remove flagged droplets, or regress cell-cycle scores out of the expression matrix, then re-cluster and re-check markers on the result.
Caveat: Cell-cycle regression can erase real proliferative subpopulations, such as progenitor or stem-like states where cycling is the biology, not noise. Don't apply this blindly to tissues where proliferation is expected to matter.
When not to "fix" it
Don't collapse clusters just because you have more of them than you expected. Neuronal subtypes genuinely number in the dozens: interneuron subclasses, cortical layer identities, and region-specific populations can each carry distinguishable markers and still add up to a UMAP with thirty or more real islands. If a cluster passes the marker check, holds up across samples, and isn't explained by depth, ambient contamination, cell cycle, or doublets, a high cluster count is the correct answer, not evidence of overclustering. Forcing the resolution down to match a scRNA-seq-derived expectation of "should look like ten to fifteen clusters" will merge real, distinct cell types and cost you the resolution the experiment was designed to buy.
Five things experienced analysts do here
- Run a resolution sweep by default, not just the tool's default value; a single resolution=0.8 run on real neuronal data can be 100% false-positive-clean at three cells and 100% false-positive-riddled on a homogeneous population, and you can't tell which without the sweep.
- Never use an integrated (Harmony/scVI) embedding to answer whether a cluster is real across samples, integration is designed to smooth out the exact sample-specific signal you're trying to check.
- Track mitochondrial percentage and unspliced fraction together in snRNA-seq, not separately; the inverse relationship between them is a more reliable ambient-contamination flag than either metric alone.
- If you can't write the cluster's identity as one plain-language sentence backed by a named marker, don't put it in the figure yet, a sea of unnamed clusters is descriptive output, not a finding.
- Keep a written log of every resolution you tried and every merge/split decision you made, tied to which check failed; six months later you or a reviewer will ask why cluster 14 became part of cluster 9, and "it looked right" is not an answer.
Questions people ask
- What resolution should I use for FindClusters on snRNA-seq data?
There's no universal number. Benchmarks show Seurat's default resolution of 0.8 produces false-positive clusters in 100% of replicates when run on data with only one true cell type. Sweep a range (roughly 0.4 to 1.2 is a common starting band for a few thousand cells) and validate every cluster the sweep produces with markers rather than trusting one default value.
- How many clusters should I expect from snRNA-seq of brain or heart tissue?
There's no fixed target count. Neuronal tissue genuinely contains dozens of real subtypes, so a large cluster count by itself is not evidence of overclustering. Judge each cluster on whether it has a nameable marker and holds up across samples, not on whether the total count matches an intuition borrowed from PBMC data.
- Why is mitochondrial percentage supposed to be near zero in snRNA-seq?
Isolated nuclei don't carry mitochondria, so a clean nucleus should show almost no mitochondrial transcript counts. Any measurable mito percentage signals that cytoplasm, and the ambient RNA it releases, contaminated the droplet, which is the opposite interpretation of a mito filter in whole-cell scRNA-seq.
- How do I tell an overclustered artifact from a real rare cell type?
Check whether the cluster has markers you can name, whether it appears consistently across biological samples without integration, and whether it's explained away by sequencing depth, ambient RNA, cell cycle, or doublets. A rare but real cell type passes all of these; an artifact usually fails at least one.
- Should I integrate my samples before checking if a cluster is real?
No. Integration methods like Harmony and scVI are built to align cell types across samples and will smooth out sample-specific differences, which is exactly the signal you need intact to tell whether a cluster exists in every sample or just one.
Related pages
- Guide · How to Tell If You Overclustered in Single-Cell RNA-seq
- Guide · How to Tell If You Overclustered in Spatial Transcriptomics
- Guide · How to Avoid Pseudoreplication in Single-Nucleus RNA-seq
- Guide · How to Choose a Normalization Method in Spatial Transcriptomics
- Guide · How to Choose Cell QC Thresholds in Single-Cell RNA-seq
- Glossary · Count matrix
- Glossary · Spatial transcriptomics
Related reading on the blog
Sources
- Chapter 19: Single-nuclei RNA-seq processing | Orchestrating Single-Cell Analysis with Bioconductor — snRNA-seq nuclei should show near-zero mitochondrial transcripts; elevated mito% signals contamination
- Resolution Tradeoffs in Modularity Clustering with Application to Single Cell RNA-seq — Louvain/Leiden split any point cloud as resolution rises; Type I vs Type II error tradeoff in resolution choice
- A knockoff calibration method to avoid over-clustering in single-cell RNA-sequencing — Default resolution 0.8 produces 100% false-positive cluster rate on single-true-cell-type simulated data
- QClus: a droplet filtering algorithm for enhanced snRNA-seq data quality in challenging samples — Inverse relationship between unspliced fraction and mito% flags ambient contamination; six-metric QC approach
- Scanpy clustering tutorial — Multi-resolution Leiden sweep code and rank_genes_groups/marker_gene_overlap usage
- Seurat v5 Documentation — FindClusters resolution parameter and standard Seurat clustering workflow
- Fine tune the best clustering resolution for scRNAseq data: trying out callback — Overclustering risk from high dimensionality and technical noise; automated resolution tuning caveats
- DoubletFinder: Doublet Detection in Single-Cell RNA Sequencing Data Using Artificial Nearest Neighbors — Cell cycle as a confounder producing false-positive doublet detection signals
Part of the Overclustering series.