Twelve Plasmid Classifiers on 432 Clinical Isolates
How often short-read plasmid prediction is right, and where it breaks: contigs the plasmid database has never seen.
Vahhab Piranfar 
Most bacterial genomes in public databases are short-read assemblies. Whether a resistance gene in one of them sits on a plasmid or on the chromosome matters for how it spreads (1), but it has to be guessed from its contig, and many tools make that guess (2,3). I ran twelve of them on the same isolates and checked every call against an answer that no plasmid database had touched.
The short version: on contigs that resemble known plasmids, most tools do well. On contigs that do not, several of the most widely used tools call far more chromosome as plasmid than they find real plasmid. On contigs that carry a resistance gene, eight tools are right in 92 to 95 of every 100 plasmid calls.
The isolates and the answer key
The isolates are 432 carbapenem-resistant Enterobacterales from one New York City hospital, collected in 2009-2018 and sequenced with both Illumina and Oxford Nanopore by Gomez-Simmonds et al. (4) (BioProject PRJNA1133668). Most are Klebsiella (313), followed by Enterobacter (66), Escherichia coli (17), Citrobacter (16) and Serratia (16).
Each tool saw only the Illumina assembly. The answer key came from a hybrid assembly of the same isolate, built with Unicycler (5) from the long and short reads:
- the longest sequence, and any other sequence of at least 1 Mb, is chromosome;
- any other closed circular sequence is plasmid;
- any other sequence connected to the chromosome in the assembly graph is chromosome.
Each short-read contig of at least 1 kb was aligned with minimap2 (6) to its own isolate’s hybrid assembly and took the label of the sequence it matched, if the match was clear. Of 35,805 contigs, 30,986 got a label; 8,743 of them are plasmid. Contigs without a clear match were left out of every count. All tool calls were fixed before any long read was downloaded.
Results
Precision is the share of a tool’s plasmid calls that were right. Recall is the share of plasmid contigs it found.
| Tool | Version | Precision | Recall | F1 | Precision, not in DB | Recall, not in DB | Disk (GB) |
|---|---|---|---|---|---|---|---|
| Platon | 1.6 | 0.932 | 0.870 | 0.900 | 0.768 | 0.742 | 4.8 |
| gplas2 | 1.1.0 | 0.919 | 0.859 | 0.888 | 0.642 | 0.749 | 3.0 |
| PlasmidEC * | 1.3 | 0.897 | 0.958 | 0.926 | 0.625 | 0.793 | 12.3 |
| MOB-recon | 3.1.7 | 0.871 | 0.937 | 0.903 | 0.774 | 0.647 | 10.9 |
| plASgraph2 | 1.0.0 | 0.919 | 0.784 | 0.846 | 0.515 | 0.776 | 3.0 |
| HyAsP | 1.0.0 | 0.872 | 0.893 | 0.882 | 0.401 | 0.611 | 1.9 |
| RFPlasmid | 0.0.18 | 0.874 | 0.880 | 0.877 | 0.521 | 0.724 | 7.0 |
| Plasmer | 0.1 | 0.824 | 0.987 | 0.898 | 0.792 | 0.823 | 71.8 |
| PlaScope * | Centrifuge 1.0.4 | 0.595 | 0.985 | 0.742 | 0.088 | 0.882 | 1.9 |
| PLASMe | 1.1 | 0.553 | 0.995 | 0.711 | 0.118 | 0.892 | 21.9 |
| geNomad | 1.7.1 | 0.586 | 0.657 | 0.619 | 0.085 | 0.742 | 5.2 |
| PlasmidFinder | 2.0.1 | 0.981 | 0.203 | 0.337 | 0.993 | 0.320 | 2.0 |
* PlasmidEC and PlaScope were run with their E. coli settings, which do not suit a cohort that is mostly Klebsiella. Their numbers here are not a fair test of either tool. “Not in DB”: contigs that do not match the plasmid database (next section). Figure 1 shows 95% intervals from resampling clusters of closely related isolates (7), defined by average nucleotide identity (8); the CSV lists them.
On all contigs, PlasmidFinder is almost always right when it calls plasmid (0.981) but finds only one plasmid contig in five, because it looks only for replicon genes (9). Platon, gplas2, MOB-recon, RFPlasmid, HyAsP and plASgraph2 sit close together, with precision of 0.87-0.93 and recall of 0.78-0.94. PLASMe and Plasmer find nearly everything but pay for it in false calls: PLASMe called 7,025 chromosomal contigs plasmid.
Contigs the database has never seen
I measured how well each contig is represented in a public plasmid database (the PLSDB-derived set distributed with PLASMe, 33,125 plasmids (10,11)). A contig counted as in the database if, in a BLAST+ search (12), at least 95% of it matched a database plasmid at 99% identity or more. By this measure, 18,516 contigs were not in the database. Only 434 of them are plasmid; the rest are chromosome.
This is where the tools come apart, as an earlier benchmark across Enterobacterales and enterococci also found (2). Apart from PlasmidFinder, precision on these contigs ranged from 0.085 to 0.792. geNomad, PlaScope and PLASMe made 3,466, 3,963 and 2,894 false plasmid calls here, against fewer than 450 real plasmid contigs. For a gene on one of these contigs, a plasmid call from those three tools is wrong far more often than it is right.
Some of this drop is arithmetic: precision falls when positives are rare (13), and only 2.3% of these contigs are plasmid. But that is also the situation a user faces with a novel contig, so it is the number that matters in practice.
Recall drops too, for ten of the twelve tools:
The two exceptions are geNomad and PlasmidFinder, which found a larger share of the plasmids that are not in the database than of those that are.
The database used here is one of several, and none of the plasmids in it is longer than 350 kb. So “not in the database” means not well represented in this one database. It is not a measure of true novelty.
Contigs that carry a resistance gene
For most users the question is narrower: is this resistance gene on a plasmid? I annotated resistance genes with AMRFinderPlus v4.2.7 (database 2026-08-07.1) (14) and counted a contig if it carried at least one core AMR determinant other than an efflux gene. There were 3,005 such contigs, and 1,810 of them (60%) are plasmid.
| Tool | Precision | Recall | False plasmid calls | Missed plasmid contigs |
|---|---|---|---|---|
| PlasmidFinder | 1.000 | 0.081 | 0 | 1,664 |
| plASgraph2 | 0.951 | 0.847 | 79 | 277 |
| MOB-recon | 0.939 | 0.968 | 114 | 58 |
| gplas2 | 0.935 | 0.930 | 117 | 126 |
| HyAsP | 0.927 | 0.956 | 136 | 80 |
| Plasmer | 0.926 | 0.998 | 144 | 3 |
| Platon | 0.926 | 0.924 | 134 | 138 |
| PlasmidEC * | 0.925 | 0.999 | 146 | 1 |
| RFPlasmid | 0.925 | 0.988 | 145 | 21 |
| PLASMe | 0.862 | 0.999 | 290 | 1 |
| geNomad | 0.820 | 0.614 | 244 | 698 |
| PlaScope * | 0.645 | 0.998 | 993 | 3 |
Here the tools look better. Eight of them had precision between 0.925 and 0.951, and Plasmer, PlasmidEC, PLASMe, RFPlasmid and MOB-recon each found 96.8% or more of the plasmid-borne resistance contigs. PlaScope still made 993 false plasmid calls, and geNomad missed 698 of the 1,810 plasmid contigs.
Part of the gain is the same arithmetic as before, in the other direction: 60% of these contigs are plasmid, against 28% of all contigs, so a plasmid call is more likely to be right. Even the best tools here called 79-146 of the 1,195 chromosomal resistance-gene contigs plasmid. Chromosomal copies are not rare even for genes usually thought of as plasmid genes: of the 383 labeled contigs carrying blaKPC, 14 are chromosomal.
By genus
Precision was lowest in Serratia for eleven of the twelve tools. The Escherichia, Citrobacter and Serratia groups each have only 16-17 isolates, so their numbers are rough.
Disk space
Running all twelve takes about 146 GB, and Plasmer alone takes about half of that. Several of the best-balanced tools here (Platon, gplas2, HyAsP, plASgraph2) need 5 GB or less.
What about combining them?
I also fitted a model that combines the twelve calls (15). Against a plain majority vote of the same twelve tools, it traded recall for precision: it made 431 false plasmid calls and missed 713 plasmid contigs, where the vote made 852 and missed 307. The difference in F1 was close to zero (-0.002; 95% CI -0.011 to 0.009). On contigs absent from the database it made 6 false plasmid calls where the vote made 128. If you already run several of these tools, a majority vote is a reasonable baseline before reaching for anything fitted.
Tools, versions and settings
Each tool ran in a pinned container with a pinned database, with the same settings for every isolate. Contigs a tool left undecided, or that it failed on, count as not plasmid in its recall; they do not affect its precision.
| Tool | Version | Input | Settings | Database or model | Contigs decided |
|---|---|---|---|---|---|
| Platon (16) | 1.6 | Contigs | Default settings | Platon database v1.5.0 (link) | 100.0% |
| gplas2 (17) | 1.1.0 | Assembly graph | PlasmidEC predictions (E. coli preset) as the initial classification; default settings | None | 88.7% |
| PlasmidEC (17) | 1.3 | Assembly graph | E. coli preset: Centrifuge, Platon and RFPlasmid (Enterobacteriaceae model), majority vote | PlaScope E. coli database v1.1; Platon database v1.5.0 | 100.0% |
| MOB-recon (18) | 3.1.7 | Contigs | Default settings | Database bundled in the MOB-suite 3.1.7 container | 99.1% |
| plASgraph2 (19) | 1.0.0 (commit 7bc4e92) | Assembly graph | Default settings | ESKAPEE model distributed with plASgraph2 1.0.0 | 96.0% |
| HyAsP (20) | 1.0.0 | Assembly graph | Maximum plasmid length 750 kb; otherwise default settings | HyAsP NCBI RefSeq database (link) | 100.0% |
| RFPlasmid (21) | 0.0.18 | Contigs | Enterobacteriaceae model; Jellyfish k-mer counting | Models distributed with RFPlasmid 0.0.18 | 100.0% |
| Plasmer (22) | 0.1 | Contigs | Minimum length 0 bp; chromosome length threshold 100 Mb | Plasmer database v1.0 (link) | 100.0% |
| PlaScope (23,24) | Centrifuge 1.0.4 | Contigs | Up to 1,000 assignments per contig | PlaScope E. coli database v1.1 (link) | 97.0% |
| PLASMe (11) | 1.1 (commit ef0409b) | Contigs | Coverage 0.9, identity 0.9, probability 0.5 | PLASMe database (link) | 100.0% |
| geNomad (25) | 1.7.1 | Contigs | End-to-end workflow; default settings | geNomad database v1.5 (link) | 100.0% |
| PlasmidFinder (9) | 2.0.1 | Contigs | BLAST method; default thresholds | PlasmidFinder database 2.2.0 (commit 49c24c1) | 99.6% |
gplas2 starts from PlasmidEC’s E. coli predictions, so it shares part of that mismatch.
Limits
- One hospital, and the isolates are closely related: the largest cluster holds 66% of them. Results in other settings may differ.
- Contigs whose label was unclear (about one in seven) were left out, and they may be harder than the rest.
- Every tool was run once, with one version and one database. Newer versions or other settings may do better.
- PlasmidEC and PlaScope were run with E. coli settings (see above).
Data
- Per-tool numbers behind this page: plasmid_classifier_benchmark.csv
- Assemblies, labels and every tool call (26): figshare, doi:10.6084/m9.figshare.34021401
- Native output of all twelve tools (27): Harvard Dataverse, doi:10.7910/DVN/Q5JNX9
- Analysis code (28): figshare, doi:10.6084/m9.figshare.34063578
- Full methods (15): Research Square, doi:10.21203/rs.3.rs-11651433/v1
- Why the tools fail, contig by contig: Why Plasmid Classifiers Fail on Short-Read Contigs
How to cite
Piranfar V. Twelve Plasmid Classifiers on 432 Clinical Isolates [Internet]. Jersey City (NJ): Vahhab Piranfar; 2026 Nov 10. Available from: https://vahhabpiranfar.com/posts/plasmid-classifier-benchmark/