← Benchmarks / Archived comparison

LFQBench / OE480

Three archived reports, the same six input files. A and B each contain three replicate runs.

This is not a current-release ranking. Tool labels preserve source-folder labels; exact releases, execution dates, hardware and full search settings are not reconciled. Matching filenames do not establish identical search settings.
Precursor q-value < 0.01; decoys excluded. IDs are distinct modified-sequence and charge pairs across all six runs.
Tool / source label Precursor IDs Detected in all six Median CV: A Median CV: B
SynapSpec (v092 folder) 81,559 86.1%70,239 of 81,559 12.5%n = 68,458 13.5%n = 69,516
DIA-NN (archived) 108,447 77.1%83,600 of 108,447 9.5%n = 90,293 10.5%n = 88,675
Spectronaut (v21_0 folder) 104,099 85.2%88,726 of 104,099 15.8%n = 94,534 17.0%n = 93,616

CV uses sample standard deviation / mean within one condition, only for precursors with positive finite MS2 quantities in all three replicates. Each tool has its own eligible population. Missing observations are not imputed.

Accuracy vs. depth

Same layout as ProteoBench’s LFQ accuracy modules (proteobench.org): each point is one tool’s full run on these six files, not one point per precursor. Horizontal position is the median absolute error between measured and expected log₂ fold change across the three species (species-weighted, lower is more accurate); vertical position is how many precursor ions that run identified. Up and to the left is more precursors identified with less error — neither axis alone ranks a tool.

80k 100k 120k 0.0 0.09 0.19 0.28 0.38 SynapSpec: 81559 precursors, median |error| 0.1846 DIA-NN: 108447 precursors, median |error| 0.1757 Spectronaut: 104099 precursors, median |error| 0.3294 Median absolute log₂ fold-change error (species-weighted)
  • SynapSpec
  • DIA-NN
  • Spectronaut

Identification, repeatability and LFQ ratios

More IDs means broader coverage, not automatically better quantification. Lower CV means more consistent repeat measurements. For LFQ, a median closer to the dashed expected line and a narrower interquartile range indicate closer agreement and less spread.

LFQBench / OE480: three-tool precursor counts, six-run completeness, within-condition CV, and species-specific LFQ median and interquartile range. Exact values follow below. Download comparison PNG
Precursor MS2 log₂(mean A / mean B), using observed values; entities require positive means in both conditions. Mixed-species annotations are excluded. Intervals are quartiles, not confidence intervals.
ToolSpeciesExpectedMedian25–75%Precursors
SynapSpec (v092 folder) HUMAN 0.0 -0.092 -0.331 – 0.037 61,721
SynapSpec (v092 folder) ECOLI -2.0 -2.194 -2.687 – -1.546 1,606
SynapSpec (v092 folder) YEAS8 1.0 1.185 0.997 – 1.373 14,794
DIA-NN (archived) HUMAN 0.0 -0.094 -0.223 – 0.016 78,862
DIA-NN (archived) ECOLI -2.0 -2.23 -2.61 – -1.918 2,113
DIA-NN (archived) YEAS8 1.0 1.176 1.035 – 1.422 18,739
Spectronaut (v21_0 folder) HUMAN 0.0 -0.051 -0.275 – 0.087 80,190
Spectronaut (v21_0 folder) ECOLI -2.0 -2.544 -3.188 – -2.031 2,237
Spectronaut (v21_0 folder) YEAS8 1.0 1.329 1.12 – 1.762 19,152

Shared precursor IDs

Exact modified-sequence and charge matches, across all six runs; pairwise counts overlap and must not be added together.

  • synapspec / diann: 73,857
  • synapspec / spectronaut: 68,717
  • diann / spectronaut: 88,863
Input files and per-file IDs
Input stemSynapSpec (v092 folder)DIA-NN (archived)Spectronaut (v21_0 folder)
20240910_LFQBench_120min_A_R1 77,839 96,182 99,121
20240910_LFQBench_120min_A_R2 78,092 97,875 99,153
20240910_LFQBench_120min_A_R3 78,087 98,034 99,099
20240910_LFQBench_120min_B_R1 78,355 95,805 98,935
20240910_LFQBench_120min_B_R2 78,496 97,306 99,236
20240910_LFQBench_120min_B_R3 78,101 97,128 98,474
Limitations and reproducibility
  • No analysis engine was rerun. These are archived report extracts, imported on 2026-09-08.
  • Protein entities use tool-specific grouping definitions and are not presented as a cross-tool score.
  • Separate SynapSpec MBR files are not included. MBR settings in the base reports remain unverified.
  • Expected ratios follow the existing BION LFQ preset: human 1:1, yeast 2:1, E. coli 1:4 (A:B). Sample preparation records and full settings still need owner review.
  • Release, execution time, resource and analysis date: not verified. No metadata is borrowed from the September 7 run.
  • These internal LFQBench experiments are not labeled PXD028735; a provenance mapping is not established.