We opened 792 real slide files and published every result

What does this benchmark actually prove?

That a named file, from a named scanner, at its real size, opens in the shipping build and puts pixels on screen. Nothing more and nothing less. A format-support list claims a capability; this measures one. If a vendor layout defeats the decoder, that file appears in the failure table with the error text a user would see, not as a missing row.

It is a vendor test — we wrote the software and we ran the test. What makes it worth reading anyway is that you can check it: the corpora are public, the runner is in our desktop repository, and the raw per-file results are on this page.

How the files were tested

Each file is handed to a sacrificial child process that builds the same loader the application builds, reads the image dimensions, decodes a plane, and converts it through the same Qt render path the viewer uses. A malformed file that aborts a native codec kills only that child; the supervisor records which file was in flight and carries on. That design is why a benchmark over multi-gigabyte vendor scans can finish at all, and why "crashed" is a category this table can honestly report as zero.

• Runner: slidescope-desktop/scripts/publish_format_benchmark.py — one command, in the SlideScope desktop repository, that runs all three passes and writes the exact JSON this page renders: python scripts/publish_format_benchmark.py --dataset <corpus> --output format-benchmark.json
• Corpora: the Open Microscopy Environment public reference corpus, OpenSlide's published test slides, the czitools and nd2 library test data, and vendor samples whose sha256 we verified against the publisher.
• Scope: whole slide and vendor microscopy formats. Loose single-plane TIFFs from bulk vendor dumps are excluded — they say nothing a pyramidal WSI TIFF does not say better.
• Recorded per file: detected format, loader, decoded T/Z/C/Y/X dimensions, pixel type, rendered size, and on failure the exception type and the user-facing message.

Whole slide image formats

These are the scanner formats — the files that arrive from an Aperio, Hamamatsu, 3DHISTECH, Leica, Ventana, Akoya or Agilent instrument and routinely run to several gigabytes.

Microscopy and medical imaging formats

Multi-dimensional acquisition formats, where the interesting question is not file size but how many channels, Z planes and timepoints survive the trip to the screen.

Every file that did not open, and why

25 of the 792 files did not open, and they are not one story but two. 9 are real limitations: a genuine image we cannot currently read. The other 16 are files that contain no image in the first place, where refusing is the correct answer. Lumping the two together would flatter us — the honest failure count for this run is 9, not 25.

Real limitations: 9 files we cannot read

Every one of these is pristine — checksums match the publisher — so these are decoder and vendor-layout limits, not damaged samples. They stay in the corpus permanently as regression coverage, which is how we find out the day one of them starts working.

The pattern is vendor layout, not modality. Fluorescence NDPI and fluorescence MIRAX both open normally; it is specific structural variants — a SCN with no identifiable main image, a BIF with an unexpected direction attribute, a pre-HDF5 "Imaris Classic" container — that the backends decline. Each message names the cause rather than calling the file corrupt, because in every one of these cases the file is fine and we are not.

Not failures: 16 objects that carry no image

DICOM is a document format as much as an image format. A DICOMDIR is an index of a disc; a presentation state records how someone once windowed a study; other objects hold reports or measurements. None contains pixels. A viewer that opened them would have nothing to show, so SlideScope declines with an explanation — the right behaviour, counted separately so it cannot pad either column.

Named in full so nothing is hidden behind a category: DICOMDIR, PS_0026, PS_0108, PS_0128, PS_0154, PS_0181, XX_0027, XX_0109, XX_0155, XX_0182, XX_0183.

Where SlideScope opens a file and shows the wrong thing

This is the failure mode that matters most and the one format tables never admit to, so here is ours.

A multiplex Akoya .qptiff scan — the Vectra seven-colour reference file — opens with no error at all. The file holds five full-resolution single-channel fluorescence pages (DAPI, FITC, CY3, Texas Red, CY5) at 34,560 × 24,960 across 33 pages, confirmed independently with tifffile. SlideScope routes .qptiff to its OpenSlide backend, which is a brightfield reader, so it reports one channel and returns an RGB slice. The fluorescence channels are unreachable and the channel controls have nothing to offer.

Nothing warns you. The brightfield H&E QPTIFF scans in the same corpus open perfectly, which is exactly why this went unnoticed. If you work with multiplex QPTIFF, this is a real limitation today and it is on the roadmap rather than in the product.

Opening is proved by this run. Pixel correctness is a separate question, and we test it separately: a second sweep decodes the same planes through the underlying library and compares them against what the loader returns. That is how the QPTIFF case was caught.

What SlideScope refuses to open

There is no generic fallback reader, and how a viewer handles a file it was never built for matters as much as the files it was. We took the smallest real file of every unsupported extension in the reference corpus — 93 of them — and put each one through the shipping loader factory.

93 of 93 were refused with a clear message. None crashed. None were half-opened into something that looked like an image but was not. The list includes formats people plausibly try — .jpg, .lof, .nii, .oib, .oir, .png, .psd, .raw, .vsi — and yes, that covers .png and .jpg: SlideScope is a scientific-image viewer, not a picture viewer, and it says so rather than guessing.

What this benchmark does not tell you

• Two formats have no real-file coverage at all. No public .vmu or .svslide sample exists — OpenSlide publishes neither — so those loaders are exercised only by synthetic edge cases. We are not claiming tested support for them.
• It is not a speed benchmark. Time-to-first-pixel depends on your disk, and these runs were local.
• It is not an accuracy audit beyond the pixel-comparison sweep described above, and it is not a clinical validation. SlideScope is for research, education and review — not diagnosis.
• It is a snapshot. Dated 2026-09-18 against the release current on that day. Decoder upgrades move these numbers in both directions.