zarr-pyramid-audit — Don't train on lies

The live defect

A PHerc0814 surface-volume pyramid in the open-data S3 bucket has valid headers at all six levels and zero chunk objects. Every reader opens it fine and gets plausible all-zero voxels — no error, no warning. Proven by direct read; independently confirms villa #1892.

Grand Prize CT preflight

13 / 13 exact prize-listed CT roots were found in the frozen S3 audit. 13 / 13 have all declared levels present, 13 / 13 have no chunkless level, 13 / 13 have no unknown chunk-presence state, and 13 / 13 have zero header findings. This is input-integrity evidence, not scan/surface/ink readiness. Exact-volume artifact →

20-day regression

The identical audit re-ran over the identical 241-root dl list on 2026-09-29. Strict diff on every finding field: empty. 18 defective pyramids, 50 actionable findings — zero fixed, zero new. Published defects don't get fixed after the fact; the publish-time gate is the lever.

Training volumes probed

548 volcomp inner chunks decoded with real content across all 64 scroll volumes, 1795 raw/Blosc chunks across 128 dl roots — 0 all-empty levels in the training data. One present-but-empty mesh derivative caught in other/dev/ (medium, human review).

Surface-volume window evidence

zpa-surface-depth-profile checks the rendered ink-model input stack itself: expected slice count, deterministic per-depth signal/texture, sampled all-zero layers, duplicate sampled-layer digests, and exact source-volume lineage. It reports evidence rather than pretending these observations prove ink.

tifxyz surfaces2026-10-01 · `zpa-tifxyz --content` on every surface patch in the S3 bucket

1539 surfaces audited: 1539 integrity PASS, 0 WARN, 0 UNKNOWN, 0 FAIL. Findings at medium or above: 0. Content tier skipped on 148 surfaces (channels over the 32 MiB cap) — reported as a coverage gap, not a pass.

code	surfaces	meaning
`TIFXYZ_NEGATIVE_COORDINATE`	161	valid points with a negative coordinate (outside any volume)
`TIFXYZ_CONTENT_UNDECODED`	148	content tier skipped (size cap or layout): coverage gap
`TIFXYZ_BBOX_MISMATCH`	30	declared bbox differs from the stored coordinates

Run artifacts →

60-second evaluator pathEvidence first: inspect it, reproduce it, then watch it test live data.

1 · Inspect frozen evidence

Every headline number on this page is generated from committed run artifacts. Pages CI regenerates the dashboard and rejects drift from the evidence.

Run artifacts → · Known-defect registry → · Submission criteria → evidence map →

2 · Reproduce a verdict

Install the public tool and run the same fail-closed gate used for publication checks.

pip install git+https://github.com/Svyable/zarr-pyramid-audit.git
zpa-gate --base <store> --root <volume.zarr>

3 · Watch real-data CI

The scheduled workflow exercises anonymous public data: a clean gate pass, a known-defect rejection, and a sampled chunk-content decode.

Real-data verification workflow →

Grand Prize preflight: use the same gate on the exact prize-eligible CT, surface, and derived Zarr inputs before expensive geometry or ink work. A silent storage defect should fail before it can contaminate an unrolling campaign. Before 2.5D ink inference, run zpa-surface-depth-profile on the rendered stack to pin the slice count, source volume, sampled depth integrity, and depth profile. Protocol →

Against the tools people already use2026-10-03 · zarr-python 3.1.6, ome-zarr-models 1.7, yaozarrs 0.3.2

On the live defect, all three baselines treat the PHerc0814 -L1 pyramid as healthy: both OME-NGFF validators accept it, and zarr-python reads a level-5 window as all zeros without an error. The ZPA gate rejects it (6 × LEVEL_NO_CHUNKS).

On the fixture corpus (ground truth assigned from what each fixture was built to contain), flagged / total:

tool	defects	suspicious content	false alarms on valid pyramids	out-of-model nodes
zarr-python (open + read every level)	8 / 26	0 / 3	0 / 10	1 / 4
ome-zarr-models (OME-NGFF validator)	14 / 26	0 / 3	0 / 10	4 / 4
yaozarrs (OME-NGFF metadata + structure)	13 / 26	0 / 3	0 / 10	4 / 4
ZPA header audit	25 / 26	0 / 3	1 / 10	4 / 4
ZPA + sampled chunk probe	26 / 26	3 / 3	2 / 10	4 / 4

Selection bias: the corpus was written around ZPA’s failure classes, so read it per defect class, not as a score. The validators check the NGFF spec, which ZPA does not attempt; run both kinds. ZPA’s false alarms are a legal compressor drift (low, integrity stays PASS) and a sampled probe on a mostly empty level.

Comparison, method and caveats →

Audit before publishfail closed on defects, stay honest about unknown evidence

Put the gate in front of publication

zpa-gate reads metadata, not payloads, and returns a non-zero exit code when a root crosses the chosen severity threshold.

zpa-gate --base s3://my-bucket/staging/
  --roots publish.jsonl --fail-on high

Absence requires evidence

Metadata and listings now carry an explicit PRESENT / ABSENT / UNKNOWN state. A timeout, 403, rate limit, or server error cannot be silently promoted into a “missing level” or “empty store” claim.

Known-defect registry → · Run artifacts →

How this fits with ScrolIQTwo repos, one question: should anyone spend GPU or expert time on this volume?

zarr-pyramid-audit · this repo

Don't train on lies.

Integrity. Is the data what its metadata claims? Header-only pyramid audit, sampled chunk-content probes, a fail-closed publish gate, and surface-input evidence tools. Read-only, ~KB per pyramid. A high finding means do not train, do not publish. Says nothing about scan quality.

ScrolIQ · companion repo

Find the bottleneck.

Diagnostics. A diagnostic layer for the Challenge's 2026 open problems: a 0–100 scan-health triage score and the “🎯 label next” coverage join, plus the diagnostic passport, spatial scan map, mesh / winding / ink audits, and the 2027 Grand Prize recto-coverage and provenance gates. Its score is not readability or prize readiness; unmeasured stages stay unknown.

repo · live survey · September writeup

1 · Integrity zpa-gate / zpa-audit — are levels, headers and chunks real?

2 · Stop or continue A high-severity finding ends it: DO NOT TRAIN

3 · Scan health ScrolIQ scores sampled real voxels, decoded through the libvolcomp vendored here

4 · Diagnose Passport, scan map, mesh / winding / ink audits, Grand Prize provenance gate

One verdict per volume: `scrollq-health`

integrity (this repo)	quality (ScrolIQ)	verdict
FAIL: any high finding, or the audit itself crashed	not consulted	DO NOT TRAIN
UNKNOWN: unreadable level, absent root, nothing auditable	not consulted	DO NOT TRAIN
WARN: medium finding(s)	not consulted	CAUTION
PASS	unscorable, or score below 40	CAUTION
PASS	score 40 or above	TRAIN

Rules as implemented in ScrolIQ's health.py (checked 2026-10-01): integrity comes from this repo's zpa.report.audit_root and follows RECOMMENDED_CONSUMER_VERDICT, so missing evidence fails closed (ScrolIQ #55; until then UNKNOWN read as a pass). All outcomes were run on live data (ScrolIQ evidence): DO NOT TRAIN on the defective PHerc0814 pyramid — its quality is unscorable, so the verdict comes from this audit alone — TRAIN on a healthy PHerc0813 dl volume, CAUTION on the v2 dev mesh, and DO NOT TRAIN on a root that does not exist.

Where ScrolIQ builds on this repo

Shared foundation. ScrolIQ imports this package's HTTP/S3 store, header parser, audit and vendored libvolcomp decoder (zpa.httpstore, zpa.zarrmeta, zpa.audit_pyramid, zpa.volcomp). It declares zarr-pyramid-audit as a dependency, so installing ScrolIQ installs this. The dependency runs one way: this repo never imports ScrolIQ.
Integrity before quality. A scan-health score is only worth trusting on a pyramid that passes integrity, so scrollq-health runs this audit alongside the score and lets a high finding, or missing evidence, override it.
Grand Prize evidence chain. ScrolIQ's scroliq-provenance gate requires a zarr_audit record for the CT volume — tool name, audit-manifest SHA-256, and a root naming the exact eligible volume — and its probe protocol makes this repo's audit Stage A.
Shared discovery data. ScrolIQ's label-coverage join reads the discover_zarr.roots.jsonl that zpa-discover produces for the S3 bucket.

Naming. ScrolIQ is spelled with a capital I (as in Mesh IQ and Ink IQ) and was previously ScrollQ. Its Python package and the scrollq-* commands keep the old name for compatibility; the newer diagnostics ship as scroliq-*. Repo and site URLs are unchanged.

# integrity only (this repo)
zpa-gate --base <store> --root <volume.zarr>

# integrity + scan quality → one verdict (installs both packages)
pip install git+https://github.com/Svyable/scrollq.git
scrollq-health --root <volume>

🚨 The defective pyramids3://vesuvius-challenge-open-data

PHerc0814/segments/20260226123353-auto_grown_20260226123353106/surface-volumes/1.129um-0.22m-59keV-volume-20260521123630-L1.zarr

All six pyramid levels carry valid array headers but zero chunk objects. Any reader — viewer, registration, model — opens it successfully and gets back plausible all-zero voxels. We proved it empirically: a full-extent read returned all zeros while a control region on a sibling volume returned 64,498 nonzero voxels.

The defect is isolated with unusual sharpness: the same filename stem exists in 19 segment runs and exactly one copy — the 20260226123353 run — is header-only. The other 18 are clean, and a sampled chunk-content probe confirmed they hold real data, not present-but-empty shells.

The fix ships with the repo: zpa-gate is a publish-time metadata gate that audits roots before they publish and fails closed (exit 1) on any finding at or above the severity threshold — with GitHub Actions annotations for CI. When used, the gate blocks this defect class before publication.

Still live: re-audited 2026-09-30 ~22:25 UTC — 6 high-severity LEVEL_NO_CHUNKS findings, identical signature. Unrepaired for the full September window.

Findings by check codeClick a column to sort · filter by severity · S3 = open-data bucket, dl = dl.ash2txt.org

check code	severity	S3	dl	meaning
`CHUNK_EXCEEDS_SHAPE`	info	0	78	fixed chunk shape carried below the array — informational
`BARE_ARRAY`	info	0	40	valid single-scale Zarr array, not a pyramid
`LEVEL_NO_CHUNKS`	high	6	11	valid header, zero chunk objects — reads return fill_value silently
`SCALE_SHAPE_MISMATCH`	high	0	15	level shape matches neither ceil nor floor of base/factor
`LEVEL_MISSING`	high	0	13	declared level has no readable array header
`COMPRESSOR_DRIFT`	low	0	9	codec changes between levels
`NOT_MULTISCALE`	info	0	6	valid Zarr group, never claimed to be a pyramid
`CONTAINER_NO_GROUP_HEADER`	low	0	1	children are Zarr nodes but root has no group header
`HEADERLESS_CHUNK_STORE`	high	0	1	chunk keys present but no decodable header
`NOT_A_ZARR_GROUP`	info	0	1	no .zgroup / zarr.json and nothing Zarr-like inside

S3: 6 actionable findings (all LEVEL_NO_CHUNKS — the six levels of the defective pyramid), 0 informational. dl: 50 actionable, 125 informational across 18 defective pyramids (villa #1755–#1760).

Chunk-content probe2026-09-30 · sampled decode, not just headers

The header-only audit answers "are chunk keys present?" — the probe answers the next question: "do the present chunks hold data?" This closes the next silent-corruption class: chunks that exist but decode to all fill_value.

campaign	roots	levels	chunks decoded	populated	all-empty levels
S3 v2 (zarr-python windows)	58	291	788	788	0
S3 v3 sharded (zarr-python windows)	70	420	1260	1224	0
dl volcomp (vendored libvolcomp, HTTP byte ranges)	64	384	2666	548	0
dl v2 raw/Blosc	128	748	1812	1795	7

One real catch: other/dev/meshes/20231022170900-ome.zarr L1–L7 decode to all zeros while L0 holds mesh data — a dev-directory mesh derivative, not a scroll. Flagged medium for human review, with caveats. This is the probe working as designed.

The present-but-empty class was not observed in the S3 or training-volume samples — reported as evidence of absence in the sample, not proof of absence in the corpus. The tool is published for anyone to rerun at larger sample sizes.

Mirror fidelity: S3 vs dl.ash2txt.orgsame names, not copies

64 same-named volumes exist in both stores. They are format migrations of identical voxel grids at all six levels:

	dl.ash2txt.org	S3 open-data
Zarr format	v3	v2
chunking	1024³ shards	128³ raw
codec	sharding_indexed + lossy volcomp (q=1.0)	none (raw uint8)

Byte-level comparison (volcomp decoded from source) shows quantization noise only: max absolute difference 30–35, mean below 0.1, ~2–4% of voxels differ. Reproducibility work must pin the store, not just the filename.

Known silent-zero volumesdl.ash2txt.org · header-only levels confirmed by direct read

volume	header-only levels
`community-uploads/bruniss/labels/surfaces/archive/1-voxel-sheet_slices-closed.zarr`	1–5 (level 0 populated)
`other/dev/inked_zarrs/3336_predictions.zarr`	all six

Unchanged between the 2026-09-09 and 2026-09-29 audits. Full machine-readable kill list: data/known-defects.json (19 entries).