Z zarr-pyramid-audit
Vesuvius Challenge · 2027 Grand Prize

Integrity is upstream
of every readable letter.

zarr-pyramid-audit is not an unrolling or ink-reading system. Its Grand Prize role is narrower and load-bearing: detect silent data defects, preserve evidence semantics, audit TIFXYZ surfaces, and make corrupted or unverified inputs fail loudly before they contaminate segmentation, training, rendering, or submission artifacts.

Why this matters for the Grand PrizeOfficial rules checked 2026-10-02; the Challenge page remains authoritative.

The 2027 Grand Prize requires reviewers to reproduce a complete pipeline, inspect TIFXYZ meshes, verify generated images, and trust that the submitted result is supported by the CT data. Silent all-zero or malformed pyramids can invalidate that chain without producing an obvious reader error.

Official Grand Prize requirements →

Scope boundary: a clean integrity audit does not prove correct sheet identity, full recto coverage, legible ink, or Grand Prize readiness. It proves only the checks actually run over the evidence available.

Where the integrity gate fitsThe right-hand column is intentionally blunt about what belongs elsewhere in the winning pipeline.

Grand Prize requirementzarr-pyramid-audit contributionStatusOutside this tool
Fully reproducible public pipelineMIT-licensed CLI, deterministic reports, schemas, fixtures, manifests, and committed campaign artifacts make integrity checks replayable.IMPLEMENTEDThe final Grand Prize pipeline still needs its own end-to-end runner, Docker image, and system requirements.
Reliable CT / Zarr inputszpa-audit, zpa-gate, and chunk-content probes distinguish malformed metadata, missing levels, header-only levels, unknown evidence, and sampled present-but-empty content. A Grand Prize exact-volume preflight matches all 13 prize-listed CT roots into the frozen S3 audit: 13 / 13 have all six levels present, no chunkless or unknown-presence level, and zero header findings. Separately, a 64-volume live volcomp campaign checked 2,008 shard indexes and independently confirmed 331 CRC32C mismatches as writer non-conformance; the indexes still parse and chunks decode, so the condition is retained as a medium warning rather than mislabeled voxel corruption.IMPLEMENTED + MEASUREDThis is dated structural/input-format evidence, not semantic voxel validation, scan quality, surface identification, or unrolling correctness.
VC3D integrationMachine-readable reports and TIFXYZ audits are designed for modular integration; review coordinates and standard formats can feed downstream tooling.PARTIALA seamless in-VC3D user workflow is not demonstrated here.
100% recto unrolledTIFXYZ files can be structurally audited before acceptance into a larger coverage ledger.SUPPORTINGSegmentation, sheet identity, flattening, and proof of whole-scroll recto coverage.
Per-column TIFXYZ submission mesheszpa-tifxyz checks native TIFXYZ structure/provenance and optional content-tier consistency.AUDIT EXISTSColumn boundaries, low-distortion isometry quality, sequential naming, and final package completeness.
Visible and legible inkIntegrity gating can prevent known-bad input volumes from entering training or inference.OUT OF SCOPEInk detection, legibility, papyrological evaluation, and line counts.
No train/prediction overlapCan ensure the underlying stores and surface files used by an external split are the expected artifacts.OUT OF SCOPESpatial split definition and leakage auditing belong in ScrolIQ / the ink pipeline.
False-positive mitigationFail-closed tri-state evidence prevents timeouts/403s/unknown listings from being misreported as absence; direct chunk reads test whether “present” data actually contain voxels.IMPLEMENTED FOR DATAInk hallucination controls and held-out text validation.
Held-out validationOffline fixtures, positive controls, and live known-clean / known-defective controls validate integrity logic.DOMAIN-SPECIFICThe Challenge's held-out ink/ground-truth validation must be satisfied by the unrolling/ink method itself.

Controls that reduce reviewer risk

Unknown is not absent

Timeouts, permission failures, rate limits, and unsupported listings remain UNKNOWN; they cannot silently become a missing-data finding.

Report contract →

Headers are not content

The fast audit checks metadata and chunk presence; the sampled content probe separately decodes real voxels to test a different silent-corruption class.

Chunk probe evidence →

Fail closed before publish

zpa-gate converts severity policy into a non-zero CI exit code so known defect classes can block a publication step.

CI example →

Surface files get their own audit

TIFXYZ checks are separate from Zarr checks because a valid CT store does not imply a valid submitted surface.

TIFXYZ usage →

Claims stay dated

Live-store findings are tied to dated artifacts so “still broken” is never presented as an eternal fact.

Artifact campaigns →

Shard checksums are independently verified

Live volcomp CRC32C mismatches were reproduced with a separate bit-by-bit implementation over the exact fetched index bytes, preventing a parser assumption from being promoted into a corruption claim.

CRC evidence →

Companion boundary

ScrolIQ owns downstream quality/evidence judgments; this repository supplies integrity primitives and does not infer readability.

ScrolIQ Grand Prize map →

What a winning pipeline should do with this

Recommended integration point: run the integrity gate before expensive segmentation/training and again before freezing final inputs; run TIFXYZ audits before accepting meshes into the column/coverage manifest. Persist the exact report hashes beside the downstream artifact so a reviewer can reconstruct which data passed which checks.

Return to the live integrity dashboard → · October evidence plan → · Source repository →