Storage Proofs: The Cryptography

Why a Checksum Isn’t Enough

This is why proof-of-storage schemes are interactive(or use pseudorandom challenges tied to a fresh seed): the prover can’t know in advance which blocks will be asked about, so they must keep all of them.

Key Papers

Twenty years of cryptography research from multiple independent groups underlies this work. These papers tackle different problems — some are theoretical foundations, others are practical constructions — and Pinion’s implementation draws directly from them.

From Theory to IPFS: Blocks, Tags, and Sparse Arrays

The papers above were designed with traditional file storage in mind — fixed-size blocks, sequential integer indices. The question for Pinion is: how do you apply these proofs to IPFS, where blocks aren’t numbered integers but 256-bit content hashes arranged in a Merkle DAG?

Standard proof-of-storage blocks

The papers above describe files as flat arrays of fixed-size blocks. Block 1, block 2, block 3 — sequential integer indices. The protocols are designed around this model: you can challenge block 47, compute its tag, and the index is part of the tag computation. Simple, but rigid.

IPFS blocks

IPFS stores content differently. Data is organized as a Merkle DAG — a directed acyclic graph of content-addressed blocks, each identified by its CID (a cryptographic hash of its contents). A file is a root CID; walk the DAG and you discover every leaf block. Block identifiers aren’t sequential integers — they’re 256-bit hashes. And two different files that share content share the same block, identified by the same CID. Deduplication is structural.

To use proof-of-storage on IPFS data, we re-imagine the DAG’s blocks as entries in a very large sparse array indexed by CID bytes. Not every element is populated — the CID space is astronomically large. But for any given root, walking the DAG gives us the complete set of populated entries.

Not every proof-of-storage protocol handles sparse, arbitrary block IDs. The schemes that require sequential integer indices for their tag computation can’t be used directly. Pinion uses only protocols where the block identifier can be arbitrary bytes — fed into a PRF or hash — so that CID bytes work as naturally as sequential integers.

The Line Protocol: one interface, five implementations

Pinion wraps all five schemes behind a single Go interface called the line protocol:

Tagger    → TagBlocks(store)              → []Tag
Challenger → Challenge(blockIDs)           → (challenge, validator)
Prover    → Prove(challenge, blockStore)  → proof
Validator  → Verify(challenge, proof)     → bool

Switching protocols requires changing only the string you pass to POST /challenge-key. Everything else — how you generate challenges, submit them, and verify responses — is identical. The interface is protocol-agnostic by design.

The Seed Trick: Stateless, Unpredictable Challenges

Each audit round starts with a fresh random 32-byte seed S. Everything about the challenge — which blocks to sample, and with what coefficients — is derived deterministically from that seed using a pseudorandom function (PRF):

The same seed produces the same challenge every time — so both the client (who generates the challenge) and the server (who constructs the proof) can independently compute which blocks are involved. No shared state. No coordination beyond passing the seed. Fresh seed each round means fresh, unpredictable challenge each round.

Because block IDs are arbitrary bytes, the PRF ranking works identically whether you’re challenging 3 blocks or 3 million. The IPFS sparse-array representation slots in naturally: CID bytes become the items being ranked, and the top-c selection is your challenge set.

The Math of Catching a Lie

Here’s the key theorem. If a fraction p of your blocks are sampled each round, and each round is independent, then after k rounds your confidence that no corruption has gone undetected is:

Each round is independent — like asking about a different random page each time. The probability of missingcorruption multiplies: if you have a 10% chance of catching it each round, after 10 rounds you’ve had 10 independent shots at it, and the chance of missing all of them is only 0.9¹⁰≈ 35%. After 44 rounds at 10% sampling, you’re above 99% confidence.

The exponential convergence means you don’t need to download the file to trust it. You just need to keep asking — and the math does the rest.

Choosing a Protocol

Pinion implements all five schemes. The right choice depends on what properties you need:

Challenge Us

Here’s the part that distinguishes a proof system from a promise: the POST /prover/prove endpoint requires no authentication. Anyone who obtains a client_setup and a list of block_ids can issue a challenge and verify the response locally — no Pinion involvement in the verification step.

Any Pinion account holder can share their key_id with you. Call GET /prover/api/v1/setup?key_id=<id> and you get back everything you need to construct and verify a challenge.

# 1. Get the client setup and block list
curl https://<env>.pinion.build/prover/api/v1/setup?key_id=<key_id>

# 2. Derive a challenge from a fresh random seed (client-side, offline)
#    → produces: challenge bytes + a local validator

# 3. Submit the challenge (no authentication needed)
curl -X POST https://<env>.pinion.build/prover/prove \
  -H "Content-Type: application/json" \
  -d '{"key_id": "<key_id>", "roots": [], "challenge": <base64>}'

# 4. Verify the proof locally against your validator
#    → true = data intact | false = tampering detected

Our storage-proofs benchmark site shows how each protocol implementation compares to theoretical predictions from the papers — detection probability curves measured against simulated corruption scenarios, timing data for setup, prove, and verify operations, and extraction success rates across varying file sizes. The curves you see there are the same formula as the graph above, measured rather than modeled.