Skip to content
Ratify Protocol

Conformance suite

Every Ratify SDK ships with a test that loads 59 canonical fixtures from testvectors/v1/ and runs them through every protocol operation. If all 59 pass in your SDK, your SDK is byte-for-byte interoperable with every other Ratify SDK on the planet. That is the contract.

What’s in the suite

Section titled “What’s in the suite”
   testvectors/v1/
   ├── delegation_signing_*.json      (cert signing — 12 fixtures)
   ├── challenge_signing_*.json       (challenge signing — 6 fixtures)
   ├── verify_valid_*.json            (full bundle verification, positive — 14 fixtures)
   ├── verify_invalid_*.json          (verification negative cases — 11 fixtures)
   ├── revocation_*.json              (revocation list scenarios — 4 fixtures)
   ├── scope_*.json                   (scope expansion / intersection — 6 fixtures)
   ├── constraint_*.json              (geo / temporal / version — 6 fixtures)
   └── (total: 59)

Each fixture is a deterministic JSON file with:

  1. Inputs. Keys (seeded so they’re reproducible), cert/bundle/list shape, verifier context.
  2. Expected canonical bytes. What delegationSignBytes or challengeSignBytes should produce (hex-encoded). Any SDK that produces different bytes here has a non-byte-identical canonicalizer and is rejected.
  3. Expected verification outcome. For verify fixtures: the exact VerifyResult struct the verifier should return, including identity_status, granted_scope, and error_reason.

The fixtures are generated by cmd/ratify-testvectors in the protocol repo. The Go reference is authoritative; every other SDK must produce identical output.

Running the suite

Section titled “Running the suite”
Terminal window
# Go
cd ratify-protocol
go test ./...


# TypeScript
cd sdks/typescript
npm run test:conformance


# Python
cd sdks/python
pytest


# Rust
cd sdks/rust
cargo test

Expected output: 59 passed. Any failure means the SDK has drifted — file a bug.

What the negative fixtures cover

Section titled “What the negative fixtures cover”

Half the value is in the rejection paths. The 11 verify_invalid_* fixtures check that the verifier rejects:

  • Tampered cert (scope modified after signing)
  • Tampered cert (expiry modified after signing)
  • Tampered cert (subject modified after signing)
  • Expired cert (now > expires_at)
  • Out-of-scope request (requested scope not in granted chain)
  • Wrong issuer key in cert
  • Wrong subject key in cert
  • Challenge signature by wrong key
  • Stale challenge (older than freshness window)
  • Future-dated challenge (clock skew beyond tolerance)
  • Replay (same challenge bytes seen earlier — strict mode)

Each rejection is checked for the exact identity_status value. The verifier must say bad_signature for a tampered cert, not scope_denied or some generic “invalid.” This strictness is what makes integrators able to write deterministic error-handling code.

Hybrid signature corner cases

Section titled “Hybrid signature corner cases”

The 6 hybrid-edge-case fixtures verify:

   Ed25519 valid + ML-DSA-65 valid     → ✓ accept
   Ed25519 valid + ML-DSA-65 invalid   → ✗ reject (bad_signature)
   Ed25519 invalid + ML-DSA-65 valid   → ✗ reject (bad_signature)
   Ed25519 invalid + ML-DSA-65 invalid → ✗ reject (bad_signature)
   Ed25519 valid + ML-DSA-65 missing   → ✗ reject (malformed)
   Ed25519 missing + ML-DSA-65 valid   → ✗ reject (malformed)

This is the structural defense in depth — both components must verify, both components must be present, no degradation paths.

Regenerating the fixtures

Section titled “Regenerating the fixtures”
Terminal window
cd ratify-protocol
go run ./cmd/ratify-testvectors -out /tmp/regen
diff -rq testvectors/v1/ /tmp/regen/
# expected: identical

The CI workflow runs this exact diff. If a contributor’s change causes the regenerated fixtures to differ from the committed ones, the change is non-deterministic (map iteration, RNG without a seed, time.Now() without a fixed clock) — and the PR fails.

This is how the project guarantees that fixture bytes are stable across rebuilds, across runners, across machines. Anyone can reproduce the bytes from the deterministic seeds.

Adding a new SDK

Section titled “Adding a new SDK”

To add a new language SDK (Swift, Java, C, etc.):

  1. Open a new-SDK coordination issue.
  2. Implement the SDK following SPEC.md.
  3. Implement a conformance test that loads testvectors/v1/*.json and runs every fixture.
  4. All 59 must pass before the SDK is merged. No exceptions, no negotiated drift.

See docs/SDKS.md in the protocol repo for the full conformance contract.

Where to next

Section titled “Where to next”