We counted every constraint in OSCAL 1.2.2. All 348 of them.

Every OSCAL validator claims to "validate OSCAL." Almost none of them can tell you what that sentence means.

NIST's OSCAL models are authored in Metaschema – eight root modules plus shared imports, thousands of lines of XML that define not just document shape but semantics: allowed value vocabularies, uniqueness rules, cross-reference indexes, cardinality requirements. A JSON Schema catches the shape. The semantics are where compliance documents actually go wrong – and where "we validate OSCAL" quietly becomes "we validate some of OSCAL, we're not sure which part."

We wanted to be sure which part. So we built our validator the way you'd audit a system: start from the sources, enumerate everything, and account for every single item.

Pinning the standard to bytes

The first decision was the important one: the NIST sources are the authority – not the reference implementation. We pinned usnistgov/OSCAL at commit 21403b4a… (v1.2.2), cached every Metaschema byte-exact with SHA-256 hashes, and treated the venerable Java OSCAL CLI as what it actually is: a comparator to cross-examine, not a truth to copy. (It embeds OSCAL 1.2.1 bindings, for one thing – it can process 1.2.2 documents, but it can't speak 1.2.2.)

Then we lexically inventoried every constraint occurrence in the pinned sources. Not constraint types – occurrences, identified by file and line, so two constraints that happen to share an ID never collapse into one. The count came to 348.

Proving each one

For every occurrence, our build must hold exactly one of three verdicts, enforced by a generator that recomputes the ledger on every run:

• Proven enforced (303): an occurrence-exact test drives the real evaluator over a minimal document and asserts that this file-and-line occurrence was evaluated. The test's AST is machine-verified – the subject must execute exactly once, its result must flow into exactly one assertion – and the test file plus its transitive imports are hash-pinned. Change the evaluator, and the evidence goes stale until re-derived.
• Schema-enforced (1): provably covered by the release JSON Schema.
• Excluded with rationale (44): explicitly out, each with a written, reviewable reason.

Unaccounted for: zero.

That last bucket – the exclusions – is where it got interesting.

The standard has bugs

When you check 348 constraints one by one, you find things. Three of our exclusions are defects in the NIST sources themselves:

• An orphaned constraint. oscal_mapping-common_metaschema.xml line 657 constrains a flag that's defined five lines earlier – and then never referenced by anything. No document that can exist carries this flag. The constraint is unsatisfiable.
• An impossible predicate. A rule for inventory-item prop names applies only when @type is software, hardware, or service – but inventory-item declares no type flag at all. (Its neighbor, asset-id, is itself commented out in the source.) The predicate can never match.
• Constraints inside comments. Five constraint occurrences live inside comment blocks – including an entire value vocabulary for profile with-child-controls. A human reading the source sees them; the compiled schema never enforces them. All three findings are reported upstream: usnistgov/OSCAL#2254, #2255, #2256.

And the audit cut both ways: our own evaluator had a bug where constraints targeting flags (like the allowed values of action/@type) compiled into the inventory but silently never fired. We didn't find it by luck – the closure gate refused to accept evidence for those two occurrences, which is precisely the failure mode the whole system is designed to expose.

Then we ran the comparison for real

Claiming "the reference implementation misses things" from source inspection alone is exactly the kind of unproven assertion this project exists to kill. So we downloaded the Java OSCAL CLI 3.2.0 from Maven Central, hash-verified it, and ran both validators over a fixture family whose baseline documents pass cleanly on both sides – so every disagreement is attributable to one injected change.

OSCAL 1.2.2 changed exactly two constraints beyond version bumps (both in the SSP model). The Java CLI – which embeds 1.2.1 bindings – is on the wrong side of every observable consequence:

• An SSP with a dangling rel="validation" component link violates 1.2.2. Java: valid, exit 0.
• An SSP with a dangling rel="validated-by" link is fine per 1.2.2 (the constraint was deleted). Java: invalid.
• A responsible-role without the optional party-uuid is fine per 1.2.2 – NIST added the predicate specifically to fix this. Java errors with, literally, Key reference [null] not found.

Then one that isn't about version skew at all: a catalog whose metadata action declares "type": "invented-type" – forbidden at line 877 of the metadata Metaschema in both 1.2.2 and the 1.2.1 sources Java embeds. Java declares it valid, in JSON and XML. The constraint targets a flag; flag-targeted constraints silently not firing is precisely the bug class our completeness gate caught in our own evaluator. The reference implementation has the same class of bug – and no closure gate to catch it. We reported it upstream as metaschema-framework/oscal-cli#279. (We mapped the class mechanically: exactly 2 of 200 allowed-values occurrences target flags, and the only error-grade member is the one we live-tested. No untested remainder.)

And the honesty beat, because the run cut both ways: our validator rejected NIST's own SP 800-53 rev5 catalog. Twenty errors, all one defect – our index evaluator treated a missing optional key field as a violation where the Metaschema spec says it's a null key. Java accepted the file; we were wrong; we fixed it the same day, and 800-53 validates clean. The run later caught a second one: we kept enforcing the OSCAL action-type vocabulary even when action/@system declared a custom URI – the exact per-organization segmentation NIST's own remarks call for. Also fixed. A differential harness that only ever finds the other side's bugs isn't a harness, it's marketing.

(Also filed under "both sides": neither resolver validated its own output semantically. Java's resolve-profile will happily write a catalog that Java's own validate then rejects – and ours did the same, schema-gated but not semantics-gated. Ours now refuses to write semantically invalid resolutions; Java still can't, because closing that hole requires a semantic layer you trust.)

Validating the graph, not just the file

The second thing missing from existing tooling: OSCAL documents don't live alone. An assessment result imports an assessment plan, which imports an SSP, which imports a profile, which resolves against catalogs. The model documentation is full of relationships that no single-file validator can check – "this finding's target must resolve to a statement in the baseline," "this POA&M must identify its system."

So we built cross-document validation that resolves those edges for real. Ask it to check an SSP, and it resolves the imported baseline through the full (draft-spec) profile-resolution pipeline down to the catalogs, computes the selected control set, and tells you:

{
  "ruleId": "SSP-003.b",
  "severity": "warning",
  "path": "/…/implemented-requirements/1/control-id",
  "message": "implemented control 'ac-99' is not supplied by the resolved import-profile 'file:///…/baseline.json'"
}

With one crucial design rule: these relationship checks are interpretations of documented prose, and we treat them that way – off by default, warnings not errors, each field's resolution scope pinned to its source line, and anything merely inferable excluded by name. Where the standard hedges, a validator shouldn't bluff. (Our favorite example: the POA&M spec says an SSP import or a system-id is required – "Both may be present." So we warn only when neither exists. No invented exclusive-or.)

Why TypeScript?

Because the documents live in browsers, serverless functions, CLIs, and databases – and we wanted one validator in all of them, not a JVM sidecar next to each. The validators are precompiled Ajv standalone modules: plain generated JavaScript, zero runtime schema compilation, byte-stable output. The same code that powers our validation surfaces runs unchanged in your browser tab and in our CLI. And determinism isn't just an ops nicety – it's what makes hash-pinned evidence possible at all.

import { createDiagnosticsOscalProcessor } from "@secani/oscal/diagnostics";
const result = createDiagnosticsOscalProcessor({ maxIssues: 50 }).validate(doc);

Where this lands

• Full OSCAL 1.1.2 → 1.2.2 support, 1.2.2-native semantics
• Complete profile resolution per the draft spec, with fail-closed handling of everything the draft leaves undefined – and a resolver that refuses to write semantically invalid output
• The Java CLI's entire OSCAL-facing surface, plus the graph validation it never had (validate --resolve-imports --interpretive all)
• And a claim you can audit: 348/348 constraint occurrences accounted for, in a machine-readable ledger our CI recomputes on every commit

The full technical report – authority model, completeness closure, the differential receipts table, and what we deliberately do not claim – lives in the validator documentation. The validator is in private staging ahead of an open-source release; the hosted validation API exposes the schema level today, with the semantic layer to follow.