diff --git a/README.md b/README.md
index 27a9f06..36becc1 100644
--- a/README.md
+++ b/README.md
@@ -18,6 +18,10 @@ their outputs.
 You can also create your own model-graded evaluations with Autoevals. It's easy to add custom prompts, parse outputs,
 and manage exceptions.
 
+To measure whether repeated categorical judgments agree on the same cases, use
+the offline [judge consistency utilities](docs/JUDGE_CONSISTENCY.md). Both Python
+and TypeScript report agreement, missing coverage and optional reference accuracy.
+
 <div className="hidden">
 
 ### Requirements
diff --git a/docs/JUDGE_CONSISTENCY.md b/docs/JUDGE_CONSISTENCY.md
new file mode 100644
index 0000000..e8d7b8d
--- /dev/null
+++ b/docs/JUDGE_CONSISTENCY.md
@@ -0,0 +1,188 @@
+# Repeated judge consistency
+
+Similar average scores can hide different behavior: one judge can repeatedly
+change its answer on the same case while another gives stable answers. The
+consistency utilities analyze **one categorical criterion** over an explicit
+case-by-rater panel. A rater can mean a named model, human annotator, or fixed
+repeat slot of the same judge. This is an offline analysis API, not a Scorer
+that sends another model request.
+
+Consistency is not correctness. A panel can agree perfectly on an incorrect
+answer. Optional reference labels provide a separate accuracy report. Shared
+errors, correlated repeat slots and unrepresentative benchmarks remain study
+design concerns; the library cannot establish an independent ground truth.
+
+## Python
+
+```python
+from autoevals import (
+    ConsistencyGate, ConsistencyPlan, Rating, analyze_consistency,
+    consistency_scores, render_consistency_markdown,
+)
+
+plan = ConsistencyPlan(
+    score_name="answer_supported",
+    raters=("repeat-1", "repeat-2", "repeat-3"),
+    categories=("no", "yes"),
+    cases=("case-1", "case-2"),
+    gates=(ConsistencyGate("raw_agreement", minimum=0.9),),
+)
+ratings = [
+    Rating("case-1", "repeat-1", "yes"),
+    Rating("case-1", "repeat-2", "yes"),
+    Rating("case-1", "repeat-3", "no"),
+    Rating("case-2", "repeat-1", status="error"),
+]
+report = analyze_consistency(
+    ratings, plan=plan, references={"case-1": "yes", "case-2": "no"}
+)
+print(render_consistency_markdown(report))
+scores = consistency_scores(report)
+```
+
+## TypeScript
+
+```typescript
+import {
+  analyzeConsistency,
+  consistencyScores,
+  renderConsistencyMarkdown,
+} from "autoevals";
+
+const report = analyzeConsistency(
+  [
+    { case_id: "case-1", rater_id: "repeat-1", category: "yes" },
+    { case_id: "case-1", rater_id: "repeat-2", category: "yes" },
+    { case_id: "case-1", rater_id: "repeat-3", category: "no" },
+    { case_id: "case-2", rater_id: "repeat-1", status: "error" },
+  ],
+  {
+    plan: {
+      score_name: "answer_supported",
+      raters: ["repeat-1", "repeat-2", "repeat-3"],
+      categories: ["no", "yes"],
+      cases: ["case-1", "case-2"],
+      gates: [{ statistic: "raw_agreement", minimum: 0.9 }],
+    },
+    references: { "case-1": "yes", "case-2": "no" },
+  },
+);
+console.log(renderConsistencyMarkdown(report));
+const scores = consistencyScores(report);
+```
+
+Both examples return an `inconclusive` requirement verdict: the missing
+observations prevent the stated support and coverage requirements from being
+met. They are not silently discarded to make the panel look reliable.
+
+## Collecting native Scores
+
+Use `ScoreMapping` and `rating_from_score` in Python, or `ratingFromScore` with a
+`ScoreMapping` in TypeScript. Supply exact numeric/category pairs, for example
+`[(0, "no"), (1, "yes")]` / `[[0, "no"], [1, "yes"]]`. The adapter checks the
+Score name and rejects unmapped scores, booleans and numeric strings. It does
+not infer thresholds or round continuous scores into labels. Define any
+discretization policy explicitly in your application before using this API.
+
+A native Score with `score=None` / `null` becomes `missing`. A non-null legacy
+Score `error` takes precedence over its numeric value and becomes `error`.
+Modern scorers throw exceptions: catch them in your collection code and call
+`error_rating` / `errorRating`. Error messages and provider metadata are never
+copied into a Rating. Do not use the same rater slot twice for a case; record a
+new repeat with a different declared rater identity.
+
+An explicit `cases` manifest retains cases for which **no judge returned**.
+Omitting it uses the observed union and records `cohort_provenance=observed_union`;
+the API cannot discover entirely absent cases. References and groups must be
+subsets of the cohort, so use a manifest if they include unobserved cases.
+`missing`, `error` and an absent cell have separate counts. Only `scored` cells
+enter categorical statistics. Inputs must use valid Unicode strings; identifiers
+are compared exactly and sorted by UTF-8 bytes without Unicode normalization.
+
+## Statistics and denominators
+
+| Output               | Supporting observations                          | Interpretation                                                                                                          |
+| -------------------- | ------------------------------------------------ | ----------------------------------------------------------------------------------------------------------------------- |
+| Raw agreement        | Cases with at least two scored ratings           | Mean per-case ordered-pair agreement; each supporting case has equal weight                                             |
+| Fleiss kappa         | Complete declared panels only                    | Fixed-panel coefficient with pooled category marginals                                                                  |
+| Nominal alpha        | All cases with at least two scored ratings       | Krippendorff nominal coincidence weighting; singleton ratings are excluded from both observed and expected disagreement |
+| Pairwise Cohen kappa | Overlap for that fixed rater pair                | Each marginal uses the same pairwise overlap; confusion cells are sparse                                                |
+| Consensus accuracy   | Reference-labeled cases with a decided consensus | Conditional accuracy, reported together with labeled/evaluated counts and reference coverage                            |
+
+Each coefficient includes supporting cases and ratings, denominator cases,
+weighting, and an explicit undefined reason. Zero expected disagreement is
+undefined, not perfect agreement. A constant-varying pair may have defined
+zero Cohen kappa. Negative coefficients remain negative.
+
+Fleiss follows the fixed-panel formulation in
+[statsmodels](https://www.statsmodels.org/stable/generated/statsmodels.stats.inter_rater.fleiss_kappa.html).
+Nominal alpha follows the coincidence construction in
+[Krippendorff, Computing Alpha-Reliability](https://www.asc.upenn.edu/sites/default/files/2021-03/Computing%20Krippendorff's%20Alpha-Reliability.pdf).
+The shared tests include that paper's missing-data example.
+
+Consensus requires strictly more than half the **declared** rater slots to
+select one category, plus `minimum_consensus_ratings` (default 2) scored
+ratings. Missing slots cannot make one remaining judge unanimous. Ties,
+insufficient ratings, and absence of a declared-panel majority are distinct
+abstention reasons. Reference labels never break a tie.
+
+## Requirements, groups and diagnostic intervals
+
+Predeclare `groups` as named lists of case IDs; `overall` is reserved. Groups
+may overlap. A gate selects one of `raw_agreement`, `fleiss_kappa`,
+`nominal_alpha`, or `consensus_accuracy`, a `minimum` value, and optionally a
+group. Defaults require at least two supporting cases, 100% statistic coverage
+and 100% scored-cell coverage. Override them explicitly for a different study.
+Accuracy's statistic coverage denominator is reference-labeled cases, not all
+cases; its rating coverage denominator still includes every planned cell.
+
+An undefined value or insufficient support/coverage produces `inconclusive`.
+Only sufficiently supported observations can pass or fail their value threshold.
+Any failed gate makes the overall verdict `fail`; otherwise any inconclusive
+gate makes it `inconclusive`. With no gates the verdict is `not_evaluated`.
+These are requirements on the supplied finite benchmark, not population-level
+confidence claims or a universal definition of an acceptable judge.
+
+`ConsistencyBootstrap` / the plan's `bootstrap` enables marginal diagnostic
+percentile intervals (`resamples=0` disables them). Whole cases, including their
+missingness and references, are resampled with replacement. Duplicate sampled
+occurrences remain distinct. Each group's UTF-8 name and the configured seed
+initialize a specified FNV1a/xorshift32 generator identically in both languages;
+global RNG state is untouched. Intervals use linear interpolation between
+ordered samples. Defaults require at least 20 supporting cases and at least
+20 expected draws in each tail. Any undefined draw or degenerate distribution
+makes the interval unavailable, with attempted/valid/invalid counts retained.
+Intervals never determine gate decisions and are not simultaneous intervals
+across multiple groups or statistics.
+
+## Replaying and sharing evidence
+
+`input_to_json` / `inputToJSON` serializes selected observations, the normalized
+plan and references. `analyze_consistency_json` / `analyzeConsistencyJSON`
+validates that versioned bundle and recomputes its report offline. Unknown
+fields, duplicate keys (including escaped duplicates), invalid Unicode and
+nonfinite values are rejected. The wire format uses snake_case in both
+languages. `report_to_json` / `reportToJSON` exports a bounded finite report;
+report export is not a signature or authenticity proof.
+
+`consistency_scores` / `consistencyScores` exposes agreement, rating coverage
+and consensus accuracy as native Scores. Undefined values stay null. Signed
+kappa and alpha are not squeezed into Score's [0,1] range. Compact support
+metadata is included; full case-level report metadata requires explicit
+`include_report=True` / `{ includeReport: true }`. Case IDs, labels and reference
+answers can themselves be sensitive even when provider metadata is excluded.
+
+`ConsistencyLimits` is application-owned and is not read from portable input.
+Default ceilings cover rating count, panel dimensions, group membership work,
+pairwise and bootstrap work, selected bytes, JSON bytes/depth/nodes and report
+details. Analysis checks declared dimensions before reading observations and
+checks cumulative work before constructing matrices or sampling. `detail_limit`
+is shared across overall and group sections, so omitted cases are counted.
+Callers processing unusually large studies should set their own limits and
+partition the study intentionally. Runtime depends on cases, raters, groups
+and resamples; the API performs no scheduling or concurrent model execution.
+
+Runnable offline examples are in `examples/judge_consistency.py` and
+`examples/judge_consistency.ts`. They exercise native Score adaptation,
+unavailable results, requirements, replay and summary projection without
+requiring API credentials.
diff --git a/examples/judge_consistency.py b/examples/judge_consistency.py
new file mode 100644
index 0000000..02db421
--- /dev/null
+++ b/examples/judge_consistency.py
@@ -0,0 +1,53 @@
+"""Run with uv run python examples/judge_consistency.py; no model calls."""
+
+from autoevals import (
+    ConsistencyGate,
+    ConsistencyPlan,
+    Score,
+    ScoreMapping,
+    analyze_consistency,
+    analyze_consistency_json,
+    consistency_scores,
+    error_rating,
+    input_to_json,
+    rating_from_score,
+    render_consistency_markdown,
+)
+
+
+def main():
+    plan = ConsistencyPlan(
+        "supported",
+        ("repeat-1", "repeat-2", "repeat-3"),
+        ("no", "yes"),
+        cases=("case-1", "case-2", "case-3"),
+        gates=(ConsistencyGate("raw_agreement", 0.9),),
+    )
+    mapping = ScoreMapping("supported", ((0, "no"), (1, "yes")))
+    # Substitute stored native Scores from your existing judge calls here.
+    stored = {
+        "case-1": (1, 1, 1),
+        "case-2": (0, 1, 0),
+        "case-3": (None, 1, None),
+    }
+    ratings = []
+    for case_id, values in stored.items():
+        for rater_id, value in zip(plan.raters, values):
+            if case_id == "case-3" and rater_id == "repeat-3":
+                ratings.append(error_rating(case_id, rater_id))
+            else:
+                score = Score("supported", value, metadata={"rationale": "not exported"})
+                ratings.append(rating_from_score(case_id, rater_id, score, mapping=mapping))
+    references = {"case-1": "yes", "case-2": "no", "case-3": "yes"}
+    report = analyze_consistency(ratings, plan=plan, references=references)
+    bundle = input_to_json(ratings, plan=plan, references=references)
+    assert analyze_consistency_json(bundle) == report
+    assert "rationale" not in bundle
+    assert report["verdict"] == "inconclusive"
+    print(render_consistency_markdown(report))
+    for score in consistency_scores(report):
+        print(score.name, score.score)
+
+
+if __name__ == "__main__":
+    main()
diff --git a/examples/judge_consistency.ts b/examples/judge_consistency.ts
new file mode 100644
index 0000000..7d3e803
--- /dev/null
+++ b/examples/judge_consistency.ts
@@ -0,0 +1,60 @@
+/** Run with pnpm exec tsx examples/judge_consistency.ts. No model calls. */
+import {
+  analyzeConsistency,
+  analyzeConsistencyJSON,
+  consistencyScores,
+  errorRating,
+  inputToJSON,
+  ratingFromScore,
+  renderConsistencyMarkdown,
+  type ConsistencyPlan,
+  type Rating,
+  type Score,
+  type ScoreMapping,
+} from "autoevals";
+
+const plan: ConsistencyPlan = {
+  score_name: "supported",
+  raters: ["repeat-1", "repeat-2", "repeat-3"],
+  categories: ["no", "yes"],
+  cases: ["case-1", "case-2", "case-3"],
+  gates: [{ statistic: "raw_agreement", minimum: 0.9 }],
+};
+const mapping: ScoreMapping = {
+  score_name: "supported",
+  entries: [
+    [0, "no"],
+    [1, "yes"],
+  ],
+};
+const stored: [string, (number | null)[]][] = [
+  ["case-1", [1, 1, 1]],
+  ["case-2", [0, 1, 0]],
+  ["case-3", [null, 1, null]],
+];
+const ratings: Rating[] = [];
+for (const [caseId, values] of stored) {
+  plan.raters.forEach((raterId, i) => {
+    if (caseId === "case-3" && raterId === "repeat-3") {
+      ratings.push(errorRating(caseId, raterId));
+    } else {
+      const score: Score = {
+        name: "supported",
+        score: values[i],
+        metadata: { rationale: "not exported" },
+      };
+      ratings.push(ratingFromScore(caseId, raterId, score, mapping));
+    }
+  });
+}
+const references = { "case-1": "yes", "case-2": "no", "case-3": "yes" };
+const report = analyzeConsistency(ratings, { plan, references });
+const bundle = inputToJSON(ratings, { plan, references });
+if (JSON.stringify(analyzeConsistencyJSON(bundle)) !== JSON.stringify(report))
+  throw new Error("replay changed evidence");
+if (bundle.includes("rationale"))
+  throw new Error("unexpected provider metadata");
+if (report.verdict !== "inconclusive")
+  throw new Error("missing coverage must not pass");
+console.log(renderConsistencyMarkdown(report));
+console.log(consistencyScores(report));
diff --git a/fixtures/consistency/studies.json b/fixtures/consistency/studies.json
new file mode 100644
index 0000000..5c810fe
--- /dev/null
+++ b/fixtures/consistency/studies.json
@@ -0,0 +1,1054 @@
+[
+  {
+    "name": "complete disagreement",
+    "input": {
+      "format": "autoevals.consistency.input",
+      "version": 1,
+      "plan": {
+        "score_name": "judge",
+        "raters": [
+          "a",
+          "b"
+        ],
+        "categories": [
+          "no",
+          "yes"
+        ]
+      },
+      "ratings": [
+        {
+          "case_id": "0",
+          "rater_id": "a",
+          "category": "yes"
+        },
+        {
+          "case_id": "0",
+          "rater_id": "b",
+          "category": "no"
+        },
+        {
+          "case_id": "1",
+          "rater_id": "a",
+          "category": "no"
+        },
+        {
+          "case_id": "1",
+          "rater_id": "b",
+          "category": "yes"
+        }
+      ],
+      "references": {}
+    },
+    "expected": {
+      "verdict": "not_evaluated",
+      "groups": [
+        {
+          "statistics": {
+            "raw_agreement": {
+              "value": 0
+            },
+            "fleiss_kappa": {
+              "value": -1
+            },
+            "nominal_alpha": {
+              "value": -0.5
+            }
+          },
+          "consensus": {
+            "decided_cases": 0,
+            "abstained_cases": 2
+          }
+        }
+      ]
+    }
+  },
+  {
+    "name": "declared missing cohort",
+    "input": {
+      "format": "autoevals.consistency.input",
+      "version": 1,
+      "plan": {
+        "score_name": "judge",
+        "raters": [
+          "a",
+          "b"
+        ],
+        "categories": [
+          "no",
+          "yes"
+        ],
+        "cases": [
+          "0",
+          "1",
+          "2"
+        ],
+        "groups": [
+          {
+            "name": "absent",
+            "cases": [
+              "2"
+            ]
+          }
+        ],
+        "gates": [
+          {
+            "statistic": "raw_agreement",
+            "minimum": 0.9
+          }
+        ]
+      },
+      "ratings": [
+        {
+          "case_id": "0",
+          "rater_id": "a",
+          "category": "yes"
+        },
+        {
+          "case_id": "0",
+          "rater_id": "b",
+          "category": "yes"
+        },
+        {
+          "case_id": "1",
+          "rater_id": "a",
+          "category": "no"
+        }
+      ],
+      "references": {
+        "0": "yes",
+        "1": "no",
+        "2": "yes"
+      }
+    },
+    "expected": {
+      "verdict": "inconclusive",
+      "groups": [
+        {
+          "coverage": {
+            "expected_cells": 6,
+            "scored_cells": 3,
+            "absent_cells": 3
+          },
+          "consensus": {
+            "reference": {
+              "accuracy": 1,
+              "coverage": 0.3333333333333333
+            }
+          }
+        },
+        {
+          "coverage": {
+            "zero_rating_cases": 1
+          }
+        }
+      ]
+    }
+  },
+  {
+    "name": "prototype and unicode",
+    "input": {
+      "format": "autoevals.consistency.input",
+      "version": 1,
+      "plan": {
+        "score_name": "judge",
+        "raters": [
+          "__proto__",
+          "constructor"
+        ],
+        "categories": [
+          "no",
+          "yes"
+        ],
+        "cases": [
+          "",
+          "𐀀",
+          "__proto__"
+        ]
+      },
+      "ratings": [
+        {
+          "case_id": "__proto__",
+          "rater_id": "constructor",
+          "category": "yes"
+        },
+        {
+          "case_id": "__proto__",
+          "rater_id": "__proto__",
+          "status": "error"
+        }
+      ],
+      "references": {
+        "__proto__": "yes"
+      }
+    },
+    "expected": {
+      "groups": [
+        {
+          "coverage": {
+            "scored_cells": 1,
+            "error_cells": 1,
+            "absent_cells": 4
+          }
+        }
+      ]
+    }
+  },
+  {
+    "name": "all unavailable",
+    "input": {
+      "format": "autoevals.consistency.input",
+      "version": 1,
+      "plan": {
+        "score_name": "judge",
+        "raters": [
+          "a",
+          "b"
+        ],
+        "categories": [
+          "no",
+          "yes"
+        ]
+      },
+      "ratings": [
+        {
+          "case_id": "x",
+          "rater_id": "a",
+          "status": "missing"
+        },
+        {
+          "case_id": "x",
+          "rater_id": "b",
+          "status": "error"
+        }
+      ],
+      "references": {}
+    },
+    "expected": {
+      "groups": [
+        {
+          "statistics": {
+            "raw_agreement": {
+              "value": null
+            },
+            "fleiss_kappa": {
+              "value": null
+            },
+            "nominal_alpha": {
+              "value": null
+            }
+          },
+          "coverage": {
+            "rating_coverage": 0
+          }
+        }
+      ]
+    }
+  },
+  {
+    "name": "seeded repeated cases",
+    "input": {
+      "format": "autoevals.consistency.input",
+      "version": 1,
+      "plan": {
+        "score_name": "judge",
+        "raters": [
+          "a",
+          "b"
+        ],
+        "categories": [
+          "no",
+          "yes"
+        ],
+        "bootstrap": {
+          "resamples": 500,
+          "confidence": 0.8,
+          "minimum_cases": 2,
+          "seed": 17
+        }
+      },
+      "ratings": [
+        {
+          "case_id": "0",
+          "rater_id": "a",
+          "category": "no"
+        },
+        {
+          "case_id": "0",
+          "rater_id": "b",
+          "category": "yes"
+        },
+        {
+          "case_id": "1",
+          "rater_id": "a",
+          "category": "yes"
+        },
+        {
+          "case_id": "1",
+          "rater_id": "b",
+          "category": "yes"
+        },
+        {
+          "case_id": "2",
+          "rater_id": "a",
+          "category": "yes"
+        },
+        {
+          "case_id": "2",
+          "rater_id": "b",
+          "category": "no"
+        },
+        {
+          "case_id": "3",
+          "rater_id": "a",
+          "category": "no"
+        },
+        {
+          "case_id": "3",
+          "rater_id": "b",
+          "category": "yes"
+        },
+        {
+          "case_id": "4",
+          "rater_id": "a",
+          "category": "yes"
+        },
+        {
+          "case_id": "4",
+          "rater_id": "b",
+          "category": "yes"
+        },
+        {
+          "case_id": "5",
+          "rater_id": "a",
+          "category": "yes"
+        },
+        {
+          "case_id": "5",
+          "rater_id": "b",
+          "category": "no"
+        },
+        {
+          "case_id": "6",
+          "rater_id": "a",
+          "category": "no"
+        },
+        {
+          "case_id": "6",
+          "rater_id": "b",
+          "category": "yes"
+        },
+        {
+          "case_id": "7",
+          "rater_id": "a",
+          "category": "yes"
+        },
+        {
+          "case_id": "7",
+          "rater_id": "b",
+          "category": "yes"
+        },
+        {
+          "case_id": "8",
+          "rater_id": "a",
+          "category": "yes"
+        },
+        {
+          "case_id": "8",
+          "rater_id": "b",
+          "category": "no"
+        },
+        {
+          "case_id": "9",
+          "rater_id": "a",
+          "category": "no"
+        },
+        {
+          "case_id": "9",
+          "rater_id": "b",
+          "category": "yes"
+        },
+        {
+          "case_id": "10",
+          "rater_id": "a",
+          "category": "yes"
+        },
+        {
+          "case_id": "10",
+          "rater_id": "b",
+          "category": "yes"
+        },
+        {
+          "case_id": "11",
+          "rater_id": "a",
+          "category": "yes"
+        },
+        {
+          "case_id": "11",
+          "rater_id": "b",
+          "category": "no"
+        },
+        {
+          "case_id": "12",
+          "rater_id": "a",
+          "category": "no"
+        },
+        {
+          "case_id": "12",
+          "rater_id": "b",
+          "category": "yes"
+        },
+        {
+          "case_id": "13",
+          "rater_id": "a",
+          "category": "yes"
+        },
+        {
+          "case_id": "13",
+          "rater_id": "b",
+          "category": "yes"
+        },
+        {
+          "case_id": "14",
+          "rater_id": "a",
+          "category": "yes"
+        },
+        {
+          "case_id": "14",
+          "rater_id": "b",
+          "category": "no"
+        },
+        {
+          "case_id": "15",
+          "rater_id": "a",
+          "category": "no"
+        },
+        {
+          "case_id": "15",
+          "rater_id": "b",
+          "category": "yes"
+        },
+        {
+          "case_id": "16",
+          "rater_id": "a",
+          "category": "yes"
+        },
+        {
+          "case_id": "16",
+          "rater_id": "b",
+          "category": "yes"
+        },
+        {
+          "case_id": "17",
+          "rater_id": "a",
+          "category": "yes"
+        },
+        {
+          "case_id": "17",
+          "rater_id": "b",
+          "category": "no"
+        },
+        {
+          "case_id": "18",
+          "rater_id": "a",
+          "category": "no"
+        },
+        {
+          "case_id": "18",
+          "rater_id": "b",
+          "category": "yes"
+        },
+        {
+          "case_id": "19",
+          "rater_id": "a",
+          "category": "yes"
+        },
+        {
+          "case_id": "19",
+          "rater_id": "b",
+          "category": "yes"
+        },
+        {
+          "case_id": "20",
+          "rater_id": "a",
+          "category": "yes"
+        },
+        {
+          "case_id": "20",
+          "rater_id": "b",
+          "category": "no"
+        },
+        {
+          "case_id": "21",
+          "rater_id": "a",
+          "category": "no"
+        },
+        {
+          "case_id": "21",
+          "rater_id": "b",
+          "category": "yes"
+        },
+        {
+          "case_id": "22",
+          "rater_id": "a",
+          "category": "yes"
+        },
+        {
+          "case_id": "22",
+          "rater_id": "b",
+          "category": "yes"
+        },
+        {
+          "case_id": "23",
+          "rater_id": "a",
+          "category": "yes"
+        },
+        {
+          "case_id": "23",
+          "rater_id": "b",
+          "category": "no"
+        },
+        {
+          "case_id": "24",
+          "rater_id": "a",
+          "category": "no"
+        },
+        {
+          "case_id": "24",
+          "rater_id": "b",
+          "category": "yes"
+        },
+        {
+          "case_id": "25",
+          "rater_id": "a",
+          "category": "yes"
+        },
+        {
+          "case_id": "25",
+          "rater_id": "b",
+          "category": "yes"
+        },
+        {
+          "case_id": "26",
+          "rater_id": "a",
+          "category": "yes"
+        },
+        {
+          "case_id": "26",
+          "rater_id": "b",
+          "category": "no"
+        },
+        {
+          "case_id": "27",
+          "rater_id": "a",
+          "category": "no"
+        },
+        {
+          "case_id": "27",
+          "rater_id": "b",
+          "category": "yes"
+        },
+        {
+          "case_id": "28",
+          "rater_id": "a",
+          "category": "yes"
+        },
+        {
+          "case_id": "28",
+          "rater_id": "b",
+          "category": "yes"
+        },
+        {
+          "case_id": "29",
+          "rater_id": "a",
+          "category": "yes"
+        },
+        {
+          "case_id": "29",
+          "rater_id": "b",
+          "category": "no"
+        }
+      ],
+      "references": {}
+    },
+    "expected": {
+      "groups": [
+        {
+          "coverage": {
+            "complete_cases": 30
+          },
+          "statistics": {
+            "raw_agreement": {
+              "value": 0.3333333333333333
+            }
+          }
+        }
+      ]
+    }
+  },
+  {
+    "name": "zero cohort",
+    "input": {
+      "format": "autoevals.consistency.input",
+      "version": 1,
+      "plan": {
+        "score_name": "judge",
+        "raters": [
+          "a",
+          "b"
+        ],
+        "categories": [
+          "no",
+          "yes"
+        ],
+        "cases": []
+      },
+      "ratings": [],
+      "references": {}
+    },
+    "expected": {
+      "groups": [
+        {
+          "coverage": {
+            "expected_cells": 0,
+            "rating_coverage": null
+          }
+        }
+      ]
+    }
+  },
+  {
+    "name": "fractional agreement threshold 15",
+    "input": {
+      "format": "autoevals.consistency.input",
+      "version": 1,
+      "plan": {
+        "score_name": "judge",
+        "raters": [
+          "a",
+          "b",
+          "c"
+        ],
+        "categories": [
+          "no",
+          "yes"
+        ],
+        "gates": [
+          {
+            "statistic": "raw_agreement",
+            "minimum": 0.3333333333333333
+          }
+        ]
+      },
+      "ratings": [
+        {
+          "case_id": "0",
+          "rater_id": "a",
+          "category": "yes"
+        },
+        {
+          "case_id": "0",
+          "rater_id": "b",
+          "category": "yes"
+        },
+        {
+          "case_id": "0",
+          "rater_id": "c",
+          "category": "no"
+        },
+        {
+          "case_id": "1",
+          "rater_id": "a",
+          "category": "yes"
+        },
+        {
+          "case_id": "1",
+          "rater_id": "b",
+          "category": "yes"
+        },
+        {
+          "case_id": "1",
+          "rater_id": "c",
+          "category": "no"
+        },
+        {
+          "case_id": "2",
+          "rater_id": "a",
+          "category": "yes"
+        },
+        {
+          "case_id": "2",
+          "rater_id": "b",
+          "category": "yes"
+        },
+        {
+          "case_id": "2",
+          "rater_id": "c",
+          "category": "no"
+        },
+        {
+          "case_id": "3",
+          "rater_id": "a",
+          "category": "yes"
+        },
+        {
+          "case_id": "3",
+          "rater_id": "b",
+          "category": "yes"
+        },
+        {
+          "case_id": "3",
+          "rater_id": "c",
+          "category": "no"
+        },
+        {
+          "case_id": "4",
+          "rater_id": "a",
+          "category": "yes"
+        },
+        {
+          "case_id": "4",
+          "rater_id": "b",
+          "category": "yes"
+        },
+        {
+          "case_id": "4",
+          "rater_id": "c",
+          "category": "no"
+        },
+        {
+          "case_id": "5",
+          "rater_id": "a",
+          "category": "yes"
+        },
+        {
+          "case_id": "5",
+          "rater_id": "b",
+          "category": "yes"
+        },
+        {
+          "case_id": "5",
+          "rater_id": "c",
+          "category": "no"
+        },
+        {
+          "case_id": "6",
+          "rater_id": "a",
+          "category": "yes"
+        },
+        {
+          "case_id": "6",
+          "rater_id": "b",
+          "category": "yes"
+        },
+        {
+          "case_id": "6",
+          "rater_id": "c",
+          "category": "no"
+        },
+        {
+          "case_id": "7",
+          "rater_id": "a",
+          "category": "yes"
+        },
+        {
+          "case_id": "7",
+          "rater_id": "b",
+          "category": "yes"
+        },
+        {
+          "case_id": "7",
+          "rater_id": "c",
+          "category": "no"
+        },
+        {
+          "case_id": "8",
+          "rater_id": "a",
+          "category": "yes"
+        },
+        {
+          "case_id": "8",
+          "rater_id": "b",
+          "category": "yes"
+        },
+        {
+          "case_id": "8",
+          "rater_id": "c",
+          "category": "no"
+        },
+        {
+          "case_id": "9",
+          "rater_id": "a",
+          "category": "yes"
+        },
+        {
+          "case_id": "9",
+          "rater_id": "b",
+          "category": "yes"
+        },
+        {
+          "case_id": "9",
+          "rater_id": "c",
+          "category": "no"
+        },
+        {
+          "case_id": "10",
+          "rater_id": "a",
+          "category": "yes"
+        },
+        {
+          "case_id": "10",
+          "rater_id": "b",
+          "category": "yes"
+        },
+        {
+          "case_id": "10",
+          "rater_id": "c",
+          "category": "no"
+        },
+        {
+          "case_id": "11",
+          "rater_id": "a",
+          "category": "yes"
+        },
+        {
+          "case_id": "11",
+          "rater_id": "b",
+          "category": "yes"
+        },
+        {
+          "case_id": "11",
+          "rater_id": "c",
+          "category": "no"
+        },
+        {
+          "case_id": "12",
+          "rater_id": "a",
+          "category": "yes"
+        },
+        {
+          "case_id": "12",
+          "rater_id": "b",
+          "category": "yes"
+        },
+        {
+          "case_id": "12",
+          "rater_id": "c",
+          "category": "no"
+        },
+        {
+          "case_id": "13",
+          "rater_id": "a",
+          "category": "yes"
+        },
+        {
+          "case_id": "13",
+          "rater_id": "b",
+          "category": "yes"
+        },
+        {
+          "case_id": "13",
+          "rater_id": "c",
+          "category": "no"
+        },
+        {
+          "case_id": "14",
+          "rater_id": "a",
+          "category": "yes"
+        },
+        {
+          "case_id": "14",
+          "rater_id": "b",
+          "category": "yes"
+        },
+        {
+          "case_id": "14",
+          "rater_id": "c",
+          "category": "no"
+        }
+      ]
+    },
+    "expected": {
+      "verdict": "pass"
+    }
+  },
+  {
+    "name": "fractional agreement threshold 10",
+    "input": {
+      "format": "autoevals.consistency.input",
+      "version": 1,
+      "plan": {
+        "score_name": "judge",
+        "raters": [
+          "a",
+          "b",
+          "c"
+        ],
+        "categories": [
+          "no",
+          "yes"
+        ],
+        "gates": [
+          {
+            "statistic": "raw_agreement",
+            "minimum": 0.33333333333333337
+          }
+        ]
+      },
+      "ratings": [
+        {
+          "case_id": "0",
+          "rater_id": "a",
+          "category": "yes"
+        },
+        {
+          "case_id": "0",
+          "rater_id": "b",
+          "category": "yes"
+        },
+        {
+          "case_id": "0",
+          "rater_id": "c",
+          "category": "no"
+        },
+        {
+          "case_id": "1",
+          "rater_id": "a",
+          "category": "yes"
+        },
+        {
+          "case_id": "1",
+          "rater_id": "b",
+          "category": "yes"
+        },
+        {
+          "case_id": "1",
+          "rater_id": "c",
+          "category": "no"
+        },
+        {
+          "case_id": "2",
+          "rater_id": "a",
+          "category": "yes"
+        },
+        {
+          "case_id": "2",
+          "rater_id": "b",
+          "category": "yes"
+        },
+        {
+          "case_id": "2",
+          "rater_id": "c",
+          "category": "no"
+        },
+        {
+          "case_id": "3",
+          "rater_id": "a",
+          "category": "yes"
+        },
+        {
+          "case_id": "3",
+          "rater_id": "b",
+          "category": "yes"
+        },
+        {
+          "case_id": "3",
+          "rater_id": "c",
+          "category": "no"
+        },
+        {
+          "case_id": "4",
+          "rater_id": "a",
+          "category": "yes"
+        },
+        {
+          "case_id": "4",
+          "rater_id": "b",
+          "category": "yes"
+        },
+        {
+          "case_id": "4",
+          "rater_id": "c",
+          "category": "no"
+        },
+        {
+          "case_id": "5",
+          "rater_id": "a",
+          "category": "yes"
+        },
+        {
+          "case_id": "5",
+          "rater_id": "b",
+          "category": "yes"
+        },
+        {
+          "case_id": "5",
+          "rater_id": "c",
+          "category": "no"
+        },
+        {
+          "case_id": "6",
+          "rater_id": "a",
+          "category": "yes"
+        },
+        {
+          "case_id": "6",
+          "rater_id": "b",
+          "category": "yes"
+        },
+        {
+          "case_id": "6",
+          "rater_id": "c",
+          "category": "no"
+        },
+        {
+          "case_id": "7",
+          "rater_id": "a",
+          "category": "yes"
+        },
+        {
+          "case_id": "7",
+          "rater_id": "b",
+          "category": "yes"
+        },
+        {
+          "case_id": "7",
+          "rater_id": "c",
+          "category": "no"
+        },
+        {
+          "case_id": "8",
+          "rater_id": "a",
+          "category": "yes"
+        },
+        {
+          "case_id": "8",
+          "rater_id": "b",
+          "category": "yes"
+        },
+        {
+          "case_id": "8",
+          "rater_id": "c",
+          "category": "no"
+        },
+        {
+          "case_id": "9",
+          "rater_id": "a",
+          "category": "yes"
+        },
+        {
+          "case_id": "9",
+          "rater_id": "b",
+          "category": "yes"
+        },
+        {
+          "case_id": "9",
+          "rater_id": "c",
+          "category": "no"
+        }
+      ]
+    },
+    "expected": {
+      "verdict": "fail"
+    }
+  }
+]
diff --git a/js/consistency-fixtures.test.ts b/js/consistency-fixtures.test.ts
new file mode 100644
index 0000000..2575025
--- /dev/null
+++ b/js/consistency-fixtures.test.ts
@@ -0,0 +1,31 @@
+import { readFileSync } from "node:fs";
+import { describe, expect, it } from "vitest";
+import { analyzeConsistencyJSON } from "./consistency";
+
+const studies = JSON.parse(
+  readFileSync("fixtures/consistency/studies.json", "utf8"),
+);
+
+function subset(actual: unknown, expected: unknown): void {
+  if (typeof expected === "number") expect(actual).toBeCloseTo(expected, 12);
+  else if (Array.isArray(expected)) {
+    expect(Array.isArray(actual)).toBe(true);
+    expected.forEach((value, i) => subset((actual as unknown[])[i], value));
+  } else if (expected !== null && typeof expected === "object") {
+    for (const [key, value] of Object.entries(expected))
+      subset((actual as Record<string, unknown>)[key], value);
+  } else expect(actual).toEqual(expected);
+}
+
+describe("shared Python/TypeScript study fixtures", () => {
+  for (const study of studies)
+    it(study.name, () => {
+      const report = analyzeConsistencyJSON(JSON.stringify(study.input));
+      subset(report, study.expected);
+      const reversed = {
+        ...study.input,
+        ratings: [...study.input.ratings].reverse(),
+      };
+      expect(analyzeConsistencyJSON(JSON.stringify(reversed))).toEqual(report);
+    });
+});
diff --git a/js/consistency-io.test.ts b/js/consistency-io.test.ts
new file mode 100644
index 0000000..1c32b5a
--- /dev/null
+++ b/js/consistency-io.test.ts
@@ -0,0 +1,90 @@
+import { describe, expect, it } from "vitest";
+import {
+  analyzeConsistency,
+  analyzeConsistencyJSON,
+  inputToJSON,
+  reportToJSON,
+  consistencyScores,
+  renderConsistencyMarkdown,
+} from "./consistency";
+
+const plan = {
+  score_name: "judge",
+  raters: ["a", "b"],
+  categories: ["no", "yes"],
+};
+const ratings = [
+  { case_id: "x", rater_id: "a", category: "yes" },
+  { case_id: "x", rater_id: "b", category: "no" },
+];
+
+describe("portable consistency evidence", () => {
+  it("renders Markdown image syntax and backslashes literally", () => {
+    const report = analyzeConsistency([], {
+      plan: {
+        ...plan,
+        score_name: "![external](https://example.com/track)\\*",
+      },
+    });
+    expect(renderConsistencyMarkdown(report)).toContain(
+      "\\!\\[external\\]\\(https://example\\.com/track\\)\\\\\\*",
+    );
+  });
+  it("round trips selected data and recomputes its report", () => {
+    const references = new Map([["x", "yes"]]);
+    const report = analyzeConsistency(ratings, { plan, references });
+    expect(
+      analyzeConsistencyJSON(inputToJSON(ratings, { plan, references })),
+    ).toEqual(report);
+    expect(JSON.parse(reportToJSON(report))).toEqual(report);
+  });
+  it.each([
+    '{"version":1,"version":1}',
+    '{"x":NaN}',
+    '{"x":1e999}',
+    '{"x":{"a":1,"\\u0061":2}}',
+    '{"x":"\\ud800"}',
+  ])("rejects invalid JSON evidence: %s", (content) => {
+    expect(() => analyzeConsistencyJSON(content)).toThrow();
+  });
+  it("rejects unknown fields and excessive structure", () => {
+    const data = JSON.parse(inputToJSON(ratings, { plan }));
+    data.plan.unknown = 1;
+    expect(() => analyzeConsistencyJSON(JSON.stringify(data))).toThrow(
+      /unknown/,
+    );
+    expect(() =>
+      analyzeConsistencyJSON(inputToJSON(ratings, { plan }), {
+        max_json_nodes: 2,
+      }),
+    ).toThrow(/structure/);
+  });
+  it("keeps prototype-looking identifiers as ordinary data", () => {
+    const unusual = { ...plan, cases: ["__proto__"] };
+    const references = new Map([["__proto__", "yes"]]);
+    const content = inputToJSON([], { plan: unusual, references });
+    expect(
+      analyzeConsistencyJSON(content).groups[0].consensus.reference
+        .labeled_cases,
+    ).toBe(1);
+  });
+  it("projects bounded scores and leaves absent accuracy undefined", () => {
+    const report = analyzeConsistency(ratings, { plan });
+    const scores = consistencyScores(report);
+    expect(scores.map((score) => score.score)).toEqual([0, 1, null]);
+    expect(scores.every((score) => !score.metadata?.report)).toBe(true);
+    const included = consistencyScores(report, { includeReport: true });
+    report.score_name = "changed";
+    expect((included[0].metadata?.report as typeof report).score_name).toBe(
+      "judge",
+    );
+  });
+  it("escapes labels in human reports", () => {
+    const report = analyzeConsistency([], {
+      plan: { ...plan, score_name: "judge|<script>\u001b[31m" },
+    });
+    const text = renderConsistencyMarkdown(report);
+    expect(text).toContain("judge\\|&lt;script&gt;");
+    expect(text).not.toContain("\u001b");
+  });
+});
diff --git a/js/consistency-limits.test.ts b/js/consistency-limits.test.ts
new file mode 100644
index 0000000..06a0603
--- /dev/null
+++ b/js/consistency-limits.test.ts
@@ -0,0 +1,48 @@
+import { describe, expect, it } from "vitest";
+import { analyzeConsistency } from "./consistency";
+
+const plan = {
+  score_name: "j",
+  raters: ["a", "b"],
+  categories: ["n", "y"],
+  cases: ["x"],
+};
+
+describe("bounded selected evidence", () => {
+  it("charges empty pairwise report sections", () => {
+    expect(() =>
+      analyzeConsistency([], {
+        plan: { ...plan, cases: [], groups: [{ name: "empty", cases: [] }] },
+        limits: { max_pairwise_work: 1 },
+      }),
+    ).toThrow(/pairwise/);
+  });
+  it("charges declared names even with no observations", () => {
+    expect(() =>
+      analyzeConsistency([], {
+        plan: { ...plan, cases: ["long-case-identifier"] },
+        limits: { max_bytes: 10 },
+      }),
+    ).toThrow(/byte/);
+  });
+  it("charges references to the same evidence budget", () => {
+    expect(() =>
+      analyzeConsistency([], {
+        plan,
+        references: { x: "y" },
+        limits: { max_bytes: 7 },
+      }),
+    ).toThrow(/byte/);
+  });
+  it("preflights declared group dimensions before consuming input", () => {
+    function* observations(): Generator<never> {
+      throw new Error("should not consume observations");
+    }
+    expect(() =>
+      analyzeConsistency(observations(), {
+        plan: { ...plan, groups: [{ name: "slice", cases: ["x", "y"] }] },
+        limits: { max_group_cells: 3 },
+      }),
+    ).toThrow(/group cell/);
+  });
+});
diff --git a/js/consistency-statistics.test.ts b/js/consistency-statistics.test.ts
new file mode 100644
index 0000000..ee5a1eb
--- /dev/null
+++ b/js/consistency-statistics.test.ts
@@ -0,0 +1,110 @@
+import { describe, expect, it } from "vitest";
+import { analyzeConsistency } from "./consistency";
+import { cohenKappa, nominalAlpha } from "./consistency/statistics";
+import { CaseRandom } from "./consistency/bootstrap";
+import type { PanelCase } from "./consistency/types";
+
+function row(categories: (string | null)[]): PanelCase {
+  return {
+    case_id: "case",
+    categories,
+    statuses: categories.map((value) => (value === null ? "absent" : "scored")),
+    reference: null,
+  };
+}
+
+describe("statistical interpretation", () => {
+  it("reproduces the published nominal alpha missing-data example", () => {
+    const raters = [
+      [1, 2, 3, 3, 2, 1, 4, 1, 2, null, null, null],
+      [1, 2, 3, 3, 2, 2, 4, 1, 2, 5, null, 3],
+      [null, 3, 3, 3, 2, 3, 4, 2, 2, 5, 1, null],
+      [1, 2, 3, 3, 2, 4, 4, 1, 2, 5, 1, null],
+    ];
+    const cases = raters[0].map((_, i) =>
+      row(raters.map((rater) => (rater[i] === null ? null : String(rater[i])))),
+    );
+    const result = nominalAlpha(cases);
+    expect(result.value).toBeCloseTo(0.7434210526315789, 12);
+    expect(result.support_cases).toBe(11);
+    expect(result.support_ratings).toBe(40);
+    expect(nominalAlpha(cases.slice(0, -1)).value).toBe(result.value);
+  });
+  it("distinguishes defined constant-varying kappa from zero expected disagreement", () => {
+    expect(
+      cohenKappa([row(["yes", "yes"]), row(["yes", "no"])], 0, 1).statistic
+        .value,
+    ).toBe(0);
+    expect(cohenKappa([row(["yes", "yes"])], 0, 1).statistic.reason).toBe(
+      "zero_expected_disagreement",
+    );
+  });
+  it("preserves signed systematic disagreement", () => {
+    const cases = [row(["yes", "no"]), row(["no", "yes"])];
+    expect(cohenKappa(cases, 0, 1).statistic.value).toBe(-1);
+    expect(nominalAlpha(cases).value).toBe(-0.5);
+  });
+  it("does not call a minority of declared slots a consensus", () => {
+    const plan = {
+      score_name: "judge",
+      raters: ["a", "b", "c", "d", "e"],
+      categories: ["no", "yes"],
+      cases: ["x", "y"],
+    };
+    const ratings = ["a", "b"]
+      .map((rater_id) => ({ case_id: "x", rater_id, category: "yes" }))
+      .concat(
+        ["a", "b", "c"].map((rater_id) => ({
+          case_id: "y",
+          rater_id,
+          category: "yes",
+        })),
+      );
+    const group = analyzeConsistency(ratings, {
+      plan,
+      references: { x: "yes", y: "yes" },
+    }).groups[0];
+    expect(group.cases.map((item) => item.consensus)).toEqual([null, "yes"]);
+    expect(group.consensus.reference.coverage).toBe(0.5);
+    expect(group.consensus.reference.accuracy).toBe(1);
+    expect(group.statistics.fleiss_kappa.value).toBeNull();
+  });
+  it("uses the specified cross-language PRNG vector", () => {
+    const rng = new CaseRandom(17, "overall");
+    expect(Array.from({ length: 5 }, () => rng.nextU32())).toEqual([
+      4291421669, 2916200383, 2632349228, 374880413, 943298360,
+    ]);
+  });
+  it("preserves undefined bootstrap draws instead of conditioning on valid draws", () => {
+    const plan = {
+      score_name: "judge",
+      raters: ["a", "b"],
+      categories: ["no", "yes"],
+      bootstrap: { resamples: 1000, confidence: 0.9, minimum_cases: 2 },
+    };
+    const ratings = ["x", "y"].flatMap((case_id) =>
+      plan.raters.map((rater_id) => ({
+        case_id,
+        rater_id,
+        category: case_id === "x" ? "yes" : "no",
+      })),
+    );
+    const intervals = analyzeConsistency(ratings, { plan }).groups[0].intervals;
+    const alpha = intervals.find((item) => item.statistic === "nominal_alpha")!;
+    expect(alpha.invalid).toBeGreaterThan(0);
+    expect(alpha.lower).toBeNull();
+    expect(alpha.reason).toBe("undefined_resamples");
+  });
+  it("bounds work before any bootstrap draws", () => {
+    const plan = {
+      score_name: "judge",
+      raters: ["a", "b", "c"],
+      categories: ["no", "yes"],
+      cases: ["x", "y"],
+      groups: [{ name: "slice", cases: ["x", "y"] }],
+    };
+    expect(() =>
+      analyzeConsistency([], { plan, limits: { max_pairwise_work: 10 } }),
+    ).toThrow(/pairwise/);
+  });
+});
diff --git a/js/consistency.test.ts b/js/consistency.test.ts
new file mode 100644
index 0000000..eeeb0b0
--- /dev/null
+++ b/js/consistency.test.ts
@@ -0,0 +1,53 @@
+import { describe, expect, it } from "vitest";
+import { analyzeConsistency, ratingFromScore } from "./consistency";
+
+const plan = {
+  score_name: "judge",
+  raters: ["a", "b", "c"],
+  categories: ["no", "yes"],
+  cases: ["one", "two"],
+};
+
+describe("categorical judge consistency", () => {
+  it("keeps wholly absent cases and declared rater slots", () => {
+    const report = analyzeConsistency(
+      [{ case_id: "one", rater_id: "a", category: "yes" }],
+      { plan },
+    );
+    expect(report.groups[0].coverage.expected_cells).toBe(6);
+    expect(report.groups[0].coverage.absent_cells).toBe(5);
+    expect(report.groups[0].cases[0].consensus).toBeNull();
+  });
+  it("adapts native scores using explicit exact labels without copying metadata", () => {
+    const rating = ratingFromScore(
+      "one",
+      "a",
+      { name: "judge", score: 1, metadata: { private: "secret" } },
+      {
+        score_name: "judge",
+        entries: [
+          [0, "no"],
+          [1, "yes"],
+        ],
+      },
+    );
+    expect(rating).toEqual({
+      case_id: "one",
+      rater_id: "a",
+      category: "yes",
+      status: "scored",
+    });
+  });
+  it("rejects duplicated cells and boolean scores", () => {
+    const rating = { case_id: "one", rater_id: "a", category: "yes" };
+    expect(() => analyzeConsistency([rating, rating], { plan })).toThrow(
+      /duplicate/,
+    );
+    expect(() =>
+      ratingFromScore("one", "a", { name: "judge", score: true } as never, {
+        score_name: "judge",
+        entries: [[1, "yes"]],
+      }),
+    ).toThrow();
+  });
+});
diff --git a/js/consistency/adapters.ts b/js/consistency/adapters.ts
new file mode 100644
index 0000000..14f2bcb
--- /dev/null
+++ b/js/consistency/adapters.ts
@@ -0,0 +1,71 @@
+import type { Score } from "../score";
+import { array, finite, identifier, normalizeRating, object } from "./config";
+import type { Rating, ScoreMapping } from "./types";
+
+/** Explicitly skipped evaluation, distinct from an absent cell. */
+export function missingRating(
+  caseId: string,
+  raterId: string,
+): Required<Rating> {
+  return normalizeRating({
+    case_id: caseId,
+    rater_id: raterId,
+    status: "missing",
+  });
+}
+
+/** Record a caught scorer error without retaining exception text or metadata. */
+export function errorRating(caseId: string, raterId: string): Required<Rating> {
+  return normalizeRating({
+    case_id: caseId,
+    rater_id: raterId,
+    status: "error",
+  });
+}
+
+/**
+ * Convert a native Score using exact caller-declared labels. Scores that raise
+ * must be handled explicitly by the caller; no evaluation or retry is started.
+ * The deprecated error field takes precedence over a simultaneous numeric value.
+ */
+export function ratingFromScore(
+  caseId: string,
+  raterId: string,
+  score: Score,
+  mapping: ScoreMapping,
+): Required<Rating> {
+  const config = object(
+    mapping,
+    ["score_name", "entries"],
+    ["score_name", "entries"],
+    "mapping",
+  );
+  const name = identifier(config.score_name, "score_name");
+  const entries = array(config.entries, "mapping entries", 1000);
+  if (!entries.length) throw new Error("mapping entries must be nonempty");
+  const labels = new Map<number, string>();
+  for (const entry of entries) {
+    const pair = array(entry, "mapping entry", 2);
+    if (pair.length !== 2)
+      throw new Error("mapping entries require score/category pairs");
+    const numeric = finite(pair[0], "mapped score", 0, 1);
+    if (labels.has(numeric)) throw new Error("duplicate mapped numeric score");
+    labels.set(numeric, identifier(pair[1], "category"));
+  }
+  if (
+    score === null ||
+    typeof score !== "object" ||
+    !Object.prototype.hasOwnProperty.call(score, "score")
+  )
+    throw new Error("expected a native Score with name and score fields");
+  if (score.name !== name)
+    throw new Error("score name does not match the selected criterion");
+  if (score.error !== undefined && score.error !== null)
+    return errorRating(caseId, raterId);
+  if (score.score === null) return missingRating(caseId, raterId);
+  const numeric = finite(score.score, "score", 0, 1);
+  const category = labels.get(numeric);
+  if (category === undefined)
+    throw new Error("numeric score is not mapped to a category");
+  return normalizeRating({ case_id: caseId, rater_id: raterId, category });
+}
diff --git a/js/consistency/alignment.ts b/js/consistency/alignment.ts
new file mode 100644
index 0000000..aa9cd57
--- /dev/null
+++ b/js/consistency/alignment.ts
@@ -0,0 +1,141 @@
+import {
+  compareNames,
+  identifier,
+  normalizeRating,
+  selectedPlanBytes,
+} from "./config";
+import type {
+  ConsistencyLimits,
+  NormalizedPlan,
+  PanelCase,
+  Rating,
+  References,
+} from "./types";
+
+export interface Panel {
+  cases: readonly PanelCase[];
+  groups: readonly (readonly [string, readonly number[]])[];
+  provenance: "expected_manifest" | "observed_union";
+}
+
+/** Freeze observations and group memberships before eligibility filtering. */
+export function align(
+  ratings: Iterable<Rating>,
+  plan: NormalizedPlan,
+  references: References | null | undefined,
+  limits: Required<ConsistencyLimits>,
+): Panel {
+  const knownRaters = new Set(plan.raters),
+    categories = new Set(plan.categories);
+  const declared = plan.cases === null ? null : new Set(plan.cases);
+  const observed = new Map<string, Map<string, Required<Rating>>>();
+  const encoder = new TextEncoder();
+  let count = 0,
+    selectedBytes = selectedPlanBytes(plan);
+  for (const input of ratings) {
+    if (++count > limits.max_ratings)
+      throw new Error("consistency rating limit exceeded");
+    const rating = normalizeRating(input);
+    if (!knownRaters.has(rating.rater_id)) throw new Error("unknown rater_id");
+    if (rating.category !== null && !categories.has(rating.category))
+      throw new Error("unknown rating category");
+    if (declared !== null && !declared.has(rating.case_id))
+      throw new Error("case_id is outside the expected manifest");
+    let cells = observed.get(rating.case_id);
+    if (!cells) {
+      cells = new Map();
+      observed.set(rating.case_id, cells);
+    }
+    if (cells.has(rating.rater_id))
+      throw new Error("duplicate case/rater cell");
+    if (observed.size > limits.max_cases)
+      throw new Error("consistency case limit exceeded");
+    for (const value of [
+      rating.case_id,
+      rating.rater_id,
+      rating.category ?? "",
+      rating.status,
+    ])
+      selectedBytes += encoder.encode(value).length;
+    if (selectedBytes > limits.max_bytes)
+      throw new Error("consistency selected byte limit exceeded");
+    cells.set(rating.rater_id, rating);
+  }
+  const caseIds = plan.cases ?? [...observed.keys()].sort(compareNames);
+  if (caseIds.length * plan.raters.length > limits.max_cells)
+    throw new Error("consistency cell limit exceeded");
+  const knownCases = new Set(caseIds);
+  const copiedReferences = new Map<string, string>();
+  if (references !== null && references !== undefined) {
+    if (typeof references !== "object" || Array.isArray(references))
+      throw new Error("references must be a case/category mapping");
+    const entries =
+      references instanceof Map
+        ? [...references.entries()]
+        : Object.entries(references);
+    if (entries.length > limits.max_cases)
+      throw new Error("references must be a bounded case/category mapping");
+    for (const [caseId, category] of entries) {
+      identifier(caseId, "reference case_id");
+      identifier(category, "reference category");
+      if (!knownCases.has(caseId))
+        throw new Error("reference case_id is outside the cohort");
+      if (!categories.has(category))
+        throw new Error("unknown reference category");
+      selectedBytes +=
+        encoder.encode(caseId).length + encoder.encode(category).length;
+      if (selectedBytes > limits.max_bytes)
+        throw new Error("consistency selected byte limit exceeded");
+      copiedReferences.set(caseId, category);
+    }
+  }
+  const groupCases =
+    caseIds.length +
+    plan.groups.reduce((total, group) => total + group.cases.length, 0);
+  if (groupCases * plan.raters.length > limits.max_group_cells)
+    throw new Error("consistency group cell limit exceeded");
+  const pairCount = (plan.raters.length * (plan.raters.length - 1)) / 2;
+  if (
+    (groupCases + plan.groups.length + 1) * pairCount >
+    limits.max_pairwise_work
+  )
+    throw new Error("consistency pairwise work limit exceeded");
+  if (
+    groupCases * plan.raters.length * plan.bootstrap.resamples >
+    limits.max_bootstrap_work
+  )
+    throw new Error("consistency bootstrap work limit exceeded");
+  const positions = new Map(caseIds.map((id, i) => [id, i]));
+  const groups: [string, readonly number[]][] = [
+    ["overall", caseIds.map((_, i) => i)],
+  ];
+  for (const group of plan.groups) {
+    const indices = group.cases.map((id) => {
+      const index = positions.get(id);
+      if (index === undefined)
+        throw new Error(
+          "group cases are outside the cohort; declare expected cases",
+        );
+      return index;
+    });
+    groups.push([group.name, Object.freeze(indices)]);
+  }
+  const cases = caseIds.map((caseId) => {
+    const cells = observed.get(caseId);
+    return Object.freeze({
+      case_id: caseId,
+      categories: Object.freeze(
+        plan.raters.map((rater) => cells?.get(rater)?.category ?? null),
+      ),
+      statuses: Object.freeze(
+        plan.raters.map((rater) => cells?.get(rater)?.status ?? "absent"),
+      ),
+      reference: copiedReferences.get(caseId) ?? null,
+    });
+  });
+  return {
+    cases: Object.freeze(cases),
+    groups: Object.freeze(groups),
+    provenance: declared === null ? "observed_union" : "expected_manifest",
+  };
+}
diff --git a/js/consistency/analysis.ts b/js/consistency/analysis.ts
new file mode 100644
index 0000000..3fc69e0
--- /dev/null
+++ b/js/consistency/analysis.ts
@@ -0,0 +1,321 @@
+import { align } from "./alignment";
+import { bootstrapIntervals } from "./bootstrap";
+import { compareNames, normalizeLimits, normalizePlan } from "./config";
+import {
+  cohenKappa,
+  counts,
+  fleissKappa,
+  nominalAlpha,
+  pairAgreement,
+  rawAgreement,
+  sum,
+} from "./statistics";
+import type {
+  CaseReport,
+  ConsistencyGate,
+  ConsistencyLimits,
+  ConsistencyPlan,
+  ConsistencyReport,
+  Coverage,
+  GateResult,
+  GroupReport,
+  NormalizedPlan,
+  PanelCase,
+  Rating,
+  RaterReport,
+  ReferenceAccuracy,
+  References,
+  Statistic,
+  StatisticName,
+} from "./types";
+
+/** A strict majority includes all declared slots in the denominator. */
+export function consensus(
+  item: PanelCase,
+  plan: NormalizedPlan,
+): [string | null, string | null] {
+  const categories = counts(item.categories);
+  if (sum(categories.values()) < plan.minimum_consensus_ratings)
+    return [null, "insufficient_ratings"];
+  const maximum = Math.max(...categories.values());
+  const winners = [...categories]
+    .filter(([, n]) => n === maximum)
+    .map(([category]) => category);
+  if (winners.length !== 1) return [null, "tie"];
+  if (maximum * 2 <= plan.raters.length)
+    return [null, "no_declared_panel_majority"];
+  return [winners[0], null];
+}
+
+export function referenceAccuracy(
+  cases: readonly PanelCase[],
+  values: readonly (string | null)[],
+): ReferenceAccuracy {
+  let labeled = 0,
+    evaluated = 0,
+    correct = 0;
+  for (let i = 0; i < cases.length; i++) {
+    if (cases[i].reference === null) continue;
+    labeled++;
+    if (values[i] !== null) {
+      evaluated++;
+      if (values[i] === cases[i].reference) correct++;
+    }
+  }
+  return {
+    labeled_cases: labeled,
+    evaluated_cases: evaluated,
+    correct_cases: correct,
+    unavailable_cases: labeled - evaluated,
+    coverage: labeled ? evaluated / labeled : null,
+    accuracy: evaluated ? correct / evaluated : null,
+    reason: !labeled
+      ? "no_references"
+      : !evaluated
+        ? "no_evaluated_references"
+        : null,
+  };
+}
+
+function consensusStatistic(
+  cases: readonly PanelCase[],
+  plan: NormalizedPlan,
+): Statistic {
+  const values = cases.map((item) => consensus(item, plan)[0]);
+  const accuracy = referenceAccuracy(cases, values);
+  let ratings = 0;
+  for (let i = 0; i < cases.length; i++)
+    if (cases[i].reference !== null && values[i] !== null)
+      ratings += cases[i].categories.filter((value) => value !== null).length;
+  return {
+    value: accuracy.accuracy,
+    support_cases: accuracy.evaluated_cases,
+    support_ratings: ratings,
+    denominator_cases: accuracy.labeled_cases,
+    observed_agreement: null,
+    expected_agreement: null,
+    weighting: "evaluated_reference_cases",
+    reason: accuracy.reason,
+  };
+}
+
+export function coefficients(
+  cases: readonly PanelCase[],
+  plan: NormalizedPlan,
+): Record<StatisticName, Statistic> {
+  return {
+    raw_agreement: rawAgreement(cases),
+    fleiss_kappa: fleissKappa(cases, plan.raters.length),
+    nominal_alpha: nominalAlpha(cases),
+    consensus_accuracy: consensusStatistic(cases, plan),
+  };
+}
+
+function coverage(cases: readonly PanelCase[], plan: NormalizedPlan): Coverage {
+  const states = counts(cases.flatMap((item) => [...item.statuses]));
+  const available = cases.map(
+    (item) => item.categories.filter((value) => value !== null).length,
+  );
+  const expected = cases.length * plan.raters.length;
+  const scored = states.get("scored") ?? 0,
+    absent = states.get("absent") ?? 0;
+  return {
+    total_cases: cases.length,
+    expected_cells: expected,
+    present_cells: expected - absent,
+    scored_cells: scored,
+    missing_cells: states.get("missing") ?? 0,
+    error_cells: states.get("error") ?? 0,
+    absent_cells: absent,
+    rating_coverage: expected ? scored / expected : null,
+    zero_rating_cases: available.filter((n) => n === 0).length,
+    singleton_cases: available.filter((n) => n === 1).length,
+    pairable_cases: available.filter((n) => n >= 2).length,
+    complete_cases: available.filter((n) => n === plan.raters.length).length,
+  };
+}
+
+function raterReport(
+  cases: readonly PanelCase[],
+  index: number,
+  rater: string,
+): RaterReport {
+  const states = counts(cases.map((item) => item.statuses[index]));
+  const values = cases.map((item) => item.categories[index]);
+  const reference = referenceAccuracy(cases, values);
+  reference.missing_cases = cases.filter(
+    (item) => item.reference !== null && item.statuses[index] === "missing",
+  ).length;
+  reference.error_cases = cases.filter(
+    (item) => item.reference !== null && item.statuses[index] === "error",
+  ).length;
+  reference.absent_cases = cases.filter(
+    (item) => item.reference !== null && item.statuses[index] === "absent",
+  ).length;
+  const scored = states.get("scored") ?? 0;
+  return {
+    rater_id: rater,
+    scored_cases: scored,
+    missing_cases: states.get("missing") ?? 0,
+    error_cases: states.get("error") ?? 0,
+    absent_cases: states.get("absent") ?? 0,
+    coverage: cases.length ? scored / cases.length : null,
+    category_counts: [...counts(values)]
+      .sort(([a], [b]) => compareNames(a, b))
+      .map(([category, count]) => ({ category, count })),
+    reference,
+  };
+}
+
+function caseReport(item: PanelCase, plan: NormalizedPlan): CaseReport {
+  const categories = counts(item.categories),
+    [winner, reason] = consensus(item, plan);
+  return {
+    case_id: item.case_id,
+    scored_ratings: sum(categories.values()),
+    category_counts: plan.categories.map((category) => ({
+      category,
+      count: categories.get(category) ?? 0,
+    })),
+    raw_agreement: pairAgreement(categories),
+    consensus: winner,
+    consensus_reason: reason,
+    reference: item.reference,
+    consensus_correct:
+      winner !== null && item.reference !== null
+        ? winner === item.reference
+        : null,
+    unavailable_raters: plan.raters.flatMap((rater_id, i) =>
+      item.statuses[i] === "scored"
+        ? []
+        : [{ rater_id, status: item.statuses[i] }],
+    ),
+  };
+}
+
+function gateResult(
+  gate: Required<ConsistencyGate>,
+  statistic: Statistic,
+  availability: Coverage,
+): GateResult {
+  const reasons: string[] = [];
+  const supportCoverage = statistic.denominator_cases
+    ? statistic.support_cases / statistic.denominator_cases
+    : null;
+  const ratingCoverage = availability.rating_coverage;
+  if (statistic.value === null)
+    reasons.push(statistic.reason ?? "undefined_statistic");
+  if (statistic.support_cases < gate.minimum_support)
+    reasons.push("insufficient_support");
+  if (supportCoverage === null || supportCoverage < gate.minimum_coverage)
+    reasons.push("insufficient_statistic_coverage");
+  if (ratingCoverage === null || ratingCoverage < gate.minimum_rating_coverage)
+    reasons.push("insufficient_rating_coverage");
+  let verdict: GateResult["verdict"];
+  if (reasons.length) verdict = "inconclusive";
+  else if (statistic.value !== null && statistic.value < gate.minimum) {
+    verdict = "fail";
+    reasons.push("below_observed_threshold");
+  } else {
+    verdict = "pass";
+    reasons.push("meets_observed_threshold");
+  }
+  return {
+    ...gate,
+    value: statistic.value,
+    support_cases: statistic.support_cases,
+    denominator_cases: statistic.denominator_cases,
+    support_coverage: supportCoverage,
+    rating_coverage: ratingCoverage,
+    verdict,
+    reasons,
+  };
+}
+
+/**
+ * Analyze one criterion without invoking models, backends or global randomness.
+ * Missing observations remain in availability denominators. Each coefficient
+ * labels its own supporting cohort. Consistency alone is not correctness.
+ */
+export function analyzeConsistency(
+  ratings: Iterable<Rating>,
+  options: {
+    plan: ConsistencyPlan;
+    references?: References | null;
+    limits?: ConsistencyLimits;
+  },
+): ConsistencyReport {
+  const limits = normalizeLimits(options.limits),
+    plan = normalizePlan(options.plan, limits);
+  const panel = align(ratings, plan, options.references, limits);
+  const groups: GroupReport[] = [],
+    gates: GateResult[] = [];
+  let remainingDetails = plan.detail_limit;
+  for (const [name, indices] of panel.groups) {
+    const cases = indices.map((index) => panel.cases[index]);
+    const stats = coefficients(cases, plan),
+      availability = coverage(cases, plan);
+    const pairwise: GroupReport["pairwise"] = [];
+    for (let left = 0; left < plan.raters.length; left++)
+      for (let right = left + 1; right < plan.raters.length; right++)
+        pairwise.push({
+          left_rater: plan.raters[left],
+          right_rater: plan.raters[right],
+          ...cohenKappa(cases, left, right),
+        });
+    const decisions = cases.map((item) => consensus(item, plan));
+    const reference = referenceAccuracy(
+      cases,
+      decisions.map(([value]) => value),
+    );
+    reference.abstained_cases = reference.unavailable_cases;
+    const reasons = counts(decisions.map(([, reason]) => reason));
+    groups.push({
+      name,
+      coverage: availability,
+      statistics: stats,
+      raters: plan.raters.map((rater, index) =>
+        raterReport(cases, index, rater),
+      ),
+      pairwise,
+      consensus: {
+        rule: "strict_majority_of_declared_panel",
+        decided_cases: decisions.filter(([value]) => value !== null).length,
+        abstained_cases: decisions.filter(([value]) => value === null).length,
+        reason_counts: [...reasons]
+          .sort(([a], [b]) => compareNames(a, b))
+          .map(([reason, count]) => ({ reason, count })),
+        reference,
+      },
+      intervals: bootstrapIntervals(cases, plan, name, stats),
+      cases: cases
+        .slice(0, remainingDetails)
+        .map((item) => caseReport(item, plan)),
+      omitted_cases: Math.max(0, cases.length - remainingDetails),
+    });
+    remainingDetails = Math.max(0, remainingDetails - cases.length);
+    for (const gate of plan.gates)
+      if (gate.group === name)
+        gates.push(gateResult(gate, stats[gate.statistic], availability));
+  }
+  const verdict = !gates.length
+    ? "not_evaluated"
+    : gates.some((gate) => gate.verdict === "fail")
+      ? "fail"
+      : gates.some((gate) => gate.verdict === "inconclusive")
+        ? "inconclusive"
+        : "pass";
+  return {
+    format: "autoevals.consistency.report",
+    version: 1,
+    score_name: plan.score_name,
+    cohort_provenance: panel.provenance,
+    categories: [...plan.categories],
+    raters: [...plan.raters],
+    interpretation: "observed_finite_benchmark",
+    interval_interpretation: "marginal_diagnostic_case_bootstrap",
+    groups,
+    gates,
+    verdict,
+  };
+}
diff --git a/js/consistency/bootstrap.ts b/js/consistency/bootstrap.ts
new file mode 100644
index 0000000..14106e6
--- /dev/null
+++ b/js/consistency/bootstrap.ts
@@ -0,0 +1,116 @@
+import { coefficients } from "./analysis";
+import type {
+  ConsistencyInterval,
+  NormalizedPlan,
+  PanelCase,
+  Statistic,
+  StatisticName,
+} from "./types";
+
+/** Specified FNV1a namespace plus unsigned xorshift32, identical in Python. */
+export class CaseRandom {
+  private state: number;
+  constructor(seed: number, namespace: string) {
+    let state = (2166136261 ^ seed) >>> 0;
+    for (const value of new TextEncoder().encode(namespace))
+      state = Math.imul(state ^ value, 16777619) >>> 0;
+    this.state = state || 0x9e3779b9;
+  }
+  nextU32(): number {
+    let value = this.state;
+    value ^= value << 13;
+    value ^= value >>> 17;
+    value ^= value << 5;
+    this.state = value >>> 0;
+    return this.state;
+  }
+  /** Rejection sampling avoids modulo bias. */
+  index(size: number): number {
+    const ceiling = Math.floor(2 ** 32 / size) * size;
+    for (;;) {
+      const value = this.nextU32();
+      if (value < ceiling) return value % size;
+    }
+  }
+}
+
+function percentile(ordered: readonly number[], probability: number): number {
+  const position = (ordered.length - 1) * probability;
+  const low = Math.floor(position),
+    high = Math.min(low + 1, ordered.length - 1);
+  const weight = position - low;
+  return ordered[low] * (1 - weight) + ordered[high] * weight;
+}
+
+/**
+ * Preserve repeated case occurrences during resampling. An undefined draw
+ * invalidates the interval; never discard it and claim an unconditional CI.
+ */
+export function bootstrapIntervals(
+  cases: readonly PanelCase[],
+  plan: NormalizedPlan,
+  group: string,
+  observed: Record<StatisticName, Statistic>,
+): ConsistencyInterval[] {
+  const options = plan.bootstrap;
+  if (!options.resamples) return [];
+  const results: ConsistencyInterval[] = [],
+    eligible: ConsistencyInterval[] = [];
+  const distributions = new Map<StatisticName, number[]>();
+  for (const name of options.statistics) {
+    const statistic = observed[name];
+    let reason: string | null = null;
+    if (statistic.value === null) reason = "undefined_observed_statistic";
+    else if (statistic.support_cases < options.minimum_cases)
+      reason = "insufficient_support";
+    else if ((options.resamples * (1 - options.confidence)) / 2 < 20)
+      reason = "insufficient_tail_resamples";
+    const result: ConsistencyInterval = {
+      statistic: name,
+      confidence: options.confidence,
+      resamples: options.resamples,
+      attempted: 0,
+      valid: 0,
+      invalid: 0,
+      lower: null,
+      upper: null,
+      reason,
+      seed: options.seed,
+      algorithm: "case-xorshift32-percentile-v1",
+    };
+    results.push(result);
+    if (reason === null) {
+      eligible.push(result);
+      distributions.set(name, []);
+    }
+  }
+  if (!eligible.length) return results;
+  const rng = new CaseRandom(options.seed, group);
+  for (let draw = 0; draw < options.resamples; draw++) {
+    const sample = cases.map(() => cases[rng.index(cases.length)]);
+    const values = coefficients(sample, plan);
+    for (const result of eligible) {
+      result.attempted++;
+      const value = values[result.statistic].value;
+      if (value === null) result.invalid++;
+      else {
+        result.valid++;
+        distributions.get(result.statistic)!.push(value);
+      }
+    }
+  }
+  for (const result of eligible) {
+    const distribution = distributions
+      .get(result.statistic)!
+      .sort((a, b) => a - b);
+    if (result.invalid) result.reason = "undefined_resamples";
+    else if (distribution[0] === distribution[distribution.length - 1])
+      result.reason = "degenerate_resamples";
+    else {
+      const alpha = (1 - options.confidence) / 2;
+      result.lower = percentile(distribution, alpha);
+      result.upper = percentile(distribution, 1 - alpha);
+    }
+  }
+  return results;
+}
diff --git a/js/consistency/config.ts b/js/consistency/config.ts
new file mode 100644
index 0000000..2790645
--- /dev/null
+++ b/js/consistency/config.ts
@@ -0,0 +1,380 @@
+import type {
+  ConsistencyBootstrap,
+  ConsistencyGate,
+  ConsistencyLimits,
+  ConsistencyPlan,
+  NormalizedPlan,
+  Rating,
+  StatisticName,
+} from "./types";
+
+export const statistics: readonly StatisticName[] = [
+  "raw_agreement",
+  "fleiss_kappa",
+  "nominal_alpha",
+  "consensus_accuracy",
+];
+const encoder = new TextEncoder();
+
+/** Python code-point and JavaScript UTF-16 ordering differ; use UTF-8 explicitly. */
+export function compareNames(left: string, right: string): number {
+  const a = encoder.encode(left),
+    b = encoder.encode(right);
+  for (let i = 0; i < Math.min(a.length, b.length); i++)
+    if (a[i] !== b[i]) return a[i] - b[i];
+  return a.length - b.length;
+}
+
+export function identifier(value: unknown, name: string): string {
+  if (typeof value !== "string" || !value)
+    throw new Error(`${name} must be a nonempty string`);
+  for (let i = 0; i < value.length; i++) {
+    const code = value.charCodeAt(i);
+    if (code >= 0xd800 && code <= 0xdbff) {
+      const next = value.charCodeAt(++i);
+      if (!(next >= 0xdc00 && next <= 0xdfff))
+        throw new Error(`${name} contains an invalid Unicode surrogate`);
+    } else if (code >= 0xdc00 && code <= 0xdfff)
+      throw new Error(`${name} contains an invalid Unicode surrogate`);
+  }
+  if (encoder.encode(value).length > 4096)
+    throw new Error(`${name} exceeds the string limit`);
+  return value;
+}
+
+export function integer(
+  value: unknown,
+  name: string,
+  low: number,
+  high: number,
+): number {
+  if (
+    !Number.isSafeInteger(value) ||
+    (value as number) < low ||
+    (value as number) > high
+  )
+    throw new Error(`${name} must be an integer between ${low} and ${high}`);
+  return value as number;
+}
+
+export function finite(
+  value: unknown,
+  name: string,
+  low: number,
+  high: number,
+): number {
+  if (
+    typeof value !== "number" ||
+    !Number.isFinite(value) ||
+    value < low ||
+    value > high
+  )
+    throw new Error(
+      `${name} must be a finite number between ${low} and ${high}`,
+    );
+  return value;
+}
+
+export function array(
+  value: unknown,
+  name: string,
+  maximum: number,
+): unknown[] {
+  if (!Array.isArray(value) || value.length > maximum)
+    throw new Error(`${name} must be a bounded array`);
+  return value;
+}
+
+export function names(
+  value: unknown,
+  name: string,
+  minimum = 0,
+  maximum = 100000,
+): readonly string[] {
+  const result = array(value, name, maximum).map((item) =>
+    identifier(item, name),
+  );
+  if (result.length < minimum) throw new Error(`${name} has too few entries`);
+  if (new Set(result).size !== result.length)
+    throw new Error(`duplicate ${name}`);
+  return Object.freeze(result.sort(compareNames));
+}
+
+export function object(
+  value: unknown,
+  allowed: readonly string[],
+  required: readonly string[],
+  name: string,
+): Record<string, unknown> {
+  if (value === null || typeof value !== "object" || Array.isArray(value))
+    throw new Error(`${name} must be an object`);
+  const record = value as Record<string, unknown>;
+  if (Object.keys(record).some((key) => !allowed.includes(key)))
+    throw new Error(`${name} contains unknown fields`);
+  if (
+    required.some((key) => !Object.prototype.hasOwnProperty.call(record, key))
+  )
+    throw new Error(`${name} is missing required fields`);
+  return record;
+}
+
+function option(
+  record: Record<string, unknown>,
+  key: string,
+  fallback: unknown,
+): unknown {
+  return record[key] === undefined ? fallback : record[key];
+}
+
+const defaultLimits: Required<ConsistencyLimits> = {
+  max_ratings: 1000000,
+  max_cases: 100000,
+  max_raters: 64,
+  max_categories: 100,
+  max_groups: 50,
+  max_cells: 1000000,
+  max_pairwise_work: 10000000,
+  max_group_cells: 2000000,
+  max_bootstrap_work: 100000000,
+  max_details: 10000,
+  max_bytes: 64 * 1024 * 1024,
+  max_json_nodes: 2000000,
+  max_json_depth: 20,
+};
+
+export function normalizeLimits(
+  input: ConsistencyLimits = {},
+): Required<ConsistencyLimits> {
+  const data = object(input, Object.keys(defaultLimits), [], "limits");
+  const result = { ...defaultLimits };
+  for (const key of Object.keys(result) as (keyof ConsistencyLimits)[])
+    result[key] = integer(
+      option(data, key, result[key]),
+      key,
+      1,
+      Number.MAX_SAFE_INTEGER,
+    );
+  return Object.freeze(result);
+}
+
+function normalizeBootstrap(value: unknown): Required<ConsistencyBootstrap> {
+  const data = object(
+    value,
+    ["resamples", "confidence", "seed", "minimum_cases", "statistics"],
+    [],
+    "bootstrap",
+  );
+  const confidence = finite(
+    option(data, "confidence", 0.95),
+    "confidence",
+    0,
+    1,
+  );
+  if (confidence === 0 || confidence === 1)
+    throw new Error("confidence must be strictly between zero and one");
+  const selected = names(
+    option(data, "statistics", statistics.slice(0, 3)),
+    "bootstrap statistics",
+    1,
+    4,
+  );
+  if (selected.some((name) => !statistics.includes(name as StatisticName)))
+    throw new Error("unknown bootstrap statistic");
+  return Object.freeze({
+    resamples: integer(option(data, "resamples", 0), "resamples", 0, 100000),
+    confidence,
+    seed: integer(option(data, "seed", 0), "seed", 0, 0xffffffff),
+    minimum_cases: integer(
+      option(data, "minimum_cases", 20),
+      "minimum_cases",
+      2,
+      100000,
+    ),
+    statistics: selected as readonly StatisticName[],
+  });
+}
+
+function normalizeGate(value: unknown): Required<ConsistencyGate> {
+  const data = object(
+    value,
+    [
+      "statistic",
+      "minimum",
+      "group",
+      "minimum_support",
+      "minimum_coverage",
+      "minimum_rating_coverage",
+    ],
+    ["statistic", "minimum"],
+    "gate",
+  );
+  if (!statistics.includes(data.statistic as StatisticName))
+    throw new Error("unknown gate statistic");
+  const minimum = finite(data.minimum, "minimum", -1, 1);
+  if (
+    ["raw_agreement", "consensus_accuracy"].includes(
+      data.statistic as string,
+    ) &&
+    minimum < 0
+  )
+    throw new Error("agreement and accuracy thresholds must be nonnegative");
+  return Object.freeze({
+    statistic: data.statistic as StatisticName,
+    minimum,
+    group: identifier(option(data, "group", "overall"), "gate group"),
+    minimum_support: integer(
+      option(data, "minimum_support", 2),
+      "minimum_support",
+      1,
+      100000,
+    ),
+    minimum_coverage: finite(
+      option(data, "minimum_coverage", 1),
+      "minimum_coverage",
+      0,
+      1,
+    ),
+    minimum_rating_coverage: finite(
+      option(data, "minimum_rating_coverage", 1),
+      "minimum_rating_coverage",
+      0,
+      1,
+    ),
+  });
+}
+
+/** Detach caller-owned arrays and normalize every runtime field, not just TS types. */
+export function normalizePlan(
+  input: ConsistencyPlan,
+  limits: Required<ConsistencyLimits>,
+): NormalizedPlan {
+  const data = object(
+    input,
+    [
+      "score_name",
+      "raters",
+      "categories",
+      "cases",
+      "groups",
+      "gates",
+      "minimum_consensus_ratings",
+      "detail_limit",
+      "bootstrap",
+    ],
+    ["score_name", "raters", "categories"],
+    "plan",
+  );
+  const raters = names(data.raters, "raters", 2, 1000);
+  const categories = names(data.categories, "categories", 2, 1000);
+  const cases =
+    data.cases === undefined || data.cases === null
+      ? null
+      : names(data.cases, "cases");
+  const groups = array(option(data, "groups", []), "groups", 1000)
+    .map((value) => {
+      const group = object(
+        value,
+        ["name", "cases"],
+        ["name", "cases"],
+        "group",
+      );
+      const name = identifier(group.name, "group name");
+      if (name === "overall") throw new Error("overall is reserved");
+      return Object.freeze({ name, cases: names(group.cases, "group cases") });
+    })
+    .sort((a, b) => compareNames(a.name, b.name));
+  if (new Set(groups.map((group) => group.name)).size !== groups.length)
+    throw new Error("duplicate group name");
+  const gates = array(option(data, "gates", []), "gates", 1000)
+    .map(normalizeGate)
+    .sort(
+      (a, b) =>
+        compareNames(a.group, b.group) ||
+        compareNames(a.statistic, b.statistic),
+    );
+  const knownGroups = new Set([
+    "overall",
+    ...groups.map((group) => group.name),
+  ]);
+  const gateKeys = new Set<string>();
+  for (const gate of gates) {
+    if (!knownGroups.has(gate.group))
+      throw new Error("gate references an unknown group");
+    const key = JSON.stringify([gate.group, gate.statistic]);
+    if (gateKeys.has(key)) throw new Error("duplicate gate statistic/group");
+    gateKeys.add(key);
+  }
+  const result = Object.freeze({
+    score_name: identifier(data.score_name, "score_name"),
+    raters,
+    categories,
+    cases,
+    groups: Object.freeze(groups),
+    gates: Object.freeze(gates),
+    minimum_consensus_ratings: integer(
+      option(data, "minimum_consensus_ratings", 2),
+      "minimum_consensus_ratings",
+      2,
+      raters.length,
+    ),
+    detail_limit: integer(
+      option(data, "detail_limit", 100),
+      "detail_limit",
+      0,
+      100000,
+    ),
+    bootstrap: normalizeBootstrap(option(data, "bootstrap", {})),
+  });
+  for (const [actual, limit, label] of [
+    [raters.length, limits.max_raters, "raters"],
+    [categories.length, limits.max_categories, "categories"],
+    [cases?.length ?? 0, limits.max_cases, "cases"],
+    [groups.length, limits.max_groups, "groups"],
+    [result.detail_limit, limits.max_details, "details"],
+  ] as const)
+    if (actual > limit) throw new Error(`consistency ${label} limit exceeded`);
+  if ((cases?.length ?? 0) * raters.length > limits.max_cells)
+    throw new Error("consistency cell limit exceeded");
+  const declaredGroupCases =
+    (cases?.length ?? 0) +
+    groups.reduce((total, group) => total + group.cases.length, 0);
+  if (declaredGroupCases * raters.length > limits.max_group_cells)
+    throw new Error("consistency group cell limit exceeded");
+  if (selectedPlanBytes(result) > limits.max_bytes)
+    throw new Error("consistency selected byte limit exceeded");
+  return result;
+}
+
+/** Count declared identities even if every planned observation is absent. */
+export function selectedPlanBytes(plan: NormalizedPlan): number {
+  let total = encoder.encode(plan.score_name).length;
+  for (const values of [plan.raters, plan.categories, plan.cases ?? []])
+    for (const value of values) total += encoder.encode(value).length;
+  for (const group of plan.groups) {
+    total += encoder.encode(group.name).length;
+    for (const value of group.cases) total += encoder.encode(value).length;
+  }
+  return total;
+}
+
+export function normalizeRating(value: Rating): Required<Rating> {
+  const data = object(
+    value,
+    ["case_id", "rater_id", "category", "status"],
+    ["case_id", "rater_id"],
+    "rating",
+  );
+  const status = option(data, "status", "scored");
+  if (!["scored", "missing", "error"].includes(status as string))
+    throw new Error("invalid rating status");
+  const category = option(data, "category", null);
+  if (status === "scored") identifier(category, "category");
+  else if (category !== null)
+    throw new Error("unavailable ratings cannot contain a category");
+  return Object.freeze({
+    case_id: identifier(data.case_id, "case_id"),
+    rater_id: identifier(data.rater_id, "rater_id"),
+    category: category as string | null,
+    status: status as Required<Rating>["status"],
+  });
+}
diff --git a/js/consistency/index.ts b/js/consistency/index.ts
new file mode 100644
index 0000000..adeda56
--- /dev/null
+++ b/js/consistency/index.ts
@@ -0,0 +1,26 @@
+export { analyzeConsistency } from "./analysis";
+export { ratingFromScore, missingRating, errorRating } from "./adapters";
+export {
+  analyzeConsistencyJSON,
+  inputToJSON,
+  reportToJSON,
+} from "./serialization";
+export { consistencyScores, renderConsistencyMarkdown } from "./reporting";
+export type {
+  Rating,
+  ScoreMapping,
+  ConsistencyPlan,
+  ConsistencyLimits,
+  ConsistencyGroup,
+  ConsistencyGate,
+  ConsistencyBootstrap,
+  ConsistencyReport,
+  Statistic,
+  StatisticName,
+  ConsistencyInterval,
+  Coverage,
+  GroupReport,
+  GateResult,
+  ReferenceAccuracy,
+  References,
+} from "./types";
diff --git a/js/consistency/reporting.ts b/js/consistency/reporting.ts
new file mode 100644
index 0000000..1ae351f
--- /dev/null
+++ b/js/consistency/reporting.ts
@@ -0,0 +1,142 @@
+import type { Score } from "../score";
+import type { ConsistencyReport, ReferenceAccuracy } from "./types";
+
+function safeText(value: string): string {
+  return value
+    .replace(
+      /[\u0000-\u001f\u007f-\u009f\u202a-\u202e\u2066-\u2069]/g,
+      (char) => `\\u${char.charCodeAt(0).toString(16).padStart(4, "0")}`,
+    )
+    .replace(/[\\`*_{}\[\]()#+\-.!|]/g, (char) => `\\${char}`)
+    .replace(/&/g, "&amp;")
+    .replace(/</g, "&lt;")
+    .replace(/>/g, "&gt;")
+    .replace(/"/g, "&quot;")
+    .replace(/'/g, "&#x27;");
+}
+
+function number(value: number | null): string {
+  return value === null ? "undefined" : Number(value.toPrecision(6)).toString();
+}
+
+/** Render uncertainty, unavailable evidence and conditional accuracy explicitly. */
+export function renderConsistencyMarkdown(report: ConsistencyReport): string {
+  const lines = [
+    `# Judge consistency: ${safeText(report.score_name)}`,
+    "",
+    `Observed benchmark verdict: **${report.verdict}**. Cohort: ${report.cohort_provenance}.`,
+    "",
+    "Consistency is not correctness. Gates describe the supplied finite benchmark; bootstrap intervals are marginal diagnostics and do not determine gate decisions.",
+    "",
+  ];
+  for (const group of report.groups) {
+    const coverage = group.coverage;
+    lines.push(
+      `## ${safeText(group.name)}`,
+      "",
+      `Scored ratings: ${coverage.scored_cells}/${coverage.expected_cells}; absent: ${coverage.absent_cells}; skipped: ${coverage.missing_cells}; errors: ${coverage.error_cells}.`,
+      "",
+      "| Statistic | Value | Supporting cases | Denominator | Weighting / undefined reason |",
+      "| --- | ---: | ---: | ---: | --- |",
+    );
+    for (const [name, statistic] of Object.entries(group.statistics))
+      lines.push(
+        `| ${safeText(name)} | ${number(statistic.value)} | ${statistic.support_cases} | ${statistic.denominator_cases} | ${statistic.weighting} / ${statistic.reason ?? "defined"} |`,
+      );
+    lines.push(
+      "",
+      "### Reference accuracy",
+      "",
+      "| Rater | Correct / evaluated / labeled | Conditional accuracy | Reference coverage |",
+      "| --- | ---: | ---: | ---: | --- |",
+    );
+    const references: [string, ReferenceAccuracy][] = group.raters.map(
+      (rater) => [rater.rater_id, rater.reference],
+    );
+    references.push(["consensus", group.consensus.reference]);
+    for (const [name, reference] of references)
+      lines.push(
+        `| ${safeText(name)} | ${reference.correct_cases} / ${reference.evaluated_cases} / ${reference.labeled_cases} | ${number(reference.accuracy)} | ${number(reference.coverage)} |`,
+      );
+    lines.push(
+      "",
+      "### Pairwise overlap",
+      "",
+      "| Left rater | Right rater | Cohen kappa | Overlap cases | Reason |",
+      "| --- | --- | ---: | ---: | --- |",
+    );
+    for (const pair of group.pairwise)
+      lines.push(
+        `| ${safeText(pair.left_rater)} | ${safeText(pair.right_rater)} | ${number(pair.statistic.value)} | ${pair.statistic.support_cases} | ${pair.statistic.reason ?? "defined"} |`,
+      );
+    if (group.intervals.length) {
+      lines.push("", "### Marginal bootstrap diagnostics", "");
+      for (const interval of group.intervals)
+        lines.push(
+          `- ${interval.statistic}: [${number(interval.lower)}, ${number(interval.upper)}]; valid/attempted=${interval.valid}/${interval.attempted}; reason=${interval.reason ?? "defined"}.`,
+        );
+    }
+    lines.push(
+      "",
+      `Consensus decided/abstained: ${group.consensus.decided_cases}/${group.consensus.abstained_cases}. Omitted case details: ${group.omitted_cases}.`,
+      "",
+    );
+    for (const row of group.cases)
+      lines.push(
+        `- ${safeText(row.case_id)}: consensus=${row.consensus === null ? "abstained" : safeText(row.consensus)}; agreement=${number(row.raw_agreement)}; reason=${row.consensus_reason ?? "decided"}.`,
+      );
+    lines.push("");
+  }
+  if (report.gates.length) {
+    lines.push("## Observed requirements", "");
+    for (const gate of report.gates)
+      lines.push(
+        `- ${safeText(gate.group)}/${gate.statistic}: ${gate.verdict}; value=${number(gate.value)}, minimum=${number(gate.minimum)}; ${gate.reasons.join(", ")}.`,
+      );
+  }
+  return lines.join("\n") + "\n";
+}
+
+/**
+ * Native Score projections are bounded agreement, coverage and accuracy only.
+ * Signed chance-corrected coefficients stay in the full report. Null means
+ * undefined, never a fabricated zero. Full report metadata is explicit opt-in.
+ */
+export function consistencyScores(
+  report: ConsistencyReport,
+  options: { includeReport?: boolean } = {},
+): Score[] {
+  const includeReport = options.includeReport ?? false;
+  if (typeof includeReport !== "boolean")
+    throw new Error("includeReport must be boolean");
+  const results: Score[] = [];
+  for (const group of report.groups) {
+    const summaries: [string, number | null, object][] = [
+      [
+        "agreement",
+        group.statistics.raw_agreement.value,
+        group.statistics.raw_agreement,
+      ],
+      ["coverage", group.coverage.rating_coverage, group.coverage],
+      [
+        "consensus_accuracy",
+        group.consensus.reference.accuracy,
+        group.consensus.reference,
+      ],
+    ];
+    for (const [suffix, score, support] of summaries) {
+      const metadata: Record<string, unknown> = {
+        cohort_provenance: report.cohort_provenance,
+        support: { ...support },
+        interpretation: "observed_finite_benchmark",
+      };
+      if (includeReport) metadata.report = JSON.parse(JSON.stringify(report));
+      results.push({
+        name: `${report.score_name}/${group.name}/${suffix}`,
+        score,
+        metadata,
+      });
+    }
+  }
+  return results;
+}
diff --git a/js/consistency/serialization.ts b/js/consistency/serialization.ts
new file mode 100644
index 0000000..1ddc27d
--- /dev/null
+++ b/js/consistency/serialization.ts
@@ -0,0 +1,179 @@
+import { align } from "./alignment";
+import { analyzeConsistency } from "./analysis";
+import {
+  array,
+  normalizeLimits,
+  normalizePlan,
+  normalizeRating,
+  object,
+} from "./config";
+import type {
+  ConsistencyLimits,
+  ConsistencyPlan,
+  ConsistencyReport,
+  Rating,
+  References,
+} from "./types";
+
+const encoder = new TextEncoder();
+
+function validUnicode(value: string): void {
+  for (let i = 0; i < value.length; i++) {
+    const code = value.charCodeAt(i);
+    if (code >= 0xd800 && code <= 0xdbff) {
+      const next = value.charCodeAt(++i);
+      if (!(next >= 0xdc00 && next <= 0xdfff))
+        throw new Error("invalid Unicode in JSON");
+    } else if (code >= 0xdc00 && code <= 0xdfff)
+      throw new Error("invalid Unicode in JSON");
+  }
+}
+
+/** Bound parsed trees, including object keys. Never silently serialize NaN as null. */
+function structure(value: unknown, limits: Required<ConsistencyLimits>): void {
+  const stack: [unknown, number][] = [[value, 0]];
+  let count = 0;
+  while (stack.length) {
+    const [item, depth] = stack.pop()!;
+    if (++count > limits.max_json_nodes || depth > limits.max_json_depth)
+      throw new Error("consistency JSON structure limit exceeded");
+    if (typeof item === "string") {
+      validUnicode(item);
+      if (encoder.encode(item).length > 16384)
+        throw new Error("consistency JSON string limit exceeded");
+    } else if (typeof item === "number") {
+      if (!Number.isFinite(item))
+        throw new Error("nonfinite JSON numbers are not supported");
+    } else if (Array.isArray(item)) {
+      for (const child of item) stack.push([child, depth + 1]);
+    } else if (item !== null && typeof item === "object") {
+      for (const [key, child] of Object.entries(item))
+        stack.push([key, depth + 1], [child, depth + 1]);
+    } else if (item !== null && typeof item !== "boolean") {
+      throw new Error("consistency data must be finite JSON");
+    }
+  }
+}
+
+/**
+ * JSON.parse validates grammar first. This bounded lexical pass only retains
+ * object key sets, detecting escaped duplicate names before evidence is used.
+ * Values and numeric syntax are left to the standard parser.
+ */
+function uniqueKeys(
+  content: string,
+  limits: Required<ConsistencyLimits>,
+): void {
+  const stack: (Set<string> | null)[] = [];
+  for (let i = 0; i < content.length; i++) {
+    const char = content[i];
+    if (char === "{" || char === "[") {
+      stack.push(char === "{" ? new Set() : null);
+      if (stack.length > limits.max_json_depth + 1)
+        throw new Error("consistency JSON structure limit exceeded");
+    } else if (char === "}" || char === "]") {
+      stack.pop();
+    } else if (char === '"') {
+      const start = i;
+      while (++i < content.length) {
+        if (content[i] === "\\") i++;
+        else if (content[i] === '"') break;
+      }
+      let next = i + 1;
+      while (next < content.length && /\s/.test(content[next])) next++;
+      if (content[next] === ":") {
+        const key: string = JSON.parse(content.slice(start, i + 1));
+        const keys = stack[stack.length - 1];
+        if (!keys) throw new Error("invalid JSON object key");
+        if (keys.has(key)) throw new Error("duplicate JSON key");
+        keys.add(key);
+      }
+    }
+  }
+}
+
+function encode(value: unknown, limits: Required<ConsistencyLimits>): string {
+  structure(value, limits);
+  const content = JSON.stringify(value);
+  if (encoder.encode(content).length > limits.max_bytes)
+    throw new Error("consistency output byte limit exceeded");
+  return content;
+}
+
+/** Recompute a versioned portable evidence bundle entirely offline. */
+export function analyzeConsistencyJSON(
+  content: string,
+  limitsInput: ConsistencyLimits = {},
+): ConsistencyReport {
+  const limits = normalizeLimits(limitsInput);
+  if (typeof content !== "string")
+    throw new Error("input JSON must be a string");
+  validUnicode(content);
+  if (encoder.encode(content).length > limits.max_bytes)
+    throw new Error("consistency input byte limit exceeded");
+  const value: unknown = JSON.parse(content);
+  uniqueKeys(content, limits);
+  structure(value, limits);
+  const data = object(
+    value,
+    ["format", "version", "plan", "ratings", "references"],
+    ["format", "version", "plan", "ratings"],
+    "input bundle",
+  );
+  if (data.format !== "autoevals.consistency.input" || data.version !== 1)
+    throw new Error("unsupported consistency input format/version");
+  const plan = normalizePlan(data.plan as ConsistencyPlan, limits);
+  const ratings = array(data.ratings, "ratings", limits.max_ratings).map(
+    (row) => normalizeRating(row as Rating),
+  );
+  return analyzeConsistency(ratings, {
+    plan,
+    references: data.references as References | undefined,
+    limits,
+  });
+}
+
+/** Snapshot only selected observations; provider metadata and error messages are excluded. */
+export function inputToJSON(
+  ratings: Iterable<Rating>,
+  options: {
+    plan: ConsistencyPlan;
+    references?: References | null;
+    limits?: ConsistencyLimits;
+  },
+): string {
+  const limits = normalizeLimits(options.limits);
+  const plan = normalizePlan(options.plan, limits);
+  const collected: Required<Rating>[] = [];
+  for (const rating of ratings) {
+    if (collected.length >= limits.max_ratings)
+      throw new Error("consistency rating limit exceeded");
+    collected.push(normalizeRating(rating));
+  }
+  const panel = align(collected, plan, options.references, limits);
+  const references = Object.fromEntries(
+    panel.cases
+      .filter((row) => row.reference !== null)
+      .map((row) => [row.case_id, row.reference]),
+  );
+  return encode(
+    {
+      format: "autoevals.consistency.input",
+      version: 1,
+      plan,
+      ratings: collected,
+      references,
+    },
+    limits,
+  );
+}
+
+/** Serialize finite evidence; the format is not a proof of authenticity. */
+export function reportToJSON(
+  report: ConsistencyReport,
+  limits: ConsistencyLimits = {},
+): string {
+  if (report.format !== "autoevals.consistency.report" || report.version !== 1)
+    throw new Error("unsupported consistency report");
+  return encode(report, normalizeLimits(limits));
+}
diff --git a/js/consistency/statistics.ts b/js/consistency/statistics.ts
new file mode 100644
index 0000000..649f598
--- /dev/null
+++ b/js/consistency/statistics.ts
@@ -0,0 +1,254 @@
+import { compareNames } from "./config";
+import type { PanelCase, PairwiseReport, Statistic } from "./types";
+
+export function counts(
+  values: readonly (string | null)[],
+): Map<string, number> {
+  const result = new Map<string, number>();
+  for (const value of values)
+    if (value !== null) result.set(value, (result.get(value) ?? 0) + 1);
+  return result;
+}
+
+export function sum(values: Iterable<number>): number {
+  // Explicit Neumaier accumulation, shared with Python across its versions.
+  // Native sum implementations differ enough to change boundary gate verdicts.
+  let result = 0,
+    correction = 0;
+  for (const value of values) {
+    const next = result + value;
+    correction +=
+      Math.abs(result) >= Math.abs(value)
+        ? result - next + value
+        : value - next + result;
+    result = next;
+  }
+  return result + correction;
+}
+
+/** Equal unordered-pair agreement within one case. */
+export function pairAgreement(categories: Map<string, number>): number | null {
+  const n = sum(categories.values());
+  if (n < 2) return null;
+  return (
+    sum([...categories.values()].map((count) => count * (count - 1))) /
+    (n * (n - 1))
+  );
+}
+
+function result(
+  value: number | null,
+  support: number,
+  ratings: number,
+  denominator: number,
+  observed: number | null,
+  expected: number | null,
+  weighting: string,
+  reason: string | null = null,
+): Statistic {
+  return {
+    value,
+    support_cases: support,
+    support_ratings: ratings,
+    denominator_cases: denominator,
+    observed_agreement: observed,
+    expected_agreement: expected,
+    weighting,
+    reason,
+  };
+}
+
+/** Equal-case raw agreement, distinct from alpha's observation weighting. */
+export function rawAgreement(cases: readonly PanelCase[]): Statistic {
+  const eligible = cases
+    .map((item) => counts(item.categories))
+    .filter((item) => sum(item.values()) >= 2);
+  if (!eligible.length)
+    return result(
+      null,
+      0,
+      0,
+      cases.length,
+      null,
+      null,
+      "equal_pairable_cases",
+      "no_pairable_cases",
+    );
+  const observed =
+    sum(eligible.map((item) => pairAgreement(item) ?? 0)) / eligible.length;
+  return result(
+    observed,
+    eligible.length,
+    sum(eligible.map((item) => sum(item.values()))),
+    cases.length,
+    observed,
+    null,
+    "equal_pairable_cases",
+  );
+}
+
+/** Fleiss pooled-marginal coefficient uses complete, fixed-size panels only. */
+export function fleissKappa(
+  cases: readonly PanelCase[],
+  raterCount: number,
+): Statistic {
+  const complete = cases.filter((item) =>
+    item.categories.every((value) => value !== null),
+  );
+  if (!complete.length)
+    return result(
+      null,
+      0,
+      0,
+      cases.length,
+      null,
+      null,
+      "complete_panels",
+      "no_complete_panels",
+    );
+  const pooled = new Map<string, number>();
+  const agreements: number[] = [];
+  for (const item of complete) {
+    const categories = counts(item.categories);
+    for (const [category, count] of categories)
+      pooled.set(category, (pooled.get(category) ?? 0) + count);
+    agreements.push(pairAgreement(categories) ?? 0);
+  }
+  const n = complete.length * raterCount;
+  const observed = sum(agreements) / complete.length;
+  const expected = sum(
+    [...pooled.values()].map((count) => (count / n) * (count / n)),
+  );
+  const value = expected >= 1 ? null : (observed - expected) / (1 - expected);
+  return result(
+    value,
+    complete.length,
+    n,
+    cases.length,
+    observed,
+    expected,
+    "complete_panels",
+    value === null ? "zero_expected_disagreement" : null,
+  );
+}
+
+/**
+ * Nominal Krippendorff alpha: fractional within-case coincidences and N(N−1)
+ * finite-sample expected disagreement. Singletons contribute to neither term.
+ * Summed disagreement avoids allocating a vocabulary-by-vocabulary matrix.
+ */
+export function nominalAlpha(cases: readonly PanelCase[]): Statistic {
+  const pooled = new Map<string, number>();
+  const disagreements: number[] = [];
+  let support = 0;
+  for (const item of cases) {
+    const categories = counts(item.categories),
+      n = sum(categories.values());
+    if (n < 2) continue;
+    support++;
+    for (const [category, count] of categories)
+      pooled.set(category, (pooled.get(category) ?? 0) + count);
+    disagreements.push(
+      (n * n - sum([...categories.values()].map((count) => count * count))) /
+        (n - 1),
+    );
+  }
+  const n = sum(pooled.values());
+  if (!support)
+    return result(
+      null,
+      0,
+      0,
+      cases.length,
+      null,
+      null,
+      "pairable_observations",
+      "no_pairable_cases",
+    );
+  const observedDisagreement = sum(disagreements) / n;
+  const expectedDisagreement =
+    (n * n - sum([...pooled.values()].map((count) => count * count))) /
+    (n * (n - 1));
+  const value =
+    expectedDisagreement <= 0
+      ? null
+      : 1 - observedDisagreement / expectedDisagreement;
+  return result(
+    value,
+    support,
+    n,
+    cases.length,
+    1 - observedDisagreement,
+    1 - expectedDisagreement,
+    "pairable_observations",
+    value === null ? "zero_expected_disagreement" : null,
+  );
+}
+
+/** Cohen uses both marginals from the same overlapping scored-case cohort. */
+export function cohenKappa(
+  cases: readonly PanelCase[],
+  left: number,
+  right: number,
+): Pick<PairwiseReport, "statistic" | "confusion"> {
+  const matrix = new Map<string, Map<string, number>>();
+  const leftCounts = new Map<string, number>(),
+    rightCounts = new Map<string, number>();
+  let total = 0,
+    equal = 0;
+  for (const item of cases) {
+    const a = item.categories[left],
+      b = item.categories[right];
+    if (a === null || b === null) continue;
+    let row = matrix.get(a);
+    if (!row) {
+      row = new Map();
+      matrix.set(a, row);
+    }
+    row.set(b, (row.get(b) ?? 0) + 1);
+    leftCounts.set(a, (leftCounts.get(a) ?? 0) + 1);
+    rightCounts.set(b, (rightCounts.get(b) ?? 0) + 1);
+    total++;
+    if (a === b) equal++;
+  }
+  const confusion: PairwiseReport["confusion"] = [];
+  for (const a of [...matrix.keys()].sort(compareNames))
+    for (const b of [...matrix.get(a)!.keys()].sort(compareNames))
+      confusion.push({ left: a, right: b, count: matrix.get(a)!.get(b)! });
+  if (!total)
+    return {
+      statistic: result(
+        null,
+        0,
+        0,
+        cases.length,
+        null,
+        null,
+        "pairwise_overlap",
+        "no_overlap",
+      ),
+      confusion,
+    };
+  const observed = equal / total;
+  const expected =
+    sum(
+      [...leftCounts].map(
+        ([category, count]) => count * (rightCounts.get(category) ?? 0),
+      ),
+    ) /
+    (total * total);
+  const value = expected >= 1 ? null : (observed - expected) / (1 - expected);
+  return {
+    statistic: result(
+      value,
+      total,
+      2 * total,
+      cases.length,
+      observed,
+      expected,
+      "pairwise_overlap",
+      value === null ? "zero_expected_disagreement" : null,
+    ),
+    confusion,
+  };
+}
diff --git a/js/consistency/types.ts b/js/consistency/types.ts
new file mode 100644
index 0000000..0ce12ba
--- /dev/null
+++ b/js/consistency/types.ts
@@ -0,0 +1,217 @@
+/** A single criterion's categorical result; unavailable cells carry no label. */
+export interface Rating {
+  case_id: string;
+  rater_id: string;
+  category?: string | null;
+  status?: "scored" | "missing" | "error";
+}
+
+export interface ScoreMapping {
+  score_name: string;
+  /** Exact numeric mapping entries, not stringified JSON object keys. */
+  entries: readonly (readonly [number, string])[];
+}
+
+export type StatisticName =
+  | "raw_agreement"
+  | "fleiss_kappa"
+  | "nominal_alpha"
+  | "consensus_accuracy";
+
+export interface ConsistencyGroup {
+  name: string;
+  cases: readonly string[];
+}
+
+/** Observed finite-benchmark requirement. Bootstrap intervals never gate. */
+export interface ConsistencyGate {
+  statistic: StatisticName;
+  minimum: number;
+  group?: string;
+  minimum_support?: number;
+  minimum_coverage?: number;
+  minimum_rating_coverage?: number;
+}
+
+/** Marginal diagnostic case-bootstrap intervals, disabled by default. */
+export interface ConsistencyBootstrap {
+  resamples?: number;
+  confidence?: number;
+  seed?: number;
+  minimum_cases?: number;
+  statistics?: readonly StatisticName[];
+}
+
+export interface ConsistencyPlan {
+  score_name: string;
+  raters: readonly string[];
+  categories: readonly string[];
+  /** Omit for observed-union provenance; an empty array declares an empty cohort. */
+  cases?: readonly string[] | null;
+  groups?: readonly ConsistencyGroup[];
+  gates?: readonly ConsistencyGate[];
+  minimum_consensus_ratings?: number;
+  detail_limit?: number;
+  bootstrap?: ConsistencyBootstrap;
+}
+
+/** Trusted application limits; portable input files cannot raise these. */
+export interface ConsistencyLimits {
+  max_ratings?: number;
+  max_cases?: number;
+  max_raters?: number;
+  max_categories?: number;
+  max_groups?: number;
+  max_cells?: number;
+  max_pairwise_work?: number;
+  max_group_cells?: number;
+  max_bootstrap_work?: number;
+  max_details?: number;
+  max_bytes?: number;
+  max_json_nodes?: number;
+  max_json_depth?: number;
+}
+
+export interface Statistic {
+  value: number | null;
+  support_cases: number;
+  support_ratings: number;
+  denominator_cases: number;
+  observed_agreement: number | null;
+  expected_agreement: number | null;
+  weighting: string;
+  reason: string | null;
+}
+
+export interface ReferenceAccuracy {
+  labeled_cases: number;
+  evaluated_cases: number;
+  correct_cases: number;
+  unavailable_cases: number;
+  coverage: number | null;
+  accuracy: number | null;
+  reason: string | null;
+  missing_cases?: number;
+  error_cases?: number;
+  absent_cases?: number;
+  abstained_cases?: number;
+}
+
+export interface Coverage {
+  total_cases: number;
+  expected_cells: number;
+  present_cells: number;
+  scored_cells: number;
+  missing_cells: number;
+  error_cells: number;
+  absent_cells: number;
+  rating_coverage: number | null;
+  zero_rating_cases: number;
+  singleton_cases: number;
+  pairable_cases: number;
+  complete_cases: number;
+}
+
+export interface RaterReport {
+  rater_id: string;
+  scored_cases: number;
+  missing_cases: number;
+  error_cases: number;
+  absent_cases: number;
+  coverage: number | null;
+  category_counts: { category: string; count: number }[];
+  reference: ReferenceAccuracy;
+}
+
+export interface CaseReport {
+  case_id: string;
+  scored_ratings: number;
+  category_counts: { category: string; count: number }[];
+  raw_agreement: number | null;
+  consensus: string | null;
+  consensus_reason: string | null;
+  reference: string | null;
+  consensus_correct: boolean | null;
+  unavailable_raters: { rater_id: string; status: string }[];
+}
+
+export interface PairwiseReport {
+  left_rater: string;
+  right_rater: string;
+  statistic: Statistic;
+  /** Sparse nonzero cells: rows are the left rater, columns the right. */
+  confusion: { left: string; right: string; count: number }[];
+}
+
+export interface ConsistencyInterval {
+  statistic: StatisticName;
+  confidence: number;
+  resamples: number;
+  attempted: number;
+  valid: number;
+  invalid: number;
+  lower: number | null;
+  upper: number | null;
+  reason: string | null;
+  seed: number;
+  algorithm: "case-xorshift32-percentile-v1";
+}
+
+export interface GroupReport {
+  name: string;
+  coverage: Coverage;
+  statistics: Record<StatisticName, Statistic>;
+  raters: RaterReport[];
+  pairwise: PairwiseReport[];
+  consensus: {
+    rule: "strict_majority_of_declared_panel";
+    decided_cases: number;
+    abstained_cases: number;
+    reason_counts: { reason: string; count: number }[];
+    reference: ReferenceAccuracy;
+  };
+  intervals: ConsistencyInterval[];
+  cases: CaseReport[];
+  omitted_cases: number;
+}
+
+export interface GateResult extends Required<ConsistencyGate> {
+  value: number | null;
+  support_cases: number;
+  denominator_cases: number;
+  support_coverage: number | null;
+  rating_coverage: number | null;
+  verdict: "pass" | "fail" | "inconclusive";
+  reasons: string[];
+}
+
+export interface ConsistencyReport {
+  format: "autoevals.consistency.report";
+  version: 1;
+  score_name: string;
+  cohort_provenance: "expected_manifest" | "observed_union";
+  categories: string[];
+  raters: string[];
+  interpretation: "observed_finite_benchmark";
+  interval_interpretation: "marginal_diagnostic_case_bootstrap";
+  groups: GroupReport[];
+  gates: GateResult[];
+  verdict: "not_evaluated" | "pass" | "fail" | "inconclusive";
+}
+
+export interface NormalizedPlan extends Required<ConsistencyPlan> {
+  cases: readonly string[] | null;
+  gates: readonly Required<ConsistencyGate>[];
+  bootstrap: Required<ConsistencyBootstrap>;
+}
+
+export interface PanelCase {
+  readonly case_id: string;
+  readonly categories: readonly (string | null)[];
+  readonly statuses: readonly string[];
+  readonly reference: string | null;
+}
+
+export type References =
+  | ReadonlyMap<string, string>
+  | Readonly<Record<string, string>>;
diff --git a/js/index.ts b/js/index.ts
index 40a2a54..1093982 100644
--- a/js/index.ts
+++ b/js/index.ts
@@ -39,6 +39,7 @@ export * from "./json";
 export * from "./templates";
 export * from "./ragas";
 export * from "./value";
+export * from "./consistency";
 export { Evaluators } from "./manifest";
 export { makePartial, ScorerWithPartial } from "./partial";
 export {
diff --git a/py/autoevals/__init__.py b/py/autoevals/__init__.py
index 0a5841a..4422b3a 100644
--- a/py/autoevals/__init__.py
+++ b/py/autoevals/__init__.py
@@ -125,6 +125,7 @@ async def evaluate_qa():
 See individual module documentation for detailed usage and options.
 """
 
+from .consistency import *
 from .json import *
 from .list import *
 from .litellm import AsyncLiteLLMClient, LiteLLMClient
diff --git a/py/autoevals/consistency/__init__.py b/py/autoevals/consistency/__init__.py
new file mode 100644
index 0000000..d6eddc8
--- /dev/null
+++ b/py/autoevals/consistency/__init__.py
@@ -0,0 +1,26 @@
+"""Offline consistency of repeated categorical judge results, distinct from accuracy."""
+
+from .adapters import ScoreMapping, error_rating, missing_rating, rating_from_score
+from .analysis import analyze_consistency
+from .config import ConsistencyBootstrap, ConsistencyGate, ConsistencyGroup, ConsistencyLimits, ConsistencyPlan, Rating
+from .reporting import consistency_scores, render_consistency_markdown
+from .serialization import analyze_consistency_json, input_to_json, report_to_json
+
+__all__ = [
+    "ConsistencyBootstrap",
+    "ConsistencyGate",
+    "ConsistencyGroup",
+    "ConsistencyLimits",
+    "ConsistencyPlan",
+    "Rating",
+    "ScoreMapping",
+    "analyze_consistency",
+    "error_rating",
+    "missing_rating",
+    "rating_from_score",
+    "analyze_consistency_json",
+    "input_to_json",
+    "report_to_json",
+    "consistency_scores",
+    "render_consistency_markdown",
+]
diff --git a/py/autoevals/consistency/adapters.py b/py/autoevals/consistency/adapters.py
new file mode 100644
index 0000000..06fbe69
--- /dev/null
+++ b/py/autoevals/consistency/adapters.py
@@ -0,0 +1,77 @@
+"""Convert native single-result Scores into explicit categorical observations."""
+
+from collections.abc import Mapping
+from dataclasses import dataclass
+from typing import Any
+
+from ..score import Score
+from .config import Rating, finite, identifier
+
+
+@dataclass(frozen=True)
+class ScoreMapping:
+    """Explicit exact numeric-to-category mapping; no inferred thresholds.
+
+    Use entries rather than JSON object keys so numeric types remain numeric.
+    Multiple numeric values may map to one category. Each numeric value appears
+    once, and an unmapped finite score is an error rather than a missing vote.
+    """
+
+    score_name: str
+    entries: tuple[tuple[float, str], ...]
+
+    def __post_init__(self) -> None:
+        identifier(self.score_name, "score_name")
+        if not isinstance(self.entries, (tuple, list)) or not 1 <= len(self.entries) <= 1000:
+            raise ValueError("mapping entries must be a nonempty bounded sequence")
+        normalized = []
+        for entry in self.entries:
+            if not isinstance(entry, (tuple, list)) or len(entry) != 2:
+                raise ValueError("mapping entries require score/category pairs")
+            normalized.append((finite(entry[0], "mapped score", 0, 1), identifier(entry[1], "category")))
+        if len({value for value, _ in normalized}) != len(normalized):
+            raise ValueError("duplicate mapped numeric score")
+        object.__setattr__(self, "entries", tuple(sorted(normalized)))
+
+
+def missing_rating(case_id: str, rater_id: str) -> Rating:
+    """Record an explicitly skipped evaluation, distinct from an absent cell."""
+    return Rating(case_id, rater_id, status="missing")
+
+
+def error_rating(case_id: str, rater_id: str) -> Rating:
+    """Record a caught scorer exception without retaining its message or payload."""
+    return Rating(case_id, rater_id, status="error")
+
+
+def rating_from_score(
+    case_id: str, rater_id: str, score: Score | Mapping[str, Any], *, mapping: ScoreMapping
+) -> Rating:
+    """Adapt one native result, independently validating its numeric contract.
+
+    AutoEvals scorers may raise: callers decide whether to propagate or explicitly
+    record ``error_rating``. The deprecated live ``Score.error`` takes precedence
+    over its numeric value. ``Score.as_dict`` omits that field; adapt live results
+    before serialization when error information matters. Metadata is never read.
+    """
+    if not isinstance(mapping, ScoreMapping):
+        raise ValueError("mapping must be a ScoreMapping")
+    if isinstance(score, Score):
+        name, value, error = score.name, score.score, score.error
+    elif isinstance(score, Mapping):
+        if "name" not in score or "score" not in score:
+            raise ValueError("score requires name and score fields")
+        name, value, error = score["name"], score["score"], score.get("error")
+    else:
+        raise ValueError("expected a native Score or score mapping")
+    if name != mapping.score_name:
+        raise ValueError("score name does not match the selected criterion")
+    if error is not None:
+        return error_rating(case_id, rater_id)
+    if value is None:
+        return missing_rating(case_id, rater_id)
+    numeric = finite(value, "score", 0, 1)
+    for expected, category in mapping.entries:
+        if numeric == expected:
+            return Rating(case_id, rater_id, category)
+    raise ValueError("numeric score is not mapped to a category")
diff --git a/py/autoevals/consistency/alignment.py b/py/autoevals/consistency/alignment.py
new file mode 100644
index 0000000..8425456
--- /dev/null
+++ b/py/autoevals/consistency/alignment.py
@@ -0,0 +1,115 @@
+"""Freeze declared panels before any missing-outcome filtering."""
+
+from collections.abc import Iterable, Mapping
+from dataclasses import dataclass
+
+from .config import ConsistencyLimits, ConsistencyPlan, Rating, identifier
+
+
+@dataclass(frozen=True)
+class PanelCase:
+    """An immutable row in a declared-rater matrix, including absent slots."""
+
+    case_id: str
+    categories: tuple[str | None, ...]
+    statuses: tuple[str, ...]
+    reference: str | None
+
+
+@dataclass(frozen=True)
+class Panel:
+    """Canonical case vectors and fixed group indices for one criterion."""
+
+    cases: tuple[PanelCase, ...]
+    groups: tuple[tuple[str, tuple[int, ...]], ...]
+    provenance: str
+
+
+def align(
+    ratings: Iterable[Rating],
+    plan: ConsistencyPlan,
+    references: Mapping[str, str] | None,
+    limits: ConsistencyLimits,
+) -> Panel:
+    """Validate every observation before allocating the complete panel.
+
+    Groups and references cannot introduce cases missing from observed data. A
+    planned case manifest is required to account for such unobserved cases.
+    Every absent slot remains distinct from an explicitly missing/error result.
+    """
+    plan.check_limits(limits)
+    raters = set(plan.raters)
+    categories = set(plan.categories)
+    declared = set(plan.cases) if plan.cases is not None else None
+    observed: dict[str, dict[str, Rating]] = {}
+    selected_bytes = plan.selected_bytes()
+    for index, rating in enumerate(ratings):
+        if index >= limits.max_ratings:
+            raise ValueError("consistency rating limit exceeded")
+        if not isinstance(rating, Rating):
+            raise ValueError("ratings must contain Rating objects")
+        if rating.rater_id not in raters:
+            raise ValueError("unknown rater_id")
+        if rating.category is not None and rating.category not in categories:
+            raise ValueError("unknown rating category")
+        if declared is not None and rating.case_id not in declared:
+            raise ValueError("case_id is outside the expected manifest")
+        cells = observed.setdefault(rating.case_id, {})
+        if rating.rater_id in cells:
+            raise ValueError("duplicate case/rater cell")
+        if len(observed) > limits.max_cases:
+            raise ValueError("consistency case limit exceeded")
+        selected_bytes += sum(
+            len(value.encode("utf-8"))
+            for value in (rating.case_id, rating.rater_id, rating.category or "", rating.status)
+        )
+        if selected_bytes > limits.max_bytes:
+            raise ValueError("consistency selected byte limit exceeded")
+        cells[rating.rater_id] = rating
+    case_ids = (
+        plan.cases if plan.cases is not None else tuple(sorted(observed, key=lambda value: value.encode("utf-8")))
+    )
+    if len(case_ids) * len(plan.raters) > limits.max_cells:
+        raise ValueError("consistency cell limit exceeded")
+    known_cases = set(case_ids)
+    if references is None:
+        references = {}
+    if not isinstance(references, Mapping) or len(references) > limits.max_cases:
+        raise ValueError("references must be a bounded case/category mapping")
+    copied_references = {}
+    for case_id, category in references.items():
+        identifier(case_id, "reference case_id")
+        identifier(category, "reference category")
+        if case_id not in known_cases:
+            raise ValueError("reference case_id is outside the cohort")
+        if category not in categories:
+            raise ValueError("unknown reference category")
+        selected_bytes += len(case_id.encode("utf-8")) + len(category.encode("utf-8"))
+        if selected_bytes > limits.max_bytes:
+            raise ValueError("consistency selected byte limit exceeded")
+        copied_references[case_id] = category
+    group_cases = len(case_ids) + sum(len(group.cases) for group in plan.groups)
+    if group_cases * len(plan.raters) > limits.max_group_cells:
+        raise ValueError("consistency group cell limit exceeded")
+    pair_count = len(plan.raters) * (len(plan.raters) - 1) // 2
+    if (group_cases + len(plan.groups) + 1) * pair_count > limits.max_pairwise_work:
+        raise ValueError("consistency pairwise work limit exceeded")
+    bootstrap_work = group_cases * len(plan.raters) * plan.bootstrap.resamples
+    if bootstrap_work > limits.max_bootstrap_work:
+        raise ValueError("consistency bootstrap work limit exceeded")
+    positions = {case_id: index for index, case_id in enumerate(case_ids)}
+    groups = [("overall", tuple(range(len(case_ids))))]
+    for group in plan.groups:
+        if set(group.cases) - known_cases:
+            raise ValueError("group cases are outside the cohort; declare expected cases")
+        groups.append((group.name, tuple(positions[case_id] for case_id in group.cases)))
+    cases = []
+    for case_id in case_ids:
+        supplied = observed.get(case_id, {})
+        values, statuses = [], []
+        for rater_id in plan.raters:
+            rating = supplied.get(rater_id)
+            values.append(rating.category if rating else None)
+            statuses.append(rating.status if rating else "absent")
+        cases.append(PanelCase(case_id, tuple(values), tuple(statuses), copied_references.get(case_id)))
+    return Panel(tuple(cases), tuple(groups), "expected_manifest" if declared is not None else "observed_union")
diff --git a/py/autoevals/consistency/analysis.py b/py/autoevals/consistency/analysis.py
new file mode 100644
index 0000000..0b0c2a8
--- /dev/null
+++ b/py/autoevals/consistency/analysis.py
@@ -0,0 +1,274 @@
+"""Analyze repeated categorical judgments without running or tuning a scorer."""
+
+from collections import Counter
+from collections.abc import Iterable, Mapping
+from dataclasses import asdict
+from typing import Any, Sequence
+
+from .alignment import PanelCase, align
+from .config import ConsistencyGate, ConsistencyLimits, ConsistencyPlan, Rating
+from .statistics import (
+    Statistic,
+    category_counts,
+    cohen_kappa,
+    fleiss_kappa,
+    nominal_alpha,
+    pair_agreement,
+    raw_agreement,
+)
+
+
+def consensus(case: PanelCase, plan: ConsistencyPlan) -> tuple[str | None, str | None]:
+    """Require a strict majority of the entire declared panel, including gaps."""
+    counts = category_counts(case)
+    if sum(counts.values()) < plan.minimum_consensus_ratings:
+        return None, "insufficient_ratings"
+    maximum = max(counts.values())
+    winners = [category for category, count in counts.items() if count == maximum]
+    if len(winners) != 1:
+        return None, "tie"
+    if maximum * 2 <= len(plan.raters):
+        return None, "no_declared_panel_majority"
+    return winners[0], None
+
+
+def reference_accuracy(cases: Sequence[PanelCase], values: Sequence[str | None]) -> dict[str, Any]:
+    """Describe evaluated-reference accuracy and abstentions separately."""
+    labeled = evaluated = correct = 0
+    for case, value in zip(cases, values):
+        if case.reference is None:
+            continue
+        labeled += 1
+        if value is not None:
+            evaluated += 1
+            correct += value == case.reference
+    return {
+        "labeled_cases": labeled,
+        "evaluated_cases": evaluated,
+        "correct_cases": correct,
+        "unavailable_cases": labeled - evaluated,
+        "coverage": evaluated / labeled if labeled else None,
+        "accuracy": correct / evaluated if evaluated else None,
+        "reason": "no_references" if not labeled else "no_evaluated_references" if not evaluated else None,
+    }
+
+
+def consensus_statistic(cases: Sequence[PanelCase], plan: ConsistencyPlan) -> Statistic:
+    """The reference-conditioned consensus statistic used by explicit gates."""
+    values = [consensus(case, plan)[0] for case in cases]
+    accuracy = reference_accuracy(cases, values)
+    support_ratings = sum(
+        sum(category is not None for category in case.categories)
+        for case, value in zip(cases, values)
+        if case.reference is not None and value is not None
+    )
+    return Statistic(
+        accuracy["accuracy"],
+        accuracy["evaluated_cases"],
+        support_ratings,
+        accuracy["labeled_cases"],
+        None,
+        None,
+        "evaluated_reference_cases",
+        accuracy["reason"],
+    )
+
+
+def coefficients(cases: Sequence[PanelCase], plan: ConsistencyPlan) -> dict[str, Statistic]:
+    """Compute the four group-level statistics on their explicitly different supports."""
+    return {
+        "raw_agreement": raw_agreement(cases),
+        "fleiss_kappa": fleiss_kappa(cases, len(plan.raters)),
+        "nominal_alpha": nominal_alpha(cases),
+        "consensus_accuracy": consensus_statistic(cases, plan),
+    }
+
+
+def coverage(cases: Sequence[PanelCase], plan: ConsistencyPlan) -> dict[str, Any]:
+    """Count additive cell states, including cases never observed at all."""
+    states: Counter[str] = Counter(status for case in cases for status in case.statuses)
+    counts = [sum(category is not None for category in case.categories) for case in cases]
+    expected = len(cases) * len(plan.raters)
+    return {
+        "total_cases": len(cases),
+        "expected_cells": expected,
+        "present_cells": expected - states["absent"],
+        "scored_cells": states["scored"],
+        "missing_cells": states["missing"],
+        "error_cells": states["error"],
+        "absent_cells": states["absent"],
+        "rating_coverage": states["scored"] / expected if expected else None,
+        "zero_rating_cases": sum(count == 0 for count in counts),
+        "singleton_cases": sum(count == 1 for count in counts),
+        "pairable_cases": sum(count >= 2 for count in counts),
+        "complete_cases": sum(count == len(plan.raters) for count in counts),
+    }
+
+
+def _rater_report(cases: Sequence[PanelCase], index: int, rater: str) -> dict[str, Any]:
+    states = Counter(case.statuses[index] for case in cases)
+    values = [case.categories[index] for case in cases]
+    reference = reference_accuracy(cases, values)
+    for status in ("missing", "error", "absent"):
+        reference[f"{status}_cases"] = sum(
+            case.reference is not None and case.statuses[index] == status for case in cases
+        )
+    return {
+        "rater_id": rater,
+        "scored_cases": states["scored"],
+        "missing_cases": states["missing"],
+        "error_cases": states["error"],
+        "absent_cases": states["absent"],
+        "coverage": states["scored"] / len(cases) if cases else None,
+        "category_counts": [
+            {"category": category, "count": count}
+            for category, count in sorted(
+                Counter(value for value in values if value is not None).items(),
+                key=lambda item: item[0].encode("utf-8"),
+            )
+        ],
+        "reference": reference,
+    }
+
+
+def _case_report(case: PanelCase, plan: ConsistencyPlan) -> dict[str, Any]:
+    counts = category_counts(case)
+    winner, reason = consensus(case, plan)
+    return {
+        "case_id": case.case_id,
+        "scored_ratings": sum(counts.values()),
+        "category_counts": [{"category": category, "count": counts[category]} for category in plan.categories],
+        "raw_agreement": pair_agreement(counts),
+        "consensus": winner,
+        "consensus_reason": reason,
+        "reference": case.reference,
+        "consensus_correct": winner == case.reference if winner is not None and case.reference is not None else None,
+        "unavailable_raters": [
+            {"rater_id": rater, "status": status}
+            for rater, status in zip(plan.raters, case.statuses)
+            if status != "scored"
+        ],
+    }
+
+
+def _gate_result(gate: ConsistencyGate, statistic: Statistic, group_coverage: dict) -> dict[str, Any]:
+    reasons = []
+    support_coverage = statistic.support_cases / statistic.denominator_cases if statistic.denominator_cases else None
+    rating_coverage = group_coverage["rating_coverage"]
+    if statistic.value is None:
+        reasons.append(statistic.reason or "undefined_statistic")
+    if statistic.support_cases < gate.minimum_support:
+        reasons.append("insufficient_support")
+    if support_coverage is None or support_coverage < gate.minimum_coverage:
+        reasons.append("insufficient_statistic_coverage")
+    if rating_coverage is None or rating_coverage < gate.minimum_rating_coverage:
+        reasons.append("insufficient_rating_coverage")
+    if reasons:
+        verdict = "inconclusive"
+    elif statistic.value is not None and statistic.value < gate.minimum:
+        verdict = "fail"
+        reasons.append("below_observed_threshold")
+    else:
+        verdict = "pass"
+        reasons.append("meets_observed_threshold")
+    return {
+        **asdict(gate),
+        "value": statistic.value,
+        "support_cases": statistic.support_cases,
+        "denominator_cases": statistic.denominator_cases,
+        "support_coverage": support_coverage,
+        "rating_coverage": rating_coverage,
+        "verdict": verdict,
+        "reasons": reasons,
+    }
+
+
+def analyze_consistency(
+    ratings: Iterable[Rating],
+    *,
+    plan: ConsistencyPlan,
+    references: Mapping[str, str] | None = None,
+    limits: ConsistencyLimits | None = None,
+) -> dict[str, Any]:
+    """Analyze one categorical criterion across repeated judges or trials.
+
+    No evaluator, model, backend or global random generator is invoked. Missing
+    cells stay in coverage denominators. Agreement coefficients use different
+    documented supports; consistency alone says nothing about correctness.
+    Optional gates describe this finite benchmark. Bootstrap intervals are
+    marginal diagnostics, never inputs to gate decisions.
+    """
+    from .bootstrap import bootstrap_intervals
+
+    if not isinstance(plan, ConsistencyPlan):
+        raise ValueError("plan must be a ConsistencyPlan")
+    limits = limits or ConsistencyLimits()
+    panel = align(ratings, plan, references, limits)
+    groups = []
+    gates = []
+    remaining_details = plan.detail_limit
+    for name, indices in panel.groups:
+        cases = tuple(panel.cases[index] for index in indices)
+        stats = coefficients(cases, plan)
+        availability = coverage(cases, plan)
+        pairwise = []
+        for left, left_name in enumerate(plan.raters):
+            for right in range(left + 1, len(plan.raters)):
+                statistic, cells = cohen_kappa(cases, left, right)
+                pairwise.append(
+                    {
+                        "left_rater": left_name,
+                        "right_rater": plan.raters[right],
+                        "statistic": statistic.to_dict(),
+                        "confusion": [{"left": a, "right": b, "count": count} for a, b, count in cells],
+                    }
+                )
+        decisions = [consensus(case, plan) for case in cases]
+        consensus_reference = reference_accuracy(cases, [value for value, _ in decisions])
+        consensus_reference["abstained_cases"] = consensus_reference["unavailable_cases"]
+        groups.append(
+            {
+                "name": name,
+                "coverage": availability,
+                "statistics": {key: value.to_dict() for key, value in stats.items()},
+                "raters": [_rater_report(cases, index, rater) for index, rater in enumerate(plan.raters)],
+                "pairwise": pairwise,
+                "consensus": {
+                    "rule": "strict_majority_of_declared_panel",
+                    "decided_cases": sum(value is not None for value, _ in decisions),
+                    "abstained_cases": sum(value is None for value, _ in decisions),
+                    "reason_counts": [
+                        {"reason": reason, "count": count}
+                        for reason, count in sorted(Counter(reason for _, reason in decisions if reason).items())
+                    ],
+                    "reference": consensus_reference,
+                },
+                "intervals": bootstrap_intervals(cases, plan, name, stats),
+                "cases": [_case_report(case, plan) for case in cases[:remaining_details]],
+                "omitted_cases": max(0, len(cases) - remaining_details),
+            }
+        )
+        remaining_details = max(0, remaining_details - len(cases))
+        gates.extend(
+            _gate_result(gate, stats[gate.statistic], availability) for gate in plan.gates if gate.group == name
+        )
+    verdict = "not_evaluated"
+    if gates:
+        verdict = (
+            "fail"
+            if any(gate["verdict"] == "fail" for gate in gates)
+            else ("inconclusive" if any(gate["verdict"] == "inconclusive" for gate in gates) else "pass")
+        )
+    return {
+        "format": "autoevals.consistency.report",
+        "version": 1,
+        "score_name": plan.score_name,
+        "cohort_provenance": panel.provenance,
+        "categories": list(plan.categories),
+        "raters": list(plan.raters),
+        "interpretation": "observed_finite_benchmark",
+        "interval_interpretation": "marginal_diagnostic_case_bootstrap",
+        "groups": groups,
+        "gates": gates,
+        "verdict": verdict,
+    }
diff --git a/py/autoevals/consistency/bootstrap.py b/py/autoevals/consistency/bootstrap.py
new file mode 100644
index 0000000..8462544
--- /dev/null
+++ b/py/autoevals/consistency/bootstrap.py
@@ -0,0 +1,113 @@
+"""Reproducible marginal case-bootstrap diagnostics shared with TypeScript."""
+
+from typing import Any, Sequence
+
+from .alignment import PanelCase
+from .config import ConsistencyPlan
+from .statistics import Statistic
+
+
+class CaseRandom:
+    """Specified xorshift32 stream; seed zero maps to a fixed nonzero state."""
+
+    def __init__(self, seed: int, namespace: str) -> None:
+        state = (2166136261 ^ seed) & 0xFFFFFFFF
+        for value in namespace.encode("utf-8"):
+            state = ((state ^ value) * 16777619) & 0xFFFFFFFF
+        self.state = state or 0x9E3779B9
+
+    def next_u32(self) -> int:
+        """Return the next unsigned32-bit value without process-global state."""
+        value = self.state
+        value ^= (value << 13) & 0xFFFFFFFF
+        value ^= value >> 17
+        value ^= (value << 5) & 0xFFFFFFFF
+        self.state = value & 0xFFFFFFFF
+        return self.state
+
+    def index(self, size: int) -> int:
+        """Rejection-sample indices without a modulo bias."""
+        ceiling = (2**32 // size) * size
+        while True:
+            value = self.next_u32()
+            if value < ceiling:
+                return value % size
+
+
+def percentile(values: Sequence[float], probability: float) -> float:
+    """Type7 quantile of a sorted nonempty finite sequence."""
+    position = (len(values) - 1) * probability
+    low = int(position)
+    high = min(low + 1, len(values) - 1)
+    weight = position - low
+    return values[low] * (1 - weight) + values[high] * weight
+
+
+def bootstrap_intervals(
+    cases: Sequence[PanelCase], plan: ConsistencyPlan, group: str, observed: dict[str, Statistic]
+) -> list[dict[str, Any]]:
+    """Resample whole case occurrences; never deduplicate repeated sampled IDs.
+
+    Undefined draws are counted but never dropped to construct a conditional
+    confidence interval. Any such draw makes that interval unavailable. This is
+    intentionally conservative for degenerate chance-corrected coefficients.
+    """
+    from .analysis import coefficients
+
+    options = plan.bootstrap
+    if not options.resamples:
+        return []
+    results = []
+    eligible = []
+    distributions: dict[str, list[float]] = {}
+    for name in options.statistics:
+        statistic = observed[name]
+        reason = None
+        if statistic.value is None:
+            reason = "undefined_observed_statistic"
+        elif statistic.support_cases < options.minimum_cases:
+            reason = "insufficient_support"
+        elif options.resamples * (1 - options.confidence) / 2 < 20:
+            reason = "insufficient_tail_resamples"
+        result = {
+            "statistic": name,
+            "confidence": options.confidence,
+            "resamples": options.resamples,
+            "attempted": 0,
+            "valid": 0,
+            "invalid": 0,
+            "lower": None,
+            "upper": None,
+            "reason": reason,
+            "seed": options.seed,
+            "algorithm": "case-xorshift32-percentile-v1",
+        }
+        results.append(result)
+        if reason is None:
+            eligible.append(result)
+            distributions[name] = []
+    if not eligible:
+        return results
+    rng = CaseRandom(options.seed, group)
+    for _ in range(options.resamples):
+        sample = tuple(cases[rng.index(len(cases))] for _ in cases)
+        values = coefficients(sample, plan)
+        for result in eligible:
+            result["attempted"] += 1
+            value = values[result["statistic"]].value
+            if value is None:
+                result["invalid"] += 1
+            else:
+                result["valid"] += 1
+                distributions[result["statistic"]].append(value)
+    for result in eligible:
+        distribution = sorted(distributions[result["statistic"]])
+        if result["invalid"]:
+            result["reason"] = "undefined_resamples"
+        elif distribution[0] == distribution[-1]:
+            result["reason"] = "degenerate_resamples"
+        else:
+            alpha = (1 - options.confidence) / 2
+            result["lower"] = percentile(distribution, alpha)
+            result["upper"] = percentile(distribution, 1 - alpha)
+    return results
diff --git a/py/autoevals/consistency/config.py b/py/autoevals/consistency/config.py
new file mode 100644
index 0000000..f9d4914
--- /dev/null
+++ b/py/autoevals/consistency/config.py
@@ -0,0 +1,230 @@
+"""Explicit categorical repeated-judge study contracts and trusted ceilings."""
+
+import math
+from dataclasses import dataclass, field
+from typing import Any
+
+STATISTICS = ("raw_agreement", "fleiss_kappa", "nominal_alpha", "consensus_accuracy")
+
+
+def identifier(value: Any, name: str) -> str:
+    """Validate portable Unicode identifiers without coercion or normalization."""
+    if type(value) is not str or not value:
+        raise ValueError(f"{name} must be a nonempty string")
+    try:
+        size = len(value.encode("utf-8"))
+    except UnicodeError as exc:
+        raise ValueError(f"{name} contains an invalid Unicode surrogate") from exc
+    if size > 4096:
+        raise ValueError(f"{name} exceeds the string limit")
+    return value
+
+
+def integer(value: Any, name: str, low: int, high: int) -> int:
+    """Restrict counts to integers exactly representable in both runtimes."""
+    if type(value) is not int or not low <= value <= high:
+        raise ValueError(f"{name} must be an integer between {low} and {high}")
+    return value
+
+
+def finite(value: Any, name: str, low: float, high: float) -> float:
+    """Reject coercible strings and boolean scores, including Python bools."""
+    if type(value) not in (int, float):
+        raise ValueError(f"{name} must be a finite number")
+    if not math.isfinite(value) or not low <= value <= high:
+        raise ValueError(f"{name} must be between {low} and {high}")
+    return float(value)
+
+
+def names(values: Any, name: str, *, minimum: int = 0, maximum: int = 100000) -> tuple[str, ...]:
+    """Copy bounded unique names in UTF-8 lexicographic order."""
+    if not isinstance(values, (tuple, list)) or not minimum <= len(values) <= maximum:
+        raise ValueError(f"{name} must be a bounded sequence")
+    result = tuple(identifier(value, name) for value in values)
+    if len(set(result)) != len(result):
+        raise ValueError(f"duplicate {name}")
+    return tuple(sorted(result, key=lambda value: value.encode("utf-8")))
+
+
+@dataclass(frozen=True)
+class ConsistencyLimits:
+    """Application-owned resource ceilings; serialized inputs cannot raise them."""
+
+    max_ratings: int = 1000000
+    max_cases: int = 100000
+    max_raters: int = 64
+    max_categories: int = 100
+    max_groups: int = 50
+    max_cells: int = 1000000
+    max_pairwise_work: int = 10000000
+    max_group_cells: int = 2000000
+    max_bootstrap_work: int = 100000000
+    max_details: int = 10000
+    max_bytes: int = 64 * 1024 * 1024
+    max_json_nodes: int = 2000000
+    max_json_depth: int = 20
+
+    def __post_init__(self) -> None:
+        for key, value in vars(self).items():
+            integer(value, key, 1, 2**53 - 1)
+
+
+@dataclass(frozen=True)
+class Rating:
+    """One explicit case/rater cell, with an outcome separate from availability."""
+
+    case_id: str
+    rater_id: str
+    category: str | None = None
+    status: str = "scored"
+
+    def __post_init__(self) -> None:
+        identifier(self.case_id, "case_id")
+        identifier(self.rater_id, "rater_id")
+        if self.status not in ("scored", "missing", "error"):
+            raise ValueError("rating status must be scored, missing or error")
+        if self.status == "scored":
+            identifier(self.category, "category")
+        elif self.category is not None:
+            raise ValueError("unavailable ratings cannot contain a category")
+
+
+@dataclass(frozen=True)
+class ConsistencyGroup:
+    """A predeclared subset of case identities, independent of scored outcomes."""
+
+    name: str
+    cases: tuple[str, ...]
+
+    def __post_init__(self) -> None:
+        identifier(self.name, "group name")
+        if self.name == "overall":
+            raise ValueError("overall is reserved")
+        object.__setattr__(self, "cases", names(self.cases, "group cases"))
+
+
+@dataclass(frozen=True)
+class ConsistencyGate:
+    """An observed finite-benchmark requirement, never a confidence guarantee.
+
+    Support coverage uses the statistic's own cohort: pairable cases for raw
+    agreement/alpha, complete panels for Fleiss, and evaluated reference cases
+    for consensus accuracy. Rating coverage independently counts all cells.
+    """
+
+    statistic: str
+    minimum: float
+    group: str = "overall"
+    minimum_support: int = 2
+    minimum_coverage: float = 1.0
+    minimum_rating_coverage: float = 1.0
+
+    def __post_init__(self) -> None:
+        if self.statistic not in STATISTICS:
+            raise ValueError("unknown gate statistic")
+        identifier(self.group, "gate group")
+        object.__setattr__(self, "minimum", finite(self.minimum, "minimum", -1, 1))
+        if self.statistic in ("raw_agreement", "consensus_accuracy") and self.minimum < 0:
+            raise ValueError("agreement and accuracy thresholds must be nonnegative")
+        integer(self.minimum_support, "minimum_support", 1, 100000)
+        for key in ("minimum_coverage", "minimum_rating_coverage"):
+            object.__setattr__(self, key, finite(getattr(self, key), key, 0, 1))
+
+
+@dataclass(frozen=True)
+class ConsistencyBootstrap:
+    """Marginal diagnostic case-bootstrap intervals; disabled by default."""
+
+    resamples: int = 0
+    confidence: float = 0.95
+    seed: int = 0
+    minimum_cases: int = 20
+    statistics: tuple[str, ...] = ("raw_agreement", "fleiss_kappa", "nominal_alpha")
+
+    def __post_init__(self) -> None:
+        integer(self.resamples, "resamples", 0, 100000)
+        integer(self.seed, "seed", 0, 2**32 - 1)
+        integer(self.minimum_cases, "minimum_cases", 2, 100000)
+        confidence = finite(self.confidence, "confidence", 0, 1)
+        if confidence in (0, 1):
+            raise ValueError("confidence must be strictly between zero and one")
+        object.__setattr__(self, "confidence", confidence)
+        statistics = names(self.statistics, "bootstrap statistics", minimum=1, maximum=4)
+        if set(statistics) - set(STATISTICS):
+            raise ValueError("unknown bootstrap statistic")
+        object.__setattr__(self, "statistics", statistics)
+
+
+@dataclass(frozen=True)
+class ConsistencyPlan:
+    """Freeze label vocabulary, repeat panel and reporting policy before analysis."""
+
+    score_name: str
+    raters: tuple[str, ...]
+    categories: tuple[str, ...]
+    cases: tuple[str, ...] | None = None
+    groups: tuple[ConsistencyGroup, ...] = ()
+    gates: tuple[ConsistencyGate, ...] = ()
+    minimum_consensus_ratings: int = 2
+    detail_limit: int = 100
+    bootstrap: ConsistencyBootstrap = field(default_factory=ConsistencyBootstrap)
+
+    def __post_init__(self) -> None:
+        identifier(self.score_name, "score_name")
+        object.__setattr__(self, "raters", names(self.raters, "raters", minimum=2, maximum=1000))
+        object.__setattr__(self, "categories", names(self.categories, "categories", minimum=2, maximum=1000))
+        if self.cases is not None:
+            object.__setattr__(self, "cases", names(self.cases, "cases"))
+        for key, cls in (("groups", ConsistencyGroup), ("gates", ConsistencyGate)):
+            values = getattr(self, key)
+            if not isinstance(values, (list, tuple)) or len(values) > 1000:
+                raise ValueError(f"{key} must be a bounded sequence")
+            if not all(isinstance(value, cls) for value in values):
+                raise ValueError(f"invalid {key} object")
+            object.__setattr__(self, key, tuple(values))
+        if len({group.name for group in self.groups}) != len(self.groups):
+            raise ValueError("duplicate group name")
+        object.__setattr__(self, "groups", tuple(sorted(self.groups, key=lambda group: group.name.encode("utf-8"))))
+        known_groups = {"overall", *(group.name for group in self.groups)}
+        if any(gate.group not in known_groups for gate in self.gates):
+            raise ValueError("gate references an unknown group")
+        gate_keys = [(gate.group, gate.statistic) for gate in self.gates]
+        if len(set(gate_keys)) != len(gate_keys):
+            raise ValueError("duplicate gate statistic/group")
+        object.__setattr__(
+            self, "gates", tuple(sorted(self.gates, key=lambda gate: (gate.group.encode("utf-8"), gate.statistic)))
+        )
+        integer(self.minimum_consensus_ratings, "minimum_consensus_ratings", 2, len(self.raters))
+        integer(self.detail_limit, "detail_limit", 0, 100000)
+        if not isinstance(self.bootstrap, ConsistencyBootstrap):
+            raise ValueError("bootstrap must be a ConsistencyBootstrap object")
+
+    def check_limits(self, limits: ConsistencyLimits) -> None:
+        """Check fixed dimensions before consuming observations."""
+        for actual, limit, name in (
+            (len(self.raters), limits.max_raters, "raters"),
+            (len(self.categories), limits.max_categories, "categories"),
+            (len(self.cases or ()), limits.max_cases, "cases"),
+            (len(self.groups), limits.max_groups, "groups"),
+            (self.detail_limit, limits.max_details, "details"),
+        ):
+            if actual > limit:
+                raise ValueError(f"consistency {name} limit exceeded")
+        if len(self.cases or ()) * len(self.raters) > limits.max_cells:
+            raise ValueError("consistency cell limit exceeded")
+        if (len(self.cases or ()) + sum(len(group.cases) for group in self.groups)) * len(
+            self.raters
+        ) > limits.max_group_cells:
+            raise ValueError("consistency group cell limit exceeded")
+        if self.selected_bytes() > limits.max_bytes:
+            raise ValueError("consistency selected byte limit exceeded")
+
+    def selected_bytes(self) -> int:
+        """Include declared identities in the direct API's selected-data budget."""
+        total = len(self.score_name.encode("utf-8"))
+        for values in (self.raters, self.categories, self.cases or ()):
+            total += sum(len(value.encode("utf-8")) for value in values)
+        for group in self.groups:
+            total += len(group.name.encode("utf-8"))
+            total += sum(len(value.encode("utf-8")) for value in group.cases)
+        return total
diff --git a/py/autoevals/consistency/reporting.py b/py/autoevals/consistency/reporting.py
new file mode 100644
index 0000000..b71ef7d
--- /dev/null
+++ b/py/autoevals/consistency/reporting.py
@@ -0,0 +1,150 @@
+"""Human-readable and native Score projections with explicit interpretation."""
+
+import html
+from typing import Any
+
+from ..score import Score
+
+
+def safe_text(value: str) -> str:
+    """Escape terminal controls, bidirectional controls and Markdown table syntax."""
+    clean = "".join(
+        (
+            f"\\u{ord(char):04x}"
+            if ord(char) < 32
+            or 127 <= ord(char) <= 159
+            or 0x202A <= ord(char) <= 0x202E
+            or 0x2066 <= ord(char) <= 0x2069
+            else char
+        )
+        for char in value
+    )
+    literal = "".join("\\" + char if char in "\\`*_{}[]()#+-.!|" else char for char in clean)
+    return html.escape(literal, quote=True)
+
+
+def _number(value: float | None) -> str:
+    return "undefined" if value is None else f"{value:.6g}"
+
+
+def render_consistency_markdown(report: dict[str, Any]) -> str:
+    """Render bounded report details without converting missing statistics to zero."""
+    lines = [
+        f"# Judge consistency: {safe_text(report['score_name'])}",
+        "",
+        f"Observed benchmark verdict: **{report['verdict']}**. Cohort: {report['cohort_provenance']}.",
+        "",
+        "Consistency is not correctness. Gates describe the supplied finite benchmark; "
+        "bootstrap intervals are marginal diagnostics and do not determine gate decisions.",
+        "",
+    ]
+    for group in report["groups"]:
+        coverage = group["coverage"]
+        lines.extend(
+            [
+                f"## {safe_text(group['name'])}",
+                "",
+                f"Scored ratings: {coverage['scored_cells']}/{coverage['expected_cells']}; "
+                f"absent: {coverage['absent_cells']}; skipped: {coverage['missing_cells']}; errors: {coverage['error_cells']}.",
+                "",
+                "| Statistic | Value | Supporting cases | Denominator | Weighting / undefined reason |",
+                "| --- | ---: | ---: | ---: | --- |",
+            ]
+        )
+        for name, statistic in group["statistics"].items():
+            lines.append(
+                f"| {safe_text(name)} | {_number(statistic['value'])} | {statistic['support_cases']} | "
+                f"{statistic['denominator_cases']} | {statistic['weighting']} / {statistic['reason'] or 'defined'} |"
+            )
+        lines.extend(
+            [
+                "",
+                "### Reference accuracy",
+                "",
+                "| Rater | Correct / evaluated / labeled | Conditional accuracy | Reference coverage |",
+                "| --- | ---: | ---: | ---: |",
+            ]
+        )
+        reference_rows = [(rater["rater_id"], rater["reference"]) for rater in group["raters"]]
+        reference_rows.append(("consensus", group["consensus"]["reference"]))
+        for name, reference in reference_rows:
+            lines.append(
+                f"| {safe_text(name)} | {reference['correct_cases']} / {reference['evaluated_cases']} / "
+                f"{reference['labeled_cases']} | {_number(reference['accuracy'])} | {_number(reference['coverage'])} |"
+            )
+        lines.extend(
+            [
+                "",
+                "### Pairwise overlap",
+                "",
+                "| Left rater | Right rater | Cohen kappa | Overlap cases | Reason |",
+                "| --- | --- | ---: | ---: | --- |",
+            ]
+        )
+        for pair in group["pairwise"]:
+            statistic = pair["statistic"]
+            lines.append(
+                f"| {safe_text(pair['left_rater'])} | {safe_text(pair['right_rater'])} | "
+                f"{_number(statistic['value'])} | {statistic['support_cases']} | {statistic['reason'] or 'defined'} |"
+            )
+        if group["intervals"]:
+            lines.extend(["", "### Marginal bootstrap diagnostics", ""])
+            for interval in group["intervals"]:
+                lines.append(
+                    f"- {interval['statistic']}: [{_number(interval['lower'])}, {_number(interval['upper'])}]; "
+                    f"valid/attempted={interval['valid']}/{interval['attempted']}; "
+                    f"reason={interval['reason'] or 'defined'}."
+                )
+        lines.extend(
+            [
+                "",
+                f"Consensus decided/abstained: {group['consensus']['decided_cases']}/"
+                f"{group['consensus']['abstained_cases']}. Omitted case details: {group['omitted_cases']}.",
+                "",
+            ]
+        )
+        for case in group["cases"]:
+            label = "abstained" if case["consensus"] is None else safe_text(case["consensus"])
+            lines.append(
+                f"- {safe_text(case['case_id'])}: consensus={label}; "
+                f"agreement={_number(case['raw_agreement'])}; reason={case['consensus_reason'] or 'decided'}."
+            )
+        lines.append("")
+    if report["gates"]:
+        lines.extend(["## Observed requirements", ""])
+        for gate in report["gates"]:
+            lines.append(
+                f"- {safe_text(gate['group'])}/{gate['statistic']}: {gate['verdict']}; "
+                f"value={_number(gate['value'])}, minimum={_number(gate['minimum'])}; {', '.join(gate['reasons'])}."
+            )
+    return "\n".join(lines) + "\n"
+
+
+def consistency_scores(report: dict[str, Any], *, include_report: bool = False) -> list[Score]:
+    """Project bounded agreement/coverage/accuracy into native Scores.
+
+    Signed kappa and alpha coefficients deliberately remain in the full report,
+    because AutoEvals Score values are restricted to[0,1]. Undefined summaries
+    use None, never zero. Full case-level report metadata is explicit opt-in.
+    """
+    if type(include_report) is not bool:
+        raise ValueError("include_report must be boolean")
+    results = []
+    for group in report["groups"]:
+        for suffix, value, support in (
+            ("agreement", group["statistics"]["raw_agreement"]["value"], group["statistics"]["raw_agreement"]),
+            ("coverage", group["coverage"]["rating_coverage"], group["coverage"]),
+            ("consensus_accuracy", group["consensus"]["reference"]["accuracy"], group["consensus"]["reference"]),
+        ):
+            metadata = {
+                "cohort_provenance": report["cohort_provenance"],
+                "support": dict(support),
+                "interpretation": "observed_finite_benchmark",
+            }
+            if include_report:
+                # Detach metadata rather than exposing the caller's mutable report.
+                import copy
+
+                metadata["report"] = copy.deepcopy(report)
+            results.append(Score(f"{report['score_name']}/{group['name']}/{suffix}", value, metadata))
+    return results
diff --git a/py/autoevals/consistency/serialization.py b/py/autoevals/consistency/serialization.py
new file mode 100644
index 0000000..9cc91cc
--- /dev/null
+++ b/py/autoevals/consistency/serialization.py
@@ -0,0 +1,163 @@
+"""Bounded non-executable input bundles and versioned JSON reports."""
+
+import json
+import math
+from collections.abc import Iterable, Mapping
+from dataclasses import asdict
+from typing import Any
+
+from .analysis import analyze_consistency
+from .config import ConsistencyBootstrap, ConsistencyGate, ConsistencyGroup, ConsistencyLimits, ConsistencyPlan, Rating
+
+
+def _object(value: Any, allowed: set[str], required: set[str], name: str) -> dict:
+    if not isinstance(value, dict):
+        raise ValueError(f"{name} must be an object")
+    if set(value) - allowed:
+        raise ValueError(f"{name} contains unknown fields")
+    if required - set(value):
+        raise ValueError(f"{name} is missing required fields")
+    return dict(value)
+
+
+def _list(value: Any, maximum: int, name: str) -> list:
+    if not isinstance(value, list) or len(value) > maximum:
+        raise ValueError(f"{name} must be a bounded array")
+    return value
+
+
+def _unique_object(entries: list[tuple[str, Any]]) -> dict:
+    value = {}
+    for key, item in entries:
+        if key in value:
+            raise ValueError("duplicate JSON key")
+        value[key] = item
+    return value
+
+
+def _constant(value: str) -> None:
+    raise ValueError("nonfinite JSON constants are not supported")
+
+
+def _structure(value: Any, limits: ConsistencyLimits) -> None:
+    stack = [(value, 0)]
+    count = 0
+    while stack:
+        item, depth = stack.pop()
+        count += 1
+        if count > limits.max_json_nodes or depth > limits.max_json_depth:
+            raise ValueError("consistency JSON structure limit exceeded")
+        if isinstance(item, dict):
+            stack.extend((key, depth + 1) for key in item)
+            stack.extend((child, depth + 1) for child in item.values())
+        elif isinstance(item, (tuple, list)):
+            stack.extend((child, depth + 1) for child in item)
+        elif type(item) is float and not math.isfinite(item):
+            raise ValueError("nonfinite JSON numbers are not supported")
+        elif isinstance(item, str):
+            try:
+                size = len(item.encode("utf-8"))
+            except UnicodeError as exc:
+                raise ValueError("invalid Unicode in JSON") from exc
+            if size > 16384:
+                raise ValueError("consistency JSON string limit exceeded")
+
+
+def plan_from_dict(value: Any, *, limits: ConsistencyLimits | None = None) -> ConsistencyPlan:
+    """Read only declared plan fields; no inherited permissive model decoder."""
+    limits = limits or ConsistencyLimits()
+    data = _object(value, set(ConsistencyPlan.__dataclass_fields__), {"score_name", "raters", "categories"}, "plan")
+    groups = _list(data.get("groups", []), limits.max_groups, "groups")
+    data["groups"] = tuple(
+        ConsistencyGroup(**_object(group, {"name", "cases"}, {"name", "cases"}, "group")) for group in groups
+    )
+    gates = _list(data.get("gates", []), (limits.max_groups + 1) * 4, "gates")
+    data["gates"] = tuple(
+        ConsistencyGate(**_object(gate, set(ConsistencyGate.__dataclass_fields__), {"statistic", "minimum"}, "gate"))
+        for gate in gates
+    )
+    bootstrap = _object(data.get("bootstrap", {}), set(ConsistencyBootstrap.__dataclass_fields__), set(), "bootstrap")
+    data["bootstrap"] = ConsistencyBootstrap(**bootstrap)
+    plan = ConsistencyPlan(**data)
+    plan.check_limits(limits)
+    return plan
+
+
+def analyze_consistency_json(content: str, *, limits: ConsistencyLimits | None = None) -> dict[str, Any]:
+    """Read a strict portable input bundle and recompute evidence locally."""
+    limits = limits or ConsistencyLimits()
+    if type(content) is not str:
+        raise ValueError("input JSON must be a string")
+    try:
+        size = len(content.encode("utf-8"))
+    except UnicodeError as exc:
+        raise ValueError("invalid Unicode in JSON") from exc
+    if size > limits.max_bytes:
+        raise ValueError("consistency input byte limit exceeded")
+    try:
+        value = json.loads(content, object_pairs_hook=_unique_object, parse_constant=_constant)
+    except (json.JSONDecodeError, RecursionError) as exc:
+        raise ValueError("invalid consistency JSON") from exc
+    _structure(value, limits)
+    data = _object(
+        value,
+        {"format", "version", "plan", "ratings", "references"},
+        {"format", "version", "plan", "ratings"},
+        "input bundle",
+    )
+    if data["format"] != "autoevals.consistency.input" or type(data["version"]) is not int or data["version"] != 1:
+        raise ValueError("unsupported consistency input format/version")
+    plan = plan_from_dict(data["plan"], limits=limits)
+    ratings = tuple(
+        Rating(**_object(row, set(Rating.__dataclass_fields__), {"case_id", "rater_id"}, "rating"))
+        for row in _list(data["ratings"], limits.max_ratings, "ratings")
+    )
+    return analyze_consistency(ratings, plan=plan, references=data.get("references"), limits=limits)
+
+
+def _encode(value: Any, limits: ConsistencyLimits) -> str:
+    _structure(value, limits)
+    try:
+        content = json.dumps(value, ensure_ascii=False, allow_nan=False, sort_keys=True, separators=(",", ":"))
+    except (TypeError, ValueError) as exc:
+        raise ValueError("consistency data must be finite JSON") from exc
+    if len(content.encode("utf-8")) > limits.max_bytes:
+        raise ValueError("consistency output byte limit exceeded")
+    return content
+
+
+def input_to_json(
+    ratings: Iterable[Rating],
+    *,
+    plan: ConsistencyPlan,
+    references: Mapping[str, str] | None = None,
+    limits: ConsistencyLimits | None = None,
+) -> str:
+    """Snapshot selected observations into a reusable input bundle, without metadata."""
+    limits = limits or ConsistencyLimits()
+    collected = []
+    for index, rating in enumerate(ratings):
+        if index >= limits.max_ratings:
+            raise ValueError("consistency rating limit exceeded")
+        collected.append(rating)
+    # Validate without performing optional resampling or report construction.
+    from .alignment import align
+
+    align(collected, plan, references, limits)
+    return _encode(
+        {
+            "format": "autoevals.consistency.input",
+            "version": 1,
+            "plan": asdict(plan),
+            "ratings": [asdict(rating) for rating in collected],
+            "references": dict(references or {}),
+        },
+        limits,
+    )
+
+
+def report_to_json(report: dict[str, Any], *, limits: ConsistencyLimits | None = None) -> str:
+    """Serialize a report as bounded finite JSON; this is not an authenticity proof."""
+    if report.get("format") != "autoevals.consistency.report" or report.get("version") != 1:
+        raise ValueError("unsupported consistency report")
+    return _encode(report, limits or ConsistencyLimits())
diff --git a/py/autoevals/consistency/statistics.py b/py/autoevals/consistency/statistics.py
new file mode 100644
index 0000000..79acbf0
--- /dev/null
+++ b/py/autoevals/consistency/statistics.py
@@ -0,0 +1,187 @@
+"""Nominal agreement coefficients with their own support and chance models."""
+
+from collections import Counter
+from dataclasses import asdict, dataclass
+from typing import Iterable, Sequence
+
+from .alignment import PanelCase
+
+
+def stable_sum(values: Iterable[float]) -> float:
+    """Explicit Neumaier accumulation, identical to the TypeScript algorithm.
+
+    Python's built-in float sum changed in 3.12. Specify the operations here so
+    supported runtimes make the same decision at a configured gate boundary.
+    """
+    result, correction = 0.0, 0.0
+    for value in values:
+        next_value = result + value
+        if abs(result) >= abs(value):
+            correction += (result - next_value) + value
+        else:
+            correction += (value - next_value) + result
+        result = next_value
+    return result + correction
+
+
+@dataclass(frozen=True)
+class Statistic:
+    """Undefined statistics retain a reason and never impersonate perfect agreement."""
+
+    value: float | None
+    support_cases: int
+    support_ratings: int
+    denominator_cases: int
+    observed_agreement: float | None
+    expected_agreement: float | None
+    weighting: str
+    reason: str | None = None
+
+    def to_dict(self) -> dict:
+        return asdict(self)
+
+
+def category_counts(case: PanelCase) -> Counter[str]:
+    """Count only available categorical observations, never missing slots."""
+    return Counter(category for category in case.categories if category is not None)
+
+
+def pair_agreement(counts: Counter[str]) -> float | None:
+    """Fraction of equal unordered pairs within one case."""
+    total = sum(counts.values())
+    if total < 2:
+        return None
+    return sum(count * (count - 1) for count in counts.values()) / (total * (total - 1))
+
+
+def raw_agreement(cases: Sequence[PanelCase]) -> Statistic:
+    """Equal-case mean of available-pair agreement; no chance correction."""
+    eligible = [(category_counts(case), case) for case in cases]
+    eligible = [(counts, case) for counts, case in eligible if sum(counts.values()) >= 2]
+    if not eligible:
+        return Statistic(None, 0, 0, len(cases), None, None, "equal_pairable_cases", "no_pairable_cases")
+    agreement = stable_sum(pair_agreement(counts) or 0 for counts, _ in eligible) / len(eligible)
+    return Statistic(
+        agreement,
+        len(eligible),
+        sum(sum(counts.values()) for counts, _ in eligible),
+        len(cases),
+        agreement,
+        None,
+        "equal_pairable_cases",
+    )
+
+
+def fleiss_kappa(cases: Sequence[PanelCase], rater_count: int) -> Statistic:
+    """Pooled-marginal Fleiss coefficient on complete fixed-size panels only.
+
+    This describes repeated categorical coding. Interpreting it as reliability
+    over randomly sampled raters requires that study design; named persistent
+    judges additionally have pairwise Cohen statistics with separate marginals.
+    """
+    complete = [case for case in cases if all(category is not None for category in case.categories)]
+    if not complete:
+        return Statistic(None, 0, 0, len(cases), None, None, "complete_panels", "no_complete_panels")
+    pooled: Counter[str] = Counter()
+    agreements = []
+    for case in complete:
+        counts = category_counts(case)
+        pooled.update(counts)
+        agreements.append(pair_agreement(counts) or 0)
+    count = len(complete) * rater_count
+    observed = stable_sum(agreements) / len(complete)
+    expected = stable_sum((value / count) * (value / count) for value in pooled.values())
+    denominator = 1 - expected
+    value = None if denominator <= 0 else (observed - expected) / denominator
+    return Statistic(
+        value,
+        len(complete),
+        count,
+        len(cases),
+        observed,
+        expected,
+        "complete_panels",
+        "zero_expected_disagreement" if value is None else None,
+    )
+
+
+def nominal_alpha(cases: Sequence[PanelCase]) -> Statistic:
+    """Krippendorff nominal alpha with pairable-observation weighting.
+
+    A unit with m observations contributes n_c(n_k-delta_ck)/(m-1) to
+    coincidence counts. Summing disagreement directly avoids a vocabulary²
+    matrix. Singletons contribute neither observed nor expected coincidences.
+    The finite-sample expected disagreement denominator is N(N-1), not N².
+    """
+    pooled: Counter[str] = Counter()
+    disagreements = []
+    support = 0
+    for case in cases:
+        counts = category_counts(case)
+        count = sum(counts.values())
+        if count < 2:
+            continue
+        support += 1
+        pooled.update(counts)
+        disagreements.append((count * count - sum(value * value for value in counts.values())) / (count - 1))
+    total = sum(pooled.values())
+    if not support:
+        return Statistic(None, 0, 0, len(cases), None, None, "pairable_observations", "no_pairable_cases")
+    observed_disagreement = stable_sum(disagreements) / total
+    expected_disagreement = (total * total - sum(value * value for value in pooled.values())) / (total * (total - 1))
+    value = None if expected_disagreement <= 0 else 1 - observed_disagreement / expected_disagreement
+    return Statistic(
+        value,
+        support,
+        total,
+        len(cases),
+        1 - observed_disagreement,
+        1 - expected_disagreement,
+        "pairable_observations",
+        "zero_expected_disagreement" if value is None else None,
+    )
+
+
+def cohen_kappa(
+    cases: Sequence[PanelCase], left: int, right: int
+) -> tuple[Statistic, tuple[tuple[str, str, int], ...]]:
+    """Cohen kappa and sparse left-row/right-column confusion cells.
+
+    Both marginals use the same overlap cohort. A constant rater compared with
+    a varying rater remains defined; only zero expected disagreement is singular.
+    """
+    matrix: Counter[tuple[str, str]] = Counter()
+    left_counts: Counter[str] = Counter()
+    right_counts: Counter[str] = Counter()
+    for case in cases:
+        a, b = case.categories[left], case.categories[right]
+        if a is None or b is None:
+            continue
+        matrix[(a, b)] += 1
+        left_counts[a] += 1
+        right_counts[b] += 1
+    total = sum(matrix.values())
+    cells = tuple(
+        (a, b, count)
+        for (a, b), count in sorted(
+            matrix.items(), key=lambda item: (item[0][0].encode("utf-8"), item[0][1].encode("utf-8"))
+        )
+    )
+    if not total:
+        return Statistic(None, 0, 0, len(cases), None, None, "pairwise_overlap", "no_overlap"), cells
+    observed = sum(count for (a, b), count in matrix.items() if a == b) / total
+    expected = sum(count * right_counts[category] for category, count in left_counts.items()) / (total * total)
+    value = None if expected >= 1 else (observed - expected) / (1 - expected)
+    return (
+        Statistic(
+            value,
+            total,
+            2 * total,
+            len(cases),
+            observed,
+            expected,
+            "pairwise_overlap",
+            "zero_expected_disagreement" if value is None else None,
+        ),
+        cells,
+    )
diff --git a/py/autoevals/test_consistency.py b/py/autoevals/test_consistency.py
new file mode 100644
index 0000000..7563f2b
--- /dev/null
+++ b/py/autoevals/test_consistency.py
@@ -0,0 +1,54 @@
+import math
+
+import pytest
+
+from autoevals import Score
+from autoevals.consistency import (
+    ConsistencyPlan,
+    Rating,
+    ScoreMapping,
+    analyze_consistency,
+    rating_from_score,
+)
+
+
+def plan(**kwargs):
+    return ConsistencyPlan(score_name="judge", raters=("a", "b", "c"), categories=("no", "yes"), **kwargs)
+
+
+def test_missing_manifest_cases_and_cells_are_retained():
+    result = analyze_consistency([Rating("one", "a", "yes")], plan=plan(cases=("one", "two")))
+    coverage = result["groups"][0]["coverage"]
+    assert coverage["expected_cells"] == 6
+    assert coverage["absent_cells"] == 5
+    assert coverage["scored_cells"] == 1
+    assert result["groups"][0]["cases"][0]["consensus"] is None
+
+
+def test_duplicate_cells_are_invalid_even_when_identical():
+    with pytest.raises(ValueError, match="duplicate"):
+        analyze_consistency([Rating("one", "a", "yes")] * 2, plan=plan())
+
+
+def test_score_adapter_selects_explicit_exact_values_and_omits_metadata():
+    mapping = ScoreMapping("judge", ((0, "no"), (1, "yes")))
+    rating = rating_from_score("one", "a", Score("judge", 1, {"secret": object()}), mapping=mapping)
+    assert rating.category == "yes"
+    assert "secret" not in repr(rating)
+    assert rating_from_score("one", "a", Score("judge", None), mapping=mapping).status == "missing"
+    with pytest.raises(ValueError, match="mapped"):
+        rating_from_score("one", "a", Score("judge", 0.5), mapping=mapping)
+
+
+@pytest.mark.parametrize("value", [math.nan, math.inf, True, "1", -1, 2])
+def test_score_adapter_rejects_invalid_numeric_shapes(value):
+    mapping = ScoreMapping("judge", ((0, "no"), (1, "yes")))
+    with pytest.raises(ValueError):
+        rating_from_score("one", "a", {"name": "judge", "score": value}, mapping=mapping)
+
+
+def test_unknown_rater_or_category_rejects():
+    with pytest.raises(ValueError, match="rater"):
+        analyze_consistency([Rating("one", "other", "yes")], plan=plan())
+    with pytest.raises(ValueError, match="category"):
+        analyze_consistency([Rating("one", "a", "unknown")], plan=plan())
diff --git a/py/autoevals/test_consistency_fixtures.py b/py/autoevals/test_consistency_fixtures.py
new file mode 100644
index 0000000..f32fadf
--- /dev/null
+++ b/py/autoevals/test_consistency_fixtures.py
@@ -0,0 +1,29 @@
+import json
+from pathlib import Path
+
+import pytest
+
+from autoevals import analyze_consistency_json
+
+STUDIES = json.loads((Path(__file__).parents[2] / "fixtures/consistency/studies.json").read_text())
+
+
+def subset(actual, expected):
+    if type(expected) in (float, int):
+        assert actual == pytest.approx(expected, abs=1e-12)
+    elif isinstance(expected, list):
+        for index, value in enumerate(expected):
+            subset(actual[index], value)
+    elif isinstance(expected, dict):
+        for key, value in expected.items():
+            subset(actual[key], value)
+    else:
+        assert actual == expected
+
+
+@pytest.mark.parametrize("study", STUDIES, ids=lambda study: study["name"])
+def test_shared_studies(study):
+    report = analyze_consistency_json(json.dumps(study["input"]))
+    subset(report, study["expected"])
+    reversed_input = {**study["input"], "ratings": list(reversed(study["input"]["ratings"]))}
+    assert analyze_consistency_json(json.dumps(reversed_input)) == report
diff --git a/py/autoevals/test_consistency_io.py b/py/autoevals/test_consistency_io.py
new file mode 100644
index 0000000..a30725f
--- /dev/null
+++ b/py/autoevals/test_consistency_io.py
@@ -0,0 +1,56 @@
+import json
+
+import pytest
+
+from autoevals import Score
+from autoevals.consistency import ConsistencyPlan, Rating, analyze_consistency
+from autoevals.consistency.reporting import consistency_scores, render_consistency_markdown
+from autoevals.consistency.serialization import analyze_consistency_json, input_to_json, report_to_json
+
+
+def test_input_bundle_roundtrip_and_strict_schema():
+    plan = ConsistencyPlan("judge", ("a", "b"), ("no", "yes"))
+    ratings = [Rating("x", "a", "yes"), Rating("x", "b", "no")]
+    encoded = input_to_json(ratings, plan=plan, references={"x": "yes"})
+    expected = analyze_consistency(ratings, plan=plan, references={"x": "yes"})
+    assert analyze_consistency_json(encoded) == expected
+    assert json.loads(report_to_json(expected)) == expected
+    payload = json.loads(encoded)
+    payload["plan"]["unknown"] = 1
+    with pytest.raises(ValueError, match="unknown"):
+        analyze_consistency_json(json.dumps(payload))
+
+
+@pytest.mark.parametrize("content", ['{"version":1,"version":1}', '{"x":NaN}', '{"x":Infinity}'])
+def test_duplicate_keys_and_nonfinite_json_constants_are_invalid(content):
+    with pytest.raises(ValueError):
+        analyze_consistency_json(content)
+
+
+def test_native_summary_does_not_put_signed_kappa_in_score_or_fake_missing_zero():
+    plan = ConsistencyPlan("judge", ("a", "b"), ("no", "yes"))
+    ratings = [Rating("x", "a", "yes"), Rating("x", "b", "no"), Rating("y", "a", "no"), Rating("y", "b", "yes")]
+    report = analyze_consistency(ratings, plan=plan)
+    scores = consistency_scores(report)
+    assert all(isinstance(score, Score) for score in scores)
+    assert {score.name for score in scores} == {
+        "judge/overall/agreement",
+        "judge/overall/coverage",
+        "judge/overall/consensus_accuracy",
+    }
+    assert scores[-1].score is None
+    assert all("report" not in score.metadata for score in scores)
+
+
+def test_markdown_escapes_untrusted_labels_and_has_no_ansi_controls():
+    plan = ConsistencyPlan("judge|<script>\x1b[31m", ("a", "b"), ("no", "yes"))
+    text = render_consistency_markdown(analyze_consistency([], plan=plan))
+    assert "<script>" not in text
+    assert "\x1b" not in text
+    assert "judge\\|&lt;script&gt;" in text
+
+
+def test_markdown_labels_cannot_load_remote_images_or_override_escapes():
+    plan = ConsistencyPlan(r"![external](https://example.com/track)\*", ("a", "b"), ("no", "yes"))
+    text = render_consistency_markdown(analyze_consistency([], plan=plan))
+    assert r"\!\[external\]\(https://example\.com/track\)\\\*" in text
diff --git a/py/autoevals/test_consistency_limits.py b/py/autoevals/test_consistency_limits.py
new file mode 100644
index 0000000..ad83a35
--- /dev/null
+++ b/py/autoevals/test_consistency_limits.py
@@ -0,0 +1,33 @@
+import pytest
+
+from autoevals.consistency import ConsistencyGroup, ConsistencyLimits, ConsistencyPlan, Rating, analyze_consistency
+
+
+def test_declared_identity_bytes_count_even_when_no_scores_arrive():
+    plan = ConsistencyPlan("judge", ("a", "b"), ("no", "yes"), cases=("long-case-identifier",))
+    with pytest.raises(ValueError, match="byte"):
+        analyze_consistency([], plan=plan, limits=ConsistencyLimits(max_bytes=20))
+
+
+def test_empty_groups_still_charge_pairwise_report_work():
+    plan = ConsistencyPlan("j", ("a", "b"), ("n", "y"), cases=(), groups=(ConsistencyGroup("empty", ()),))
+    with pytest.raises(ValueError, match="pairwise"):
+        analyze_consistency([], plan=plan, limits=ConsistencyLimits(max_pairwise_work=1))
+
+
+def test_reference_bytes_share_the_selected_evidence_budget():
+    plan = ConsistencyPlan("j", ("a", "b"), ("n", "y"), cases=("x",))
+    # Plan identifiers total six bytes; reference identifiers add two.
+    with pytest.raises(ValueError, match="byte"):
+        analyze_consistency([], plan=plan, references={"x": "y"}, limits=ConsistencyLimits(max_bytes=7))
+
+
+def test_invalid_group_work_rejects_before_consuming_observations():
+    plan = ConsistencyPlan("j", ("a", "b"), ("n", "y"), groups=(ConsistencyGroup("slice", ("x", "y")),))
+
+    def ratings():
+        raise AssertionError("observations should not be consumed")
+        yield Rating("x", "a", "y")
+
+    with pytest.raises(ValueError, match="group cell"):
+        analyze_consistency(ratings(), plan=plan, limits=ConsistencyLimits(max_group_cells=3))
diff --git a/py/autoevals/test_consistency_statistics.py b/py/autoevals/test_consistency_statistics.py
new file mode 100644
index 0000000..9d04dd6
--- /dev/null
+++ b/py/autoevals/test_consistency_statistics.py
@@ -0,0 +1,142 @@
+import random
+
+import pytest
+
+from autoevals.consistency import (
+    ConsistencyBootstrap,
+    ConsistencyGate,
+    ConsistencyGroup,
+    ConsistencyLimits,
+    ConsistencyPlan,
+    Rating,
+    analyze_consistency,
+)
+from autoevals.consistency.alignment import PanelCase
+from autoevals.consistency.bootstrap import CaseRandom
+from autoevals.consistency.statistics import cohen_kappa, nominal_alpha
+
+
+def case(values):
+    return PanelCase(
+        "case", tuple(values), tuple("scored" if value is not None else "absent" for value in values), None
+    )
+
+
+def test_published_nominal_alpha_missing_data_example():
+    # Krippendorff (2013), Computing Alpha-Reliability, pp4-5.
+    rows = [
+        [1, 2, 3, 3, 2, 1, 4, 1, 2, None, None, None],
+        [1, 2, 3, 3, 2, 2, 4, 1, 2, 5, None, 3],
+        [None, 3, 3, 3, 2, 3, 4, 2, 2, 5, 1, None],
+        [1, 2, 3, 3, 2, 4, 4, 1, 2, 5, 1, None],
+    ]
+    cases = [case([str(value) if value is not None else None for value in column]) for column in zip(*rows)]
+    result = nominal_alpha(cases)
+    assert result.value == pytest.approx(0.7434210526315789)
+    assert result.support_cases == 11
+    assert result.support_ratings == 40  # singleton's rating is excluded
+    assert result.observed_agreement == pytest.approx(0.8)
+    assert nominal_alpha(cases[:-1]).value == result.value
+
+
+def test_constant_varying_cohen_is_defined_but_both_constant_is_not():
+    statistic, confusion = cohen_kappa([case(["yes", "yes"]), case(["yes", "no"])], 0, 1)
+    assert statistic.value == 0
+    assert confusion == (("yes", "no", 1), ("yes", "yes", 1))
+    statistic, _ = cohen_kappa([case(["yes", "yes"])], 0, 1)
+    assert statistic.value is None
+    assert statistic.reason == "zero_expected_disagreement"
+
+
+def test_systematic_disagreement_remains_negative():
+    statistic, _ = cohen_kappa([case(["yes", "no"]), case(["no", "yes"])], 0, 1)
+    assert statistic.value == -1
+    assert nominal_alpha([case(["yes", "no"]), case(["no", "yes"])]).value == -0.5
+
+
+def test_missingness_support_and_consensus_use_declared_panel():
+    plan = ConsistencyPlan("criterion", ("a", "b", "c", "d", "e"), ("no", "yes"), cases=("x", "y"))
+    ratings = [
+        Rating("x", "a", "yes"),
+        Rating("x", "b", "yes"),
+        Rating("y", "a", "yes"),
+        Rating("y", "b", "yes"),
+        Rating("y", "c", "yes"),
+    ]
+    result = analyze_consistency(ratings, plan=plan, references={"x": "yes", "y": "yes"})["groups"][0]
+    assert result["cases"][0]["consensus"] is None
+    assert result["cases"][1]["consensus"] == "yes"
+    assert result["consensus"]["reference"]["accuracy"] == 1
+    assert result["consensus"]["reference"]["coverage"] == 0.5
+    assert result["statistics"]["fleiss_kappa"]["value"] is None
+
+
+def test_low_reference_coverage_cannot_pass_even_with_perfect_observed_accuracy():
+    plan = ConsistencyPlan(
+        "criterion",
+        ("a", "b", "c"),
+        ("no", "yes"),
+        cases=("x", "y"),
+        gates=(ConsistencyGate("consensus_accuracy", 0.9, minimum_rating_coverage=0.5),),
+    )
+    ratings = [Rating("x", rater, "yes") for rater in plan.raters]
+    report = analyze_consistency(ratings, plan=plan, references={"x": "yes", "y": "yes"})
+    assert report["verdict"] == "inconclusive"
+    assert "insufficient_statistic_coverage" in report["gates"][0]["reasons"]
+
+
+def test_whole_case_resampling_is_seeded_without_global_rng_mutation():
+    plan = ConsistencyPlan(
+        "criterion",
+        ("a", "b"),
+        ("no", "yes"),
+        bootstrap=ConsistencyBootstrap(resamples=1000, confidence=0.9, minimum_cases=2),
+    )
+    ratings = [
+        Rating(str(i), rater, "yes" if (i + j) % 3 else "no") for i in range(30) for j, rater in enumerate(plan.raters)
+    ]
+    state = random.getstate()
+    report = analyze_consistency(ratings, plan=plan)
+    assert random.getstate() == state
+    assert analyze_consistency(list(reversed(ratings)), plan=plan) == report
+    intervals = report["groups"][0]["intervals"]
+    assert all(item["attempted"] == 1000 and item["valid"] == 1000 for item in intervals)
+    assert all(item["lower"] < item["upper"] for item in intervals)
+
+
+def test_undefined_bootstrap_draws_do_not_become_conditional_intervals():
+    plan = ConsistencyPlan(
+        "criterion",
+        ("a", "b"),
+        ("no", "yes"),
+        bootstrap=ConsistencyBootstrap(resamples=1000, confidence=0.9, minimum_cases=2),
+    )
+    ratings = [Rating("x", rater, "yes") for rater in plan.raters] + [
+        Rating("y", rater, "no") for rater in plan.raters
+    ]
+    results = analyze_consistency(ratings, plan=plan)["groups"][0]["intervals"]
+    alpha = next(item for item in results if item["statistic"] == "nominal_alpha")
+    assert alpha["invalid"] > 0
+    assert alpha["lower"] is None
+    assert alpha["reason"] == "undefined_resamples"
+
+
+def test_pairwise_and_bootstrap_resource_checks_precede_sampling():
+    plan = ConsistencyPlan(
+        "criterion", ("a", "b", "c"), ("no", "yes"), cases=("x", "y"), groups=(ConsistencyGroup("slice", ("x", "y")),)
+    )
+    with pytest.raises(ValueError, match="pairwise"):
+        analyze_consistency([], plan=plan, limits=ConsistencyLimits(max_pairwise_work=10))
+
+
+def test_unicode_order_is_utf8_and_invalid_surrogates_reject():
+    plan = ConsistencyPlan("criterion", ("a", "b"), ("no", "yes"), cases=("\U00010000", "\ue000"))
+    assert plan.cases == ("\ue000", "\U00010000")
+    with pytest.raises(ValueError, match="surrogate"):
+        Rating("\ud800", "a", "yes")
+
+
+def test_prng_fixed_vector_for_typescript_parity():
+    rng = CaseRandom(17, "overall")
+    # Independently calculated with JavaScript Math.imul/unsigned shifts.
+    assert [rng.next_u32() for _ in range(5)] == [4291421669, 2916200383, 2632349228, 374880413, 943298360]