10/15/2026

Proposal: a small corpus of algorithm counterexamples

A design for retaining representative cases without confusing a fixture set with a universal proof.

proposalregression-testsalgorithms

Proposal: a small corpus of algorithm counterexamples

A compact regression corpus could connect each algorithm test to the failure it is meant to detect. This is a proposed authoring convention; no new testing service or generated benchmark is being claimed.

One record per behavior

{
  "id": "diff-one-insertion",
  "purpose": "Keep shared-line numbers stable when one line is added.",
  "left": "a\nc",
  "right": "a\nb\nc",
  "expectedKinds": ["equal", "add", "equal"]
}

Keep a few exact reference cases: empty inputs, one insertion, one deletion, repeated lines and a replacement. Add a new case when it represents a different failure mode rather than only increasing the file count.

How to judge the idea

Run the corpus against the real function that the page imports. Deliberately break one operation and confirm that a relevant case fails. Preserve the expected result as reviewed data, not output regenerated by the same implementation being tested.

A fixture corpus documents observed contracts. It does not prove performance for very large inputs or correctness for every possible string. Benchmarking, fuzzing and accessibility checks answer different questions and should report their own evidence.

Related reading: Reproducible examples.