Gate Falsification Kit

We found this one in our own house. During an internal review of our CI, we discovered that a parity gate — a check we believed was standing guard over a critical invariant — had been green over nothing. It had passed on every run because it was structurally incapable of failing; the condition it claimed to verify was never actually exercised. Every green tick it had ever produced was worthless, and we had been reading those ticks as evidence for months. The lesson generalizes uncomfortably well: a gate that has only ever passed proves nothing. Passing is what a working gate does, and it is also what a broken one does.

The Gate Falsification Kit is the service and toolkit we built from that discovery: an engagement that treats every gate in your pipeline as a claim to be falsified, not a fixture to be trusted.

What it does

• Falsification drills. For each gate under audit, we plant a violation the gate exists to catch — a real defect of the class it guards against — and observe whether the gate fails. The drills are self-reverting: the planted violation is removed automatically whether or not it is caught, so the exercise leaves no residue in your codebase. A gate that catches its planted violation has demonstrated, not asserted, that it can fail.
• A receipt taxonomy. Every gate receives one of three classifications. BEHAVIORAL: the gate was drilled and observably caught a planted violation — the strongest receipt. STRUCTURAL: the gate’s failure path was verified by inspection of its mechanism, but not exercised end to end. DECLARED-GAP: the gate could not be drilled or verified, and this is recorded as an open gap — never rounded up to a pass.
• The deliverable: a per-gate receipt ledger. The engagement produces a ledger listing every gate, its classification, and the drill evidence behind it — a document that turns “our CI is green” from a mood into a claim with receipts. Aggregate findings across engagements: NOT YET PUBLISHED.
• The toolkit itself. The drill harness ships with the engagement, so falsification becomes a discipline you re-run — because a gate proven able to fail last quarter can be quietly broken by this quarter’s refactor.

What it does not do

It does not certify a gate it could not drill. Gates that resist falsification — too entangled to plant against, too opaque to inspect — are classified DECLARED-GAP and reported as such, even when the client is confident they work. Confidence is what we had in our parity gate. The kit also does not confuse coverage with proof: a BEHAVIORAL receipt attests that the gate caught the violation we planted, not every violation conceivable.

Who it is for

Teams whose CI has been green for months — which is precisely the condition under which this audit finds things. Engineering leads who inherited a pipeline of checks nobody remembers writing, and anyone whose deployment decisions rest on ticks that have never once been observed to fail.