Skip to content
Embabel Worlds

Coverage — completeness is a fact in the graph, not a hope

A mirrored source records what it actually holds, per partition:

(:Coverage:Public {
   source:        'nsw-planning-register',
   partition:     'Inner West Council',
   declaredTotal: 12905,          // what the source says exists
   ingested:      12905,          // what we hold
   completeAsAt:  '2026-07-29T04:10:00Z',
   status:        'COMPLETE'      // COMPLETE | PARTIAL | STALE | FAILED
})

A coverage record describes one world’s mirror. It is kept per world, source and partition, and another world’s walk of a source by the same name never answers for it.

Because the engine already knows which labels a query touches, it can attach the matching coverage to every result through the existing {rows, warnings} envelope — no new plumbing, and no realm has to remember to do it. A street-level question carries “Inner West complete as at 29 Jul”; a statewide aggregate carries “3 of 128 councils ingested — this is not a statewide figure”.

Two rules follow, and they are not the same rule:

  1. Never block a query on coverage. Always qualify the answer. Refusing is brittle and teaches users to distrust the surface; qualifying composes and stays honest.
  2. Withhold a DERIVED STATISTIC when coverage is inadequate. An approval rate computed over whichever partitions happen to be ingested is not a weak figure, it is a wrong one. This is the same discipline as withholding a percentage over a tiny denominator — the denominator here is partitions, not rows.

Coverage is what makes lazy ingestion safe. Without it, every aggregate over a partially-mirrored source is quietly incorrect, and the surface cannot tell. With it, on-first-use ingestion is honest and a full statewide mirror becomes an optimisation rather than a correctness requirement — so build the coverage record BEFORE building any ingestion.