Skip to content

9.1 Fills — driving a large materialization slowly

Some materializations are too big for one query’s patience: a 12-month history over many anchors, an open catalog sweep. A fill drives one durably, slowly, and idempotently. An operator starts one with a query (cypher), a per-pass call budget (budgetPerTick) and a label; lists fills with their id, state, ticks, liveCallsTotal, lastError and warning; and cancels one by id, which keeps completed work cached and rolls nothing back. How an operator reaches those three operations is the host’s surface; the reference host offers them under /api/v1/admin/kg/fills.

The engine re-runs the fill’s query on a schedule, each pass under a call budget: at most budgetPerTick producer calls, stopped CLEANLY when spent — everything fetched stands, nothing is recorded for the skipped remainder. Because finished work is cached (and closed periods never expire), each pass advances past everything already done and spends its whole budget on new ground. A pass that completes with ZERO live calls is the finish line: the fill turns DONE.

Guarantees

  • Idempotent and resumable by construction. The fill’s progress IS the caches. Restart the process, cancel and restart the fill, run the same query by hand in between — nothing is done twice and nothing is lost beyond the tick in flight.
  • Rate-limited twice over. The per-tick budget bounds each pass; within a pass, every producer’s own declared cost.rate paces its calls.
  • An error never finishes a fill. A failing pass records its error and stays RUNNING — transient source trouble is retried on the next tick, and the error is visible on the fill.
  • A budget-exhausted ordinary query says so. Outside fills, the same budgeted stop surfaces as a warning naming what was not fetched — a partial result is announced, never passed off as complete.

A fill reaches DONE only if its producers keep what they fetch

DONE is a pass that makes no live call at all, and work already done is skipped on the next pass only while it is still fresh. So a producer whose records expire between passes is re-read every pass: the fill keeps working, it never runs out of live calls to make, and it never turns DONE. Nothing is broken — it has simply become a slow permanent poller. Whether a fill can terminate is decided entirely by its producers’ cache: declarations, and by nothing about the query, the budget or the schedule.

That is a division of labour, not a limitation:

  • A fill converges a COLD materialization of something that does not change once mined: a closed month of history, a catalog, a backfill of records that are what they are. Declare those producers so their records outlive the whole sweep — cache: { kind: immutable } for reference data, or a ttl in days — and the fill climbs to DONE and stops. periods: already does this for you: a closed period is held effectively forever whatever the producer’s own ttl says, which is why a 12-month history fill finishes. Only the open period rides the ordinary TTL — and it is also the reason a periods: fill over a producer with a short TTL does not terminate either, since the open period expires under the sweep while the sweep is still running.
  • Re-assessing something that keeps changing — a live queue, prices, today’s state — is scheduled work, not a fill: a Realm Function on a schedule, or a routine bound to a signal. Those are built to re-run for ever and have no finish line to miss. A fill aimed at a live source only imitates one, at a fill’s pacing and without a schedule’s triggers.

A fill started over producers that cannot outlive a sweep is accepted and carries a warning naming them, visible on the fill in the listing above. It is a warning rather than a refusal because a slow permanent poll is sometimes exactly what someone wants; it should be chosen, not stumbled into.

A realm ships the sweep its own producer needs

The calls above are the operator’s door. A realm that declares an expensive producer should not have to hope somebody finds out that fills exist, works out the right query and POSTs it: it declares the sweep as a sweeps: entry on one of its own agents (README, agents/), each naming a view of the realm’s that takes no arguments. The host then runs it as a fill with everything above true of it — budgeted, paced, resumable, cancellable, warned about where it cannot converge.

What a realm gets by declaring one, and what it does not:

  • Installing the realm starts nothing. A realm agent arrives off duty with no sponsor and no signature, so there is no sweep to run. The declaration is visible on the agent’s card — the sweep, the view it fills, the budget it asks for — and nothing is bought until somebody sponsors the agent, signs it and puts it on duty. Those are the same three acts every realm agent already needs; a sweep adds no consent step of its own, and the install is not one of them.
  • It spends the AGENT’S budget and reaches only what the AGENT may reach — the budget and authority.reach its sponsor signed, never the installing operator’s own allowance. An agent whose reach does not name the API behind a producer the sweep plans calls nothing: the pass is refused before it begins.
  • Anybody who can stop the agent stops the sweep. Standing the agent down, retiring it or throwing the world’s kill switch stops it; so does removing the realm, which cancels the fill rather than leaving one retrying a query whose realm is gone. Completed work stays cached, so putting the agent back on duty resumes rather than restarts.
  • A realm update cannot change what an adopted sweep asks. The view’s text is part of what the sponsor signed, so rewriting it shows as an unsigned change and the signed text keeps running until somebody signs again — as a routine’s body does.
  • It is not a schedule. A sweep converges a cold materialization and stops, under exactly the rules above; re-assessing something that keeps changing is a routine on a schedule.
agents/surveyor.yml
name: surveyor
job: Keep the crime history mined, so a question about a street answers from the graph.
sweeps:
- name: crime-history # stable within the agent; the fill is found by it
view: street-crime-sweep # a view of this realm's, taking no arguments
budgetPerTick: 60 # optional; the paid calls one pass may make
Sweep field Required Meaning
name yes Stable id within the agent. Asking for the same sweep twice is one fill, not two.
view yes A saved view this world has, declaring no params. Its body is the query each pass runs.
budgetPerTick no Producer calls, or inline-judgement groups, one pass may buy. Omitted takes the host’s default.

A sweep whose view the world does not have, or whose view takes arguments, is a need on the agent: the agent cannot be put on duty until it is fixed, so the declaration is reported before anything runs rather than failing a pass at a time. A host may cap budgetPerTick, and the number of fills it advances per tick is the host’s, so several sweeps in one world each advance more slowly.