Skip to content
Embabel Worlds

5.17 Calendar history — periods: on a remote producer

A DATE-ADDRESSABLE source — one whose operation takes a calendar period as an argument (a street-crime API’s date=YYYY-MM, a daily feed’s day=) — holds its own history, so history is a KEY dimension, not something to accumulate. periods: declares it:

- name: crimeMonths
  kind: remote
  operation: streetCrimes
  keyArgs: [lat, lng]
  echoKeyAs: key                    # REQUIRED with periods (records stamp back to the caller's key)
  cost: { rate: "1/second" }        # the fan-out respects the source's declared pace
  periods: { param: date, unit: month, count: 12, lag: 2, stampAs: month }

Each join key fans out into one call per period — count consecutive units (month | day), newest first, stepping lag periods back from now for sources that publish in arrears. Every record is stamped with its period under stampAs — declare that property on the target type, because it is how queries see time:

MATCH (p:Place)-[:HAS_CRIME_MONTH]->(m:CrimeMonth)
WITH p, m ORDER BY m.month
WITH p, collect(m) AS months
RETURN p.name, months[-1].total AS now,
       round(100.0 * (months[-1].total - months[0].total) / months[0].total, 1) AS changePct
ORDER BY changePct DESC

Guarantees

  • The past is free after the first read. A CLOSED period is immutable — June’s recorded rows do not change in September — so its result caches effectively forever (per key, per period). Only the period containing now refreshes on the producer’s ordinary cache: TTL. A cold read pays keys x count calls, paced by cost.rate; every later read pays only the open period.
  • A quiet closed period is an answer, and it is kept. An empty result for a closed period caches as long as a full one; the open period’s empty follows the ordinary negative-TTL rules.
  • A failure caches nothing. “Could not ask” never becomes “asked, nothing there” — the next read retries exactly the failed (key, period) pairs.
  • History does not backfill past the window. count bounds what exists; nothing about a query widens it. Deepening the window is a spec change, and only the ADDED periods are new cost — everything already cached stays paid.
  • periods requires echoKeyAs, and is mutually exclusive with partition: (one fans out calendar periods; the other splits a range-shaped key).

Cost every consumer should know: the fan-out multiplies the anchor set by count. Bound the anchors as usual (maxAnchors declares the join’s appetite), and for a deep window over many anchors, drive the cold fill with a fill (§9.1) instead of one long query. A realm ships that sweep WITH the producer rather than leaving it to be discovered: declare it as a sweeps: entry on one of the realm’s own agents (README, agents/), naming a view of yours that takes no arguments.

A worked example: immutable records behind a date-filterable list. GitHub’s workflow-runs endpoint is not date-addressable by design, but its created filter takes a month (2026-09) or an exact date, and a completed run never changes — so the same endpoint declared twice gives two doors to one collection (realm-github-actions, 2026-09-23):

- name: runsByRepo              # LIVE: newest first, a literal since pushed to `created`
  operation: actions/list-workflow-runs-for-repo
  pushdown: [{ property: created_at, op: GREATER_THAN_OR_EQUAL, argPath: created, clause: ">={value}" }]
  cache: { kind: ttl, seconds: 600 }
- name: runHistoryByRepo        # HISTORY: this month and last, a closed month cached 400 days
  operation: actions/list-workflow-runs-for-repo
  paging: { style: page, param: page, sizeParam: per_page, size: 100, maxPages: 15 }
  periods: { param: created, unit: month, count: 2, lag: 0, stampAs: month }
  echoKeyAs: repository
  cache: { kind: ttl, seconds: 1800 }

unit: month sends created=2026-09. Two months always cover at least the last 31 days. Measured: a cold read of one busy repository is one page per hundred runs in the two months (13 calls, ~25 s); every later read of any window up to 31 days is 0 calls and 150–600 ms, because the history door declares no rule a window could be pushed into, so every view’s read of a period is the same read whatever window the view asks for (§5.13). unit: day was tried first: 31 calls cold for the same coverage, and a bigger refresh surface. “Right now” questions can stay on the live door. State the edge in the realm’s README: a record still changing when its period closed (a run in progress when the month turned, a run re-run in a later month) keeps in history the state it had when that period was last read.