5.15 cypher — a remote graph as a keyed source, and the governance: grammar

A cypher producer joins a FOREIGN property graph — another Neo4j, Memgraph, Amazon Neptune, or any engine speaking Bolt or openCypher-over-HTTP — using an authored read query. It is the property-graph sibling of sparql, in the same language the rest of the system speaks.

- name: officersByName
  kind: cypher
  uri: "bolt://graph.example.org:7687"        # or https://…:8182/openCypher with protocol: http
  query: |
    UNWIND $keys AS key
    MATCH (o:Officer) WHERE o.name = key
    RETURN key AS name, o.countries AS countries
  connection:
    protocol: bolt            # bolt (default) | http
    username: reader
    passwordEnv: GRAPH_PASSWORD   # env var; never a credential in YAML
  governance:
    mode: governed
    expose:
      Officer: { properties: [name, countries] }
    mask: { countries: hash }
    caps: { maxRows: 5000 }

The contract:

One batch, one query per chunk. The whole key batch is bound to $keys (rename via keysParam); the natural shape is UNWIND $keys AS key MATCH … RETURN key AS <echo>, …. Rows must echo the key through the query’s own RETURN (declare recordKeyField on the join).
Structurally read-only. A query containing any write clause — or CALL — is rejected before anything is sent, regardless of engine. Server-side read modes, where the engine has them, are a second layer, never the guarantee.
Rows are the returned columns. A returned node or relationship value flattens to a map of its properties plus _labels / _type; a returned PATH is rejected (return the columns you need instead).

The governance: grammar — declarable on every producer kind, not only the two documented here. It is enforced at the dispatch seam, so a governance: block means the same thing over an API, a CSV, a feed or an index as it does over Postgres. What differs between kinds is where the rules are applied, which the source answers and the realm does not choose:

tier	what it means	kinds
compiled	exposure enters the query; unexposed data never leaves the source	`sql`
filtered	the request is narrowed before the fetch; the source may still send more than is kept
enforced	rows arrive whole and are stripped on receipt — the data crossed the wire	every other kind, `cypher` included

Only a compiled source can honour a row-level where:, because only a generated query has a clause to put it in. Declaring one anywhere else is refused when the realm loads.

One vocabulary regardless of what sits behind the source:

mode: open (default) — everything the query returns is visible. Two rules hold even here: execution is read-only, and the secret reflex — fields whose names look like credentials (password, token, api-key, private-key, ssn, …) are never returned, in any mode.
mode: governed — the default inverts: a label or field not listed under expose: does not exist. A governed query that names an unexposed label is rejected loudly. Flipping a working open configuration to governed changes no query that only touched exposed data. (Note: the key-echo alias must be listed among the exposed properties, or the join cannot form.)
mask: — per-field strategies in both modes: hash (deterministic digest — joinable, not readable), last4, drop.
caps.maxRows — exceeding it truncates AND reports truncation.
Withheld is said, never silent. Every field governance removes is reported in warnings as withheld-by-policy, so an absent field can never read as “no data”. And a governance rule the source cannot enforce (row-level where: on an authored template) is a loud error, never silently ignored — that rule is reserved for sources that generate their own queries.

Cost every consumer should know: each key chunk is one live query against an engine the realm does not control; declare a TTL cache and bound maxKeysPerCall accordingly.

5.15.1 `sql` — governance by construction, the schema miner, and stored procedures

The sql kind (see §5.3) is the first source where every governance rule is enforceable: the statement is generated, so expose decides what is SELECTed (an unexposed column never leaves the database), where: predicates are part of every query, and the caps are a limit clause. The key column is implicitly exposed — it is the identity echo, and the caller already holds every key value.

- name: ordersByCustomerEmail
  kind: sql
  datasource: warehouse          # declared in <realm>/sql/datasources.yml
  table: orders
  keyColumn: customer_email
  governance:
    mode: governed
    expose:
      orders:
        properties: [id, total, placed_at]
        where: "tenant_id = :userId"    # enforceable HERE — the template kinds must reject it
    caps: { maxRows: 2000 }

Scanning a table: scan: instead of keyColumn:. A keyed fetch answers “the rows for these anchors”, which is the right shape for a source that charges per key. A relational table is not that source, and a realm whose every producer needs keys cannot ask the table’s OWN question — the largest donations ever disclosed, the most recent returns — because there is no anchor to name. A sql producer may instead declare scan:, which reads the table ordered and bounded in ONE statement:

- name: donationsRanked
  kind: sql
  datasource: au_donations
  table: donations_made
  echoKeyAs: by                  # every row comes back carrying the pinned value
  scan:
    orderBy: value               # the DEFAULT ordering column
    descending: true
    limit: 200                   # the most rows this door will ever return
  governance:
    mode: governed
    expose:
      donations_made:
        properties: [id, donor_id, recipient_id, value, made_on]

Reached through a virtual join whose anchor is PINNED rather than matched:

MATCH (:DonationsTop {by:'value'})-[:TOP_DONATION]->(d:Donation)
RETURN d.value, d.made_on ORDER BY d.value DESC

Guarantees:

keyColumn and scan are mutually exclusive. A producer answers one question or the other; one that declared both would silently drop a contract. Declaring neither is rejected at load.
The pinned value chooses the ordering column, so one door answers more than one superlative ({by:'value'} and {by:'made_on'} on the same producer). A pin that does not name an EXPOSED column falls back to orderBy — ordering by a column the caller cannot read would make it an oracle over hidden values.
A scan is bounded by declaration. limit is required and capped again by governance.caps.maxRows, whichever is lower. There is no unbounded scan: a door that returns the table is an import, not a question.
echoKeyAs is required, and is what makes a keyless read joinable — every returned row carries the pinned value under that property, which is the join’s recordKeyField.
Exposure, where: predicates and masks apply exactly as they do to a keyed fetch.

Reading a stored procedure: procedure: instead of table:. On PostgreSQL a read-only routine is a FUNCTION, and a keyed fetch over SELECT * FROM fn() already reaches it. SQL Server and Sybase ASE have no such form: EXEC is the only way to invoke any procedure, read or write, and an operations estate is full of reporting procedures that are SELECT wrappers — queue depth right now, connections by host, items completed in the last fifteen minutes. A sql producer may declare procedure:, and the procedure’s result set becomes typed graph rows: joinable, viewable, cacheable, and served through every door the world has.

- name: robotQueueByInstance
  kind: sql
  datasource: lumina_au            # a READ-ONLY datasource: no `readOnly: false` is needed
  echoKeyAs: instance              # every row comes back carrying the pinned key
  procedure:
    name: lumina_au.dbo.sp_database_monitor_lumina_robot_queue_statistics_pending
    args: []                       # ordered; "$key" binds the anchor key, anything else is a literal
  cache: { kind: none }            # "right now" means right now
  governance:
    mode: governed
    expose:
      lumina_au.dbo.sp_database_monitor_lumina_robot_queue_statistics_pending:
        properties: [queue_item_type, queue_item_sub_type, pending_queue_item_count]

Reached through a virtual join whose anchor is PINNED — a datasource instance, a site, a region, whatever the procedure is about:

MATCH (:SybaseInstance {name:'lumina_au'})-[:ROBOT_QUEUE]->(q:RobotQueueDepth)
RETURN q.queue_item_type, q.pending_queue_item_count ORDER BY q.pending_queue_item_count DESC

Guarantees:

Placement is the volatility declaration. A producer is a read path: its rows are materialized for one query and rolled back with it. A procedure declared here runs against a read-only datasource with no opt-in. A procedure that WRITES is a verb, declared in sql/procedures.yml against a datasource that says readOnly: false, and stays gated exactly as before. “Can anything mutate this database?” is still answered by grepping for readOnly: false.
The declaration is made true, not trusted. By default (rollback: true) the call runs inside a transaction that is rolled back unconditionally after the result set is read, so a procedure that writes leaves nothing behind. This matters on engines where a procedure runs with its owner’s rights rather than its caller’s. rollback: false exists for an engine whose procedures refuse to run inside a caller’s transaction (Sybase ASE unchained-mode procedures); with it, the declaration is the only assurance, and the realm README should say so.
One call per key. A procedure argument is a scalar, so there is no batch. Every returned row carries the pinned key under echoKeyAs, which is required and is the join’s recordKeyField. A procedure with no arguments still needs a pinned anchor: the key it echoes is the door the query walked through.
The first result set is the rows. A procedure that produces none reads as no rows. A procedure the datasource cannot run is a diagnostic and no rows, never a silent zero.
procedure, keyColumn and scan are mutually exclusive, and exactly one is required. A procedure read needs no table.
Governance is enforced on receipt, and the tier says so. There is no SELECT list to compile exposure into, so the rows cross the wire whole and unexposed columns are stripped on arrival; expose is keyed by the procedure’s name. A row-level where: predicate has nowhere to go and is REFUSED at load, never silently ignored. Masks and the secret reflex apply as everywhere.
A procedure-backed label’s values are not enumerable. There is no column to list distinct values of; discovery says so rather than guessing.
SQL Server, Azure SQL and Sybase ASE datasources are supported, for procedures and tables alike. A jdbc:sqlserver: or jdbc:sybase: URL selects the T-SQL dialect on its own; dialect: sqlserver or dialect: sybase on the producer names it explicitly.

A join may declare a POLICY instead of one key. Partly implemented — see the implementation-status note at the end of this section for exactly which guarantees hold today. keyField says “match this column”; a policy says how to try, in order, and what to do when the rules disagree:

- anchorLabel: Order
  relationship: PLACED_BY
  keyField: id
  producer: customersById
  policy:
    rules:
      - key: { on: customer_id, to: id }                          # the declared key, via `producer`
      - key: { on: customer_email, to: email, ci: true, confidence: medium,
               producer: customersByEmail }                       # a DIFFERENT door, keyed by email
      - ask: "Which customer placed this order?"                  # a person, when rules disagree
      - none                                                      # no link is a legitimate answer

A key rule may name its own producer, and usually must. A producer is keyed by ONE field — keyArg for a remote op, keyColumn for SQL, keyTemplate for a query string — fixed in its own declaration. So a rule that matches on a different target field needs the door that is keyed by that field: customersById cannot be asked for a customer by email, however the rule is written. Omit producer and the rule uses the join’s, which is right for the rule whose to IS the join’s key and wrong for every other. This mirrors a resolve: chain, where canonicalEmail: { producer: … } names its door the same way and for the same reason.

Guarantees:

Rules are tried IN ORDER and the first that yields wins. A rule that yields nothing does not end the chain — that is what a fallback is for.
Each rule fetches ONCE for every anchor still unresolved, through its own producer, and anchors a rule settles leave the chain. A chain of three rules over a thousand anchors is at most three fetches, never one per anchor.
More than one match is not a match. The rows become the candidates offered to a later ask, and resolution continues rather than picking one.
ask never runs before the rules that could answer without a person, and where nobody can be asked — a scheduled run, an expired question — the chain falls through to the next rule rather than blocking.
Every resolved edge records HOW it was found: the rule that matched and that rule’s confidence, with an ask answer recorded as asserted — a person’s word, not a probability.
none ends the chain with no link, which is an answer and not a failure.
A question outlives the session. Where the run can park, an ask becomes a durable question against that run — answerable minutes later, from another device, by someone who was not the asker. It is not a modal dialog and it does not hold a transaction open.
An answered ask is not asked again. The resolved edge is written with its asserted confidence and the identity it settled, so the next query keys on it directly. A person is asked once per ambiguity, not once per query.
A declined or vetoed identity stays declined. An ask answered “not the same” records a negative assertion, and no later automatic rule may re-link what a person has separated.
A join with no policy: behaves exactly as before: key on keyField, once.

Implementation status, and what a host must do about it. In the reference host the key ladder is implemented: key rules run in order, each through its own producer, each fetching once for the anchors still unresolved, and none ends the chain. A rung goes through the ordinary fetch path, so it inherits caching, cost budgets, diagnostics — and, where the anchor is a spine, the spine’s key normalization.

Honoured. A realm may depend on these:

confidence — stamped on the resolved edge as confidence, alongside matchedBy naming the rung that matched ("email -> customerEmail"). This is what makes WHERE r.confidence = 'high' answerable, so an edge found by a name match never reads like one found by an id.
“More than one match is not a match” — enforced. A rung returning several records for one anchor settles none of them; that anchor falls through to the next rung, where a narrower key may still resolve it. Do NOT design around this by giving a fallback rung a key that is unique on the far side — that workaround was advice for an earlier host and is no longer needed.

Refused, not ignored. The host rejects these at validate/install time, naming the join, per the rule below:

ask — parking a query on a person’s answer is not implemented.
ci: false — a case-SENSITIVE match is not something this engine can offer: the link side lowercases both sides, on every rung. An explicit false is refused rather than silently given the looser comparison it asked not to have. Omitting ci says nothing and is always fine.

This block previously reported confidence and the ambiguity rule as unimplemented after both had shipped. That is the more damaging direction for a status note to be wrong in: an author who believes a safety property is absent designs around one that is already there, and declines provenance they are already entitled to.

The normative rule stands, and is why ask is refused rather than skipped:

A host that cannot honour a declared policy: rule MUST reject the realm at validate/install time, naming the join. It MUST NOT accept the declaration and silently resolve without it.

A realm author who writes a fallback ladder and an ask has said, in the only place the format lets them, that automatic rules are not enough for this join. Quietly resolving it anyway produces exactly the wrong answers the policy was written to prevent — silently, on customer data, with no warning at the point of use. Refusing the realm is loud, immediate, and fixable.

A ladder is not a substitute for a spine. A ladder answers “this join has more than one possible key”. A spine answers “these realms mean the same entity”. If the fallback rungs exist only because two systems spell one key differently, declare the spine (§5.4.1) and the ladder disappears.

Mining a database into a realm. A relational schema already IS a graph — tables are labels, primary keys identities, foreign keys edges. The host can mine a datasource’s metadata into a realm scaffold: one type per table, one sql producer + virtual join per foreign key, with the full column list pre-written into an open-mode expose: block. The scaffold is ordinary realm YAML: inspect it, prune the exposure, flip mode: governed, and no query that touched only exposed data changes.

Stored procedures. A realm may publish stored procedures as TYPED verbs in <realm>/sql/procedures.yml — the declaration IS the allowlist (a procedure the realm does not name does not exist; nothing is ever exposed by introspection), mirroring the GraphQL mutations: gate. Declared args become the verb’s typed signature on the code-mode surface (gateway.<datasource>.<name>); result rows never include credential-shaped columns. Procedures are VERBS, not joins: they are never reachable from virtual cypher.

- name: repriceOrder
  datasource: warehouse
  procedure: reprice
  description: "Reprice an order to a new total."
  args: { orderId: string, amount: number }

Datasources are read-only by default. A datasource declaration that does not say readOnly: false can only ever be SELECTed: every write path — update statements and stored-procedure calls, including DECLARED procedures — is refused, and no procedure verb is even published against it. Whether anything can mutate a database is answered by the declaration file alone, never by auditing call sites: grep the realm for readOnly: false.

5.15 cypher — a remote graph as a keyed source, and the governance: grammar

5.15.1 sql — governance by construction, the schema miner, and stored procedures

5.15.1 `sql` — governance by construction, the schema miner, and stored procedures