Skip to content
Embabel Worlds

2. The execution model

A Virtual Cypher query never writes anything that outlives it. On every surface that takes a query — asks, views, rule bodies, code, captured-realm reads — a write clause anywhere in it (CREATE, MERGE, SET, REMOVE, DELETE, FOREACH, CALL { } IN TRANSACTIONS, including after a WITH) that touches a stored or fetched type refuses the query before anything is fetched or run, naming the clause. The one write a query may make is to projected types (§7.9): nodes of a projected: true type, and relationships declared on one, created inside the run and gone when it ends — which is how a view can be a step that adds a fact for the next (a judged link such as (:Issue)-[:TRACKED_BY]->). A view whose writes are all projected validates and runs; one that writes anything else is refused at load, with the same rule the run applies. A write clause that appears only inside a string, comment or map key is data and the query runs. A realm persists changes only through its write proposals, never by writing from a query — so nothing a query read from a live source is ever stored by that query. The one surface that writes durably is the annotation surface (§12): a realm and a deployment must both opt into it, it takes its own fixed statement shape, and it writes only declared annotation properties of REAL nodes — never a fetched record.

Every Virtual Cypher query runs the same five conceptual phases, and nothing it materializes outlives the run:

   ┌─ parse + plan ──────────────────────────────────────────────────────────┐
   │                                                                          │
   │  1. PROBE    bind REAL anchors in the host-bound world                  │
   │  2. FETCH    call each producer ONCE with all anchor keys (batched)      │
   │  3. MATERIALIZE  splice fetched records in as transient :Virtual nodes   │
   │  4. RUN      run your query over the combined real + virtual graph       │
   │  5. ROLL BACK    discard everything materialized — nothing persists      │
   │                                                                          │
   └──────────────────────────────────────────────────────────────────────────┘

  1. Probe. The engine runs the real-graph prefix of your query — the part that binds the anchor the virtual data hangs off — applying your own WHERE and pinned literals, so it selects only the anchors that will survive your filter. It collects each anchor’s identity and its key field (the value a producer fetches by, e.g. an email address).

    The probe binds what it can name, and a position it cannot name only widens it. The probe binds the anchors it must project and the labelled nodes that constrain them. A position whose label it cannot name — the endpoint of a pattern predicate you left unnamed, WHERE (s)-[:SUPPLIES]->(), in any direction, negated, pinned by a property, or over a variable-length hop — is dropped from the prefix, with its relationship and any condition on it, exactly as a position only an OPTIONAL MATCH or a NOT EXISTS reaches is: a label-free MATCH (x) is a scan of the whole graph, and a scan is not a constraint. So such a filter lowers no fetch’s cost — every anchor the rest of the query binds is fetched for — and your own WHERE still decides the answer, which is the same answer as over the same records stored. A query is never refused over a position the probe chose to bind. A node YOU bind with no label is unscopable (§11), as it always was; a node only a pattern predicate reaches is yours to leave unnamed.

    An equality on a number or a boolean keeps every anchor it matches, whatever spelling holds the value. n.rating = 10 keeps a node that stores the number 10 and one that stores the text '10'; f.isStatic = true keeps a node that stores the boolean true and one that stores the text 'true'. This holds for an anchor an earlier hop of the same query fetched as much as for a stored one, so a filter on a fetched node never empties the hop that follows it. The probe can only widen here, never drop: your own WHERE still decides the answer. Only the exact words true and false are read as booleans; 'True' or 'yes' stays text.

  2. Fetch. For each virtual join the engine calls its producer once with the union of all anchor keys — never one call per anchor (no N+1). The producer is the source-specific fetcher (a REST op, a SQL query, a vector search, an in-process computation).

  3. Materialize. Each returned record becomes a transient node carrying the extra :Virtual label, a dateRetrieved timestamp, and the host-bound worldId, contextId, and access-policy revision. The engine links it to the anchor it was fetched for (keyField == recordKeyField). A record may also carry its own sub-graph (brings:), materialized in the same pass.

    A fetched node shows a query only what the same record would show stored. The :Virtual label and the run’s stamps are the engine’s bookkeeping, and introspection leaves them out: labels(n), keys(n), properties(n), n {.*} and a returned node or relationship answer as they would over the stored record, so size(labels(n)), labels(n)[0] and 'Virtual' IN labels(n) are right. A query that names the bookkeeping explicitly — n:Virtual, or a property under the engine’s reserved __vc prefix — still reads it. Both names are reserved to the engine, so hiding them never hides the caller’s data.

    A fetched record never sets the node’s scope. On every fetched node and brought child, userId, worldId, workspaceId and visibleTo are the scope the world stamps, never the record’s own fields, so no source can hand a node to whichever user it names. A record’s own field of one of those names is readable under its alias: userId as _sourceUserId, worldId as _sourceWorldId, workspaceId as _sourceWorkspaceId, and visibleTo as _sourceVisibleTo. The schema the generator is shown names them that way. A mirrored source’s persisted nodes store a record’s userId under the same name. A type keeps declaring the source’s own field name, userId. A filter written on an alias reaches the source as the field it stands for: a pushdown: rule keeps naming the source field (property: userId), and a record screened before materialization is judged by that field. A record field under the engine’s reserved __vc prefix is dropped: those names say which run may see a node, and a record does not choose its own visibility.

    A fetched type or brought child keyed by any of the four scope fields — the key would sit on a property the record never fills — or one that declares a scope field and also declares or keys on its alias, is a load problem against the declaring realm. Queries over such a realm still run, and the record’s own field is what its alias holds. A type keyed by userId is identified by _sourceUserId: records with the same source userId are one node, however often they are fetched, and a pin on _sourceUserId finds it.

  4. Run. Your full query now runs over the combined graph. WHERE, ORDER BY, RETURN, aggregates — all of Cypher — apply to virtual nodes exactly as to real ones. That includes Unicode normalization: normalize(s[, NFC|NFD|NFKC|NFKD]) and s IS [NOT] [form] NORMALIZED answer over fetched records as over the same records stored, NFC when no form is written.

  5. Roll back. The virtual nodes vanish. A re-run re-fetches (cheap, because of caching). Nothing is ever persisted by a read query.

    How they vanish is the deployment’s execution model, and a query cannot tell which is in force. Under the transient model the run’s transaction is rolled back. Under the committed model the materialized nodes are written, stamped with the run, and removed when it ends (§13.3). Overlapping runs that fetched the same record each keep seeing it for as long as they run, and each returns what it would return alone.

    The one exception: an identity bridge (writeThrough) is committed as a warm cache and re-resolved after refreshAfter (§5.2).

Evidence when a lookup returns no matching rows

Section titled “Evidence when a lookup returns no matching rows”

A fetched record is not the same thing as a matching query row. A producer may return records that a query’s predicate or join does not reach; an empty answer in that case does not prove those records are absent. The result distinguishes records returned by producers before query-side filtering, nodes made available to the query, and rows matched by the query. Cache-served records count as fetched for that execution; a source that genuinely returns zero records does not.

When a query about one identity-pinned record matches nothing, a result may include a record card with already-fetched fields and up to 20 rows per related section. The card is supporting context only: it does not answer the original filter, prove omitted or truncated related sections complete, or fetch additional sources. Its result identifies the Cypher that produced the card and marks the answer partial. An incomplete-source warning remains visible even if the card supplies useful context.

These distinctions appear in the result envelope next to rows and warnings. Each field below is omitted when it does not apply; presence can differ by surface, as noted.

Field Type Present Meaning
fetchedRecords integer on query, ask and saved-view results (the admin REST endpoints omit a zero) Records the producers returned before query-side filtering, cache-served ones included.
materializedNodes integer on query, ask and saved-view results (the admin REST endpoints omit a zero) Graph nodes made available to the query from those records.
reason string on query, ask, saved-view and admin REST results, only when the result is not a clean answer; absent on a clean answer, including a genuine zero Why there is no clean answer, as a code a caller can branch on. The evidence codes are FETCHED_NOT_MATCHED (records were fetched and none matched), PARTIAL_NOT_MATCHED (an incomplete read matched nothing and no step failed) and UNPUSHED_FILTER (a bounded read omitted a declared filter); other codes report refusals and failures. Any of them means an empty result does not prove absence.
incomplete string on query, ask and saved-view results, only when the read was incomplete; the admin REST endpoints omit it One line, such as Incomplete: based on the first 100 rows, more exist, that qualifies every row and figure in the result.
answerBoundary object {kind, unresolvedQuestion?} only on a non-empty record card kind: SUPPORTING_CONTEXT_ONLY: the rows support the lookup but do not answer its original filter. unresolvedQuestion restates what is still unanswered.
cypher string an ask result (kg_ask): on an empty result, a requested explanation, or a record card. A saved-view result: only on a requested explanation. A direct query (kg_query) never echoes it, since the caller wrote it. The admin REST query endpoints always include it. The executed query, when present. 
{ "rows": [], "warnings": [], "fetchedRecords": 12, "materializedNodes": 12,
  "reason": "FETCHED_NOT_MATCHED", "cypher": "MATCH (e:Event) WHERE e.status = 'open' RETURN e" }

Twelve records arrived and none matched. The answer is “no fetched record matched”, not “no such records exist”.

Two concepts the rest of the spec leans on

Section titled “Two concepts the rest of the spec leans on”

Bound anchor. A virtual label may only be reached by traversing a declared join from a bound anchor — a node the engine can resolve to a finite set. An anchor is bound when it is:

  • pinned by an inline literal — (p:Person {primaryEmail:'a@b.com'}), or
  • pinned by a WHERE equality — WHERE p.primaryEmail = 'a@b.com', or
  • narrowed by any property predicate — WHERE toLower(p.name) CONTAINS 'grace', or
  • the acting human principal — (me:AssistantUser) (the scope rewriter resolves the node linked to principalId; a service principal has no implicit me anchor), or
  • reachable over a required edge from another bound node.

A number pinned on an anchor key also matches the key’s text form. (q:EventQuery {eventId: 2779}) and WHERE q.eventId = 2779 find an anchor keyed 2779 or '2779', whether it is a stored node or one the engine created for the fetch; the quoted {eventId: '2779'} matches only the text key. Each number in a key list does the same: WHERE q.eventId IN [2779, 2780] finds anchors keyed 2779 or '2779' and 2780 or '2780', while a quoted list matches only text keys. This applies only to the key a join fetches by; every other property compares exactly, so {age: 40} does not match '40'.

A naked MATCH (hc:HubSpotContact) — no anchor — is rejected (§4). This is what stops Virtual Cypher from trying to fetch every contact in HubSpot.

A declared everyone-door is taken implicitly. A realm may declare a door whose identity has a default — scope: { metadata: { identity: "true", default: "all" } } — meaning “the whole population”. When exactly one of a label’s joins runs from such a door and is keyed on that identity property, a bare MATCH (i:Invoice) is planned as if (:Books {scope:'all'})-[:HAS_INVOICE]->(i:Invoice) had been written, and returns the same rows. The rule is not loosened: declaring the door is the realm’s opt-in, the same declaration that already seeds (:Books) when the door is written without a pin. Predicates on the bare node are pushed to the source exactly as they are through the written door. A label with no such door is rejected as above, and so is a label with two or more, since “everyone” would then name more than one population; the rejection names each door. A node pinned on a key a join declares as materializedKeyField is still found by that key, not by listing everyone.

Per-world and per-context scope. The world selects the outer data boundary; the context selects a confidentiality boundary within it; the principal supplies authority. The probe runs through the fail-closed scope rewriter under immutable host-bound (worldId, contextId, access-policy revision), while principalId remains separate for authorization and audit. Every anchor and virtual node belongs to that scope. The caller supplies none of these identities. A query cannot reach another context without an explicit policy-authorized bridge, and cannot reach another PRINCIPAL’s data at all. Between two worlds of the SAME principal the boundary is focus rather than confidentiality: each world decides which types, realms, producers, views and skills exist in it, and a label whose realm is not installed there cannot be named, fetched or viewed — but data the principal may see is data the principal may see. See §15. Local single-user deployments use the same path and an explicit default context.