Skip to content

Populating types from an external system (deterministic, no code)

The patterns above cover types the user creates. A realm can also declare a type that the host populates automatically from a connected external system — deterministically, with no LLM and no Kotlin/Java in the realm. A structured record (a CRM contact, an issue, a calendar attendee) is already typed at the source, so extracting it with an LLM is wasteful and error-prone; instead the realm declares a projection in property metadata: and the host’s projector does the rest.

This builds on a small canonical-entity model the host ships: within one host-bound visibility scope, a Contact is a Person resolved by email. The host-side merge key is (worldId, visibilityScopeId, type, email) even though realm YAML does not repeat the scope fields. visibilityScopeId is WORLD for world-visible data and contextId for context-private data. Declare a mirror type that parents: [Contact] and annotate each property:

# types/hubspot.yml — populated from HubSpot CRM, no Kotlin in the realm
- name: HubSpotContact
parents: [Contact]
visibility: internal # machinery, not a user-browsable repository type
userAnchor: { predicate: OWNED_BY, direction: from-user }
properties:
email: # the deterministic merge key, and the spine it keys
identity: true
hub: Person
jobtitle: # projects onto the canonical Person
metadata: { canonical: "jobTitle" }
company: # resolve + link a related entity
metadata: { relationship: "WORKS_FOR", target: "Organization", matchBy: "name" }
phone:
cardinality: LIST
metadata: { canonical: "phones", multivalued: "true" }
# undeclared fields (lifecyclestage, …) stay on the mirror, source-private
Metadata key Effect
identity: "true" This field’s value is the email portion of the merge key: records with the same (worldId, visibilityScopeId, type, email) resolve to one canonical :Person (no LLM entity resolution). Also unioned onto the Person’s emails.
hub: <Spine> The opt-in. This field’s value keys the named spine — Person, Organization, or a spine a realm declares with spine:. Without a hub: a type is never canonicalized, whatever its identity. May sit on a property other than the identity one, when what identifies the record (a CRM id) is not what identifies the entity (a website). Writable first-class (hub: Person) or under metadata:. See VIRTUAL_CYPHER.md §5.4.
canonical: <field> Project this source field onto the canonical Person’s <field>. Single-valued → a winner is chosen by host precedence then most-recent; mark multivalued: "true" (or cardinality: LIST) to union instead.
relationship: <EDGE> + target: <Label> + matchBy: <prop> Create-or-match a <Label> keyed on matchBy, and link (:Person)-[:EDGE]->(:Label).
(none) Source-private — the value lands only on the per-record mirror node.

Storage model. Each source record becomes a per-record mirror node (:<TypeName>:RemoteHandle) holding the raw fields + provenance, linked to one canonical :Person that holds the resolved values (queryable: MATCH (p:Person) WHERE p.jobTitle = 'CEO'). The mirror’s label is the namespace, so two sources never clash on a field; “what does HubSpot specifically say” is one hop to the mirror. A record with no email becomes a mirror-only orphan (reaped in the background).

When it runs. Declaring the type loads nothing. Once the user connects the account (OAuth), the host pulls on a schedule (cadence configurable per source), checkpointed by a persisted watermark so a large import drains over successive ticks and a mid-run failure safely retries the window (projection is idempotent). One-click backfill and real-time webhooks ride the same path. The realm supplies only the type (above) and the fetch (its apis/ OpenAPI op or a handler); the projector and scheduling are host-side.