Skip to content

7.6 Cost, determinism, and failure

These call an LLM, so they are the non-deterministic members of the surface (contrast §11): the same query can score two runs slightly differently, and the model — not the graph — decides. Bound the cost — each criterion is one batched call over the fetched rows (chunked for large sets), so they scale with rows fetched, not rows × 1; keep the fetched set small (an anchor, a real WHERE, a LIMIT on the fetch) before judging. Their calls are counted against the query’s model-call limit before any is made (§9), so a judgment over more rows than the query may pay for is refused rather than spent.

A model that fails refuses the query. When the call behind a judgment errors, the answer depended on that judgment, so none is returned: a retryable SOURCE_UNAVAILABLE names the primitive and the criterion. Never a 0 from count(*) over an ai.relevant filter, a column of 0.0 scores, or empty labels — each of those would read as a real judgment. A run that was cancelled, or stopped by a budget, keeps its own refusal.

Within an answer the model did return, a row it left unjudged is scored at the keep threshold (0.5), so an omission never silently hides a result. The threshold, not 1.0: a perfect score would keep the row and rank it above every row the model actually read, which is how a grants query for “youth mental health” once came back topped by aged-care infection research, at fit 1.0, with its kept count swinging across identical runs. At the threshold an unjudged row is kept and never promoted. It is also visible — a PARTIAL_RESULT (TRUNCATED) warning names how many rows of how many carry no judgment, so a caller ranking or counting them knows how much of its answer the model never read. Reach for them only when the discriminator is genuinely subjective; a property, an embedding (§6), or a real predicate is always cheaper and more repeatable.