5.19 source-tree with from: jar — the source a jar was compiled from

The source-tree kind (§5.21) with from: jar reads a jar’s source; the source kind that did before is retired: a producer file that still writes kind: source does not load, and the realm reports it as a problem naming the file. Change kind: source to kind: source-tree, from: jar and emit: callsites to emit: calls: keys, ids and columns are as they were.

A from: jar producer reads source files where they are, when a query reaches them, keyed by what a jar row (§5.18) already carries. Nothing is ingested; the rows are transient like any fetch.

- { name: javaSourceFile,  kind: source-tree, from: jar, emit: files,        roots: [src] }             # key: a sourceKey
- { name: javaSourceSpan,  kind: source-tree, from: jar, emit: spans,        roots: [src], context: 6 } # key: a spanKey
- { name: javaDeclaration, kind: source-tree, from: jar, emit: declarations, roots: [src] }             # sourceKey or spanKey
- { name: javaCallSites,   kind: source-tree, from: jar, emit: calls,        roots: [src] }             # key: a spanKey
- { name: javaLocals,      kind: source-tree, from: jar, emit: locals,       roots: [src] }             # sourceKey or spanKey
- { name: upstream,        kind: source-tree, from: jar, emit: spans,
    roots: [ { git: "https://github.com/acme/acme", ref: "v{version}" } ] }

MATCH (j:Jar {path: 'data/jars/app.jar'})-[:HAS_CALL]->(c:JavaMethodCall {calleeOwner: 'java.util.Map', calleeName: 'put'})
      -[:AT_SITE]->(cs:JavaCallSite)
RETURN c.caller, cs.receiver, cs.arguments, cs.argumentTypes

The contract:

• Five emits. files returns the file a sourceKey names: root, path, language, lines, sha256, package, candidates, and for a git root commit and url. spans returns the numbered lines within context (default 6, at most 50) of a spanKey’s line, with matches — whether the word the key expects appears on them — and inRange. declarations returns, for a sourceKey, every type, field, constructor, method and initializer the file declares, or for a spanKey the innermost one around the line: kind, name, binary owner, fromLine / toLine, signature, modifiers, annotations (names), annotationText (each in full, as written), javadoc, and for a member its numbered text. calls returns the calls on a spanKey’s line whose method (or, for a constructor, class) is the key’s word: receiver, receiverType, arguments, argumentTypes, argumentTypesResolved, enclosing, and resolution: type when the receiver’s static type is known, else name (§5.21). locals returns, for a sourceKey, every local variable the file declares, or for a spanKey those of the method, constructor or initializer around the line: kind (local, for, foreach, resource, pattern), name, line, declaredType as written, isVar, isFinal, initializer, initializerType, varEligible and varPreservesType (below), enclosing, enclosingKind, enclosingOwner, and the numbered text of the declaration. Every row carries id, sourceKey, jarPath and anchorKey, so a span or declaration joins back to its file by sourceKey.
• Where source is looked for, in order. A <jar name>-sources.jar beside the jar — for a war, <war name>-sources.jar beside the war; for a folder of classes, the project it was built in, read as a source tree (the folder above target or build: shop/target/classes finds shop/src/main/java/com/x/Cache.java; a project folder named as the key is itself the tree); then each root, in the order declared — a -sources.jar named as a root, a <jar name>-sources.jar inside a root folder, the folder itself as a source tree, or a git repository. Paths resolve as a jar path does: the world’s data/ tree, an operator-trusted root, the declaring realm’s own directory. In a tree a source path is found under any module and source folder (core/src/main/java/com/x/Cache.java for com/x/Cache.java); when more than one file matches, the first in path order is read and candidates says how many there were. The first root holding the path answers.
• A git root is read at a commit. ref is a commit, tag or branch; {version} in it is replaced by the jar’s version. A tag or branch is resolved to its commit first, and rows report the commit, so the same rows come back whatever the tag later points at. Only https URLs are accepted, held to the same address rule as any fetched URL; a repository needing credentials is refused, not waited on. The first read of a commit lists its files; a file’s content is fetched the first time it is read.
• Declarations, call sites and locals need a syntax tree. They answer for Java source; a file in another language answers files and spans, and these emits return nothing for it with a PRODUCER_ERROR warning saying so. A file with syntax errors answers around them. An argument’s type is resolved from the file alone — new X(…), a literal, a cast, a parameter, a local declared before the call, a field of an enclosing type — through the file’s own types, single-type imports, java.lang and the file’s package. A type only another file could explain is returned as written with argumentTypesResolved false; a method’s return value is unknown.
• Whether var would do. varEligible is the language’s rule: one variable, given a value that is not null, an array initializer, a lambda or a method reference (every enhanced-for variable; never a pattern binding). varPreservesType says whether var would give the variable the type it declares: true for String s = "x" or Key k = new Key(id); false when the initializer’s type is another one (long n = 1, List<X> xs = new ArrayList<>()), an anonymous class, or a diamond under a parameterised type; null when the initializer’s type is not in the file — never guessed. It is null for a local already declared with var and for one var cannot replace.
• What is honest about absence. A key whose path no root holds returns nothing — a generated class has no source. A jar for which no root at all could be found, a declared root that could not be read, a git ref that names nothing, and a file larger than maxFileKb (default 1024) are each a PRODUCER_ERROR warning naming what was missing. A source that is not the jar’s version shows as matches: false on its spans and declarations; its line numbers cannot be trusted.

Cost: a root is listed once and kept for later keys and queries (an archive until it changes on disk, a commit for good); a folder tree is listed again when a lookup misses, at most once per fetch. Each key is then one file read, and a syntax-tree emit parses a file once per fetch however many keys name it. A git root’s first query pays for a shallow, blob-less fetch of the commit; each file read from it for the first time is one more fetch.