Skip to content
Embabel Worlds

5.18 jar — a Java jar's classes as a keyed source

A jar producer reads the bytecode of a Java jar when a query reaches it, and returns what the jar declares: its packages and types, their fields and methods, which methods read and write which fields, and which methods call which — into the jar, the JDK or any other library — with source lines from the jar’s debug information. Nothing is ingested, indexed in advance or stored: the rows are transient for the query, like any fetch. No source code is needed or read.

- { name: javaJarTypes,        kind: jar, emit: types }                 # key: a jar path
- { name: javaFieldsOfType,    kind: jar, emit: fields,   by: owner }   # key: <jar path>!<owner>
- { name: javaAccessesOfField, kind: jar, emit: accesses, by: field }   # key: <jar path>!<field>
MATCH (j:Jar {path: 'data/jars/app.jar'})-[:HAS_TYPE]->(t:JavaType {name: 'com.example.Cache'})
      -[:DECLARES_FIELD]->(f:JavaField)-[:ACCESSED_BY]->(a:FieldAccess)
RETURN f.name, a.method, a.access, a.lines

The contract:

  • emit is the collection, by is the selector. emit is one of artifact, packages, types, fields, methods, accesses, calls, casts, methodBodies. With by absent (or jarPath) the key is a jar path and every row of that collection is returned. With any other by the key is <jar path>!<value>, and the rows are those whose by property equals value, or — for a list property such as a type’s interfaces — contains it. A by that is not a property of the collection, or an unknown emit, is refused when the realm loads.
  • Every row carries the key the next hop needs. id is <jar path>!<member>, jarPath is the jar path as the key gave it, and every property naming another member has an …Id twin (ownerId, methodId, fieldId, callerId, calleeId, superclassId). A row also carries anchorKey, the key it was fetched for: that is the recordKeyField a join declares. So a hop from a type to its fields keys on the type’s id with by: owner, and a hop from an access to its method keys on methodId with by: key.
  • Names are the JVM’s. A type is named by its binary name, pkg.Outer$Inner, as stack traces and heap dumps print it; an array type is written T[]. A method’s key is Owner.name(parameter types), constructors are <init> and a static initializer <clinit>.
  • Only the jar’s own classes are rows. A supertype, field or callee outside the jar (the JDK, another library) is a name on the row that refers to it — superclass, fieldOwner, calleeOwner — never a row of its own; internal: false marks those references. A call row says resolution: exact: bytecode names the method the runtime calls (§5.21). A field’s container (map, collection, array, threadlocal, reference, none) is decided by its declared type and, for a type in the jar, that type’s own supertypes down to the JDK.
  • Generics are read from the class files. A field carries genericType, its declared type with type arguments as source writes it (java.util.Map<com.x.Key, java.lang.String>), and typeArguments, those arguments erased to binary names. A map’s key and a set’s element — the type the container hashes — is keyType; a map’s value is valueType; a collection’s, array’s, ThreadLocal’s or Reference’s is elementType; each has an …Id twin. A method carries genericParameters and genericReturnType, a type its typeParameters. These hold for what a declaration says, never for what flows through it at run time.
  • A type says where its hashCode and equals come from. hashCodeFrom and equalsFrom name the class that supplies each: the type itself, an ancestor in the jar, a JDK ancestor, or java.lang.Object — identity. They are absent for an interface, and when an ancestor in another library would decide.
  • Rows carry the keys a jar’s source is read by (§5.19). Types, fields, methods, calls and accesses carry sourcePath — the package path and the source file the class file records (com/x/Cache.java; a nested class shares its outer class’s file) — and sourceKey, <jar path>!<sourcePath>. Methods, calls and accesses also carry spanKey, <sourceKey>#<line>#<name>: their first line and the word expected there (the member’s name, the class’s simple name for a constructor, static for an initializer). A class compiled without debug information has neither.
  • A method knows every line it has code on. A method row carries lines, the source lines its line-number table holds, ascending; lastLine, the last of them; and at, one Owner.name:line per line. line is only where the method starts, and a constructor’s code can start at a field initializer above another constructor, so the method a stack frame’s line was in is the one whose lines hold it: by: at with key <jar path>!<Owner>.<name>:<line> returns exactly that method, overloads included. Empty without debug information.
  • An access or a call is grouped by its target. A method that reads a field on six lines is one READ row with six lines; a WRITE is an assignment. Adding to a collection held in a field is a READ of the field plus a call (put, add) in the same method.
  • What the program casts at run time: casts. One row per checkcast and instanceof instruction in the jar’s own method bodies, never grouped: kind (cast or instanceof), targetType (binary name, an array as T[]; targetTypeId reaches the type when it is in the jar, internal says so), targetElementType (the type without its brackets) and isArray; method, the methods key of the method the instruction was written in, with methodId, methodOwner and methodName; line, the source line from debug information; and ordinal, its place among the method’s casts and instanceofs, from 1. key is <method key> <kind> <targetType> #<ordinal>. A cast inside a lambda belongs to the method the lambda is written in, as a call or access there does, and lambda names the compiled lambda (lambda$parse$0); inSyntheticMethod marks every cast the compiler placed in a method of its own making. A row carries sourceKey and, with a line, spanKey <sourceKey>#<line>#<word>, where the word is the target’s simple name (Entry for Map$Entry, int for int[]).
  • How a method’s code branches: methodBodies. One row per compiled method body — abstract and native methods have none — counting what the compiler emitted: branches (every conditional jump: if, &&, ||, ?:, a loop’s test, a null check), backEdges (jumps to earlier code: one per loop, as compiled), switches (switch instructions), handlers (distinct exception handlers) and catchAnyHandlers (those catching anything: finally, synchronized), casts, instanceofs, instructions, and line, lastLine and lineSpan from the line table. method names the methods key the body belongs to — a lambda body (isLambda) belongs to the method it is written in, so selecting by: method returns a method’s own body and one per lambda in it; sum them for the method as written. key is the body’s own: the method’s key, a lambda’s compiled name, or for a bridge (isBridge), which shares its method’s key, <method key> returns <type>. A synthetic accessor has no method.
  • Bytecode counts are not source counts. Both collections read what the compiler wrote, and a reader comparing them with the source should expect:
    • casts the source never wrote: a generic method’s erased result (String s = list.get(0) is a checkcast String), a bridge’s argument, a pattern match (o instanceof Foo f is an instanceof and a cast), an enum’s values() (a cast of the cloned array), a Kotlin smart cast;
    • no row for what source calls a conversion: widening and narrowing between primitives ((int) aLong), boxing and unboxing are other instructions, not casts;
    • a switch on a string is two switch instructions (one on the hash code, one on the position) plus an equals branch per label; a switch on an enum reads a synthetic lookup array first; a pattern switch is an invokedynamic that picks the case, a switch on its answer, and a cast per typed case;
    • &&, ||, ?: and assert are branches; for, while and do compile to one back edge each, and a continue that jumps back to a loop’s test is another;
    • finally code is copied into every exit path, so its branches and casts count once per copy;
    • a compiler other than javac (kotlinc, scalac, a bytecode weaver) makes different choices, and Kotlin’s inlined functions report line numbers past the end of the file, which stretch lastLine and lineSpan.
  • A key names a jar, a war, or a folder of classes. A key ending .jar is a jar. A key ending .war is a web application’s own code — what its WEB-INF/classes holds, the classes and the resources beside them — and not its WEB-INF/lib, which is other people’s: warLibs names the library jars to read with it, by file name glob (warLibs: [acme-*.jar]); a library’s class never replaces the application’s, nor an earlier library’s, and its META-INF is not read. Any other key is a folder of compiled classes: one module’s target/classes, or a project’s folder, whose every module’s classes are read — except test output below the key (test-classes, Gradle’s classes/<language>/test), which reads only when named as the key itself; where two modules compile one class, the first in path order is read. A war or a folder answers every collection a jar does, with the same properties, and its rows’ ids start from the key as given: apps/shop.war!com.acme.Cache, src/shop/target/classes!com.acme.Cache (a folder key’s trailing / is dropped, so both spellings give the same ids). In a folder’s member keys the archive ends at the LAST !. The artifact row says which it was: packaging is jar, war or classes; a folder’s coordinate is its project’s folder name unless its classes carry a pom.properties, and its sha256 is of the classes, not of any file.
  • Where a jar may be. A relative path resolves against the world’s data/ tree, then against the declaring realm’s own directory — so a realm can ship an example jar — and an absolute path must lie under a root the operator trusts. A key that names no readable jar, war or folder there yields no rows and a PRODUCER_ERROR naming the key; a jar or war larger than maxJarMb (default 128) is refused the same way, never half read — for a war the size is what is read, its classes and the named libraries, and for a folder the size of its class files. A folder is read again when a file in it is added, removed or changed. On an appliance, embabel data add <file|folder> --to jars puts one under the world’s data/ and prints the key to read it by.
  • Filters in the MATCH narrow the read. An equality, IN, CONTAINS or list membership on a property of the emitted collection is applied before rows reach the graph: (j:Jar {path: $jar})-[:HAS_CALL]->(c:MethodCall {calleeOwner: 'java.lang.System'}) does not materialize every call in the jar. The graph re-applies every filter, so the answer never depends on it.

Cost: a jar is read once and kept for later hops of the same and subsequent queries until it changes on disk, so the first query against a jar pays for reading it and the rest are lookups. A war or a folder costs the same plus copying its classes once each time it is read; a folder is also listed once per hop that reaches it, to see whether it changed. A whole-jar hop over a large jar is still many rows — tens of thousands of calls in a mid-sized library — and is subject to maxFanoutTotal like any other; reach those rows through a keyed hop, or narrow them in the MATCH.

Code labels are language-neutral parents. The standard model declares Package, CodeType, Field, Method, Call, FieldAccess, Source, SourceSpan, SourceDeclaration and CallSite, each with the properties every language’s rows carry; a realm declares its language’s children (JavaMethod with parents: [Method]) over its own producers, as memory systems are children of Memory (§6.8). A query naming a parent is answered by every language with a join.