Upload a file to ingest as a job

POST

/api/v1/documents/ingests

const url = 'http://localhost:11044/api/v1/documents/ingests';
const form = new FormData();
form.append('file', 'file');

const options = {method: 'POST', headers: {'X-Embabel-Api-Key': '<X-Embabel-Api-Key>'}};

options.body = form;

try {
  const response = await fetch(url, options);
  const data = await response.json();
  console.log(data);
} catch (error) {
  console.error(error);
}

curl --request POST \
  --url http://localhost:11044/api/v1/documents/ingests \
  --header 'Content-Type: multipart/form-data' \
  --header 'X-Embabel-Api-Key: <X-Embabel-Api-Key>' \
  --form file=@file

Answers 202 once the file is received, with a job to follow at GET /documents/ingests/{id}, rather than holding the request until the document is written.

Authorizations

apiKeyAuth
basicAuth

Parameters

Query Parameters

fromOrgDomain

string

Domain of the organisation the file is really from, recorded as its publisher (the upload dialog’s From field); omitted or blank records none

tags

Array<string>

Tags to stamp on the document and its chunks, as repeated tags fields or one comma-separated value; trimmed, blanks dropped, duplicates collapsed. Omitted leaves the document untagged

Request Body

multipart/form-data

object

file

required

The file to ingest (PDF, text, HTML and other formats the converter reads), spooled to disk before the 202 so the job survives a restart. Its original filename names the job and the document; a part with no filename is called upload

string format: binary

Responses

202

Accepted; the job, queued

*/*

One document ingest job as its owner sees it: its stage, embedding progress, and how it ended. Answered by POST /documents/ingests, POST /documents/ingests/url, POST /documents/embed-missing and GET /documents/ingests/{id}

object

chunks

Chunks the stored document was cut into; null until the job succeeds, and when the store does not say

integer format: int32

chunksWithoutEmbeddings

Chunks of the stored document with no embedding, which never match a semantic search; null until the job succeeds, and when the store does not say. POST /documents/embed-missing can embed them

integer format: int32

error

Why the job failed, in words; null unless state is failed

string

required

Job id, a UUID; the {id} of GET /documents/ingests/{id}

string

name

required

What the job is called: the uploaded file’s name, the URL being ingested, or for an embed-missing job the document’s title (its URI when it has none)

string

progress

Chunks embedded so far, of how many; null until the job starts embedding

object

done

required

Chunks embedded so far; never more than total

integer format: int32

total

required

Chunks to embed: an estimate from the document’s text while the job runs, corrected to the true count when it succeeds; raised rather than exceeded when the document cuts into more

integer format: int32

resumes

required

Times the appliance restarted while the job was unfinished and picked it up again under the same id; 0 for a job that ran straight through

integer format: int32

startedAt

required

When the job was submitted, ISO-8601 instant

string

state

required

Stage the job is at: queued, waiting_for_memory, converting, writing, embedding, succeeded or failed; the last two are final

string

Allowed values: queued waiting_for_memory converting writing embedding succeeded failed

title

Title of the stored document; null until the job succeeds, and when the document has none

string

updatedAt

required

When the job last changed stage or embedded a batch of chunks, ISO-8601 instant; a value that stops moving while the job is unfinished suggests it is stuck

string

uri

URI of the stored document; null until the job succeeds

string

409

Nothing can embed; not started

*/*

A 409 from an ingest-job endpoint: this appliance has no embedding model yet, so nothing was spooled or started. Not set up, rather than broken

object

embeddings

required

The embedding status this was decided from: configured, model, dimensions and reason

object

key

additional properties

message

required

Why, in the operator’s words, naming the command that sets an embedding model

string

status

required

Always unavailable

string