Working with documents

This page shows code for using the document service from your app. Documents are stored as originals with versions, and malware scanning and search indexing run asynchronously. The concepts are covered in Documents.

Accepted documents

• PDF and UTF-8 text. Office documents (Word, Excel, PowerPoint) are kept as originals and converted to PDF for indexing and extraction.
• Up to 10 MiB per file. If you send a ZIP file, each file inside becomes its own document.
• By default, only the uploader can read a document (private). Use project for documents the team shares.
• If a document with the same content (same SHA-256) already exists with the same visibility, Hane adds a receipt record to it instead of creating a new document.

Upload

An upload has three steps: prepare, send the original, and complete. The BFF computes the SHA-256 of the body it received from the browser before sending.

1. Prepare

   const digest = new Uint8Array(await crypto.subtle.digest("SHA-256", body));
   const sha256 = Array.from(digest, (b) => b.toString(16).padStart(2, "0")).join("");
   
   const session = await unwrap(
     hane.POST("/v1/documents/upload-sessions", {
       body: { filename: "lease-C-1001.pdf", content_type: "application/pdf", size: body.byteLength, sha256, visibility: "project" },
     }),
   );

   To add a new version to an existing document, also pass document_id.

2. Send the original

   await unwrap(
     hane.PUT("/v1/documents/upload-sessions/{id}/content", {
       params: { path: { id: session.id } },
       body: "",
       bodySerializer: () => body, // send the bytes as is
       headers: { "content-type": "application/pdf" },
     }),
   );

   The Content-Type must match the content_type you prepared with.

3. Complete

   const done = await unwrap(hane.POST("/v1/documents/upload-sessions/{id}/complete", { params: { path: { id: session.id } }, body: {} }));
   console.log(done.document.id, done.version.version, done.deduplicated);

   For a ZIP file, call POST /v1/documents/upload-sessions/{id}/expand instead of complete to get a result per file (accepted, merged, or rejected).

Check status

At completion, scanning has only been scheduled. Look at two statuses on the version to decide what to show.

Status	Values	Meaning
inspection_status	queued, scanning, clean, infected, failed	Malware scan. Only clean originals can be downloaded
index_status	queued, indexed, needs_ocr, needs_conversion, failed	Search index. Searchable once indexed

UTF-8 text is indexed at completion and is searchable without waiting for the scan. PDFs with embedded text become searchable after the text is extracted. Scanned PDFs are read in environments with OCR configured; without it they stay at needs_ocr.

const v = await unwrap(hane.GET("/v1/documents/{id}/versions/{version}", { params: { path: { id, version } } }));
if (v.inspection_status === "clean") {
  // the original can be fetched with GET /v1/documents/{id}/versions/{version}/content (needs documents:download)
}

Search

const hits = await unwrap(hane.POST("/v1/documents/search", { body: { query: "C-1001 expiry", limit: 20 } }));
for (const h of hits.items) console.log(h.title, h.page_start, h.snippet);

• Returns passages (chunks of 800 characters within a page) that contain every space-separated term, strongest match first. Up to 8 terms.
• Returns only the current versions of documents you can read, and never versions where malware was detected.
• It has no side effects, so no Idempotency-Key is needed.

Choose how to search with mode.

mode	How it searches	Availability
keyword (default)	Full-text search	Available
summary	The index of per-page summaries. Also finds terms that span pages	Available
semantic, hybrid	Semantic search, and its combination with full-text search	Preview CONFLICT in environments where it is not enabled

Read page text

You can read the text of the matching pages without handing out the original.

const pages = await unwrap(
  hane.GET("/v1/documents/{id}/versions/{version}/pages", {
    params: { path: { id: hit.document_id, version: hit.version }, query: { from: hit.page_start, to: hit.page_end } },
  }),
);

Confirm extracted values

With an extraction schema (ExtractionSchema), Hane extracts field values of a type from documents as extracted facts. When a person checks a fact against the original, it becomes a confirmed ledger value.

apiVersion: documents/v1alpha1
kind: ExtractionSchema
metadata:
  name: lease-contract
spec:
  target: LeaseContract
  review:
    assignment: fleet-desk
  documents:
    types: [lease-contract]
  fields:
    contract_no: { labels: [Contract number] }
    start_date: { labels: [Lease start date, Start date] }
    term_months: { labels: [Lease term, Contract term] }

// facts that need review
const facts = await unwrap(hane.GET("/v1/documents/facts", { params: { query: { status: "needs_review" } } }));

// confirm with the extracted value (pass value to correct it before confirming)
await unwrap(
  hane.POST("/v1/documents/facts/{id}/confirm", {
    params: { path: { id: fact.id }, header: { "If-Match": ifMatch(fact.version) } },
    body: {},
  }),
);

Use /reject to reject a fact, and /inquire when the value can’t be read from the document and you ask the other party. Call these with the user’s token; the user is recorded as the person who checked the value.

Tip

To let an agent search documents, add documents.search and documents.read_pages to its tools. Documents attached in the chat are passed to the chat agent’s input with format: document.

Next steps

DocumentsScanning, indexing, document types, retention

Embedded chatAttach documents and ask an agent