> ## Documentation Index > Fetch the complete documentation index at: https://help.stralos.app/llms.txt > Use this file to discover all available pages before exploring further. # Document AI pipeline > Internal architecture for the document processing pipeline The document AI pipeline is owned by `stralos-api-documents`. ## File acceptance rules `validateFile` (`src/pipeline/fileRules.ts`) is the shared resolver for `POST /api/documents` presigning and for drive sync's fetch verdict. It returns `{ ok: true, rule }` or `{ ok: false, reason }`. | Kind | Extensions | Max size | | - | - | - | | `pdf` | `.pdf` | 100 MB | | `tiff` | `.tif`, `.tiff` | 100 MB | | `image` | `.png`, `.jpg`, `.jpeg` | 50 MB; processable up to 10 MB | | `docx` | `.docx` | 50 MB | | `xlsx` | `.xlsx`, `.xlsm`, `.xltm`, `.xlsb` | 50 MB | | `csv` | `.csv` | 50 MB | | `dwg` | `.dwg`, `.dxf` | 100 MB, when the [DWG and DXF acceptance flag](#dwg-and-dxf-acceptance-flag) is on | | `email` | `.msg`, `.eml` | 50 MB, store-only | | `project_schedule` | `.mpp` | 50 MB, store-only | A file matches a rule on its extension or its content type, first rule wins. For `.dwg` and `.dxf` the extension decides, whatever content type the client declares. The `xlsx` rule covers macro-enabled workbooks, macro-enabled templates and binary workbooks. SheetJS reads all four as spreadsheet data and never executes VBA; with its default `bookVBA: false` it does not even copy the VBA project out of the package. ### Store-only documents A document is store-only when its kind is `email` or `project_schedule` (`store_only_type`), or when an image exceeds its 10 MB processable limit (`too_large_to_process`). Store-only documents are accepted, virus-scanned, stored and downloadable, and rest at `stored`. The process CAS refuses them with a `409` whose message starts `store_only:`, so they never enter `parsing`; a store-only document that reaches the parse stage by another path is parked back at `stored`, never failed. List and detail responses carry the reason in `storeOnly` (or `null`). ### Estimate-path restrictions When the upload target is an estimate, `validateFile` also rejects store-only kinds (`store_only_estimate_upload:`) and images above 10 MB, using the processable limit as a hard cap. An estimate-scoped upload is processed automatically after a clean scan, so a file that cannot be processed is refused before any bytes move. This mirrors the DWG estimate restriction. ### Temporary files `isTemporaryFile` runs before any rule lookup. Office lock files (`~$…`), `.tmp` files, `Thumbs.db`, `.DS_Store` and `desktop.ini` are rejected with `temporary_file:`. Drive sync maps that prefix to its own `temporary_file` skip reason, so these files are never linked. ## Flow 1. `POST /api/documents` validates file metadata and creates document rows. Files that fail server-side validation receive a `rejected` status immediately; no bytes move, and the row is a permanent terminal record. DWG and DXF acceptance depends on the conversion flag described below. Accepted files receive a presigned upload ticket. 2. The browser uploads files directly to S3 or LocalStack for accepted files. 3. Object-created events move uploads into scanning. 4. Scan verdicts gate every later stage. 5. Parse workers extract text. 6. Summary workers write document summaries and metadata. 7. Classification creates area runs. 8. Area workers create reviewable suggestions. 9. Users accept or reject suggestions from the estimate feature. ## DWG and DXF conversion The `STRALOS_DWG_UPLOADS_ENABLED` feature flag gates DWG and DXF acceptance. While the flag is disabled, the presign handler records a durable rejected document row with a `dwg_unsupported:` reason so demand can be measured without uploading file bytes. Rejected rows do not enter the pipeline and are not retried. `.dwg` and `.dxf` files are accepted by the upload endpoint when the `STRALOS_DWG_UPLOADS_ENABLED` flag is active. They are converted to a PDF derivative via Autodesk Platform Services (APS) Design Automation before parsing and AI processing can proceed. The original drawing file is preserved at its own S3 key for download; only the derived PDF is used for OCR, thumbnails, and area processing. ### Conversion state machine Conversion runs a separate closed state machine alongside the document lifecycle status. The document remains at `stored` throughout conversion — the conversion state is tracked independently in `docMetadata.conversion`. | State | Description | | - | - | | `unclaimed` | Conversion has been requested but not yet claimed by a worker. | | `claimed` | Worker has claimed this attempt; no workitem submitted yet. | | `submitted` | Workitem sent to APS Design Automation; worker polling for result. | | `succeeded` | Converted PDF written to `derived////converted.pdf`. | | `failed_retryable` | Attempt failed; conversion will be re-driven up to the restart limit. | | `failed_terminal` | Conversion exhausted its restart budget or encountered a deterministic failure. The document cannot be processed. | Maximum restart attempts: 2 (attempts 0 and 1 are live; reaching 2 is terminal). ### Conversion trigger When a DWG or DXF file scans clean, the scan handler writes the `stored` transition and a `documents.conversion.requested` event in the same database transaction. This prevents a crash between the two steps from silently dropping the conversion stage. The event carries an `attempt` fence token equal to the document's current `conversionRestartCount`. The claim CAS requires equality, so a dead DLQ message from attempt 0 arriving after attempt 1 is live cannot retire the active workitem. ### Processability guard `POST /:id/process` and `POST /:id/route` (with `process: true`) refuse to start AI processing on a drawing whose derivative is not yet ready. The response is `200` with `reason: "conversion_pending"` while conversion is progressing, and `409` with `reason: "conversion_failed"` when conversion is `failed_retryable` or `failed_terminal`. Once conversion succeeds, the derived PDF key is stored on the document row and subsequent calls to `/process` treat the document as a renderable PDF. ### Sweep predicates The scheduled sweep handles four stall cases for in-flight conversions: 1. **Wall-clock deadline** (checked first, order is load-bearing) — a conversion that has been running for more than 2 hours is settled as `failed_terminal`, regardless of its current state. 2. **Orphaned claim** — a claim where no workitem was ever recorded (`job_id IS NULL`) and no progress for 15 minutes. The sweep checks for existing output before paying again; if output is absent it settles `failed_retryable`. 3. **Stalled submitted workitem** — a workitem with a job ID and no progress for 10 minutes. The sweep polls APS once; a still-running result is a no-op, a finished or unknown result settles the attempt. 4. **Retryable re-drive** — a `failed_retryable` conversion not yet re-driven after 5 minutes. The sweep publishes a new `documents.conversion.requested` event to restart the attempt. The sweep predicates key on `conversion_progress_at` (last progress of any kind — claim, submit, each poll, each settle), not on `updated_at`. Folder reassignment or routing changes bump `updated_at` without touching the conversion clock, so they cannot postpone reaping a stuck conversion. Messages that first arrive after the two-hour wall-clock deadline are refused and settled as `failed_terminal`; a delayed message cannot resurrect a conversion after the client has been told it will not complete. #### Harvest failure isolation The orphaned-claim harvest (predicate 2) HEADs the `derived/` output key to check whether a workitem already completed before paying for another. This S3 HEAD requires `s3:ListBucket` on the bucket with a `s3:prefix` condition for `derived/*`. Without that grant, S3 returns `403 AccessDenied` instead of `404` for a missing key. `objectExists` re-throws every non-404 response deliberately — treating `AccessDenied` as absent would settle a workitem that was already paid for. Harvest errors are isolated per row. A failing harvest logs the error, increments the sweep's failed-harvest count, and moves on to the next orphaned claim. The rest of the pass continues: stuck-document timeouts, variation-job cleanup, sync housekeeping, and outbox dispatch all run to completion regardless of how many individual harvests fail. Once the pass is complete, the sweep Lambda throws if the failed-harvest count is greater than zero, which trips the sweep error alarm without killing any other work. A genuinely poisoned row (one past the two-hour wall-clock deadline) stops being selected by predicate 1 before it can page indefinitely. While conversion is pending, `GET /:id/page-thumbnails` returns `complete: false` with no items so the client keeps polling. After success, it returns rendered thumbnails as they become available. A terminal conversion returns `complete: true` with no items, which stops the poll. ### APS Design Automation The APS activity `stralos.DwgToPdf` is published out of band and referenced by alias. It evaluates an AutoLISP script via `accoreconsole`, enumerates paper-space layouts, fits each to the nearest ISO A-series sheet, and produces a single multi-page PDF plus an `audit.json` sidecar. The converted PDF and audit file are written to presigned S3 URLs. APS never stores the drawing file; it reads and writes through the presigned URLs only. Each page in the audit report carries: * `layoutName` — the paper-space layout name from the drawing. * `widthMm` / `heightMm` — physical page dimensions. * `isoName` — the resolved ISO paper size (e.g. `"A1 Landscape"`). The audit `pages[]` array is required for a successful conversion. An absent or unparseable array settles the attempt as `failed_retryable` because a PDF whose page geometry cannot be described is not usable for takeoff. Audit metadata fields (`xrefsMissing`, `proxyObjects`, etc.) are best-effort and their absence does not block success. ### Blank output detection After a DWG conversion succeeds and the derived PDF passes verification, the pipeline scans the PDF to check whether every page is blank. This covers drawings whose content lives entirely in model space while the layout tab carries only an empty viewport — the conversion produces a correctly sized, structurally valid PDF with no ink on it. The check is advisory, not a gate. The conversion result stays `succeeded`, the derivative is published and renderable, and the only output is a `contentWarning: "blank_output"` field on `docMetadata.conversion`. When the key is absent there is nothing to report. The frontend turns the flag into a non-blocking notice for the user; it does not trigger a re-conversion. Blank output is detected by reading the PDF content streams through pdf.js. A page counts as painted if any operator puts a visible mark on it — geometry, glyph-showing text operators, or annotations other than `Link` and `Popup`. The scan short-circuits as soon as one page paints something, because a set where only some pages are blank is ordinary draughting practice. Two bounds limit the work the scan can do: * Files larger than 16 MiB are skipped and no warning is emitted. Blank output scales at roughly a kilobyte per sheet; a file above this threshold cannot plausibly be all-blank. * The scan has a 30-second budget. If the budget is exhausted between pages the scan stops, logs a warning, and reports no content warning. A timer can also win while one page is decoding, but it does not cancel that in-page work; the decode may continue in the background until the page operation releases its resources. The detection is total with respect to failure: a pdf.js error, an unreadable page, or a scan that overruns its budget all degrade to no warning and a log entry, never to a pipeline failure. A missed warning costs a user one look at blank sheets; a warning that can fail the pipeline would cost a re-conversion that was not needed. ### Conversion provider modes | Value | Transport | | - | - | | `aps` | Autodesk Platform Services Design Automation v3 | | `fixture` | In-process deterministic provider that copies a canned PDF and makes no external calls | Set `STRALOS_CONVERT_PROVIDER` to select the provider. Terraform sets it per environment through `convert_provider`: dev and prod use `aps`, and test uses `fixture` until APS credentials are provisioned there. Local development and CI use `fixture`. APS credentials are resolved from `APS_CLIENT_ID` / `APS_CLIENT_SECRET` directly, or from SSM parameters named by `APS_CLIENT_ID_SSM_PARAMETER` / `APS_CLIENT_SECRET_SSM_PARAMETER`. ## Document ownership model Every document row is owned by exactly one of a project or a company (party). The schema enforces this with a `CHECK` constraint: `(project_id IS NOT NULL)::int + (party_id IS NOT NULL)::int = 1`. A row with both or neither is rejected at the database level. * **Project document** — `project_id` is set, `party_id` is `NULL`. Can carry an `estimate_id` when uploaded directly into an estimate area. Can be routed and processed. * **Company document** — `party_id` is set, `project_id` is `NULL`. Cannot be routed to an estimate (`POST /:id/route` returns `422`). Stops at `stored` after a clean scan. ### Document kinds and pinned slots The `doc_kind` column accepts `prequal`, `contract_agreement`, `day_rates`, and `other`. The pinned kinds — `contract_agreement` and `day_rates` — are company-only: `documents_party_kind_check` refuses them on a project document. A company has at most one current file per pinned kind, enforced by a partial unique index on `(organisation_id, party_id, doc_kind) WHERE doc_kind IN ('contract_agreement', 'day_rates') AND superseded_at IS NULL`. The slot handover works as follows: 1. A pinned upload is inserted with `superseded_at` already stamped (non-current) because the presigned POST handshake creates the row before the bytes exist. Stamping at insert rather than at completion means an abandoned upload leaves the previous file current. 2. When the scan handler marks the new file `stored`, it clears `superseded_at` on the new row and stamps the previous holder in the same database transaction. 3. Two concurrent uploads completing against an empty slot race to clear their `superseded_at`; the partial unique index rejects one side with `23505`, which rolls back cleanly and redelivers. On redelivery the winner is visible, the loser supersedes it, and the slot converges. An abandoned upload (one that never reaches `stored`) leaves the previous file current. Its `superseded_at` reads as "not current" rather than "replaced at T", which is a deliberate trade-off documented in the migration comment. ## Routing and processing A document uploaded at the project level becomes `stored` immediately after a clean scan, before parsing. It must be routed into an estimate before parsing and AI processing can begin. Company documents also become `stored` after a clean scan but cannot be routed. They have no project and `POST /:id/route` returns `422` for them. DWG and DXF files also become `stored` after a clean scan, but additionally trigger a conversion job that runs before the document can be processed. The document stays at `stored` while conversion is in progress. See [DWG and DXF conversion](#dwg-and-dxf-conversion) above. **Route**: `POST /api/documents/:id/route` links a document to an estimate. Set `estimateId: null` to unroute. Pass `process: true` to start processing in the same transaction once routing is committed. **Process**: `POST /api/documents/:id/process` starts AI processing independently. A first start requires `stored`, a clean scan verdict, a non-null `estimateId`, and — for DWG/DXF files — a completed conversion. It uses a compare-and-swap write; a retry or race that finds the clean document on the active processing path returns the benign `200 already_processing` outcome. Both the `/route` and `/process` endpoints enter the pipeline via the same scan-passed outbox event that a freshly uploaded document would produce, so the pipeline path is identical from that point on. Changing an estimate link or unroute requires a settled state (`stored`, `processed`, or `failed`). Re-routing a mid-pipeline document to its existing estimate is idempotent; moving it to a different estimate is blocked with `409`. ### Shared project document library (PRD 210 R14) A project can hold several estimates — a tender, copies, and a Delivery estimate — and they all read from the same document library. The `estimate_id` column records only the estimate a document was **first routed to**; it does not gate visibility. An estimate-scoped document list resolves the estimate's project and returns every document routed to that project, regardless of which estimate originally received it. `POST /:id/route` accepts any estimate that belongs to the document's project. Routing a document that is already in the project library — whether it was first routed to this estimate or to another estimate of the same project — is an idempotent success and never overwrites the stored first-routed value. **Reference ring on unroute and delete.** The reference check before unrouting or deleting a document queries every estimate that uses the document. This includes the first-routed estimate and any estimate the usage registry records as measuring the document (for example, a Delivery estimate that measures a document routed through the tender). A `404` from the first-routed estimate is a hard error; a `404` from a usage-registry estimate means that estimate no longer exists and carries no references. The variation assessment path is a parallel, standalone flow. The estimate service calls `POST /variation-assessments` at any point after documents are processed. The documents service enqueues a job and returns a `jobId`; the estimate service polls `GET /variation-assessments/:id` until the job reaches `complete` or `failed`. The variation assessment corpus is **project-scoped**, not estimate-scoped. The worker derives the job's project from its own stored contract documents and uses that project as the RLS scope when reading document extractions. All supplied contract documents must belong to the same project — a cross-project set is refused `422` at enqueue. ## Area modules and suggestion kinds Each area worker reads the document's extracted text, an optional context payload fetched from sibling services, and a document summary, then calls the AI gateway to produce a list of suggestion drafts. ### project\_rates area The `project_rates` area processes supplier and subcontractor quote documents. Each priced line of the quote becomes one `project_rate` suggestion. **Context reads.** Before calling the AI, the area worker fetches two context lists on the documents-worker's existing `rates:read` and `estimate:read` grants: * `GET /api/rates/rate-items?projectId=&length=200` — the Rates Library plus items owned by this specific project. Including `projectId` ensures a project-only item can be matched even when it does not appear in the shared library. * `GET /api/estimate/estimates//work-items?length=200` — the estimate's work items, so a quote line that names one can be linked to it. Each read carries a `service:documents-worker` bearer token minted by `api-global` for the document's organisation. The token is cached per org and re-minted near expiry. In deployed environments the worker invokes the `api-global-mint:mint-for-documents-worker` Lambda alias via IAM; in local development it uses the loopback mint route. Missing mint configuration is a hard failure — the area run fails with `ServiceMintNotConfiguredError` rather than reading its peers unauthenticated. A mint that is configured but fails degrades gracefully: the context reads proceed without a token, the peers reject them, and the area run gets an empty context. See [Service token authentication](https://help.stralos.app/internal/service-token-auth.md#caller-notes) for the full mint transport contract. Neither read returns supplier records. The supplier is carried as the quote's own printed words and the reviewer confirms it in the review card; this worker does not resolve the name to a Companies register party. **Suggestion payload.** Each `project_rate` suggestion carries: | Field | Description | | - | - | | `rateItemNameOrCode` | The item exactly as the supplier words it, including any supplier code. | | `matchedRateItemId` | The id of a matching rate item from the context read. Present only when the model was confident. | | `unit` | The unit price unit. | | `price` | The unit price, excluding GST where identifiable. | | `supplier` | The supplier or subcontractor name as printed on the quote. | | `supplierLocation` | The supplier branch, depot, quarry, or other supply location explicitly named for this price, copied as printed. Absent when the quote does not name one. | | `quoteReference` | The quote's own reference or number (e.g. `Q-4471`), when one is printed. | | `supplyOnly` | `true` when the quote is supply-only rather than supply-and-install. | | `validUntil` | The quote's valid-until date, normalised to `YYYY-MM-DD` (day-first, as NZ quotes print). A date that cannot be parsed is stored verbatim so the reviewer can correct it before accepting. | | `workItemRefs` | Optional array of work items the line is priced for, each carrying `number`, `name`, `matchedWorkItemId`, `quantity`, and `basis` (`per_unit` or `total`). Only included when the quote explicitly names work items. | | `notes` | Exclusions, conditions, or other notes. | | `unmatched` | `true` when no existing rate item was matched. An unmatched suggestion carries no `matchedRateItemId`. | **Date normalisation.** The AI copies the validity date exactly as printed (`validUntilAsPrinted`) and the documents worker normalises it before storing the payload. NZ quotes print dates day-first (`DD/MM/YYYY`); the normaliser also handles ISO (`YYYY-MM-DD`) and worded forms (`30 September 2026`). A date the normaliser cannot resolve is kept verbatim so the reviewer sees what the quote said. **Legacy kind.** Before S5, quote lines produced `rate_override` suggestions. Existing pending suggestions of that kind remain readable. The `payloadSchema` accepts both kinds, and the review card handles them identically. **Accepted suggestions.** Accepting a `project_rate` suggestion does not write directly into estimate data. The review card asks the user to confirm the supplier match and the rate-item match before the estimate service creates the project rate through `api-rates` and attaches it to the named work items through `api-estimate`. ### work\_items area The `work_items` area (`src/areas/workItems.ts`) extracts two kinds of suggestion from specifications, schedules, and bills of quantities: `scope_item` (a scope or BOQ line) and `testing_requirement` (a QA or testing obligation, with its clause reference and frequency). Each payload carries: | Field | Description | | - | - | | `kind` | `scope_item` or `testing_requirement`. | | `itemNumber` | The row's own reference, copied exactly as printed (`1.01`, `3.10`, `A-17`). Omitted when the row has none; never generated. | | `description` | The line or obligation. | | `quantity`, `unit` | When stated. | | `sectionHint` | The heading the row sits directly under, as number, space, title (`4.1 Stormwater`, `3.0 : Roading`). For spreadsheet sources the prompt tells the model to drop the CSV comma between a heading's number and title cells. | | `parentSectionHints` | The headings above `sectionHint`, outermost first, written the same way. Includes a heading with no rows of its own. Omitted for a top-level heading. | | `sectionOccurrence` | When two or more unnumbered headings share a title, which one (1-based) the row sits under. Omitted otherwise. | | `clauseRef` | The specification clause a testing requirement cites. Not the same as `itemNumber`. | | `frequency` | Required testing frequency, for example `per 500m2 per layer`. | | `possibleDuplicateOfId` | A suggestion this one may duplicate. | A row under `4.1 Stormwater` inside `4 Utilities` arrives with `sectionHint: "4.1 Stormwater"` and `parentSectionHints: ["4 Utilities"]`, so api-estimate can create section 4 on accept even when no row sits directly under it. Both new fields are optional: a suggestion extracted before they existed still imports, and api-estimate then works out parent sections from the number alone. ## Document deletion `DELETE /:id` is a soft-delete. The document row is stamped with `deleted_at` and `deleted_by` in a single transaction; the row, its extractions, area runs, suggestions, uploaded file, rendered pages, and conversion output are all preserved. A deleted document remains renderable from history and still resolves via `GET /:id` (with `deletedAt` set). `DELETE /:id` is blocked while the document is in any in-flight pipeline status (`scanning`, `parsing`, `summarising`, or `processing_areas`). Documents in other statuses, including `linked`, `uploading`, `stored`, `processed`, `failed`, `infected`, and `rejected`, can be deleted. **Transaction scope.** The delete handler takes a row lock on any current drive-sync mapping for the document, re-reads the document state under that lock, runs local reference checks, marks the mapping `excluded` (keeping `external_file_id` and `version_marker`), and stamps the document row in a single transaction. Clearing the mapping's `document_id` foreign key means a re-included file later lands as a new document rather than superseding the deleted one. **Reference check.** For a routed document, the handler calls the estimate service S2S to check for live references before opening its local transaction. The remote check is deliberately non-atomic with the local transaction. A routed document with live references returns `409 document_in_use`. If the estimate service is unreachable, the handler returns `502`. The local check (`localReferenceBlockers`) counts accepted suggestions from any area, pending suggestions from every area except `takeoff`, and area runs still pending or running. Pending takeoff suggestions do not block delete or unroute: no screen lists them and no review count includes them, because the takeoff decision is document-level. The same check guards unrouting. **Visibility after delete.** Document lists (`GET /api/documents?estimateId=`, `?projectId=`, `?partyId=`) and the suggestions summary (`GET /suggestions/summary`) exclude deleted documents. `GET /:id` continues to resolve the row and includes `deletedAt` in the response. Any write to a deleted document (`PATCH`, route, process, area run) returns `409` with `{ code: "deleted" }`. **Idempotency.** A repeat DELETE on an already-deleted document returns `204` with no change. ## Organisation data purge Soft-delete is the only user-facing delete. The one hard-delete path is the weekly organisation purge in the retention Lambda, which removes every documents row and every S3 object version for an organisation that has been disabled for at least 90 days (`ORG_OFFBOARDING_PURGE_AFTER_DAYS`). The cross-service policy, clock, status guard, purge order and erasure contract are on [Data retention](https://help.stralos.app/internal/data-retention.md#organisation-purge). Documents-specific details: * **Clock table.** `documents_org_offboarding` holds `organisation_id`, `first_disabled_at` and `purged_at`. It is platform-only (no runtime-role access) and is not cleared by the purge, so `purged_at` remains as the record that the data was removed. * **Row scope.** Every table is deleted on `organisation_id`, in one transaction, children before parents. The two nullable-organisation system tables, `documents_processing_events` and `documents_outbox`, also lose rows whose organisation is null but whose document belongs to the organisation. Soft-deleted documents are included. * **Legacy keys.** Documents stored before org-scoped storage keys are found through their rows, so their exact keys and per-document `thumbnails//` and `pages//` prefixes are deleted alongside the organisation prefixes. * **Per-user erasure.** The service holds only opaque user IDs, so `erase-user` keeps them and returns empty counts. For operations, see [Data retention operations](https://help.stralos.app/internal/runbooks.md#data-retention-operations). ## State machine The status model is monotonic and idempotent. Workers may receive duplicate or out-of-order events, so each stage advances from any earlier non-terminal state and no-ops when the document is already past that stage. ## Transactional outbox Workers write state changes and outbound events in the same database transaction. Dispatch happens after commit, and a sweep can redispatch undelivered outbox rows. ## Render offload Every page image and thumbnail of every renderable document is rendered ahead of time, on a dedicated render worker. The API Lambda never renders pages: a page that is not ready when a client asks for it is queued for the render worker. The one exception to not reading the PDF is the scale fill (`PUT /:id/takeoff-pages/:page/scale`), which reads page sizes from the cached `page-sizes.json` beside the renders and parses the PDF once, writing that file, when it is missing. The client-facing contract (`async=1`, `202` polling, the `503` wait for older clients) is in the [API reference](https://help.stralos.app/api/documents.md#page-render-endpoints). ### Render worker The `render-jobs` stage is its own Lambda. Each invocation handles one job: one page's full-size image and thumbnail, or one page's scale detection. An out-of-memory or timeout therefore costs one page one attempt, never the whole document. * **Memory.** A dense drawing page needs about 900 MB, and Lambda allocates CPU by memory. The Terraform variable `render_worker_memory_size` is sized for 4096 MB but defaults to 3008 MB, the Lambda memory cap on the current AWS accounts. Raise it to 4096 once the account limit allows. * **Two queues.** The worker reads an interactive queue (the `render-jobs` stage queue, for a page someone has open, with no batching wait) and a low-priority bulk queue (`render_bulk`, for upload-time and sweep work). Each queue's event source mapping has a maximum concurrency of 10 by default. The two caps bound concurrent renders and their cost; no Lambda concurrency is reserved. * **Attempts.** Each page job is attempt-counted in `docMetadata` (`pageRenders` and `scaleDetectionJobs`, keyed by page; `src/pipeline/pageJobState.ts`). An error is retried once in the same invocation, and a crash relies on SQS redelivery. After `MAX_PAGE_JOB_ATTEMPTS` (2) the page is recorded as failed. The render queues allow five receives, more than other stages, so receives spent on a throttled poller cannot exhaust a page's real attempts. * **Failure scope.** A failed page is a failed page, never a failed document. The S3 object is the completion marker. ### Render event types | Event | Handled by | What it does | | - | - | - | | `documents.render.thumbnails.requested` | Parse worker (fan-out) | Queues every missing page of the document on the bulk queue. Published when parsing completes, when an on-demand upload scans clean, and when a drawing conversion is published. | | `documents.render.pages.requested` | Parse worker (fan-out) | Queues the pages of a saved takeoff selection. Also used by the sweep's page retry. | | `documents.render.scale-detections.requested` | Parse worker (fan-out) | Queues scale detection for each page of a document the classifier found to be a drawing. | | `documents.render.page.requested` | Render worker | Renders one page's full-size image and thumbnail, and runs scale detection for a drawing. | | `documents.render.scale-detection.requested` | Render worker | Runs one page's scale detection (text, then vision). | The three fan-out events go through the transactional outbox. The per-page messages are sent straight to SQS by `src/pipeline/renderQueue.ts`. Each page is claimed before it is queued, so a page whose job is already on its way is skipped. Fan-outs are best-effort: a failed fan-out is logged and never dead-letters, because the parse DLQ would mark the document failed. The whole-document fan-out stamps `docMetadata.pageRenderFanoutAt`, which `GET /:id/thumbnails` uses to avoid queuing a second fan-out within 30 minutes. `GET /:id/page-thumbnails` keeps its own 30-second enqueue debounce. Thumbnail progress for that older endpoint is still tracked by the contiguous `thumbnailPageCount` checkpoint. **Converted drawings.** For a DWG with a `derivedPdfKey`, the render worker reads the derivative, not the original drawing. No render job is ever queued against the raw `.dwg` bytes. ### Sweep safety nets The sweep catches what the upload paths miss (`src/pipeline/pageRenderCoverage.ts`). Both nets queue onto the bulk queue, so they never sit in front of a page an estimator has open. * **Missed fan-out.** A renderable, scan-clean document uploaded in the last 24 hours with no `pageRenderFanoutAt` gets a fresh `documents.render.thumbnails.requested`. Documents younger than five minutes are left for their upload path. Each document gets at most three sweep requests, so an unreadable PDF stops being asked about. * **Failed or lost pages.** A page whose render failed more than an hour ago, or whose job has been pending for more than 15 minutes, is queued again, up to `MAX_PAGE_RENDER_AUTO_RETRIES` (3) times. Each sweep pass logs one line, `documents: sweep` followed by a JSON object of counts, including `pageRenderFanOutsRequested` and `pageRendersRetried`. Documents uploaded before ahead-of-time rendering are covered by `scripts/backfill-page-renders.mts`, run deliberately rather than by the sweep. ### Page image geometry When a full-resolution render is written to S3, the worker stores four geometry values in the object's user-metadata: `page-image-width`, `page-image-height`, `pdf-page-width-pt`, and `pdf-page-height-pt`. The first two are read from the PNG's IHDR (a direct measurement of the raster), and the latter two come from the pdf.js viewport used to produce the render (the PDF page box at scale 1 with rotation applied). `GET /:id/pages/:page/image` reads this metadata from the cached render's S3 head and returns it as `width`, `height`, `pageWidthPt`, and `pageHeightPt`. For a render produced before this metadata was introduced, the endpoint falls back to a ranged GET of the PNG header to recover `width` and `height`; the point values stay absent rather than guessed. Any field that cannot be established is omitted rather than defaulted. ### Standard fonts PDFs that name a standard font without embedding it (Helvetica, Times, and similar) rely on pdf.js's own copies of those fonts to rasterise their text. The Lambda has no system fonts, so any render that falls back to system fonts produces a white page for text-only content while line work beside it draws normally. The renderer is configured with an explicit `standardFontDataUrl` pointing at the `standard_fonts/` directory copied beside the bundle by `scripts/build-lambda.sh`, with `disableFontFace` and `useSystemFonts: false` set so that the Lambda and a local machine behave identically regardless of installed system fonts. The pipeline throws rather than rendering without the font data, because a silent white-page render is worse than a logged failure. Pages already cached white by earlier builds are not automatically re-rendered; deleting the affected S3 objects under `thumbnails/` or `pages/` causes the render worker to regenerate them with the correct fonts. ### WASM image decoders PDFs containing JPEG2000 (`/JPXDecode`), JBIG2 (`/JBIG2Decode`), or CCITT fax (`/CCITTFaxDecode`) images require pdf.js's WASM decoders to rasterise those images. Without the decoders, affected images render as white holes, including JPEG images that use a JPX soft mask, where a failed mask decode drops the entire parent image. The renderer passes a `wasmUrl` pointing at the `wasm/` directory shipped beside the Lambda bundle by `scripts/build-lambda.sh` (`openjpeg.wasm` for JPEG2000 and `jbig2.wasm` for JBIG2/CCITT). The pipeline throws at cold start if either file is absent. Pages already cached white by earlier builds are not automatically re-rendered; deleting the affected S3 objects under `thumbnails/` or `pages/` causes the render worker to regenerate them correctly. ### pdf.js RGB image patch `pdfjs-dist` 6.1.200 (the version `unpdf` 1.8.1 bundles) has a bug in `convertRGBToRGBA`: the tail loop for the last pixels of each 16-row chunk restarts from offset zero, so those pixels are copied from the top of the image and RGB images show artefacts. The service carries a `pnpm patch` (`patches/pdfjs-dist@6.1.200.patch`) for the four pdf.js build files until an `unpdf` release bundles a fixed pdf.js. **Page count backfill.** The render worker writes the authoritative `pageCount` when the parse stage never set it (guarded to `IS NULL`), ensuring the `page-thumbnails` GET can eventually report `complete`. ## Automatic scale detection The render worker reads the scale each drawing page states and sets it on the takeoff page selection when there is one clear scale (`src/pipeline/takeoffAutoScale.ts`). Specifications and contracts are never detected: their "1:N" values are slopes and ratios. **Which documents.** Two ways in: 1. **At render.** A document that is a drawing from upload is detected page by page as it renders: a takeoff-panel PDF import (the `takeoffAutoApprove` marker) and a converted DWG or DXF drawing. 2. **At classification.** For other uploads (the Documents page, drive sync), when the classifier finds the document is a drawing, the classify transaction marks it (`docMetadata.autoScaleDrawing`) and writes `documents.render.scale-detections.requested`. A document counts as a drawing when the classifier finds `takeoff` relevant at the routing threshold, or when the uploader targeted takeoff. A drive-sync new version's targets do not count, because they are copied from the previous version. Pages already rendered are not rendered again. Sync-attempt rows are never written to. Every path claims the page's one scale-detection slot and reads the cached answer first, so a page is never detected twice. **One clear scale** (`src/lib/takeoffScale/autoScale.ts`) means: * The detector's preselected candidate is the only candidate, or the only title-block scale among view titles. Two title-block scales, or a long section's horizontal and vertical scales, are not clear. * The drawing names no sheet size, names the page's own size, or names another ISO A size the page is a standard reduction or enlargement of. For example, `1:200 @ A1` on an A3 page is set as `1:400`, with `convertedFrom` recording what the drawing said. * A size mismatch on a non-standard or cropped page, sheet sizes that disagree, and "not to scale" drawings are not clear. Those detections are cached so the scale side sheet opens at once, but nothing is written. **Existing scales.** The write runs under the row lock and only applies to a page that is unscaled or has a `filled` scale. A `manual` or `auto` scale is never replaced. The page entry records `scaleSource: "auto"` and a `scaleAuto` object (`detectedFrom`, `detectedAt`). `DELETE /:id/takeoff-pages/:page/scale` clears it. **Same-size fill.** When an estimator sets a scale with `PUT /:id/takeoff-pages/:page/scale`, unscaled pages of the same sheet size are filled, except pages whose cached detection says they are not to scale. Those are reported in `skippedPages.notToScale`. Historic drawings can be given the same treatment with the backfill script's `--scale` option. ## Queues and DLQs The pipeline uses stage queues for upload events, scan results, parse jobs, parse results, summary jobs, classify jobs, area jobs, variation-assessment jobs, convert jobs, render jobs, and tender-check jobs. Render jobs run on the dedicated `render-jobs` worker, which also reads the `render_bulk` queue (see [Render offload](#render-offload)). Convert jobs run on a dedicated `convert-jobs` queue with a longer timeout to accommodate APS Design Automation processing. Text-extraction-only requests for a tender check are sent straight to the `parse-jobs` queue, without the outbox. Each queue has a DLQ and the consumer preserves payload evidence for investigation. The terminal state depends on the workload: ordinary document stages mark the document failed; conversion exhaustion leaves the original drawing stored and marks only conversion terminally failed; a variation-assessment job becomes failed; and a current drive-sync run marks its connection errored. ## Stage observability Terraform defines the following CloudWatch alarms for the service. Regional workload alarms publish to the per-environment alerts topic from the core `environment-setup` remote state. The optional `alarm_sns_topic_arn` variable can override that topic with another same-account ARN. The `us-east-1` cost alarms use the separate topic described below. **DLQ-depth alarms** (`cloudwatch.tf`) — one per stage DLQ. Any message visible on a DLQ means a document exhausted all retries. **Worker error alarms** (`cloudwatch.tf`) — one per stage worker Lambda. Trips on one or more errors in a five-minute period. **Sync Lambda error alarms** (`sync.tf`) — cover the sync-dispatch and sync-worker Lambdas. **API entrypoint, DLQ consumer, and sweep alarms** (`alarms.tf`) cover: * The HTTP API entrypoint Lambda (`-api`). Errors alarm trips at three or more errors in five minutes (absolute count); a separate error-rate alarm trips at 20% or higher when at least 10 invocations are present in the window. * Every DLQ consumer Lambda: one per stage queue, plus the sync DLQ consumer. Each trips on the first error, because a dead consumer is otherwise indistinguishable from a healthy-idle queue. * The scheduled sweep Lambda. Trips on the first error. Each of those Lambdas also carries a throttles alarm that trips on the first throttle in five minutes. **API Gateway 5xx alarm** (`alarms.tf`) — watches `AWS/ApiGateway 5XXError` scoped to the REST API stage. An HTTP 5xx from the Fastify handler is a successful Lambda invocation and is invisible to the Lambda error alarms; this alarm covers application and dependency failures that surface as HTTP 500-level responses. **Sweep dead-man's-switch** (`alarms.tf`) — uses `AWS/Events SuccessfulInvocationAttempts` (scoped to the EventBridge sweep rule) rather than a Lambda metric. Trips when fewer than one successful delivery is recorded in three consecutive ten-minute windows (\~30 minutes). This catches both a stopped schedule and a schedule that fires but fails to invoke its target, neither of which produces Lambda errors. **Alarm coverage check** (`alarms.tf`) — a Terraform `check` block reads the live Lambda list and asserts that every `{name_prefix}` or `{name_prefix}-*` function is either newly alarmed in `alarms.tf` or already covered by the worker or sync alarms. A gap surfaces as a named plan-time finding without blocking a deploy. **Cost alarms** (`cloudwatch.tf`) — A Textract `EstimatedCharges` alarm runs in `us-east-1` and uses a separate `cost_alarm_sns_topic_arn` variable because CloudWatch alarm actions must be in the same region as the alarm. A monthly AWS Budget provides a broader cost guardrail. Model-spend alarms belong to `api-assistant`, the sole deployed model caller; this service does not carry Bedrock or other model cost alarms. ## Regional storage `region_key` is the stable product-region identifier (default `au-nz`). The workload AWS region is derived from `aws_region_by_key` (`au-nz` maps to `ap-southeast-2`), so a resource's product region and provider region cannot diverge. The default S3 bucket and EventBridge bus names are `{project}-{environment}-{region_key}-documents`. Stage queues use the same region-qualified prefix with the service name, for example `{project}-{environment}-{region_key}-api-documents-`. Set `documents_bucket_name` only when a deliberate bucket-name override is required. ### Browser upload CORS The bucket CORS rule permits `POST`, `GET`, `PUT`, and `HEAD` from `cors_allow_origins`, with all request headers allowed and `ETag` exposed. When the variable is empty, the bucket fallback is `http://localhost:5173` for local development. When non-empty, the configured list is applied to the bucket and passed to the API Lambda through `CORS_ORIGIN`. The empty-list API fallback differs: the Fastify API allows HTTP(S) origins on `localhost` at any port. The per-environment `cors_allow_origins` values are: | Environment | Allowed origins | | - | - | | `dev` | `https://au-nz.dev.stralos.app`, `http://localhost:5173` | | `test` | `https://au-nz.test.stralos.app` | | `prod` | `https://au-nz.stralos.app` | Test and prod intentionally omit `localhost`. An empty `cors_allow_origins` in either of those environments would silently fall back to `localhost` only, causing presigned uploads from the deployed shell to fail with a CORS error. The values above match the shell origins in `stralos-core app-hosting/.tfvars` (`app_domain_aliases`, `shell_origins`), consistent with `stralos-api-rate-extract` and `stralos-api-assistant`. ## Peer API service discovery Documents resolves its Global, Estimate, and Rates API base URLs from: 1. A direct `GLOBAL_API_URL`, `ESTIMATE_API_URL`, or `RATES_API_URL` override. 2. The SSM parameter named by the corresponding `*_SSM_PARAMETER` variable, read through the shared five-minute cache. Workers that read a peer URL lazily retry an SSM throttle with full-jitter exponential backoff (four retries, 200 ms base; `src/config/peer-api-urls.ts`), because a burst of cold starts can exhaust Parameter Store's shared throughput. ### API Lambda SSM preload The API Lambda loads every SSM parameter it reads once at cold start, before it serves a request: `preloadApiSsmParameters()` in `src/config/ssm-preload.ts`, called from `lambda.ts` during INIT. It uses the batched `GetParameters` preload from `api-common` (0.18.0 or later), which reads up to ten names per call and retries throttling with jitter. * **Required:** the three database URL parameters and the Global and Estimate API URL parameters. If one cannot be loaded, INIT fails and Lambda retries the cold start. * **Optional:** the OCR provider, scan provider, DWG uploads and drive-sync kill switch parameters. These fail soft. After INIT, values are served from memory. An expired value is served while it refreshes in the background, so no request waits on SSM. The drive-sync kill switch therefore still reaches warm containers within its five-minute TTL. Flags that `hydrateRuntimeConfig` copies at cold start, such as the DWG uploads flag, do not. `RATES_API_URL_SSM_PARAMETER` is not on the API list: the API Lambda does not read it. `test/ssmPreload.unit.test.ts` fails when a request-path `*_SSM_PARAMETER` read is missing from the list. The API role has both `ssm:GetParameter` and `ssm:GetParameters`. There is no localhost fallback. If neither variable is set, the resolver throws a configuration error that names both variables, for example `api-global URL is not configured: set GLOBAL_API_URL or GLOBAL_API_URL_SSM_PARAMETER`. Callers resolve the URL before the `try` block around the peer request, so the error is never reported as the peer being unreachable or degraded to an empty result. `stralos-local` sets the direct variables, and `.env.example` sets them to the local ports (`3300`, `3303`, and `3301`) for standalone runs. Deployed functions receive the `*_SSM_PARAMETER` names from Terraform. Terraform derives each parameter name from `////services//api-url` unless the matching `global_api_url_ssm_parameter`, `estimate_api_url_ssm_parameter`, or `rates_api_url_ssm_parameter` variable overrides it. The `area-jobs` worker receives all three parameter names and read permissions. Other stage workers do not receive these peer-URL SSM permissions. ## Project file delivery URL origin The `/:id/content` endpoint serves checked project files (original downloads, thumbnails, and page images) through a ticket issued by the Global API. URLs pointing to this endpoint are constructed by `projectFileUrl()` in `src/routes/projectFileDelivery.ts`. In a deployed Lambda behind API Gateway, `request.host` resolves to the API Gateway's internal host. That host omits the required stage-path prefix, producing broken URLs. The `DOCUMENTS_PUBLIC_BASE_URL` environment variable overrides the origin so generated URLs use the public edge domain instead. **Resolution order for the URL origin:** 1. `DOCUMENTS_PUBLIC_BASE_URL` — explicit public origin, set by Terraform in all deployed environments. 2. Fallback: `https://{request.host}` (Lambda) or `{request.protocol}://{request.host}` (local/non-Lambda). The fallback is correct only when the service is reached directly, not through API Gateway. **Per-environment values (set in Terraform `tfvars`):** | Environment | `documents_public_base_url` | | - | - | | `dev` | `https://au-nz.dev.stralos.app` | | `test` | `https://au-nz.test.stralos.app` | | `prod` | `https://au-nz.stralos.app` | The value must be an HTTPS origin with no path, query string, or fragment. Terraform validates this at plan time. Leave `DOCUMENTS_PUBLIC_BASE_URL` unset in local development — the fallback builds the correct URL from the local server's protocol and host. ## Outbound backend contracts The service makes outbound calls to peer APIs. The contracts below are audited against the sender implementation and note where coverage limits exist. ### BC-12: Delegated project existence **Route:** `GET /api/global/projects/:id` on the Global API. **Sending implementation:** `src/sync/projects.ts`. **Existing sender tests:** `test/syncRouting.test.ts`, `test/dwgDemand.test.ts`. DB-backed sender route tests use a simulated Global project endpoint and cover known versus unknown project bindings. **Coverage limits.** The sender currently checks HTTP status only and does not parse success JSON. The stub returns `{project:{id}}`, while Global returns the project DTO at the top level. This does not break the current status-only client but cannot prove response-shape compatibility. No dedicated client unit suite exists. **Maintaining both sides.** For a change to a listed method, path, parameter, required header, status code, or response field: 1. Use the same contract ID in the sender and receiver notes and associated test descriptions. 2. Update the receiver tests to pin accepted/rejected inputs and actual response JSON. Update sender tests to assert the real outgoing request and its handling of valid, missing, null, malformed, and error responses. 3. Keep fixtures compatible with the receiving API, including UUID formats, envelopes, and optional fields. Do not only update a mock to make a caller test pass. 4. Review and run both repositories' relevant suites and update both notes. Preserve existing integration and security checks. Existing sender tests are not consistently tagged `[backend-contract]` and are run by the normal CI suite rather than a dedicated contract suite. A failure fails the sender's own test job; a receiver change does not automatically run sender CI. Independent green suites do not prove compatibility against each other's current implementation. ## DWG and DXF acceptance flag The Documents API includes a runtime flag that gates whether `.dwg` and `.dxf` uploads are accepted for conversion. The flag is off by default in every environment and is controlled independently of the DWG converter configuration. **What the flag gates.** `STRALOS_DWG_UPLOADS_ENABLED` gates `validateFile` when an upload is presigned — whether a drawing upload is accepted at all. It does not gate `fileKindFor`, which resolves the file kind unconditionally so that documents accepted while the flag was on continue to resolve correctly if the flag is later turned off. **What happens when the flag is off.** A `.dwg` or `.dxf` upload still reaches the server, is rejected with a `status: rejected` document row and `status_detail` beginning `dwg_unsupported:`, and is never presigned. No bytes are transferred. The rejected row is the demand metric used to track drawing upload interest. **Drive sync.** Drive sync links DWG and DXF files like any other file. Their fetch verdict uses the same `validateFile` check, so a drawing is processable only while the flag is on; otherwise it can be fetched and downloaded but not processed. **Estimate-scoped uploads.** Even when the flag is on, DWG and DXF files uploaded directly into an estimate area (the `auto` upload target) are rejected with `dwg_estimate_upload:`. Drawings must be uploaded to the project Documents page. **Resolving the flag value at runtime.** The service resolves the flag at cold start through `hydrateRuntimeConfig`. Resolution order: 1. `STRALOS_DWG_UPLOADS_ENABLED` direct environment variable. 2. The SSM String parameter named by `STRALOS_DWG_UPLOADS_ENABLED_SSM_PARAMETER`. 3. Default: `false`. Only the exact string `"true"` (case-insensitive, trimmed) enables the flag. Any other value — `"1"`, `"yes"`, an empty string, a typo — is treated as off. This is deliberate: the flag controls a metered conversion API and must never fail open. **Toggling in a deployed environment.** Terraform seeds the SSM parameter at `///api-documents/dwg-uploads-enabled` with value `false` and then ignores subsequent changes (`lifecycle { ignore_changes = [value] }`). To enable DWG uploads without a Terraform apply, use the AWS console or CLI to update the parameter value to `"true"`. The service reads it only when a new Lambda execution environment initialises, so recycle the API Lambda's execution environments through the supported deployment process or wait for natural replacement; invoking an existing warm environment does not reload the value. **Local development.** Set `STRALOS_DWG_UPLOADS_ENABLED=true` in `.env` to admit DWG uploads locally. Set `STRALOS_CONVERT_PROVIDER=fixture` to have conversion copy a canned PDF instead of calling Autodesk Design Automation. ## Provider modes Provider modes are configurable: * AI: central `api-assistant` gateway (Gemini, deployed and local), or fixture (CI/tests). * OCR: Textract (deployed), local parser with Gemini for scanned files (local), or fixture. * Scan: GuardDuty or stub. * Convert: APS Design Automation or fixture. ### PDF text layer extraction Before invoking any OCR engine, the parse worker checks whether a PDF already carries a usable native text layer. If it does, the text is read directly — no OCR job is started. The check applies in both `textract` and `local` OCR provider modes. In `textract` mode, Textract is invoked as `StartDocumentTextDetection` (plain text only, not `AnalyzeDocument`). For a PDF that already has a text layer, Textract would re-derive characters that are already present, at a per-page charge, through an async job. Reading the text layer directly is faster, free, and more accurate — these documents are primarily prices and quantities, where OCR guesses digits. In `local` mode there is no OCR engine: a scanned PDF that lacks a usable text layer, and any TIFF or image file, fails immediately as an unsupported file (deterministic failure, no SQS retry). Use `textract` or `fixture` for those file types. **Usability rule.** The check is per page, not an average. An average is skewed by the most common failure case: a scanned price list with one text-bearing cover page clears a 30-char-per-page average comfortably, and the scanned pages behind it are then returned as empty text rather than being OCR'd. The rule fails toward OCR — at least 80% of pages must carry 100 or more characters, or the whole document goes to OCR. A small number of blank or logo-only pages is tolerated: a one-in-five ratio of text-less pages does not trigger OCR. This is a known residual: a document whose last page is a scanned appendix is read digitally and that page's content is lost. Closing this gap requires classifying text-less pages as blank-versus-image, which the text API cannot see. | Condition | Outcome | | - | - | | PDF with usable text layer (≥80% of pages carry ≥100 chars) | Read directly; provider recorded as `pdf-text` | | PDF without usable text layer | Routed to Textract (deployed) or Gemini (local) | | TIFF or image | Routed to Textract (deployed) or Gemini (local); no text layer is possible | | Password-protected or XFA PDF | Fails fast with `unsupported_pdf`; not routed to OCR | A password-protected, encrypted, or XFA PDF is always rejected immediately. Routing it to OCR would spend a Textract call on a document nothing can read and surface the same failure later and asynchronously. DOCX, XLSX, and CSV files are always parsed natively in both provider modes and are not affected by this check. Local development should keep API behaviour, prompts, schemas, queues, and state transitions aligned with AWS mode. ### Local OCR with Gemini When `STRALOS_OCR_PROVIDER=local`, scanned PDFs, PNG/JPEG images, and TIFFs that have no usable text layer are sent to the Gemini `generateContent` API directly. This path is used for local development only; deployed environments (dev, test, prod) use Textract (`ocr-provider` SSM = `textract`) and are unaffected. The local provider calls Gemini directly rather than through the `api-assistant` structured-completion Lambda, because that protocol accepts only a string prompt and cannot carry a file attachment. **Configuration.** Set these variables in `.env` for `stralos-api-documents`: | Variable | Purpose | | - | - | | `GOOGLE_API_KEY` | Required. The Gemini API key used for local OCR. The workspace `scripts/setup-local-env.sh` writes dev's key; without dev access, paste your own Gemini key. Never commit a real key. | | `GOOGLE_MODEL_ID` | Optional. Defaults to `gemini-3.6-flash`. | **File size limit.** Files larger than 32 MB are refused deterministically. The pipeline records this as a parse failure immediately rather than retrying, because a larger file cannot succeed. **Truncation guard.** If Gemini stops generating before the end of the document (`finishReason` is not `STOP`), the response is refused and the job fails. Storing a half-extracted document would silently lose content. **Digital formats unaffected.** DOCX, XLSX, CSV, and PDFs with a usable text layer are parsed natively and never call Gemini. ### AI provider selection In deployed environments, all document AI calls (summary, classification, area extraction, and variation assessment) are routed through the `api-assistant` structured-completion Lambda gateway. The documents service invokes that function synchronously using `STRUCTURED_COMPLETION_FUNCTION_ARN`, which Terraform binds to the structured-completion function's `live` alias. The IAM policy allows both the bare function ARN and the alias ARN. The API Lambda gets the same variable as the AI workers, because tender scope drafts and filename recommendations call the gateway from the request path. Provider, model, and credential selection are owned entirely by `api-assistant` and are not configurable from `stralos-api-documents`. Local development uses the same gateway: stralos-local starts api-assistant's structured-completion container (Gemini) and the service reaches it through `STRUCTURED_COMPLETION_AWS_ENDPOINT_URL`. Set `AI_PROVIDER=fixture` to select the in-process deterministic provider instead, as CI and the automated tests do. The `fixture` value cannot be set via SSM. In deployed environments `AI_PROVIDER` is unset; the service uses the central gateway. AI worker Lambdas (summary, classify, area, and variation-assessment stages) receive an extended Lambda timeout (`ai_worker_lambda_timeout`, default 900 seconds) to accommodate two sequential gateway completions when a validation retry occurs. `STRUCTURED_COMPLETION_AWS_ENDPOINT_URL` is separate from `AWS_ENDPOINT_URL` because local document storage uses LocalStack, while the structured-completion container has its own local endpoint (`http://127.0.0.1:9001` in `.env.example`). When it is unset, the service falls back to `AWS_ENDPOINT_URL`. Terraform sets neither, so deployed environments use the standard regional Lambda endpoint. ## Variation assessment pipeline The documents service also runs contract variation assessments on behalf of the estimate service (S2S). The flow is separate from the document-upload pipeline. 1. The estimate service calls `POST /variation-assessments` with an issue description, contract profile, TAGs, work items, and contract document IDs. 2. The route validates the request, snapshots the resolved form guidance alongside the stored job row, writes the job and its outbox event in a single database transaction, then dispatches only that one event (bounded enqueue) before returning 202. No AI work runs inline. 3. A worker picks up the `documents.variation-assessment.requested` event, claims the job with an atomic lease, runs bounded retrieval over the contract document extracts, calls the AI provider, validates all citations against the supplied references, and writes the result. 4. The estimate service polls `GET /variation-assessments/:id` until the job reaches `complete` or `failed`, then persists the result. ### Guidance snapshot and provenance At enqueue the route resolves the contract-form guidance (terminology, clause hints, guidance depth) from `contractForm` and stores it in the job row as `resolvedGuidance`. At execution the worker prefers this snapshotted guidance over a fresh derivation. This means a terminology deployment between enqueue and execution cannot cause the guidance reported on the GET response to differ from the guidance that shaped the prompt. Job rows created before M34 do not carry `resolvedGuidance`; the worker derives guidance fresh for those rows. ### Verdict-aware fixture mode In `fixture` AI provider mode, the variation assessment fixture synthesises a deterministic assessment from the actual retrieved windows, TAGs, and work items at run time. Unlike the file-based fixture used by area modules, it cannot use a static JSON file because citations reference server-issued window IDs that only exist at run time. The fixture selects its verdict from the content of the issue text and supplied TAGs, making local E2E development possible without the central AI gateway. ## Tender check The documents service also runs AI tender checks for the estimate service, as an asynchronous job separate from the per-document area flow. Jobs run on the `tender-check-jobs` worker; the endpoints are listed in the [API reference](https://help.stralos.app/api/documents.md#tender-check-endpoints). For the job lifecycle, snapshot, prompt and review rules, see [Tender check](https://help.stralos.app/internal/tender-check.md). Schedule checks (shown as **Schedule review**) run on the same endpoints and worker, told apart by `checkKind: "schedule"`; see [Schedule check](https://help.stralos.app/internal/schedule-check.md). ## Takeoff review model Takeoff uses a document-level Approve/Reject decision rather than per-suggestion acceptance. * `PUT /:id/takeoff-review` writes `takeoffReviewStatus` (`approved` or `rejected`) and `takeoffReviewedAt` in an atomic conditional update that requires `status = 'processed'`. A concurrent area rerun that moves the document out of `processed` cannot slip past this precondition. * The decision is idempotent and overwritable. A later call refreshes `takeoffReviewedAt`. * The takeoff panel derives drawable documents from `takeoffReviewStatus = 'approved'`. The per-page selection stored by `PUT /:id/takeoff-pages` is independent and unchanged by a review decision. * Takeoff `page_classification` suggestions are excluded from every pending-review count (document detail `pendingSuggestions`, the suggestions list badge, and the `GET /suggestions/summary` aggregates). Only the document-level Approve/Reject decision matters for takeoff. Historical accepted/rejected takeoff suggestion rows are preserved but do not inflate pending counts. For the same reason, a pending takeoff suggestion does not block deleting or unrouting the document; an accepted one does. ### Auto-approval for takeoff-panel imports A PDF uploaded directly from the takeoff panel — an estimate-scoped upload with `targetAreas` containing `takeoff` — is eligible for automatic approval. At upload time, `POST /api/documents` stamps `docMetadata.takeoffAutoApprove: true` on the document row. When the takeoff area run completes successfully, the transaction that advances the document to `processed` also sets `takeoffReviewStatus = 'approved'`. The auto-approval is conditional on all of the following: * The document is being advanced to `processed` (i.e. its takeoff run completed successfully; a failed or skipped run does not approve). * `takeoffReviewStatus` is still `NULL` at the time of the write. A user's prior rejection is preserved; a re-run does not resurrect approval. * `docMetadata @> '{"takeoffAutoApprove": true}'` — the upload-time marker is present. * The document is not soft-deleted. The marker is set at upload time rather than derived from `target_areas` because `target_areas` cannot distinguish a takeoff-panel upload from a Documents-page multi-area import or a drive-sync re-version, both of which must keep the pending review. Only PDFs qualify. The upload endpoint checks `fileKindFor(fileName, fileType) === 'pdf'` before stamping the marker, because the takeoff canvas requires a renderable PDF. A spreadsheet or other kind whose takeoff run completes would have no pages to offer; without this guard it would have been auto-approved. The `autoApproveTakeoffImport` helper is called in two places: inside the `maybeCompleteDocument` finalisation path in `areaJobs.ts` (the normal single-area completion path) and inside the classify-job no-runnable path (a redelivery or `/retry` re-drive where every area was already terminal at classify time). ### Sync-attempt guard on takeoff writes In-flight sync provisioning creates temporary document rows (`sync_attempt = true`) that are internal to the sync engine and must never be user-mutable. The page-add and page-replace routes include `sync_attempt = false` in the predicate of their locked ownership reads, so a sync-attempt row is never locked by a user request. The takeoff-review route enforces the same boundary through an atomic conditional update, then uses an existence read only to distinguish a missing document from an invalid processing state. All three routes return `404` for a sync-attempt ID, making it indistinguishable from a document that does not exist. ## Lambda connection pool management The API Lambda creates up to three Postgres pools at cold start — owner, runtime, and platform — each capped by `LAMBDA_POOL_BOUNDS` (`max: 2`, `connectionTimeoutMillis: 5000`). A Lambda container serves one request at a time, so a pool of two is sufficient: one connection for the active transaction and one spare for a helper that needs to check out while that transaction is open. Without a cap, `pg` defaults to `max: 10`, and three pools across dozens of parallel containers can exhaust a database's connection limit within a single page load. The 5-second connection timeout surfaces a saturated database as a retryable `503` rather than hanging until API Gateway's 29-second limit. ### Idle connection release A frozen Lambda container never runs `pg`'s idle timer, so any connection kept open while the container is idle continues to hold a Postgres slot until the container is reaped. The Lambda handler releases idle connections in a `finally` block at the end of every invocation, after the response is sent and telemetry is flushed. Keep `DB_KEEP_IDLE_CONNECTIONS` unset or `false` in Lambda, including behind RDS Proxy. Tenant `set_config` calls pin a client session, so retaining idle clients would retain pinned database connections and recreate connection exhaustion behind the proxy. | Variable | Default | Purpose | | - | - | - | | `DB_KEEP_IDLE_CONNECTIONS` | unset (release on invocation end) | Leave unset or `false` in Lambda so tenant-pinned idle clients are released after each invocation. | ## Tenant isolation Twelve Documents tables enforce FORCE RLS on `organisation_id` (org-security M4). Nine tenant tables require a non-null organisation. The outbox, processing-events, and variation-assessment-job tables keep the column nullable for system rows, which remain visible only through the platform BYPASSRLS connection. Authenticated tenant ownership and read phases run on the `stralos_runtime_documents` connection with the `app.current_org` GUC set; processing transitions use the platform pool described below. Tenant writes — presign, process, route, retry, area run, suggestion accept/reject/patch — require an authenticated caller with a non-null organisation UUID. Anonymous requests receive `401`; platform or null-org service tokens receive `403`. This is enforced before any database query. The `/process`, `/route`, and `/retry` handlers use a two-pool split within one request. Ownership reads run on the runtime (`stralos_runtime_documents`) connection with the caller's `app.current_org` GUC, so FORCE RLS confines the read to that organisation. State transitions and outbox writes then run on the platform (`stralos_platform_documents`) BYPASSRLS connection. This keeps the processing CAS and its outbox side-effect atomic even though outbox rows may have a nullable `organisation_id` for system events. ### Project-membership enforcement Migration `0014_project_membership.sql` adds restrictive `project_membership` policies to every table that carries a `project_id` column. The runtime role can only read or write rows whose `project_id` is in the `app.project_ids` session array (or is `NULL`). Descendant tables without their own `project_id` inherit project scope transitively through their nearest project-owning parent. ### Cutover requirement for migration 0014 Migration `0014_project_membership.sql` is a `CUTOVER_REQUIRED` migration. Old API instances lack the actor and project scope that the new restrictive policies expect, so they would fail closed under the new RLS. Pause the API across this migration and code activation; coordinate authority, background workers, and any historical backfill before resuming traffic. See [Documents API: project-membership migration](https://help.stralos.app/internal/runbooks.md#documents-api-project-membership-migration-api-documents) in the runbooks. Pipeline workers use two database connections. * **Discovery connection** — the owner pool (`DATABASE_URL`). Workers directly query stored routing metadata to derive the organisation, project, and estimate scope for a document, connection, job, object key, or Textract job ID. The envelope's candidate `organisationId` is never authority and must agree with the stored row. * **Runtime connection** — the `stralos_runtime_documents` pool (`RUNTIME_DATABASE_URL`). All handler work runs on this FORCE-RLS-subject connection. Before the handler runs, `withWorkScope` checks out a connection, sets the session GUCs `app.current_org`, `app.project_ids`, `app.estimate_ids`, and `app.worker_scope` inside a transaction, and passes the scoped handle to the handler. For document-keyed messages, the runtime dispatcher independently calls the `resolve_org_for_document` SECURITY DEFINER function, which returns only the organisation ID; if that resolver is absent or errors, it logs and drops the message. Because the GUC is set transaction-local (`SET LOCAL`), neither a commit, an error, nor connection pooling can carry organisation context from one job to the next. After the handler returns, the connection is released. Maintenance tasks (the scheduled sweep and sync dispatch) follow the same two-pool split. `runScopedMaintenance` iterates persisted tenant and project partitions derived from the database; each partition runs independently scoped through the runtime connection. Workers do not use the `stralos_platform_documents` BYPASSRLS connection. The `app.worker_scope` GUC preserves maintenance-worker access to `documents_processing_events` for documents that have no project, such as company documents or documents mid-cutover. The scope is resolved from stored routing metadata rather than the message envelope. The `GET /api/documents/documents/status` S2S bulk-status endpoint is also org-scoped. A caller can only confirm document IDs that belong to its own organisation; foreign IDs are silently omitted from the response. Every authenticated request also enforces project-level access. Before any query runs, `runScoped` calls the Global API's `GET /api/global/projects/access-scope` endpoint and receives the caller's accessible project IDs. Those IDs are written into the `app.project_ids` transaction GUC. A restrictive RLS policy on every `project_id`-bearing table (and their FK descendants) uses the `project_scope_allows` function to check the GUC, so rows belonging to a project outside the caller's scope are invisible even when the organisation matches. If the scope check cannot reach the Global API, the request fails with `503 project_access_unavailable` and a `Retry-After: 5` header. If the caller is not a member of the requested project, the result is a `403 project_access_required` response. A row blocked by the RLS policy at query time also produces `403 project_access_required`. For the full project membership RLS architecture, see [Organisation security — project membership enforcement](https://help.stralos.app/internal/org-security.md#project-membership-enforcement). For the full isolation architecture including database principals and migration order, see [Organisation security](https://help.stralos.app/internal/org-security.md). ## Machine-readable API contract The service exposes the OpenAPI 3.1 contract for its descriptor-backed business routes at `GET /api/documents/openapi.json`. Health, version, and OpenAPI infrastructure routes are excluded. The document is built once at boot from the live Zod route descriptors (the same objects the handlers parse with) and served as cached bytes — byte-stable per deploy and never recomputed per request. This endpoint is auth-exempt and returns `cache-control: no-store`. The source repository uses two separate gates: the drift test compares descriptor-backed operations with the registered business routes, while the snapshot test requires byte equality with `src/openapi/openapi.snapshot.json`. ## Authentication The Documents API accepts EdDSA user and service tokens verified against the Global API JWKS. A service token's audience must include `stralos-api-documents`. Missing, invalid, or HS256 credentials receive `401` on protected routes. See [Authentication](https://help.stralos.app/api/authentication.md) for the token contract and [Service token authentication](https://help.stralos.app/internal/service-token-auth.md) for the caller allowlist. `AUTH_JWKS_URL` is required. `buildApp` throws `AUTH_JWKS_URL is not configured` before any plugin registers when it is unset or blank. Terraform supplies it from `var.auth_jwks_url` and `stralos-local` sets it locally. ### Billing access guard After token verification, the auth plugin parses the optional `billing` claim and runs `registerBillingAccessGuard`. When `BILLING_ENTITLEMENTS_TABLE` is configured, the guard authoritatively checks DynamoDB before user writes such as presign, process, route, retry, area runs, and suggestion changes. It refreshes stale claim state for that request and rejects an organisation beyond its grace period before document business logic runs. Platform service tokens and auth-exempt infrastructure routes do not use the organisation-level check. When the table name is empty, the guard is a no-op. Terraform's `enable_billing_enforcement` flag supplies the environment-qualified table name and `dynamodb:GetItem` permission; it is on in dev, test and prod. For the responses a client sees, see [Billing access](https://help.stralos.app/api/documents.md#billing-access). ## User-safety rules * No AI output writes directly into estimate data. * Downloads require clean scan evidence. * Infected or unsupported documents stay terminal. * Area reruns replace pending suggestions for that run but preserve accepted/rejected decisions. * Fabricated citation references and quotes are dropped before any assessment result is stored. This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.