> ## Documentation Index
> Fetch the complete documentation index at: https://help.stralos.app/llms.txt
> Use this file to discover all available pages before exploring further.

# Cloud drive sync

> Internal architecture for the cloud-drive document sync engine

Cloud drive sync lets a Stralos admin connect a Google shared drive or Microsoft SharePoint document library to a project. Stralos lists the drive's files on the project **Documents** page as links, fetches a file's bytes when someone opens or processes it, and feeds fetched files through the standard document AI pipeline.

This page covers the internal design in `stralos-api-documents`. For the admin setup steps, see [Shared drives](https://help.stralos.app/guides/admins/drive-sources.md). For the endpoint reference, see [Documents API](https://help.stralos.app/api/documents.md#cloud-drive-sync-endpoints). For the shared wire types, see [Cloud-drive document sync](https://help.stralos.app/internal/cloud-drive-sync.md).

## Architecture overview

```
Admin connects the organisation's identity (Google SA or Microsoft tenant binding)
Admin registers source → proof-of-control check → source active
Admin maps source (or folder) to project → connection active
Dispatcher (every five minutes) → enqueues run SQS event
Sync worker → acquires run → crawl and/or incremental phase
  → per file: link (reserve a key, write a `linked` row, no bytes)
  → engine finalises run stats, schedules next run

User opens or processes a linked document → POST /:id/open (or /route, /process)
  → live version check (3 s budget)
  → create or join a drive-fetch job → 202
  → drive-fetch worker: claim → fetch bytes → store → scan → promote
```

The sync engine (`src/sync/engine.ts`) is a three-layer construct:

1. **Run acquisition**: a worker-started-at CAS (`NULL` → `now()`) means a duplicate SQS delivery finds the run already claimed and drops. At most one worker executes a given run.
2. **Connection fence**: every mapping write and cursor advance re-checks that the connection's `currentRunId` still matches the run. A disabled source, deleted mapping, or concurrent pause breaks the fence and aborts the run cleanly.
3. **Mapping CAS**: every link write locks the mapping row (lock order mapping → document) and decides from its current state, so a replayed page or crashed run cannot double-write a file.

## Sync flow

### Initial crawl

A brand-new (or reactivated) connection first captures the change cursor from the provider, then performs a full crawl of the folder scope. The crawl walks the folder tree, links each file, and advances a page cursor so a crash or restart resumes from the page boundary.

### Incremental sync

After the initial crawl completes, each run reads the provider's change feed. Only changed or deleted files are processed. A deleted file removes its mapping.

The engine reads the change feed to its end within each run. Google Drive can return an empty page that still carries a `nextPageToken`, so the engine trusts the provider's `hasMore` signal rather than the page's entry count.

Folder entries in the feed are filtered before any reconciliation decision. A folder change triggers reconciliation only when the folder is inside the mapped subtree or is already recorded in the connection's folder map. Folder activity elsewhere in the same shared drive is ignored and never written to the crawl frontier.

### Reconciliation crawl

The engine seeds a reconciliation crawl when a soft-deleted mapping is reactivated, an admin re-includes a file, a folder-topology change is detected, or a provider change cursor expires. This bumps the generation and re-walks the mapped subtree from its root. The preserved or freshly seeded change cursor drives the incremental phase after reconciliation completes.

Removal by absence applies to synced mappings and to skipped mappings that still retain a visible document. Present skipped files are stamped with the current generation, so reconciliation does not remove them falsely.

## Link-first sync

The sync engine stores a **link**, not a copy. For each file it writes a `linked` document row with its S3 key reserved and no object at that key, and records the drive's latest known version on the mapping. No provider download or S3 write happens during a sync run. The drive-fetch worker fetches bytes on demand.

### What the engine writes per file

| Case | Engine action |
| - | - |
| New file that is not skipped | Insert a `linked` document row and a `synced` mapping pointing to it |
| Changed version, row never fetched | Update the linked row in place to describe the new version |
| Changed version, row holds (or held) bytes | Record the new version on the mapping only. The held copy is never replaced; a newer version arrives as a new row when someone fetches it |
| Skipped file | Mark the mapping `skipped`; no document row |

Nothing is skipped for its type or size. The engine skips only:

| Reason | Cause |
| - | - |
| `temporary_file` | Office lock files (`~$…`), `.tmp`, `Thumbs.db`, `.DS_Store`, `desktop.ini` |
| `proof_file` | The proof-of-control file |
| `shortcut` | Google Drive shortcut (target may be outside the granted drive) |
| `remote_item` | SharePoint item shared in from outside the connected library |
| `package` | OneNote notebooks and other package facets |

The mapping statuses `ingesting` and `failed` are copy-era states the engine no longer writes. The engine re-evaluates any mapping still in those states (or `skipped` or `error`) and links it unless it is skipped. The per-run file cap counts link writes (2,000 per run).

### Fetch verdict

`fetchVerdict` (`src/sync/linkFirstPolicy.ts`) classifies every linked file from its name, MIME type and size, without a drive call:

* `processable`: the upload rules accept it and it is not store-only. It can be fetched, routed and processed.
* `viewable`: anything else at or below the fetch ceiling. It can be fetched, scanned and downloaded, but `/process` answers `409`.
* `drive_only`: above the 2 GB fetch ceiling (`too_large`), or a Google-native type Stralos cannot export (`not_downloadable`). It can only be opened in the drive.

See [File acceptance rules](https://help.stralos.app/internal/document-ai-pipeline.md#file-acceptance-rules) for the processable and store-only types. A Google-native export over Google's 10 MB export cap is discovered only at fetch time and recorded as `not_downloadable`.

### Drive-fetch worker

The drive-fetch worker (`src/sync/driveFetchWorker.ts`) runs as the sync-worker role on the `drive-fetch-jobs` SQS queue. Its concurrency is capped by the queue's event source mapping (`scaling_config.maximum_concurrency`), set by the `drive_fetch_maximum_concurrency` Terraform variable: default `10`, allowed range `2` to `1000`. No reserved Lambda concurrency is set, so a burst waits in the queue rather than being throttled. It handles two message types:

* **Fetch** (`driveFetchJobId`): claim the job, read the file's current version from the drive, choose the target row, stream the bytes through the verified byte path (`sync-staging/` → the reserved `uploads/` key), and hand the object to the scan chain. The job's identity comes from rows (job → mapping → connection → source → organisation identity), never from the request.
* **Purge** (`documentId`, queued by the sweep): revert an unused copy to `linked`, or retire an older version.

The scan handler (`src/sync/driveFetchScan.ts`) finishes the job. A `first_fetch` or `refetch` fills the linked row itself. A `new_version` job creates a new row beside the old one: after a clean scan the new row becomes Version N+1, the previous row is marked superseded by it, and the mapping points at the new row. The previous row keeps its estimate routing and derived data. Only a clean scan makes a new version visible.

A threat or scan error fails the job. A threat blocks that drive version, so neither `/open` nor prefetch fetches it again until the drive publishes a newer version.

### Fetch job states

| State | Meaning |
| - | - |
| `queued` | Enqueued; not yet claimed |
| `fetching` | Worker is reading the file from the drive |
| `storing` | Bytes are being written to S3 |
| `scanning` | Awaiting the malware scan verdict |
| `ready` | Scan clean; the result row is visible |
| `failed` | Terminal failure; see `failureReason` |

Failure reasons: `too_large`, `not_downloadable`, `access_revoked`, `deleted_in_drive`, `scan_threat`, `scan_error`, `changed_during_fetch`, `provider_error`, `timeout`, `sync_disabled`. The blocking failures (`too_large`, `not_downloadable`, `scan_threat`) are recorded on the mapping and block that drive version until the drive publishes a newer one.

A revoked grant met during a fetch suspends the source exactly as a sync run would (see [Error classification](#error-classification)).

### Opening a document

`POST /:id/open` resolves an open in one of these ways:

1. **Held copy, current**: returns a download (`200`, `state: "ready"`). The response reports whether the live version check ran (`versionCheck`).
2. **Held copy with a newer drive version, used by a non-frozen entity**: returns `200` with `state: "decision_required"`, the used and latest versions, and the entities that must choose to keep or sync.
3. **Linked (no bytes held)**: creates or joins a fetch job and returns `202` with `state: "fetching"`. The client polls `GET /drive-fetches/:jobId`.
4. **Cannot be fetched**: `state: "drive_only"` (above the ceiling or not downloadable) or `state: "unavailable"` (`deleted_in_drive`, `access_revoked`, `scan_threat`, or `sync_disabled`).

The live version check has a 3-second budget. If it times out or fails, `/open` serves the held copy with `versionCheck: "unavailable"`.

`/route` with processing requested and `/process` on a `linked` row also create or join a fetch job that processes the document once it scans clean. A `drive_only` row answers `409 drive_only`, a `viewable` row `409 not_processable` (or `store_only`), and a closed kill switch `409 drive_sync_disabled`.

### Limits

| Limit | Value |
| - | - |
| Fetch ceiling | 2 GB; larger files are `drive_only` |
| Direct download | Up to 100 MB through the API; larger downloads use a presigned S3 URL |
| Background prefetch | Files up to 250 MB; 25 per call, 100 per organisation per hour, 30-minute cooldown per entity |
| User fetch jobs | 30 new jobs per user per 10 minutes (joining a live job does not count); `429 too_many_fetches` |
| Unused copy purge | A copy not used and not opened for 30 days reverts to `linked` |

The sweep also re-queues fetch jobs whose worker died (up to three attempts), fails jobs stuck in `scanning` after 30 minutes, and pauses the 30-day purge whenever the usage consumer's DLQ is not confirmed empty, because an unapplied usage event could be the one that makes a copy "used".

## Document usage registry

The usage registry (`src/usage/registry.ts`) records which module entities use which documents. It drives the 30-day purge, the newer-version prompt, the removed-from-drive flag, possible-variation hints on frozen entities, and prefetch.

The estimate module publishes `DocumentUsagesChanged` v1 events on the estate bus. The `estate-events` Lambda routes each event to `src/estate-events/documentUsagesConsumer.ts`, which applies the full usage set for the entity only when the event's `usageVersion` is higher than the one held, and then replaces that module's rows for the entity. A document absent from the set is no longer used. Documents also writes its own `routed` usage rows when `/route` runs.

Modules without a bus publisher can `PUT /usages/:module/:entityType/:entityId` with a `service:<module>` token. The registry reads and writes are never gated by the kill switch, so "used" stays accurate for the purge.

## Folder filing

When the sync engine links a file it resolves the file's source folder path and stores it as the document's `relatedFolder` field. The path is scope-relative: levels below the mapping's root folder are preserved; the mapping root itself is stripped.

The format is a breadcrumb separated by `/` (space-slash-space). A confirmed file at the mapping scope root, or a resolved path that exceeds 2048 characters, is treated as unfiled (`relatedFolder = null`). If the source path cannot be resolved, sync treats the result as unknown and preserves any existing filing.

A new version fetched beside an older one carries the older row's folder and ownership marker forward.

### Folder ownership

Two actors can set a document's folder: the sync engine and a user via `PATCH /api/documents/:id`. The `relatedFolderSetBy` marker on the document row tracks which actor last wrote it:

| Value | Meaning |
| - | - |
| `sync` | The sync engine set this path. |
| `user` | A user set this path via the API. |
| `null` | No actor marker is recorded. A non-null folder with this legacy marker is treated as user-owned. |

Sync writes the folder only when it owns the value: `relatedFolderSetBy` is `sync`, or both the marker and the folder are `null`. A user-set folder is never overwritten by sync, including an explicit clear (`relatedFolder = null` with `relatedFolderSetBy = 'user'`).

If a file moves to a different folder in the source drive and sync owns the document's filing, sync updates `relatedFolder` on the next sync cycle. If the user has already filed the document, it stays where the user put it.

### Unresolvable paths

Sync treats an unresolvable folder path as unknown rather than unfiled. An unknown path never overwrites an existing `relatedFolder` value, so the previous filing is preserved across retries and restores.

### Post-deploy backfill

Documents synced before folder filing shipped retain their previous filing. For the procedure, supported connection states, and re-run behaviour, see [Drive sync folder transfer backfill](https://help.stralos.app/internal/runbooks.md#drive-sync-sync-folder-transfer-backfill).

## File routing

Synced files land as project-scoped documents (`ai_processing = on_demand`) in the `linked` state, with no estimate attached, until an estimator routes one to an estimate. Only routed documents trigger AI area processing, and a routed link is fetched and scanned before processing starts.

## Project-status sync pause

Stralos pauses sync for a project that api-global reports as `lost` or `archived`.

A dedicated consumer (`src/estate-events/projectStatusConsumer.ts`) reads `ProjectStatusChanged` events from the `documents-project-status` SQS queue and upserts the `documents_project_statuses` projection. The upsert applies only when the event's `statusVersion` is higher than the one held, so redelivered or out-of-order events are harmless. Each event is also de-duplicated on `eventId` through `processed_events` in the same transaction. Locally the same handler runs via `pnpm worker:events:local`.

The effect (`src/sync/projectSyncPause.ts`):

* The dispatcher's candidate scan excludes connections whose project has a `lost` or `archived` projection row. A project with no projection row keeps syncing.
* A run dispatched just before the project was paused ends at start with nothing crawled.
* `POST /sync-connections/:id/sync-now` returns `409` with `project_sync_paused` rather than re-arming a claim the dispatcher would never honour.

The connection itself is untouched: its status stays `active` and its cursor and crawl state are preserved. When a later event moves the project to any other status, the next dispatcher tick resumes from where it stopped.

## Provider modes

Set `STRALOS_SYNC_PROVIDERS` (or the Terraform `sync_provider_mode` variable) to:

* `live`: calls the Google Drive API or Microsoft Graph as the organisation's own identity. Requires the platform variables in [Deploy prerequisites](#deploy-prerequisites) and an `active` organisation identity.
* `fixture`: deterministic, code-point-ordered fixture data; makes no external calls. Default for local development.

Both modes go through the same engine, link, fetch and routing code.

## Drive-sync kill switch

A kill switch stops all drive-sync activity without a deploy. The primary lever is the `drive-sync-enabled` SSM parameter (`STRALOS_DRIVE_SYNC_ENABLED_SSM_PARAMETER` names it). Only the exact string `true` opens the gate; any other value, a missing parameter or a failed read keeps it closed. `STRALOS_DRIVE_SYNC_ENABLED` overrides the parameter for local development and tests.

Terraform seeds the parameter once per environment from `drive_sync_enabled_initial` and then ignores it, so flipping it is an `aws ssm put-parameter` call, not an apply. The flag is read on every dispatch tick, worker message and API request through the shared five-minute SSM cache, so a flip reaches every warm Lambda within five minutes.

A second, independent layer is the `drive_sync_schedule_enabled` Terraform variable, which enables or disables the sync-dispatch EventBridge rule.

When the switch is closed:

* **Dispatch**: the tick claims no connections and enqueues nothing. Existing leases and run history are untouched.
* **Sync worker**: a queued job is dropped before any provider call. The run stays unstarted; once sync reopens, the next dispatch tick replaces it after the lease expires.
* **Source and connection API**: every source and connection mutation, `sync-now`, `check-access` and folder browse returns `409 drive_sync_disabled`. `onboarding-config` answers `enabled: false`, and `GET /drive-identities` answers `{ enabled: false, platform: { google: { configured: <bool>, enabled: false }, microsoft: { configured: <bool>, enabled: false } }, items: [] }`.
* **Link-first API**: `drive-version`, `prefetch` and processing recommendations return `409 drive_sync_disabled`. `/open` degrades: a link answers `unavailable` (`sync_disabled`), and a held copy is still served with `versionCheck: "unavailable"`.

Read-only list and detail routes and reads of already-synced documents are unaffected. Authentication and authorisation run before the gate, so a `401` or `403` takes precedence over `drive_sync_disabled`.

For the procedure, see [Drive sync: toggle the kill switch](https://help.stralos.app/internal/runbooks.md#drive-sync-toggle-the-kill-switch).

## Source and connection states

### Source states

| State | Meaning |
| - | - |
| `pending_proof` | Registered; waiting for the admin to drop the proof file and click Verify |
| `active` | Verified; sync runs normally |
| `auth_error` | Provider access revoked, the organisation's identity was refused, or the identity was reconnected; re-grant if needed and Verify to recover |
| `error` | Platform-side federation failure; ops must fix the platform, then Verify |
| `disabled` | Admin-disabled, or disabled by an identity disconnect; all mappings paused |
| `deleted` | Soft-deleted; all mappings must be removed first |

`pending_proof`, `auth_error`, and `error` recover only through `/verify`. `PATCH` toggles only the `active` ↔ `disabled` pair. A `pending_proof` source that is not verified within seven days is soft-deleted by the sweep; registering the same URL again reactivates it with a fresh proof file name.

### Connection (mapping) states

| State | Meaning |
| - | - |
| `active` | Running normally |
| `paused` | Admin-paused; no new runs dispatched |
| `error` | Reserved degraded mapping state in the shared contract |
| `disabled` | Reserved disabled mapping state in the shared contract |
| `deleted` | Soft-deleted |

The connection mutation route transitions only between `active` and `paused`, plus soft-delete to `deleted`. Provider access and credential failures live on the source.

## Per-organisation drive identities

Each organisation authenticates to its drive provider through its own identity. There is no shared credential.

* **Google Drive**: Stralos provisions one Google service account per organisation in the environment's drive GCP project. The Lambda authenticates keylessly: its AWS role exchanges a Workload Identity Federation token at Google STS, and the federated token mints a short-lived read-only access token for that organisation's service account. No key file is created or stored.
* **Microsoft 365**: one multi-tenant Entra app per environment. Isolation comes from binding each organisation to its tenant GUID. The GUID is learned only from a validated OIDC `id_token`, signed by that tenant's own keys, from a sign-in by a member of the tenant. The Lambda authenticates to Entra through AWS outbound web-identity federation.

A sync run or fetch loads the organisation's identity under the run's RLS context and asserts it belongs to the source's organisation and provider before any provider exists.

### Identity lifecycle

| Status | Meaning |
| - | - |
| `pending` | Provisioning in progress (Google service account being confirmed; Microsoft awaiting consent or sign-in) |
| `active` | Fully operational |
| `auth_error` | The identity failed, for example a Google provisioning timeout (`status_detail: provision_timeout`); re-run the connect flow |
| `disconnected` | Admin-disconnected; in the 30-day grace period |
| `deleted` | Grace period elapsed; identity purged |

### Identity routes

All identity routes are under `/api/documents/drive-identities`. Every route except the read and the two Microsoft redirect targets is admin-only.

| Method | Path | Purpose |
| - | - | - |
| `GET` | `/drive-identities` | The caller organisation's non-deleted identities, whether drive sync is enabled, and each provider's platform readiness. Any member may read it. |
| `POST` | `/drive-identities/google` | Provision the organisation's Google service account and queue the readiness probe. Idempotent: `pending` or `auth_error` resumes the probe; `active` is returned unchanged. Returns `202 { identity }`. |
| `POST` | `/drive-identities/microsoft/tenant-lookup` | Look up the tenant for a domain via OIDC discovery (a hint only), record the `pending` identity, and return the tenant-specific admin-consent URL. |
| `POST` | `/drive-identities/microsoft/signin-start` | Issue a single-use 15-minute PKCE sign-in state and return the authorisation URL. |
| `GET` | `/drive-identities/microsoft/consent-done` | The admin-consent redirect's landing page. Static HTML; binds nothing. |
| `GET` | `/drive-identities/microsoft/signin-callback` | Microsoft's redirect target. No JWT; the stored state is the capability. Binds the tenant and redirects to **Settings → Shared drives** with `?microsoft=<outcome>`. |
| `POST` | `/drive-identities/:id/disconnect` | Disconnect. For Google, the service account is disabled at Google before the route returns (`502` if Google does not confirm; retry). All active sources move to `disabled` and their run fences break. |
| `POST` | `/drive-identities/:id/reconnect` | Reconnect within the grace period. Google: the same service account is re-enabled and its sources wait in `auth_error` for `/verify`. Microsoft: returns a new sign-in URL whose tenant must match the bound GUID. |

When the Microsoft platform variables are not set, the identity routes answer `409 drive_platform_not_configured`.

### Platform readiness block

`GET /drive-identities` includes a `platform` block alongside `enabled` and `items`. It is present whether or not drive sync is enabled, so the UI can explain a disabled **Connect** button before the admin interacts.

```json theme={null}
{
  "enabled": true,
  "platform": {
    "google": { "configured": true, "enabled": true },
    "microsoft": { "configured": false, "enabled": true }
  },
  "items": []
}
```

* `platform.<provider>.configured` — `true` when the environment's Terraform platform variables for that provider are set (the condition behind `409 drive_platform_not_configured`).
* `platform.<provider>.enabled` — `true` when the drive-sync kill switch is open (the condition behind `409 drive_sync_disabled`). Both flags are identical across providers for a given response.

**Connect works only when both flags are `true` for the provider.** A consumer that receives a response without a `platform` key (older API version) should treat provider readiness as unknown.

### Google service account readiness

A new Google service account can take 60 seconds or more before Google accepts a token mint for it. `POST /drive-identities/google` returns immediately with the identity `pending` and queues a readiness probe on the sync-jobs queue with a 60-second delay. The probe mints a token for the service account:

* Success: the identity moves to `active` and the service account email becomes visible.
* Failure: the probe retries with backoff (30 s, 60 s, 120 s, then 180 s) within a 10-minute window, then moves the identity to `auth_error` with `status_detail: provision_timeout`. **Resume setup** re-runs `POST /drive-identities/google`.

### Microsoft tenant binding

1. The admin enters their domain. The API calls Microsoft's OIDC discovery for that domain and returns a hint GUID and the tenant-specific admin-consent URL.
2. A Microsoft admin grants consent. The consent redirect lands on `consent-done`, which renders a static page and stores nothing.
3. The Stralos admin starts the sign-in. The API issues a 15-minute PKCE state and returns the authorisation URL.
4. Microsoft redirects to `signin-callback` with an authorisation code. The callback consumes the state first (single use), redeems the code, verifies the `id_token` against the tenant's own keys, confirms the signer is a member of the tenant, checks consent, binds the tenant GUID in the organisation context the state row carries, and redirects back.

Callback outcome codes:

| Code | Meaning |
| - | - |
| `connected` | Binding succeeded |
| `consent_missing` | Admin consent not yet granted |
| `not_a_member` | The account is a guest or not a member of the target tenant |
| `tenant_mismatch` | A different tenant signed in during a reconnect |
| `tenant_unavailable` | The tenant cannot be bound, for example because another organisation already holds it |
| `state_invalid` | PKCE state expired, already used, or not found |
| `signin_failed` | Microsoft returned an error, or the sign-in could not be completed |
| `drive_sync_disabled` | Drive sync is switched off |

One Microsoft 365 tenant can be bound to only one Stralos organisation.

### Grace period and deletion

A disconnected identity stays in `disconnected` for 30 days and can be reconnected without re-sharing drives. After that the scheduled sweep deletes the Google service account at Google first (Google keeps its own 30-day undelete window), then marks the row `deleted` and nulls the service account email and unique ID, or the tenant GUID. Nulling the GUID frees the tenant for a fresh binding. A failed service account delete keeps the row `disconnected` and retries on the next sweep.

The sweep also prunes Microsoft sign-in states a day after they expire. The organisation purge deletes the organisation's Google service accounts as part of offboarding; see [Data retention](https://help.stralos.app/internal/data-retention.md#what-a-purge-deletes).

### Audit trail

Every identity mutation writes one row to `documents_sync_audit_events` in the same transaction. The table is append-only for the runtime role (`SELECT` and `INSERT` only). Actions: `identity.provisioned`, `identity.consented`, `identity.bind_refused`, `identity.disconnected`, `identity.reconnected`, `identity.deleted`, `identity.auth_error`. Detail fields record the provider, state transitions and counts, never tokens, sign-in states or PKCE verifiers.

## Check SharePoint site access

`POST /api/documents/drive-sources/check-access` (admin-only, Microsoft only) lets the **Add shared drive** panel ask, before saving, whether the organisation's identity can already read a SharePoint site. It is read-only: it reads the site and lists its libraries as the organisation's identity, and writes no source or audit row.

```json theme={null}
// Request
{ "provider": "microsoft_graph", "sourceUrl": "<SharePoint site or library URL>" }

// Access confirmed
{ "ok": true, "site": { "id": "…", "displayName": "…", "webUrl": "https://…" } }

// Refused
{ "ok": false, "code": "not_granted" }
```

| `code` when `ok` is `false` | Meaning |
| - | - |
| `url_invalid` | Not a supported SharePoint site address |
| `personal_onedrive` | A personal OneDrive (`-my.sharepoint.com` or a `/personal/` path) |
| `identity_not_connected` | The organisation's Microsoft identity is not active, or the tenant refused it |
| `not_granted` | The site exists but Stralos cannot read it or its libraries |
| `site_not_found` | Microsoft could not find the site in the organisation's tenant |

The kill switch and an unconfigured platform return `409`, like sibling routes. When Microsoft is unreachable the route returns `502 drive_check_unavailable`. The route is rate-limited to 30 requests per minute.

### SharePoint URL rules

`parseSharePointSiteUrl` (`src/sync/sharePointSiteUrl.ts`) reduces a site, library or sharing URL to a canonical `{ hostname, sitePath }`. The UI (`stralos-feature-organisation` `src/sharePointGrant.ts`) mirrors the same rules, so a URL the check accepts is one the save accepts.

Accepted:

* `https://<tenant>.sharepoint.com/sites/<name>/…` → `/sites/<name>`
* `https://<tenant>.sharepoint.com/teams/<name>/…` → `/teams/<name>`
* Other paths on `https://<tenant>.sharepoint.com/` → the root site
* **Copy link** sharing URLs: `/:x:/s/<name>` and `/:x:/t/<name>` reduce to `/sites/<name>` and `/teams/<name>`; `/:x:/r/<path>` is reduced as a full path

Refused as `url_invalid`: non-`https` schemes, embedded credentials, ports, any host other than one DNS label before `.sharepoint.com`, the `-admin.sharepoint.com` admin centre, `/_layouts`, `/_api` and `/_vti_bin` at the root, dot segments in any spelling the URL parser would resolve, and tab or newline characters anywhere in the input.

The hostname only names a site inside the organisation's bound tenant. It never selects the tenant a token is minted for, so a URL from another company's SharePoint resolves as `site_not_found`.

## Onboarding config endpoint

`GET /api/documents/drive-sources/onboarding-config` (admin-only) returns the calling organisation's provider readiness for the Settings panel.

```json theme={null}
{
  "enabled": true,
  "google": {
    "configured": true,
    "serviceAccountEmail": "<organisation's service account>",
    "identity": { "id": "…", "provider": "google_drive", "status": "active" }
  },
  "microsoft": {
    "configured": true,
    "clientId": "<entra-app-client-id>",
    "tenantId": "<bound-tenant-guid>",
    "identity": { "id": "…", "provider": "microsoft_graph", "status": "active" }
  }
}
```

`configured: true` means the environment's platform is set up and the organisation's own identity is `active`, so the organisation can register a drive of that provider now. Otherwise the provider block is `configured: false`, with an `identity` field when a non-deleted identity row exists. `serviceAccountEmail` is the organisation's own address and is included only when the identity is active, so the UI never asks an admin to share with an address Google does not yet accept.

With the kill switch closed, the endpoint answers `{ "enabled": false, "google": { "configured": false }, "microsoft": { "configured": false } }` without reading any identity.

## List drive sources

`GET /api/documents/drive-sources` returns the calling organisation's registered sources with mapping counts. When no drive has been configured it returns `200` with `{ "items": [] }`, the normal state for an organisation that has not set up a drive. Per-source routes still return `404` when the specific source does not exist.

## Configuration reads

SSM reads in `stralos-api-documents` go through `src/config/ssm.ts`, which delegates to the shared `@stralos-co-nz/api-common` parameter cache. Successful entries use a five-minute TTL by default. Empty values and Terraform placeholders are treated as unset. The drive-sync kill switch is read this way.

The API Lambda loads every SSM parameter it reads once at cold start, before it serves a request (`preloadApiSsmParameters()` in `src/config/ssm-preload.ts`). The kill switch parameter (`STRALOS_DRIVE_SYNC_ENABLED_SSM_PARAMETER`) is optional in that preload: if it cannot be read, the Lambda still starts and the gate stays closed. After the first load, an expired value keeps being served while the cache refreshes it in the background, so a flip still reaches warm containers within five minutes.

Drive identity credentials are not stored in SSM. They are minted just in time from the Lambda's AWS role through Workload Identity Federation (Google) or outbound web-identity federation (Microsoft).

## Deploy prerequisites

Before enabling sync in an environment, and before switching `sync_provider_mode` to `live`:

1. **Egress**: worker Lambdas run in private subnets. Verify NAT or equivalent in `stralos-core` reaches `googleapis.com`, `graph.microsoft.com`, and `login.microsoftonline.com`. Without it every live provider call times out silently.
2. **Platform variables**: set these `stralos-api-documents` Terraform variables before any organisation can connect. Each is wired to the Lambda environment variable shown in brackets. They ship empty (fail closed); an incomplete set makes the provider report `configured: false` and the identity routes answer `409 drive_platform_not_configured`, without crashing. None is a secret.
   * Google: `drive_gcp_project_id` (`STRALOS_DRIVE_GCP_PROJECT_ID`), `drive_gcp_project_number` (`STRALOS_DRIVE_GCP_PROJECT_NUMBER`), `drive_wif_pool_id` (`STRALOS_DRIVE_WIF_POOL_ID`), `drive_wif_provider_id` (`STRALOS_DRIVE_WIF_PROVIDER_ID`), and optionally `drive_gcp_provisioner_service_account` (`STRALOS_DRIVE_GCP_PROVISIONER_SA`) when provisioning must impersonate a provisioner service account.
   * Microsoft: `drive_entra_client_id` (`STRALOS_DRIVE_ENTRA_CLIENT_ID`), `drive_aws_outbound_audience` (`STRALOS_DRIVE_AWS_OUTBOUND_AUDIENCE`, must be `api://AzureADTokenExchange`), and `documents_public_base_url` (`DOCUMENTS_PUBLIC_BASE_URL`, the public origin for the sign-in callback and consent-done page).
3. **Kill switch**: open the `drive-sync-enabled` parameter and set `drive_sync_schedule_enabled` (see [Drive-sync kill switch](#drive-sync-kill-switch)).
4. **Auth gate**: unconditional auth enforcement (org-security M1b) must be live before any sync-capable deployment is reachable beyond the team.
5. **Multi-tenant gate**: do not onboard a second organisation until organisation-security's isolation milestone (its §7: M2–M5 complete with per-milestone tests deployed) is met.

## Error classification

Provider errors carry a stable class so the engine applies the correct blast radius. The drive-fetch worker applies the same classification.

| Class | Effect |
| - | - |
| `grant_revoked` | Moves the affected source to `auth_error` and breaks every mapping fence for that source |
| `identity_auth_error` | The organisation's identity was refused: moves every active source on that identity to `auth_error` and breaks their fences, for that organisation only |
| `platform_federation_failure` | The environment's federation failed: moves all active sources for that provider to `error`, breaks their fences, and pages |
| `throttle` | Backs off, then resumes from the committed cursor on a later run |
| `file_gone` | Settles the individual mapping as removed |
| `folder_unlistable` | Leaves a non-root crawl unsettled for retry; a denied scope root becomes `grant_revoked` |

An expired Google change token or Microsoft resync response seeds reconciliation instead of suspending the source. Excluded-file mappings remain excluded until an explicit re-include.

Auth failures are caught and recorded, so Lambda `Errors` never fires for them. The CloudWatch alarms use Logs metric filters over the structured `"syncMetric":"<name>"` token.

## Run summary log

At the end of every run the engine emits one structured JSON line with `event: "sync_run_summary"`. It records which phases ran, how many provider pages and entries were read, how many files the scope filter kept, and where they went. It logs counts and IDs only, never file names.

```json theme={null}
{
  "event": "sync_run_summary",
  "connectionId": "…",
  "runId": "…",
  "phases": ["incremental"],
  "pages": 3,
  "itemsSeen": 5,
  "filesSeen": 5,
  "inScope": 5,
  "outOfScope": 0,
  "ingested": 5,
  "superseded": 0,
  "skipped": 0,
  "skippedByReason": {},
  "errored": 0,
  "earlyStop": false,
  "reconciliationTriggered": false,
  "error": null
}
```

A run that reaches the provider but links nothing still emits this line. `inScope: 0` with `outOfScope` above zero points to a scoping mismatch rather than a provider or credential failure.

## Runbook entries

See [Runbooks](https://help.stralos.app/internal/runbooks.md) for drive-sync operational procedures.


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.