SafetyDataSheetAPI is a product of Dscvry. Explore Dscvry →

Integration Guide

EHS Platform SDS Ingestion Architecture

Technical implementation guide for ingesting SDS files into EHS platforms with queueing, validation gates, retry policy, and schema-pinned downstream contracts.

Last updated: 2026-03-10

Target architecture

Layer Role in ingestion flow Implementation note
Ingress APIReceives SDS file and metadata envelope.Attach source system ID and tenant key at ingress.
Extraction serviceParses SDS into structured section fields.Pin `schema_version` per environment.
Validation engineRuns required-field and confidence policies.Block low-confidence records from auto-publish.
Review queueRoutes warnings to compliance reviewers.Queue priority by hazard severity and site impact.
EHS writerPersists approved record to EHS data model.Use idempotent upsert by document fingerprint.
Audit storeKeeps immutable extraction/review history.Log reviewer actions and release timestamps.

End-to-end ingestion flow

  1. Upload SDS file with plant, product, and supplier metadata.
  2. Extract structured fields and generate confidence/warning metadata.
  3. Run policy checks for critical sections (2, 3, 8, 14, 15).
  4. Route passing records to EHS write path and flagged records to review queue.
  5. Emit webhook events so downstream systems can synchronize state.

Webhook payload (bulk job completion)

When a bulk job was submitted with a callback_url, the API POSTs the job status to it once every file is processed, with the headers X-Webhook-Event: bulk_extraction.completed and X-Webhook-Signature: sha256=<hmac>. Verify the signature before trusting the payload. Review routing (queues, tenants, policy checks on critical sections) is something your integration builds on top of confidence_score and warnings.

{
  "job_id": "9b1e3c0a-5d4f-4a7e-8f21-0c6d2e7a4b88",
  "status": "completed",
  "total_items": 2,
  "completed_items": 1,
  "failed_items": 1,
  "webhook_status": null,
  "webhook_attempts": 0,
  "items": [
    {
      "file_name": "supplier-a.pdf",
      "status": "success",
      "confidence_score": 0.94,
      "warnings": [],
      "data": { "product_identification": { "product_name": "..." }, "...": "16 SDS sections" }
    },
    {
      "file_name": "supplier-b.pdf",
      "status": "error",
      "confidence_score": null,
      "warnings": [],
      "data": null,
      "error_code": "provider_error",
      "error_message": "The extraction provider failed to process this document. Please retry."
    }
  ]
}

Illustrative, abbreviated payload. See the OpenAPI contract (webhooks) and the bulk and webhooks docs.

Retry and reconciliation strategy

Failure case Retry behavior Reconciliation action
Transient extraction timeout3 retries with exponential backoff.Escalate to integration alert after final retry.
Schema validation failureNo auto-retry.Route to engineering queue with payload diff.
EHS write conflictIdempotent retry after lock release.Run record-level compare and reconcile latest revision.
Webhook delivery failureA few immediate signed retries, not a persistent delayed-retry queue.Poll GET /extract-sds/bulk/:jobId for job/item status - this is the recovery path, not a dead-letter replay UI.

Security and tenancy: what the API provides and what you build

  • The API authenticates with an API key (Authorization: Bearer or X-API-Key). Self-serve accounts have their own key and enterprise customers get a dedicated deployment, so keep keys server-side and map your own tenants in your integration layer.
  • Webhook payloads are signed with X-Webhook-Signature (HMAC-SHA256 of the raw body). Each account has its own signing secret. Deliveries also carry X-Webhook-Id, X-Webhook-Timestamp and X-Webhook-Signature-V2; reject old timestamps and repeated ids, and deduplicate on job_id.
  • Tenant isolation, review queues and audit logging belong to your platform: attach your own tenant key at ingress and store the request_id with every result.
  • For the controls we document, see security and data residency.

Go-live readiness checklist

  • Schema version pinned in all ingestion environments.
  • Critical-section validation thresholds reviewed by compliance leads.
  • Review queue staffing model tested against expected warning rates.
  • Replay and disaster-recovery drills completed for webhook workflows.

FAQ

Can this flow support asynchronous ingestion at high document volume?

Yes. The architecture is queue-driven and designed for asynchronous processing with webhook completion events.

How do we avoid duplicate records during retries?

Use stable document fingerprints and idempotent upsert keys when writing into the EHS platform.

Can one tenant enforce stricter review gates than another?

Yes, in your integration. The API returns confidence_score and warnings for every document, and you set the thresholds and routing per tenant or business unit.

Related pages

Designing an SDS ingestion pipeline for a multi-tenant EHS platform? Request an EHS integration plan to validate architecture, queue design, and release controls.