MSDS to JSON: Converting Safety Data Sheet PDFs

MSDS and SDS: what the name means for conversion

"MSDS" is the older name. Hazard communication was aligned with GHS, and the document is now called a safety data sheet (SDS) with 16 sections in a defined order: in the EU, REACH Annex II sets the content and headings, and in the US the OSHA Hazard Communication Standard, updated in 2012, requires the same 16-section format. Archives often still hold older MSDS PDFs whose layout predates this, and some older layouts order the sections differently, for example composition before hazard identification.

The API returns the same 16 section keys whatever layout the source uses. How well a pre-GHS document maps onto them is something we have not published figures for, so measure it on a sample of your own archive before converting all of it.

What you get back

One JSON object per document, with a key per SDS section under outputs.json, plus confidence_score and warnings. The section keys are listed in the field coverage matrix.

{
  "ok": true,
  "request_id": "2f6c1a54-7b0e-4d0a-9d52-6a1f0f3c9b11",
  "confidence_score": 0.94,
  "document_language": "en",
  "warnings": [],
  "outputs": {
    "json": {
      "product_identification": { "product_name": "...", "supplier_name": "..." },
      "hazards_identification": { "ghs_classification": ["..."], "signal_word": "...", "h_statements": ["H225 ..."] },
      "composition": [ { "substance_name": "...", "cas_number": "67-64-1", "concentration_percent": "95-100" } ],
      "transport_information": { "un_number": "UN1090", "hazard_class": "3", "packing_group": "II" }
    },
    "xml": "<?xml version=...>",
    "csv": "field,value ..."
  }
}

Convert a whole folder

This script uses the Python client. It sends files in bulk jobs of 10 with an idempotency key, so re-running it after an interruption returns the existing jobs instead of creating duplicates. It writes one JSON file per document and a review.csv listing everything that failed, has warnings, or falls below your confidence threshold.

import csv, hashlib, json, os, pathlib
from safetydatasheetapi import Client

SOURCE = pathlib.Path("msds-archive")
TARGET = pathlib.Path("json-out")
THRESHOLD = 0.85  # set this from a measured sample of your own documents, not a default

client = Client(api_key=os.environ["SDS_API_KEY"])
TARGET.mkdir(exist_ok=True)
pdfs = sorted(SOURCE.glob("*.pdf"))
flagged = []

for start in range(0, len(pdfs), 10):  # a bulk job takes up to 10 files
    batch = pdfs[start:start + 10]
    key = hashlib.sha256("|".join(p.name for p in batch).encode()).hexdigest()[:32]
    job = client.submit_bulk(batch, idempotency_key=key)  # re-running returns the same job
    done = client.wait_for_bulk_job(job["job_id"])
    for item in done["items"]:
        if item["status"] != "success":
            flagged.append((item["file_name"], item.get("error_code", "error"), ""))
            continue
        out = TARGET / (pathlib.Path(item["file_name"]).stem + ".json")
        out.write_text(json.dumps(item["data"], indent=2), encoding="utf-8")
        if item["warnings"] or (item["confidence_score"] or 0) < THRESHOLD:
            flagged.append((item["file_name"], "needs_review", item["confidence_score"]))

with open("review.csv", "w", newline="", encoding="utf-8") as handle:
    csv.writer(handle).writerows([("file", "reason", "confidence")] + flagged)
print(f"{len(pdfs)} files processed, {len(flagged)} flagged for review (see review.csv)")

Two practical limits apply. A bulk job takes up to 10 files of up to 10 MB each, and the shared hosted endpoint defaults to 20 bulk jobs per 24 hours (about 200 files a day); the script raises a rate-limit error with a Retry-After value when you hit it. Self-serve and enterprise limits are set by the plan.

A sensible archive conversion plan

1. Pick about 50 documents that represent the archive: old and new layouts, scans, several suppliers and languages.
2. Convert them and compare a handful of fields (product name, CAS numbers, UN number, H statements) against the PDFs.
3. Choose a confidence threshold and warning policy that your reviewers can live with.
4. Convert the archive in batches and work through review.csv.
5. Spot-check a random sample of the documents that were not flagged, to make sure the threshold is not hiding errors.

JSON, XML or CSV?

Use JSON for new integrations. The same record is also returned as XML and CSV in every response; the XML page documents exactly how arrays, empty values and CSV quirks are written.

How you get access: the free sample upload on the homepage needs no key, the self-serve plan issues an API key after signup, and enterprise customers get a dedicated deployment with its own base URL. See pricing.

FAQ

Related pages

• SDS field coverage matrix
• SDS to XML converter API
• Bulk processing and webhooks
• Multilingual SDS extraction