BlackCat Bio — Case Study · Phase I / First-in-Human
A Phase 1 first-in-human protocol and SAP written by a CRO — reviewed, iterated, and finalized at a speed no human QC process could match.
Background
A sponsor needed to review and sign off on a Phase 1 first-in-human protocol and Statistical Analysis Plan drafted by a CRO. Rather than rely solely on manual review — a process known to miss subtle inconsistencies in complex clinical documents — the sponsor commissioned a BlackCat Bio AI evaluation before sign-off. The results revealed how much can slip through even experienced human review.
What the AI found: 71 issues across 5 categories
Each issue was classified by impact — High, Medium, or Low — with specific recommended language changes provided alongside every finding. Representative examples from each tier:
Protocol / SAP misalignment
Protocol referenced CTCAE v5.0; SAP referenced CTCAE v4.0 — a discrepancy that would propagate through all safety analyses if uncorrected.
Internal SAP inconsistency
"Day 1" defined differently in two separate SAP sections, creating ambiguity in the analysis window affecting primary endpoint calculations.
Likely typo — invisible to spellcheck
MAD and SAD switched in several places. Both are valid abbreviations, so standard spellcheck tools pass them without comment.
Ambiguous language
Vague eligibility criterion language open to site-level interpretation, risking enrollment inconsistency across the study.
The review process
Cycle 1
Initial AI evaluation of the full protocol and SAP
71 issues identified, each with a recommended change — delivered in under 4 hours
Cycle 2
CRO submitted revised documents; BlackCat Bio ran a second AI evaluation
Substantially reduced issue count, with lower average impact — delivered in under 4 hours
Cycle 3+
Continued iteration with progressively fewer, lower-impact findings
Each cycle completed in under 4 hours; issue count converging toward zero
Finalization
Documents finalized and signed off by sponsor
Total elapsed time: days, not the weeks a traditional review back-and-forth would require
"Standard spellcheck would never catch MAD and SAD being swapped — both are real abbreviations. That's exactly the kind of issue that survives human review and causes problems later. Our system found it on the first pass."
BlackCat Bio statistician
Outcomes
71 issues surfaced before sign-off
Issues ranging from document-altering inconsistencies to invisible typos — all caught before regulatory submission.
Finalized in days, not weeks
Sub-4-hour cycle times compressed a process that traditionally spans multiple weeks into a matter of days.
Every finding came with a fix
Recommended language changes delivered alongside each issue — the CRO could act immediately without additional interpretation.
Invisible errors caught
Domain-aware AI caught context-sensitive errors — swapped abbreviations, mismatched grading scales — no general-purpose tool would flag.
Why it matters
Phase 1 first-in-human documents carry unique risk. Errors in this class of documents — ambiguous dosing language, mismatched safety grading systems, inconsistent day definitions — can propagate into every downstream analysis and site operation. Yet these documents are also among the most likely to contain subtle inconsistencies: written under time pressure, by teams who may not cross-reference every section. BlackCat Bio's AI QA evaluation provides a systematic, exhaustive check that human review cannot replicate at this speed or consistency — transforming sign-off from a liability into a documented assurance.
See what we'd find in your documents.
Bring us a protocol or SAP and we'll run an evaluation. No commitment required.