Clinical data preparation for research
- 01Challenge
- An oncology outcomes study needed encounters, labs, medications, and clinical notes for roughly 18,000 patients, delivered to an external university partner every quarter. Each refresh relied on custom SQL, manual note review, and repeated rounds with the privacy office.
- 02Data and methods
- Structured EHR extracts and 240,000 narrative notes. Clinical NLP redaction on notes, configured transformations on structured fields, and per-patient date shifting to keep treatment intervals intact. Statistical risk analysis and documentation were prepared for the qualified expert’s determination.
- 03Validation
- Residual PHI and false redactions measured on a stratified sample of 2,000 notes. Row counts, join cardinality, and diagnosis-to-treatment intervals compared against the source before release.
Released by the data owner’s designated reviewers under the existing data use agreement.
