Real World Evidence: Relevance, Reliability, and What FDA Accepts

Real world data (RWD) is data about patient health status or care delivery collected outside a controlled trial: electronic health records, claims and billing, product and disease registries, pharmacy dispensing, and data from devices and wearables.

Real world evidence (RWE) is the clinical evidence about a product's usage, benefits or risks that comes from analyzing that data.

The terms get used interchangeably in pitches but not in submissions. RWE is the conclusion drawn from RWD, and the conclusion is what FDA reviews.

Statutory basis

The 21st Century Cures Act of 2016 directed FDA to create a program for evaluating RWE in support of new indications for approved drugs and to satisfy post-approval study requirements. FDA published its framework in December 2018 and has issued guidances since covering data standards, registries, EHR and claims data, and the regulatory considerations for non-interventional studies.

Devices have a separate track, with FDA guidance on RWE in device regulatory decision-making that predates the drug framework.

Relevance and reliability

Relevance. Does the data answer the question? That means the data contains the exposures, outcomes and covariates you need, in enough detail, on a population representative of the one your claim is about, over a long enough period.

Reliability. Can the data be trusted? This covers how it was collected, whether the collection was consistent over time and across sites, how complete it is, what the quality assurance was, and whether the provenance can be traced from the analysis back to the source record.

A dataset can be highly reliable and completely irrelevant. Claims data records what was billed with great consistency and may say nothing useful about symptom severity.

Limits

It does not lower the evidentiary standard. FDA still requires substantial evidence of effectiveness from adequate and well-controlled investigations. RWE offers a different data source for meeting that standard.

Where FDA accepts it:

• Label expansions for approved products, particularly new populations.
• Post-approval study requirements and safety commitments.
• External or historical control arms for single-arm trials, most often in rare disease and oncology where randomization is difficult.
• Natural history characterization supporting a development program.

Where sponsors overreach: using a retrospective database analysis to support a primary effectiveness claim for a new molecular entity.

Before generating RWE

Protocol and analysis plan first. Write and date them before you look at the outcome data. A retrospective analysis of a database you have already explored invites the question of how many analyses preceded the one you submitted.

Assess the data source before choosing it. Document the relevance and reliability assessment for the specific source, including what the data provider does and does not control. FDA asks sponsors to identify RWD and RWE use in the submission cover letter, along with the purpose, the design and the source.

These studies most often fail inspection on data provenance. If you cannot trace a number in the analysis back to the record it came from, the study is not verifiable, and ALCOA expectations apply to real world data as much as to trial data.