Regulatory glossary

Real World Evidence: Relevance, Reliability, and What FDA Accepts

Real world data (RWD) is data about patient health status or care delivery collected outside a controlled trial: electronic health records, claims and billing, product and disease registries, pharmacy dispensing, and data from devices and wearables.

Real world evidence (RWE) is the clinical evidence about a product's usage, benefits or risks that comes from analysing that data.

The distinction matters because the two words get used interchangeably in pitches and never in submissions. RWD is the input. RWE is the conclusion you draw, and the conclusion is what FDA reviews.

Where the programme came from

The 21st Century Cures Act of 2016 directed FDA to create a programme for evaluating RWE in support of new indications for approved drugs and to satisfy post-approval study requirements. FDA published its framework in December 2018 and has issued guidances since covering data standards, registries, EHR and claims data, and the regulatory considerations for non-interventional studies.

Devices have a separate track, with FDA guidance on RWE in device regulatory decision-making that predates the drug framework.

The two tests

Everything FDA asks about RWD reduces to two words.

Relevance. Does the data answer the question? That means the data contains the exposures, outcomes and covariates you need, in enough detail, on a population representative of the one your claim is about, over a long enough period.

Reliability. Can the data be trusted? This covers how it was collected, whether the collection was consistent over time and across sites, how complete it is, what the quality assurance was, and whether the provenance can be traced from the analysis back to the source record.

A dataset can be highly reliable and completely irrelevant. Claims data records what was billed with great consistency and may say nothing useful about symptom severity.

What RWE does not do

It does not lower the evidentiary standard. FDA still requires substantial evidence of effectiveness from adequate and well-controlled investigations. RWE is a way of meeting that standard with a different data source, not an exemption from it.

Where it is genuinely accepted:

• Label expansions for approved products, particularly new populations.
• Post-approval study requirements and safety commitments.
• External or historical control arms for single-arm trials, most often in rare disease and oncology where randomisation is difficult.
• Natural history characterisation supporting a development programme.

Where sponsors overreach: using a retrospective database analysis to support a primary effectiveness claim for a new molecular entity.

What to do before you generate any

Two things decide whether the evidence survives review, and both happen early.

Protocol and analysis plan first. Write and date them before you look at the outcome data. A retrospective analysis of a database you have already explored invites the question of how many analyses preceded the one you submitted.

Assess the data source before choosing it. Document the relevance and reliability assessment for the specific source, including what the data provider does and does not control. FDA asks sponsors to identify RWD and RWE use in the submission cover letter, along with the purpose, the design and the source.

Data provenance is where these studies fail an inspection. If you cannot trace a number in the analysis back to the record it came from, the study is not verifiable, and ALCOA expectations apply to real world data as much as to trial data.