How This Glossary Was Developed 

The terms selected for inclusion in this glossary were identified based on a targeted literature review and in the expertise of NHC and Duke-Margolis staff and expert advisors. The following process was used: 

• The NHC conducted three webinars on April 15 and 17, 2020. Webinar participants were asked to complete a survey, and the results were used to inform this list.  
• The RWE Advisory Committee reviewed and refined this list to ensure it is comprehensiveness and utility and identified priority terms to be defined.  
• Definitions for each of the priority terms were compiled from a variety of existing, trusted resources.  
• The compiled definitions were harmonized, transformed into plain language and reviewed by the RWE Advisory Committee.  
• Finally, health literacy experts reviewed the definitions for utility and readability. 

Active Comparator: A treatment used in clinical research that is compared to the “new” treatment being studied. In most studies, the active comparator is the “usual” or most typical treatment for that condition. An active comparator is different from a placebo.¹  

Administrative claims data: See Health insurance claims and billing data 

Bias: An error (typically unintended) in the research design or the method of collecting or analyzing data. The error can lead to incorrect conclusions about a study’s findings.²  

Big data: The combination and analysis of large and diverse data, including non-health and health data. For example, data from administrative claims combined with data from wearable devices that track steps.³ 

Biomarker: Features of the body that you can measure. A substance or signal that can be measured in the blood, other body fluids, or tissues that can be used to provide information about a health condition. Blood pressure is a biomarker. Other examples include lab or radiology results.⁴ 

Claims data: See Health insurance claims and billing data 

Clinical outcome: A type of health outcome. A measurable characteristic that describes or reflects how an individual feels, functions, or survives. Examples of clinical outcomes include pain, mobility, or death.⁵ Clinical outcomes are measured using biomarkers, and survival, and clinical outcome assessments (e.g., patient-reported outcomes.)

Clinical outcome assessment (COA): A type of health outcome measure that tells you something about how a patient feels or functions.⁵ COA measures are typically assessments or questionnaires reported by:  

• Patient direct report (patient-reported outcome or PRO);  
• Observations by a trained clinician (clinician-reported outcome or ClinRO);  
• Observations by someone other than the patient or a health professional (observer-reported outcome ObsRO); or 
• Observations by a trained person who asks the patient to perform a task(s) according to instructions and scores the ability to perform it (performance outcome or PerfO) 

Clinical research: Research in which people, or data or samples of tissue from people, are studied to understand health and disease. There are two broad categories of clinical research: clinical trials and observational studies.¹

Clinical trial: A type of clinical research in which participants are assigned to receive one or more treatments to learn about the safety and effectiveness of a treatment on a defined health outcome. Clinical trials are also called interventional studies.¹  

Cohort: Refers to a group of individuals who have something in common. A group of participants with a shared trait, such as age.¹ For example, an observational study on risks of cigarette smoking follows one cohort of 2019 high–school graduates over time to see who smokes and who develops certain health outcomes.   

Comparative effectiveness research (CER): Research that directly compares two or more treatments or approaches to health care to learn which treatments work best for which patients, and with what benefits and harms.⁶ 

Comparator group: Researchers compare this group of study participant’s results to the group that received the treatment being studied to learn how well the treatment works. Researchers will compare the health outcomes of two groups. One group receives the new treatment and the second group receives the comparator. Participants in the comparator group may get an active comparator (like usual care), a placebo, or no treatment.¹  

Confounding: Confounding happens due to possible “mixing” of effects. It is best explained through an example: a researcher wants to understand the effect of drinking alcohol on lung cancer. The researcher must also consider if people who drink alcohol are more likely to smoke or be around smokers. Smoking is known to increase lung cancer and it is known that people who drink alcohol are often likely to smoke or be around smokers. Because smoking is related to both alcohol consumption and lung cancer, it is considered a confounder. Confounding can lead to research bias. The researcher might conclude that drinking alcohol leads to lung cancer when it is actually the third variable, smoking, that affects the relationship between alcohol and lung cancer.⁷ 

De-identified data: When a person’s health information is intentionally disconnected from that person’s personal details, so the health information cannot be traced back to the individual. For example, removal of data such as name, phone number, address, email address, social security number, medical record number, health plan number, or biometric identifiers (voice/video recordings, fingerprints, or photographs of faces).⁸ 

Effectiveness: Describes a type of research that examines how well a treatment works in the real world under normal circumstances (outside of a traditional clinical trial).⁹ Effectiveness studies tell us: Does it work in regular, day-to-day circumstances?  

Efficacy: Describes a type of study that tells how well a treatment works under ideal circumstances, such as in a clinical trial. Efficacy studies tell us: Can it work under ideal circumstances, such as in a clinical trial? Treatments may show efficacy when used in a trial, but may not be effective in real–world use.⁹ 

Electronic health record (EHR): A patient’s medical record in digital format. It typically includes the information a paper record would have about a patient’s health status and health care, such as diagnoses, medicines, vaccine records, allergies, imaging results, and lab test results. Health care providers fill out and update EHRs at each patient visit or other contact.^6,10  

Eligibility criteria: A list of requirements for determining who can or cannot take part in a clinical trial or be included in an observational research study. Researchers set the eligibility criteria, such as age or sex or diagnosis.¹ There are two types: 

• Exclusion criteria: The factors that prevent someone from taking part in a clinical trial or being included in an observational research study.¹⁹ 
• Inclusion criteria: Factors that allow someone to take part in a clinical trial or be included in an observational research study.¹⁹

Endpoint: An event or outcome that is specifically measured over a defined period of time to identify if or how well a treatment or other intervention worked for participants.^5,11   

FDA: See U.S. Food and Drug Administration 

Health insurance claims and billing data: An electronic record of the information health care providers submit to insurers to receive payment for services provided to a patient.^10,12  

Intervention: See Treatment 

Interventional study: See Clinical trial  

Medical claims data: See Health insurance claims and billing data  

Medical device: An instrument, machine, software algorithm, or tool (such as a surgical instrument, MRI machine, or first aid kit). They may be used to diagnose, treat, or prevent health conditions without having any chemical action in the body. 

Medical product: Any substance, device, or technology used to diagnose, treat, or prevent health conditions.²⁰ 

Natural history study: A type of observational study that follows a group of people over time who have, or have a higher chance of getting, a certain health condition. Information is gathered to understand how the condition develops and changes over time, which may be useful to determine how to treat it in the real world. Data often come from a registry.^{4, 13}  

Observational study: A type of clinical research in which researchers do not assign participants to treatments; they “observe” illness and care in real-world circumstances. The data can be collected directly by the researcher or more typically it comes from other sources such as registries or health insurance claims data. If a treatment is studied, participants were already using it in their medical care; they were not assigned to the treatment. Observational studies are also called non-interventional. Observational studies are useful with rare diseases or if exposing participants to a treatment or other factor would be unethical.³  

• Prospective observational study: Researchers identify the participant population and outcome at the start of the study, and collect information from that point forward.¹²  
• Retrospective observational study: Researchers collect information about participant outcomes from data that existed before the start of the study.¹²  

Outcome: A measurable change in a patient’s health that is due to care or treatment – for example, decreased pain, lower blood pressure, and occurrence of a heart attack.¹¹  

Patient-generated health data (PGHD): Data that are created, recorded, or gathered by or from patients, family members, or caregivers. For example, data gathered through smartphone apps, websites, wearables, and surveys.¹⁴  

Patient registry: An organized collection of data about a specific patient populations’ health condition, lived experience, and treatments. People/patients volunteer to give their information to be used in research, such as in an observational study.^3,12 

Protected or Personal health information (PHI):  Any information about a patient and their health that can be used to identify them. Government laws protect PHI, such as the Health Insurance Portability and Accountability Act (HIPAA). Any information that is related to a patient’s health is PHI.⁸  

Personally identifiable information (PII):  Information that can be used to identify, contact, or find a person. It includes information that is or can be linked to a person, such as medical, educational, financial, and employment information.¹⁵  

PICOTS framework: A tool designed to help researchers clearly state a research question. “PICOTS” is an acronym that stands for the key parts of the research question— Patient population, Intervention (treatment) or issue being studied, Comparator or comparison treatments, Outcome of interest, Time, and Setting. It helps to ensure a research question is clearly explained, specific, and focused on the patient.¹⁶ 

Placebo: A comparator used in a clinical trial that often looks like the trial treatment but does not have any active ingredient in it (e.g., sugar pill). Placebos can help researchers make a comparison with a treatment.¹  

Population: The group of participants who will be studied. Researchers define the specific population for a clinical research study when they set the eligibility criteria, describing such things as age, race, or health condition.¹⁶ 

Post-market surveillance study or Phase 4 study: A type of clinical study that looks at a treatment after it has been approved by a government regulatory agency and is on the market for use. These studies are used to collect more information about side effects and safety, long-term risks and benefits, or how well the treatment works when it is used in the real world. Drug makers may conduct these studies voluntarily or as required by a regulatory agency such as FDA.^3,4  

Pragmatic clinical trial: A type of randomized, clinical trial that takes place in a real-world setting to look at the effects of one or more available treatments.⁶  Pragmatic trials often have broader eligibility criteria than traditional trials (e.g., patients may have wide age ranges, other chronic conditions). 

Randomization: The process of assigning participants to different treatment groups by random chance (like flipping a coin). This helps make sure participants are fairly assigned to treatments. It also helps make sure treatment groups are similar so researchers can compare the results as accurately as possible.⁴ 

Randomized controlled trial (RCT): A type of clinical trial where researchers randomly assign (by chance, like flipping a coin) the participants to different treatments and one of the treatments is considered a control treatment, such as placebo. This helps ensure the treatment groups are similar. It is designed to measure the effects of a treatment by fairly comparing a treatment to a control.⁶ 

Real-world data (RWD): Data about patient health and delivery of care, routinely collected as part of getting care or daily living. RWD can come from a variety of sources, such as electronic health records, health insurance claims and billing, mobile health apps, and surveys. RWD are collected outside of a clinical trial.¹² 

Real-world evidence (RWE): Clinical evidence from research studies that analyze real-world data (RWD). ¹² 

Registry: See Patient registry 

Regulatory agency: A government agency responsible for overseeing and enforcing laws for the benefit of the public. In the U.S., medical products, such as drugs and medical devices, are regulated by the U.S. Food and Drug Administration (FDA). This is the regulatory agency that has responsibility for approving new treatments. 

Sentinel Initiative: A long-term, FDA effort to create a national, electronic system that monitors the safety of medicines in use after FDA approval. The Sentinel Initiative was created in response to a requirement that the FDA work with public, academic, and private organizations to create a system to get information from electronic health care data sources, like EHRs and health insurance claims and billing data.^12,17  

Setting: The location where a study takes place, such as a hospital or nursing home. The setting can also refer to the type of study, such as a clinical trial or a real-world study.¹⁶   

Surrogate endpoint: A substitute for a direct measure of how a patient feels, functions, or survives. It predicts a clinical outcome instead of directly measuring the outcome.⁵ For example, researchers might measure a treatment’s effect on cholesterol as a surrogate endpoint for the long-term effect of death from heart disease, which would take much longer to measure. 

Treatment, also called intervention: A medical product, service, or procedure, such as a surgery, that is used in care or studied. Treatments include drugs, medical devices, procedures, vaccines, diagnostic tests, and other medical products that are either investigational (not yet approved by the FDA) or are already approved. Treatments can also include behaviors, such as education or changes to eating and exercising.¹

U.S. Food and Drug Administration (FDA): FDA is the regulatory agency responsible for protecting the public health by ensuring the safety, efficacy, and security of human and veterinary drugs, biological products, and medical devices; and by ensuring the safety of the nation’s food supply, cosmetics, and products that emit radiation.¹⁹ 

Wearable: An electronic device that people wear, such as a smartwatch. It collects data on the person’s health and activity and can create patient-generated health data (PGHD).¹⁸ 

How to cite this report: National Health Council. Real-World Evidence Glossary Published April 28, 2021. https://nationalhealthcouncil.org/real-world-evidence-glossary/

References 

1. ClinicalTrials.gov. Glossary of Common Site Terms. NIH US National Library of Medicine. Published 2019. Accessed March 9, 2021.https://clinicaltrials.gov/ct2/about-studies/glossary 
2. Schlesselman JJ. Case-Control Studies: Design, Conduct, Analysis. New York: Oxford University Press; 1982. 
3. European Patients’ Academy on Therapeutic Innovation. Glossary. Accessed March 9, 2021. https://toolbox.eupati.eu/glossary/  
4. National Cancer Institute. NCI Dictionary of Cancer Terms. Accessed March 9, 2021. https://www.cancer.gov/publications/dictionaries/cancer-terms  
5. National Institutes of Health. BEST (Biomarkers, EndpointS, and other Tools) Resource. FDA-NIH Biomarker Working Group Web site. Published 2016. Accessed March 9, 2021. https://www.ncbi.nlm.nih.gov/books/NBK338448/   
6. Patient-Centered Outcomes Research Institute. Glossary. Accessed March 9, 2021. https://www.pcori.org/glossary  
7. Understanding Health Research. Confounders. Accessed March 3, 2021. https://www.understandinghealthresearch.org/useful-information/confounders-17  
8. U.S. Health and Human Services. Guidance Regarding Methods for De-identification of Protected Health Information in Accordance with the Health Insurance Portability and Accountability Act (HIPAA) Privacy Rule. Accessed March 9, 2021. https://www.hhs.gov/hipaa/for-professionals/privacy/special-topics/de-identification/index.html#standard  
9. National Health Council. Glossary of Value Assessment Terms. Updated March 19, 2020. Accessed March 9, 2021. https://nationalhealthcouncil.org/glossary-of-value-assessment-terms/  
10. Cigna. Glossary. Accessed March 9, 2021. https://www.cigna.com/glossary  
11. National Health Council. COA Glossary. Accessed March 9, 2021. https://nationalhealthcouncil.org/blog/blog-coa-glossary-untangling-terms-third-installment-our-coa-webinar-series/  
12. U.S. Food and Drug Administration. Framework for FDA’s Real-World Evidence Program. Published December 2018. https://www.fda.gov/media/120060/download  
13. U.S. Food and Drug Administration. Rare Diseases: Natural History Studies for Drug Development. Draft Guidance for Industry. Published March 2019. https://www.fda.gov/regulatory-information/search-fda-guidance-documents/rare-diseases-natural-history-studies-drug-development  
14. Daniel G, McClellan M, Silcox C. Characterizing RWD Quality and Relevancy for Regulatory Purposes. Duke-Margolis Center for Health Policy. Published October 1, 2018. https://healthpolicy.duke.edu/publications/characterizing-rwd-quality-and-relevancy-regulatory-purposes-0  
15. University of Miami Miller School of Medicine Office of Privacy and Data Security. What is personally identifiable information (PII)? Accessed March 9, 2021. http://privacy.med.miami.edu/faq/privacy-faqs/what-is-personally-identifiable-information-pii  
16. Using the PICOTS Framework to Strengthen Evidence Gathered in Clinical Trials—Guidance from the AHRQ’s Evidence-based Practice Centers Program. Accessed March 9, 2021. https://www.fda.gov/media/109448/download  
17. U.S. Food and Drug Administration. Background. Sentinel Initiative. Accessed March 9, 2021. https://www.sentinelinitiative.org/background  
18. Phaneuf A. Latest trends in medical monitoring devices and wearable health technology. Business Insider. Published January 11, 2021. Accessed March 9, 2021 https://www.businessinsider.com/wearable-technology-healthcare-medical-devices  
19. U.S. Food and Drug Administration. Clinical Trials: What Patients Need to Know. Glossary of Terms. Accessed April 6, 2021.  https://www.fda.gov/patients/clinical-trials-what-patients-need-know/glossary-terms.  
20. PREFER Patient Preferences. Glossary. Accessed April 6, 2021. https://www.imi-prefer.eu/about/glossary/.