​The use of real world data in NICE decision-making

Written on Tuesday 7th November 2017

Fiona Glen, Healthcare Data Analyst at pH Associates (An OPEN Health company) discusses a recent International Society for Pharmacoeconomics and Outcomes Research (ISPOR) article which looks at use of RWE in HTAs across different European agencies. The findings complement pH's poster being presented this afternoon. The poster is an audit titled “The use of real world data for NICE decision-making: a review of submissions to three treatment effectiveness evaluation programmes in 2015 and 2016.

The use of real world data in NICE decision-making

Many countries have their own healthcare bodies dedicated to the systematic evaluation of new technologies, in order to provide recommendations as to their effectiveness and whether they should be financed or reimbursed by the healthcare system in that particular region. In England and Wales, The National Institute for Health and Care Excellence (NICE) develops recommendations for new technologies based on evidence submitted across a number of different evaluation programmes. For instance, the Health Technology Appraisal (HTA) programme produces guidance that primarily assesses the clinical and cost effectiveness of new pharmaceutical and biopharmaceutical products, as well as certain procedures, devices and diagnostic agents.

Diagnostic guidance produced by NICE on the other hand focuses specifically on the evaluation of innovative medical diagnostic technologies, while Medical Technologies guidance addresses specific technologies that were notified to NICE by manufacturers. NICE also produces guidance for interventional procedures and antimicrobial prescribing as well as making general evidence-based recommendations on how to prevent and manage specific conditions.

Traditionally, the ‘gold standard’ evidence source for NICE appraisals, and indeed for other international assessment bodies, has been the randomised controlled trial (RCT), a stringent and highly controlled study design whereby a carefully selected group of patients are assigned to either the experimental treatment condition or a comparator. However, due to their highly controlled nature, it is possible that results from RCTs may not generalise to the more varied and diverse patient groups seen in real-world clinical practice.

RCTs are also costly and time-consuming to run and may not always be practical or ethical to conduct; for instance, in patients with rare diseases or in situations whereby randomising patients to a less beneficial treatment could have a direct impact on their survival or another key clinical outcome. As such, real world data are now also being considered alongside RCTs as a relevant evidence source by NICE when making funding and licensing decisions.

Real world data can be gathered from retrospective or prospective studies in which the investigator observes natural events without intervention (e.g. audits, cohort or case–control studies). The inclusion of real world data in NICE submissions may help to demonstrate external validity of study findings and accelerate approval processes, thereby allowing earlier patient access to innovative technologies. SamOliver@phassociates.com)

There is, however, limited guidance available regarding the appropriate use of real world data when designing and analysing studies for assessing treatment effectiveness, and this advice varies across different NICE programmes. (1) (2)

At pH Associates, they recently carried out an audit of recent NICE guidance and evidence appraisals, in order to determine the extent to which real world data was considered in submissions and reviews for the HTA, Diagnostic Assessment and Medical Technology Evaluation programmes in 2015 and 2016. We found that evidence derived from a number of non-interventional studies, including audits, retrospective case reviews and surveys, was indeed included in recent submissions across a variety of disease areas. This evidence was carefully appraised by reviewing committees with specific comments emphasizing a need for more robust study design in many cases, including the demonstrated use of appropriate sample sizes and follow-ups, detailed descriptions of disease populations and adjustments for confounders.

There was also a trend towards specific recommendations for further real world data collection, in order to show the long-term effectiveness of technologies, the impact on the quality of life of patients, and the costs and resource use associated with utilising technologies in real-world practice.

The extent to which real world evidence was utilised within submissions also varied across programmes, with the majority of diagnostic and medical technology submissions including evidence derived from real world data compared with only 10% of HTA submissions.

These findings coincided with a recent review of six European HTA agencies which reported a large degree in variation in guidance and policies for the use of RWD across countries and contexts. (3)

Detailed guidance and collaboration across NICE programmes will be vital to not only encourage the inclusion of real world data in healthcare decision-making, but also to facilitate robust study design that both fulfils committee requirements and accurately depicts the diverse nature of real-world clinical practice.

If you would like to learn more about pH Associate's audit on the use of real world data in recent NICE submissions,their poster titled “The use of real world data for NICE decision-making: a review of submissions to three treatment effectiveness evaluation programmes in 2015 and 2016” will be presented at ISPOR on Tuesday 7th November (SESSION IV; PHP239; Board I4). Members of the pH Associates team will also be at Stand 105 throughout the conference, so please do come and say hello!

Alternatively, please contact Sam Oliver or Amanda Pulfer, Joint Managing Directors at pH Associates via email at SamOliver@phassociates.com and Amandapulfer@phassociates.com for more information, or to discuss your specific real world data needs.


  1. Faria et.al. (2015). NICE DSU technical support document 17: the use of observational data to inform estimates of treatment effectiveness in technology appraisal: methods for comparative individual patient data.
  2. Bell et al. (2016). The use of real world data for the estimation of treatment effects in NICE decision- making.
  3.  Makady et al. (2017). Policies for Use of Real-World Data in Health Technology Assessment (HTA): A Comparative Study of Six HTA Agencies. Value in Health, 20(4), 520-532.