The PSI One Day Meeting and Workshop "Real World Evidence: Generalisability of Treatment Comparisons for Decision Making" took place on Tuesday 18th September 2018 in Bad Homburg and was hosted by Lilly Germany GmbH. Over 50 participants attended the event. The morning was structured into three sessions that included several talks covering the topics “indirect comparisons with and without adjustment for patient characteristics”, “generalisability of clinical trial data into real life settings” and “cross-design approaches combining observational and clinical trial data”. After lunch, the three sessions were followed by breakout sessions that offered attendees the opportunity to further discuss relevant topics about needed research and impact for the daily work as a statistician. The main findings of the day and take-home messages were summarized in a panel discussion in the end.
| Time | Agenda |
| 08:30 - 08:50 | Registration |
| 08:50 - 09:00 | Welcome and Introduction Alexander Schacht (Lilly) |
| 09:00 - 10:00 | Indirect comparisons with and without adjustment for patient characteristics and related approaches Sarah Böhme (Pfizer) David Phillippo (University Bristol) |
| 10:00 - 11:00 | Generalizability of clinical trial data into real life settings Yann Ruffieux (University Bern) Alan Brnabic (Lilly) |
| 11:00 - 11:15 | Break |
| 11:15 - 12:15 | Cross-design approaches combining observational and clinical trial data Mark Belger (Lilly) Keith Abrams (University Leicester) |
| 12:15 - 13:15 | Lunch |
| 13:15 - 14:45 |
Break out #1: Bucher vs matching adjusted indirect comparisons and further refinements of these Break out #2: Generalizability of clinical trial data into real life settings Break out #3: Cross-design approaches combining observational and clinical trial data |
| 14:45 - 15:00 | Break |
| 15:00 - 16:00 |
Panel discussion Alexander Schacht (Lilly, moderator) |
| 16:00 | Close |
Indirect comparisons with and without adjustment for patient characteristics and related approaches - Sarah Böhme (Pfizer), David Phillippo (University Bristol) |
|
 Sarah Böhme Pfizer |
Indirect comparisons with and without adjustment for patient characteristics within the framework of AMNOG Abstract: Within the framework of the Act on the Reform of the Market for Medicinal Products (AMNOG) in Germany, indirect comparisons are allowed to assess the extent of added benefit in case of a lack of direct evidence. The method proposed by Bucher et al. has been recognized as one of the standard approaches to perform adjusted IC. Further alternative methods exist, e.g. Matching-based approaches, which aim to overcome different challenges. However, all these methods have certain limitations. In this talk the statistical properties of the Bucher approach and the Matching-adjusted indirect comparison as well as their limitations in practice will be discussed. |
|
Multilevel network meta-regression for population adjustment based on individual and aggregate level data Abstract: Standard network meta-analysis (NMA) and indirect comparisons combine aggregate data (AgD) from multiple studies on treatments of interest, assuming that any effect modifiers are balanced across populations. We can relax this assumption if individual patient data (IPD) are available from all studies by fitting an IPD meta-regression. However, in many cases IPD are only available from a subset of studies.In the simplest scenario, IPD are available for an AB study but only AgD for an AC study. Methods such as Matching Adjusted Indirect Comparison (MAIC) create a population-adjusted indirect comparison between treatments B and C. However, the resulting comparison is only valid in the AC population without additional assumptions, and the methods cannot be extended to larger treatment networks. Meta-regression-based approaches can be used in larger networks. However, these typically fit the same model at both the individual and aggregate level which incurs aggregation bias. We propose a general method for synthesising evidence from individual and aggregate data in networks of all sizes, Multilevel Network Meta-Regression, extending the standard NMA framework. An individual-level regression model is defined, and aggregate study data are fitted by integrating this model over the covariate distributions of the respective studies. Since integration is often complex or even intractable, we take a flexible numerical approach using Quasi-Monte Carlo integration, allowing for easy implementation regardless of model form or complexity. Correlation structures between covariates are accounted for using copulae. We illustrate the method using an example and compare the results to those obtained using current methods. Where heterogeneity may be explained by imbalance in effect modifiers between studies we achieve similar fit to a random effects NMA, but uncertainty is substantially reduced, and the model is more interpretable. Crucially for decision making, comparisons may be provided in any target population with a given covariate distribution. Biography: David Phillippo is a statistician at the University of Bristol. His research focuses on methodology for evidence synthesis, Bayesian Network Meta-Analysis, and indirect comparisons. He is the lead author of a recent Technical Support Document published by the NICE Decision Support Unit on population-adjusted indirect comparisons, on which he is also undertaking his PhD. Click here to view the slides. |
|
Generalizability of clinical trial data into real life settings - Yann Ruffieux (University Bern), Alan Brnabic (Lilly) |
|
|
Combining RCT efficacy data and real-world evidence to predict drug effectiveness – a case study in Rheumatoid Arthritis. Abstract: Decision-makers often need to assess the real-world effectiveness of a new drug before it is on the market. We propose a method to predict drug effectiveness pre-launch, and apply it in a case study in rheumatoid arthritis. Our approach comprises several steps: 1) identify an existing treatment similar to the new drug, 2) quantify the impact of treatment, prognostic factors, and effect modifiers on clinical outcome, 3) determine the characteristics of patients likely to receive the new drug in routine care, 4) predict treatment outcome for patients with these characteristics.Biography: Yann Ruffieux is a Statistician at the Institute of Social and Preventive Medecine (ISPM) in Bern, Switzerland. He has an MSc in Mathematical Engineering from the Swiss Federal Institute of Technology in Lausanne (EPFL). After briefly working in pharma as Biostatistician, he joined ISPM in 2015, where he has contributed to the GetReal project and to HIV-related epidemiological research. Click here to view the slides. |
|
Reweighting randomized controlled trial (RCT) evidence to better reflect real life – a case study of the Innovation in Medicine initiative using patients with non-small cell lung cancer (NSCLC) Abstract: The objective of the presentation will be to present a case study that assesses the generalizability of efficacy (overall survival [OS]) from the pivotal RCT (JMDB) comparing pemetrexed with gemicitabine to treat non-squamous non-small cell lung cancer using real-world data from a prospective observational study (FRAME) using a reweighting approach. Both inverse propensity scoring and entropy balancing were used to reweight the RCT data based on the real-world FRAME data in an attempt to mirror routine clinical practice in the trial setting.Biography: Mr. Brnabic is currently Principal Research Scientist at Eli Lilly working in Real World evidence (RWE) with a focus on specialized analysis that supports this area. Prior to this he was the Asia Pacific Director of the Health Outcomes and Health Economics, Life Sciences for OPTUM. Whilst at Eli Lilly he has been the Health Outcomes and Statistics Asia Pacific statistical sciences group leader and manager. His work has included: designing/reviewing and analyzing concepts and studies (Phase IIIb & IV observational studies), as well as leading and reviewing external methodologies/ guidelines for use within the company as well as consulting/coordinating strategy for analysis on Reimbursement dossiers & other related Health Outcome activities for countries like Australia, Canada & Korea. He worked as a Consultant Biostatistician for 5 years in Public Health NSW Health Department. Following that he was a Senior Biostatistician at the George Institute which is affiliated with UNSW where he worked on epidemiological studies and RCTs. Before joining Eli Lilly he also took a position at the NSW Department of Corrective Services as Deputy Director of the Research & Statistics, Sydney. Mr. Brnabic’s interests are in the design and analysis of observational studies with a focus on methodologies related to subgroup identification as well as selection bias adjustment tools including matching, propensity score analysis and local control. He is also interested in Health Outcomes and statistical approaches used to help support the reimbursement of medicines like matched adjusted indirect comparisons as well as mixed treatment comparisons. He has A-STAT Professional Accreditation with the Statistical Society of Australia (SSAI). He is co-chair and previous Chair for the Australian Pharmaceutical Biostatistics Group (APBG). |
|
Cross-design approaches combining observational and clinical trial data - Mark Belger (Lilly), Keith Abrams (University Leicester) |
|
|
Cross-design approaches combining observational and clinical trial data for HTA Abstract: The Innovative Medicines Initiative (IMI) “GetReal” project explored methods for combing Randomised Clinical Trials (RCT) data with non-RCT data within the same Network Meta-Analysis (NMA). Methods such as, the design-adjusted analysis, using informative priors and three-level hierarchical models have been summarised in the manuscript. “Combining randomized and nonrandomized evidence in network meta-analysis “[Orestis Efthimiou et al.]. We will discuss how to incorporate these methods within an HTA setting. Outlining the limitations in combining this type of evidence, and exploring how these methods are used to improve our understanding of how a new intervention will perform outside of the clinical trial environment. Efthimiou O, Mavridis D1, Debray TP, Samara M, Belger M, Siontis GC, Leucht S, Salanti G; GetReal Work Package 4. Combining randomized and non-randomized evidence in network meta-analysis. Stat Med. 2017 Apr 15;36(8):1210-1226. doi: 10.1002/sim.7223. Epub 2017 Jan 12 Biography: Mark Belger been a statistician for the last 34 years mainly working in the area of non-RCT studies. I joined the pharmaceutical industry 14 years ago prior to that I worked in the NHS. I draw from extensive experience of conducting studies in Non RCT populations from both an industry and non-industry perspective. My current responsibilities with Eli Lilly are to support the companies submissions to HTA’s with a focus on our Neurodegeneration and pain indications. In addition, I also lead on a number of Real World analytical methodological projects within the company. I was an active member of IMI “GetReal”, and I am currently involved in two Alzheimer’s disease IMI projects “ROADMAP” and “MOPEAD”. I have co-authored publications that focus on methods for analysing non-RCT data, and clinical papers reporting results from non-RCT studies conducted by Eli Lilly.Click here to view the slides. |
 Keith Abrams, |
Incorporating Real World Evidence (RWE) in Network Meta-Analysis (NMA) – Experiences from the Innovative Medicines Initiative (IMI) GetReal Project Abstract: In this talk the possible situations in which Real World Evidence (RWE), both comparative and single arm studies, could be included in a Network Meta-Analysis (NMA) will be described and discussed. These include; sparse networks, disconnected networks, multiple outcome networks, and the use of such NMAs in terms of decision making and designing future Randomised Controlled Trials (RCTs). In particular, methods for the allowance of potential biases and selection effects associated with RWE and how these may also be incorporated into NMAs will be discussed. The methods will be illustrated using examples from the IMI GetReal Project on patients with Multiple Sclerosis or Rheumatoid Arthritis.Biography: Keith Abrams is Professor of Medical Statistics, within the Department of Health Sciences at the University of Leicester, where he heads the Biostatistics Research Group. His research interests, for which he has an international reputation, are primarily concerned with the development and application of Bayesian statistical methods in Health Technology Assessment (HTA), in particular regarding clinical trials, evidence synthesis, and economic decision modelling, and Non-Communicable Disease (NCD) epidemiology. This work is primarily supported with funding from EU, Medical Research Council (MRC), National Institute for Health Research (NIHR) and industry (with a total value in excess of £20M over the last 5 years). Prof Abrams has been extensively involved with the UK NIHR HTA Programme and UK National Institute for Health & Care Excellence (NICE) appraisal process since their inception. He was a member of the NICE Technology Appraisals Committee for over 8 years until 2015, is a member of the NICE Decision Support Unit and NICE Technical Support Unit, acts as a consultant to the NICE Scientific Advice Programme, and is a NIHR Senior Investigator Emeritus. He is also a Fellow of the Royal Statistical Society, and a Chartered Statistician. He has published widely in both substantive and methodological areas [h-index 69] including co-authoring books on Methods for Meta-Analysis in Medical Research, Bayesian Approaches to Clinical Trials and Healthcare Evaluation, and Evidence Synthesis for Decision Making in Healthcare, in addition to co-editing one of the first texts on Methods for Evidence-based Healthcare. Prof Abrams has extensive experience over the last 25 years as a consultant to both the pharmaceutical and healthcare consultancy sectors, providing both methodological and strategic HTA advice across a wide range of therapeutic areas. Click here to view the slides. |
