In October 2014 the ICH Steering Committee endorsed a final Concept Paper with the goal of developing new regulatory guidance, suggested to be an Addendum to ICH E9, which promotes harmonised standards on the choice of estimands in clinical trials and an agreed framework for planning, conducting and interpreting sensitivity analyses of clinical trial data. An Expert Working Group sponsored by ICH has been established
to further develop the concept and is due to report findings in December
2015. Frank Bretz and Chrissie Fletcher are the EFPIA representatives on
the ICH E9(R1) Expert Working Group.
As a first step in developing the response from the European industry to any guidance issued by the ICH E9(R1) Expert Working Group, PSI/EFSPI held a 1-day workshop on estimands and sensitivity analyses in February 2015. A list of meeting attendees with affiliations is given in Appendix 1. Most participants recognised that they had not been aware of the word “estimand” - still less actually used it – before the report “The prevention and treatment of missing data in clinical trials” was published by the National Research Council in 2010. So one important aspect of this area in the next few years will be to communicate and educate both statisticians and non-statisticians in the concepts, terminology and consequences of the choice of estimands for a clinical trial. The estimand framework helps to distinguish between the target of estimation (estimand) and the method of estimation (estimator).An estimand reflects what is to be estimated to address the scientific question of interest posed by a trial.
It will be important to develop and publicise some simple but informative examples which can motivate all involved in clinical trials to become more focussed on the choice of estimands. For example two relevant classes of estimand are the ‘de jure’ estimand (“efficacy”) and the ‘de facto’ estimand (“effectiveness”).
” The efficacy is an ideal treatment effect that could have been reached if all patients had fully adhered or, in other words, the expected effect a patient may reach if he or she takes the medication as directed. Because of the lack of perfect adherence in practice, the efficacy cannot be observed. In contrast, effectiveness comprises treatment effects that occur when full adherence to treatment is lacking. Besides the sole benefit of the treatment, effectiveness also takes into account safety and tolerability issues.” (Leuchs et al, 2015). The discussion group noted that the positioning of this new document as an addendum to ICH E9 may give the impression that it is of interest only or primarily to statisticians. Yet the “Statement of the Perceived Problem” makes it clear that all concerned with clinical trials need to give this area further thought – “Incorrect choice of estimand and unclear definitions for estimands lead to problems in relation to trial design, conduct and analysis and introduce potential for inconsistencies in inference and decision making.” Although the die is cast, some members of the PSI/EFSPI discussion group felt that attaching the addendum to ICH E8 on “General Considerations For Clinical Trials” might have involved a broader leadership in making the changes in practice required. In particular E8 would benefit from the addition of a robust definition of the objectives of a study.
The group would position the estimand as a more detailed statement of the objective, formulated to address the scientific question of interest posed by the trial. It would therefore specify:
• a population of interest, e.g. the population defined by the target Summary of Product Characteristics (not to be confused with the analysis population);
• an endpoint of interest, i.e. a measurement at a specific timepoint, and
• a measure of the relevant effect of the intervention, taking into account the potential confounding due to post-randomisation events such as non-compliance, use of rescue medication or death. It was noted that while a study design follows from the objectives and estimands, feasibility aspects may result in a feedback loop to refine them. It was felt that clearly defined estimands will be needed to make regulatory claims and that each estimand involved would need to be linked back to objectives and the study design.
It was recognised that different stakeholders may be interested in different estimands, in particular distinguishing between regulatory approval and Health Technology Assessment. However any assumption that the former are more interested in efficacy estimands and the latter in effectiveness estimands is over-simplistic, as there is some evidence of regulators not finding ‘de jure’ estimands useful.
The ICH E9(R1) Concept Paper focuses equally on sensitivity analyses, but the discussion group spent less time in this area. We agreed with the Concept Paper that there is no clear framework for planning and executing sensitivity analyses and that this can lead to inconsistency in regulatory decision-making. The recent paper by Leuchs et al (2015) gives a useful distinction between the goals of internal and external validity for sensitivity analysis and this framework may form a useful foundation for further work.
While the group had differing opinions in certain areas, it was agreed that this is an important area for statisticians to be involved in as we aim to clarify the objectives and applicability of clinical trials. Attendees at the PSI conference will be able to take advantage of two sessions on this topic on the morning of Tuesday May 12th, including statistical leaders from both FDA and MHRA as well as industry and academia. In addition the discussion group has had a letter to the editor accepted commenting on the Leuchs paper and is exploring a separate publication of its own.David Morgan (Ipsen)
Reference:
International Conference on Harmonisation (2014) Final Concept Paper E9(R1):
Addendum to Statistical Principles for Clinical Trials. Geneva, Switzerland.
http://www.ich.org/fileadmin/Public_Web_Site/ICH_Products/Guidelines/
Efficacy/E9/E9__R1__Final_Concept_Paper_October_23_2014.pdf - accessed 21
April 2015.
A-K Leuchs, J Zinserling, A Brandt, D Wirtz and N Benda (2015) Choosing
Appropriate Estimands in Clinical Trials. Therapeutic Innovation & Regulatory
Science
Appendix 1: List of meeting attendees and affiliations
Juan Abellan-Andres Gruenenthal
Soren Andersen Novo Nordisk
Frank Bretz Novartis
Chrissie Fletcher Amgen
Lesley France AstraZeneca
Andrew Garrett Quintiles
Ray Harris Eisai
Oliver Keene GlaxoSmithKline
Magnus Kjaer AstraZeneca
David Morgan Ipsen
Michael O’Kelly Quintiles
Alan Phillips Icon
James Roger Live Data
In October 2014 the ICH Steering Committee endorsed a final Concept Paper with the goal of developing new regulatory guidance, suggested to be an Addendum to ICH E9, which promotes harmonised standards on the choice of estimands in clinical trials and an agreed framework for planning, conducting and interpreting sensitivity analyses of clinical trial data. An Expert Working Group sponsored by ICH has been established
to further develop the concept and is due to report findings in December
2015. Frank Bretz and Chrissie Fletcher are the EFPIA representatives on
the ICH E9(R1) Expert Working Group.
As a first step in developing the response from the European industry to any guidance issued by the ICH E9(R1) Expert Working Group, PSI/EFSPI held a 1-day workshop on estimands and sensitivity analyses in February 2015. A list of meeting attendees with affiliations is given in Appendix 1. Most participants recognised that they had not been aware of the word “estimand” - still less actually used it – before the report “The prevention and treatment of missing data in clinical trials” was published by the National Research Council in 2010. So one important aspect of this area in the next few years will be to communicate and educate both statisticians and non-statisticians in the concepts, terminology and consequences of the choice of estimands for a clinical trial. The estimand framework helps to distinguish between the target of estimation (estimand) and the method of estimation (estimator).An estimand reflects what is to be estimated to address the scientific question of interest posed by a trial.
It will be important to develop and publicise some simple but informative examples which can motivate all involved in clinical trials to become more focussed on the choice of estimands. For example two relevant classes of estimand are the ‘de jure’ estimand (“efficacy”) and the ‘de facto’ estimand (“effectiveness”).
” The efficacy is an ideal treatment effect that could have been reached if all patients had fully adhered or, in other words, the expected effect a patient may reach if he or she takes the medication as directed. Because of the lack of perfect adherence in practice, the efficacy cannot be observed. In contrast, effectiveness comprises treatment effects that occur when full adherence to treatment is lacking. Besides the sole benefit of the treatment, effectiveness also takes into account safety and tolerability issues.” (Leuchs et al, 2015). The discussion group noted that the positioning of this new document as an addendum to ICH E9 may give the impression that it is of interest only or primarily to statisticians. Yet the “Statement of the Perceived Problem” makes it clear that all concerned with clinical trials need to give this area further thought – “Incorrect choice of estimand and unclear definitions for estimands lead to problems in relation to trial design, conduct and analysis and introduce potential for inconsistencies in inference and decision making.” Although the die is cast, some members of the PSI/EFSPI discussion group felt that attaching the addendum to ICH E8 on “General Considerations For Clinical Trials” might have involved a broader leadership in making the changes in practice required. In particular E8 would benefit from the addition of a robust definition of the objectives of a study.
The group would position the estimand as a more detailed statement of the objective, formulated to address the scientific question of interest posed by the trial. It would therefore specify:
• a population of interest, e.g. the population defined by the target Summary of Product Characteristics (not to be confused with the analysis population);
• an endpoint of interest, i.e. a measurement at a specific timepoint, and
• a measure of the relevant effect of the intervention, taking into account the potential confounding due to post-randomisation events such as non-compliance, use of rescue medication or death. It was noted that while a study design follows from the objectives and estimands, feasibility aspects may result in a feedback loop to refine them. It was felt that clearly defined estimands will be needed to make regulatory claims and that each estimand involved would need to be linked back to objectives and the study design.
It was recognised that different stakeholders may be interested in different estimands, in particular distinguishing between regulatory approval and Health Technology Assessment. However any assumption that the former are more interested in efficacy estimands and the latter in effectiveness estimands is over-simplistic, as there is some evidence of regulators not finding ‘de jure’ estimands useful.
The ICH E9(R1) Concept Paper focuses equally on sensitivity analyses, but the discussion group spent less time in this area. We agreed with the Concept Paper that there is no clear framework for planning and executing sensitivity analyses and that this can lead to inconsistency in regulatory decision-making. The recent paper by Leuchs et al (2015) gives a useful distinction between the goals of internal and external validity for sensitivity analysis and this framework may form a useful foundation for further work.
While the group had differing opinions in certain areas, it was agreed that this is an important area for statisticians to be involved in as we aim to clarify the objectives and applicability of clinical trials. Attendees at the PSI conference will be able to take advantage of two sessions on this topic on the morning of Tuesday May 12th, including statistical leaders from both FDA and MHRA as well as industry and academia. In addition the discussion group has had a letter to the editor accepted commenting on the Leuchs paper and is exploring a separate publication of its own.David Morgan (Ipsen)
Reference:
International Conference on Harmonisation (2014) Final Concept Paper E9(R1):
Addendum to Statistical Principles for Clinical Trials. Geneva, Switzerland.
http://www.ich.org/fileadmin/Public_Web_Site/ICH_Products/Guidelines/
Efficacy/E9/E9__R1__Final_Concept_Paper_October_23_2014.pdf - accessed 21
April 2015.
A-K Leuchs, J Zinserling, A Brandt, D Wirtz and N Benda (2015) Choosing
Appropriate Estimands in Clinical Trials. Therapeutic Innovation & Regulatory
Science
Appendix 1: List of meeting attendees and affiliations
Juan Abellan-Andres Gruenenthal
Soren Andersen Novo Nordisk
Frank Bretz Novartis
Chrissie Fletcher Amgen
Lesley France AstraZeneca
Andrew Garrett Quintiles
Ray Harris Eisai
Oliver Keene GlaxoSmithKline
Magnus Kjaer AstraZeneca
David Morgan Ipsen
Michael O’Kelly Quintiles
Alan Phillips Icon
James Roger Live Data
In October 2014 the ICH Steering Committee endorsed a final Concept Paper with the goal of developing new regulatory guidance, suggested to be an Addendum to ICH E9, which promotes harmonised standards on the choice of estimands in clinical trials and an agreed framework for planning, conducting and interpreting sensitivity analyses of clinical trial data. An Expert Working Group sponsored by ICH has been established
to further develop the concept and is due to report findings in December
2015. Frank Bretz and Chrissie Fletcher are the EFPIA representatives on
the ICH E9(R1) Expert Working Group.
As a first step in developing the response from the European industry to any guidance issued by the ICH E9(R1) Expert Working Group, PSI/EFSPI held a 1-day workshop on estimands and sensitivity analyses in February 2015. A list of meeting attendees with affiliations is given in Appendix 1. Most participants recognised that they had not been aware of the word “estimand” - still less actually used it – before the report “The prevention and treatment of missing data in clinical trials” was published by the National Research Council in 2010. So one important aspect of this area in the next few years will be to communicate and educate both statisticians and non-statisticians in the concepts, terminology and consequences of the choice of estimands for a clinical trial. The estimand framework helps to distinguish between the target of estimation (estimand) and the method of estimation (estimator).An estimand reflects what is to be estimated to address the scientific question of interest posed by a trial.
It will be important to develop and publicise some simple but informative examples which can motivate all involved in clinical trials to become more focussed on the choice of estimands. For example two relevant classes of estimand are the ‘de jure’ estimand (“efficacy”) and the ‘de facto’ estimand (“effectiveness”).
” The efficacy is an ideal treatment effect that could have been reached if all patients had fully adhered or, in other words, the expected effect a patient may reach if he or she takes the medication as directed. Because of the lack of perfect adherence in practice, the efficacy cannot be observed. In contrast, effectiveness comprises treatment effects that occur when full adherence to treatment is lacking. Besides the sole benefit of the treatment, effectiveness also takes into account safety and tolerability issues.” (Leuchs et al, 2015). The discussion group noted that the positioning of this new document as an addendum to ICH E9 may give the impression that it is of interest only or primarily to statisticians. Yet the “Statement of the Perceived Problem” makes it clear that all concerned with clinical trials need to give this area further thought – “Incorrect choice of estimand and unclear definitions for estimands lead to problems in relation to trial design, conduct and analysis and introduce potential for inconsistencies in inference and decision making.” Although the die is cast, some members of the PSI/EFSPI discussion group felt that attaching the addendum to ICH E8 on “General Considerations For Clinical Trials” might have involved a broader leadership in making the changes in practice required. In particular E8 would benefit from the addition of a robust definition of the objectives of a study.
The group would position the estimand as a more detailed statement of the objective, formulated to address the scientific question of interest posed by the trial. It would therefore specify:
• a population of interest, e.g. the population defined by the target Summary of Product Characteristics (not to be confused with the analysis population);
• an endpoint of interest, i.e. a measurement at a specific timepoint, and
• a measure of the relevant effect of the intervention, taking into account the potential confounding due to post-randomisation events such as non-compliance, use of rescue medication or death. It was noted that while a study design follows from the objectives and estimands, feasibility aspects may result in a feedback loop to refine them. It was felt that clearly defined estimands will be needed to make regulatory claims and that each estimand involved would need to be linked back to objectives and the study design.
It was recognised that different stakeholders may be interested in different estimands, in particular distinguishing between regulatory approval and Health Technology Assessment. However any assumption that the former are more interested in efficacy estimands and the latter in effectiveness estimands is over-simplistic, as there is some evidence of regulators not finding ‘de jure’ estimands useful.
The ICH E9(R1) Concept Paper focuses equally on sensitivity analyses, but the discussion group spent less time in this area. We agreed with the Concept Paper that there is no clear framework for planning and executing sensitivity analyses and that this can lead to inconsistency in regulatory decision-making. The recent paper by Leuchs et al (2015) gives a useful distinction between the goals of internal and external validity for sensitivity analysis and this framework may form a useful foundation for further work.
While the group had differing opinions in certain areas, it was agreed that this is an important area for statisticians to be involved in as we aim to clarify the objectives and applicability of clinical trials. Attendees at the PSI conference will be able to take advantage of two sessions on this topic on the morning of Tuesday May 12th, including statistical leaders from both FDA and MHRA as well as industry and academia. In addition the discussion group has had a letter to the editor accepted commenting on the Leuchs paper and is exploring a separate publication of its own.David Morgan (Ipsen)
Reference:
International Conference on Harmonisation (2014) Final Concept Paper E9(R1):
Addendum to Statistical Principles for Clinical Trials. Geneva, Switzerland.
http://www.ich.org/fileadmin/Public_Web_Site/ICH_Products/Guidelines/
Efficacy/E9/E9__R1__Final_Concept_Paper_October_23_2014.pdf - accessed 21
April 2015.
A-K Leuchs, J Zinserling, A Brandt, D Wirtz and N Benda (2015) Choosing
Appropriate Estimands in Clinical Trials. Therapeutic Innovation & Regulatory
Science
Appendix 1: List of meeting attendees and affiliations
Juan Abellan-Andres Gruenenthal
Soren Andersen Novo Nordisk
Frank Bretz Novartis
Chrissie Fletcher Amgen
Lesley France AstraZeneca
Andrew Garrett Quintiles
Ray Harris Eisai
Oliver Keene GlaxoSmithKline
Magnus Kjaer AstraZeneca
David Morgan Ipsen
Michael O’Kelly Quintiles
Alan Phillips Icon
James Roger Live Data
In October 2014 the ICH Steering Committee endorsed a final Concept Paper with the goal of developing new regulatory guidance, suggested to be an Addendum to ICH E9, which promotes harmonised standards on the choice of estimands in clinical trials and an agreed framework for planning, conducting and interpreting sensitivity analyses of clinical trial data. An Expert Working Group sponsored by ICH has been established
to further develop the concept and is due to report findings in December
2015. Frank Bretz and Chrissie Fletcher are the EFPIA representatives on
the ICH E9(R1) Expert Working Group.
As a first step in developing the response from the European industry to any guidance issued by the ICH E9(R1) Expert Working Group, PSI/EFSPI held a 1-day workshop on estimands and sensitivity analyses in February 2015. A list of meeting attendees with affiliations is given in Appendix 1. Most participants recognised that they had not been aware of the word “estimand” - still less actually used it – before the report “The prevention and treatment of missing data in clinical trials” was published by the National Research Council in 2010. So one important aspect of this area in the next few years will be to communicate and educate both statisticians and non-statisticians in the concepts, terminology and consequences of the choice of estimands for a clinical trial. The estimand framework helps to distinguish between the target of estimation (estimand) and the method of estimation (estimator).An estimand reflects what is to be estimated to address the scientific question of interest posed by a trial.
It will be important to develop and publicise some simple but informative examples which can motivate all involved in clinical trials to become more focussed on the choice of estimands. For example two relevant classes of estimand are the ‘de jure’ estimand (“efficacy”) and the ‘de facto’ estimand (“effectiveness”).
” The efficacy is an ideal treatment effect that could have been reached if all patients had fully adhered or, in other words, the expected effect a patient may reach if he or she takes the medication as directed. Because of the lack of perfect adherence in practice, the efficacy cannot be observed. In contrast, effectiveness comprises treatment effects that occur when full adherence to treatment is lacking. Besides the sole benefit of the treatment, effectiveness also takes into account safety and tolerability issues.” (Leuchs et al, 2015). The discussion group noted that the positioning of this new document as an addendum to ICH E9 may give the impression that it is of interest only or primarily to statisticians. Yet the “Statement of the Perceived Problem” makes it clear that all concerned with clinical trials need to give this area further thought – “Incorrect choice of estimand and unclear definitions for estimands lead to problems in relation to trial design, conduct and analysis and introduce potential for inconsistencies in inference and decision making.” Although the die is cast, some members of the PSI/EFSPI discussion group felt that attaching the addendum to ICH E8 on “General Considerations For Clinical Trials” might have involved a broader leadership in making the changes in practice required. In particular E8 would benefit from the addition of a robust definition of the objectives of a study.
The group would position the estimand as a more detailed statement of the objective, formulated to address the scientific question of interest posed by the trial. It would therefore specify:
• a population of interest, e.g. the population defined by the target Summary of Product Characteristics (not to be confused with the analysis population);
• an endpoint of interest, i.e. a measurement at a specific timepoint, and
• a measure of the relevant effect of the intervention, taking into account the potential confounding due to post-randomisation events such as non-compliance, use of rescue medication or death. It was noted that while a study design follows from the objectives and estimands, feasibility aspects may result in a feedback loop to refine them. It was felt that clearly defined estimands will be needed to make regulatory claims and that each estimand involved would need to be linked back to objectives and the study design.
It was recognised that different stakeholders may be interested in different estimands, in particular distinguishing between regulatory approval and Health Technology Assessment. However any assumption that the former are more interested in efficacy estimands and the latter in effectiveness estimands is over-simplistic, as there is some evidence of regulators not finding ‘de jure’ estimands useful.
The ICH E9(R1) Concept Paper focuses equally on sensitivity analyses, but the discussion group spent less time in this area. We agreed with the Concept Paper that there is no clear framework for planning and executing sensitivity analyses and that this can lead to inconsistency in regulatory decision-making. The recent paper by Leuchs et al (2015) gives a useful distinction between the goals of internal and external validity for sensitivity analysis and this framework may form a useful foundation for further work.
While the group had differing opinions in certain areas, it was agreed that this is an important area for statisticians to be involved in as we aim to clarify the objectives and applicability of clinical trials. Attendees at the PSI conference will be able to take advantage of two sessions on this topic on the morning of Tuesday May 12th, including statistical leaders from both FDA and MHRA as well as industry and academia. In addition the discussion group has had a letter to the editor accepted commenting on the Leuchs paper and is exploring a separate publication of its own.David Morgan (Ipsen)
Reference:
International Conference on Harmonisation (2014) Final Concept Paper E9(R1):
Addendum to Statistical Principles for Clinical Trials. Geneva, Switzerland.
http://www.ich.org/fileadmin/Public_Web_Site/ICH_Products/Guidelines/
Efficacy/E9/E9__R1__Final_Concept_Paper_October_23_2014.pdf - accessed 21
April 2015.
A-K Leuchs, J Zinserling, A Brandt, D Wirtz and N Benda (2015) Choosing
Appropriate Estimands in Clinical Trials. Therapeutic Innovation & Regulatory
Science
Appendix 1: List of meeting attendees and affiliations
Juan Abellan-Andres Gruenenthal
Soren Andersen Novo Nordisk
Frank Bretz Novartis
Chrissie Fletcher Amgen
Lesley France AstraZeneca
Andrew Garrett Quintiles
Ray Harris Eisai
Oliver Keene GlaxoSmithKline
Magnus Kjaer AstraZeneca
David Morgan Ipsen
Michael O’Kelly Quintiles
Alan Phillips Icon
James Roger Live Data
In October 2014 the ICH Steering Committee endorsed a final Concept Paper with the goal of developing new regulatory guidance, suggested to be an Addendum to ICH E9, which promotes harmonised standards on the choice of estimands in clinical trials and an agreed framework for planning, conducting and interpreting sensitivity analyses of clinical trial data. An Expert Working Group sponsored by ICH has been established
to further develop the concept and is due to report findings in December
2015. Frank Bretz and Chrissie Fletcher are the EFPIA representatives on
the ICH E9(R1) Expert Working Group.
As a first step in developing the response from the European industry to any guidance issued by the ICH E9(R1) Expert Working Group, PSI/EFSPI held a 1-day workshop on estimands and sensitivity analyses in February 2015. A list of meeting attendees with affiliations is given in Appendix 1. Most participants recognised that they had not been aware of the word “estimand” - still less actually used it – before the report “The prevention and treatment of missing data in clinical trials” was published by the National Research Council in 2010. So one important aspect of this area in the next few years will be to communicate and educate both statisticians and non-statisticians in the concepts, terminology and consequences of the choice of estimands for a clinical trial. The estimand framework helps to distinguish between the target of estimation (estimand) and the method of estimation (estimator).An estimand reflects what is to be estimated to address the scientific question of interest posed by a trial.
It will be important to develop and publicise some simple but informative examples which can motivate all involved in clinical trials to become more focussed on the choice of estimands. For example two relevant classes of estimand are the ‘de jure’ estimand (“efficacy”) and the ‘de facto’ estimand (“effectiveness”).
” The efficacy is an ideal treatment effect that could have been reached if all patients had fully adhered or, in other words, the expected effect a patient may reach if he or she takes the medication as directed. Because of the lack of perfect adherence in practice, the efficacy cannot be observed. In contrast, effectiveness comprises treatment effects that occur when full adherence to treatment is lacking. Besides the sole benefit of the treatment, effectiveness also takes into account safety and tolerability issues.” (Leuchs et al, 2015). The discussion group noted that the positioning of this new document as an addendum to ICH E9 may give the impression that it is of interest only or primarily to statisticians. Yet the “Statement of the Perceived Problem” makes it clear that all concerned with clinical trials need to give this area further thought – “Incorrect choice of estimand and unclear definitions for estimands lead to problems in relation to trial design, conduct and analysis and introduce potential for inconsistencies in inference and decision making.” Although the die is cast, some members of the PSI/EFSPI discussion group felt that attaching the addendum to ICH E8 on “General Considerations For Clinical Trials” might have involved a broader leadership in making the changes in practice required. In particular E8 would benefit from the addition of a robust definition of the objectives of a study.
The group would position the estimand as a more detailed statement of the objective, formulated to address the scientific question of interest posed by the trial. It would therefore specify:
• a population of interest, e.g. the population defined by the target Summary of Product Characteristics (not to be confused with the analysis population);
• an endpoint of interest, i.e. a measurement at a specific timepoint, and
• a measure of the relevant effect of the intervention, taking into account the potential confounding due to post-randomisation events such as non-compliance, use of rescue medication or death. It was noted that while a study design follows from the objectives and estimands, feasibility aspects may result in a feedback loop to refine them. It was felt that clearly defined estimands will be needed to make regulatory claims and that each estimand involved would need to be linked back to objectives and the study design.
It was recognised that different stakeholders may be interested in different estimands, in particular distinguishing between regulatory approval and Health Technology Assessment. However any assumption that the former are more interested in efficacy estimands and the latter in effectiveness estimands is over-simplistic, as there is some evidence of regulators not finding ‘de jure’ estimands useful.
The ICH E9(R1) Concept Paper focuses equally on sensitivity analyses, but the discussion group spent less time in this area. We agreed with the Concept Paper that there is no clear framework for planning and executing sensitivity analyses and that this can lead to inconsistency in regulatory decision-making. The recent paper by Leuchs et al (2015) gives a useful distinction between the goals of internal and external validity for sensitivity analysis and this framework may form a useful foundation for further work.
While the group had differing opinions in certain areas, it was agreed that this is an important area for statisticians to be involved in as we aim to clarify the objectives and applicability of clinical trials. Attendees at the PSI conference will be able to take advantage of two sessions on this topic on the morning of Tuesday May 12th, including statistical leaders from both FDA and MHRA as well as industry and academia. In addition the discussion group has had a letter to the editor accepted commenting on the Leuchs paper and is exploring a separate publication of its own.David Morgan (Ipsen)
Reference:
International Conference on Harmonisation (2014) Final Concept Paper E9(R1):
Addendum to Statistical Principles for Clinical Trials. Geneva, Switzerland.
http://www.ich.org/fileadmin/Public_Web_Site/ICH_Products/Guidelines/
Efficacy/E9/E9__R1__Final_Concept_Paper_October_23_2014.pdf - accessed 21
April 2015.
A-K Leuchs, J Zinserling, A Brandt, D Wirtz and N Benda (2015) Choosing
Appropriate Estimands in Clinical Trials. Therapeutic Innovation & Regulatory
Science
Appendix 1: List of meeting attendees and affiliations
Juan Abellan-Andres Gruenenthal
Soren Andersen Novo Nordisk
Frank Bretz Novartis
Chrissie Fletcher Amgen
Lesley France AstraZeneca
Andrew Garrett Quintiles
Ray Harris Eisai
Oliver Keene GlaxoSmithKline
Magnus Kjaer AstraZeneca
David Morgan Ipsen
Michael O’Kelly Quintiles
Alan Phillips Icon
James Roger Live Data
In October 2014 the ICH Steering Committee endorsed a final Concept Paper with the goal of developing new regulatory guidance, suggested to be an Addendum to ICH E9, which promotes harmonised standards on the choice of estimands in clinical trials and an agreed framework for planning, conducting and interpreting sensitivity analyses of clinical trial data. An Expert Working Group sponsored by ICH has been established
to further develop the concept and is due to report findings in December
2015. Frank Bretz and Chrissie Fletcher are the EFPIA representatives on
the ICH E9(R1) Expert Working Group.
As a first step in developing the response from the European industry to any guidance issued by the ICH E9(R1) Expert Working Group, PSI/EFSPI held a 1-day workshop on estimands and sensitivity analyses in February 2015. A list of meeting attendees with affiliations is given in Appendix 1. Most participants recognised that they had not been aware of the word “estimand” - still less actually used it – before the report “The prevention and treatment of missing data in clinical trials” was published by the National Research Council in 2010. So one important aspect of this area in the next few years will be to communicate and educate both statisticians and non-statisticians in the concepts, terminology and consequences of the choice of estimands for a clinical trial. The estimand framework helps to distinguish between the target of estimation (estimand) and the method of estimation (estimator).An estimand reflects what is to be estimated to address the scientific question of interest posed by a trial.
It will be important to develop and publicise some simple but informative examples which can motivate all involved in clinical trials to become more focussed on the choice of estimands. For example two relevant classes of estimand are the ‘de jure’ estimand (“efficacy”) and the ‘de facto’ estimand (“effectiveness”).
” The efficacy is an ideal treatment effect that could have been reached if all patients had fully adhered or, in other words, the expected effect a patient may reach if he or she takes the medication as directed. Because of the lack of perfect adherence in practice, the efficacy cannot be observed. In contrast, effectiveness comprises treatment effects that occur when full adherence to treatment is lacking. Besides the sole benefit of the treatment, effectiveness also takes into account safety and tolerability issues.” (Leuchs et al, 2015). The discussion group noted that the positioning of this new document as an addendum to ICH E9 may give the impression that it is of interest only or primarily to statisticians. Yet the “Statement of the Perceived Problem” makes it clear that all concerned with clinical trials need to give this area further thought – “Incorrect choice of estimand and unclear definitions for estimands lead to problems in relation to trial design, conduct and analysis and introduce potential for inconsistencies in inference and decision making.” Although the die is cast, some members of the PSI/EFSPI discussion group felt that attaching the addendum to ICH E8 on “General Considerations For Clinical Trials” might have involved a broader leadership in making the changes in practice required. In particular E8 would benefit from the addition of a robust definition of the objectives of a study.
The group would position the estimand as a more detailed statement of the objective, formulated to address the scientific question of interest posed by the trial. It would therefore specify:
• a population of interest, e.g. the population defined by the target Summary of Product Characteristics (not to be confused with the analysis population);
• an endpoint of interest, i.e. a measurement at a specific timepoint, and
• a measure of the relevant effect of the intervention, taking into account the potential confounding due to post-randomisation events such as non-compliance, use of rescue medication or death. It was noted that while a study design follows from the objectives and estimands, feasibility aspects may result in a feedback loop to refine them. It was felt that clearly defined estimands will be needed to make regulatory claims and that each estimand involved would need to be linked back to objectives and the study design.
It was recognised that different stakeholders may be interested in different estimands, in particular distinguishing between regulatory approval and Health Technology Assessment. However any assumption that the former are more interested in efficacy estimands and the latter in effectiveness estimands is over-simplistic, as there is some evidence of regulators not finding ‘de jure’ estimands useful.
The ICH E9(R1) Concept Paper focuses equally on sensitivity analyses, but the discussion group spent less time in this area. We agreed with the Concept Paper that there is no clear framework for planning and executing sensitivity analyses and that this can lead to inconsistency in regulatory decision-making. The recent paper by Leuchs et al (2015) gives a useful distinction between the goals of internal and external validity for sensitivity analysis and this framework may form a useful foundation for further work.
While the group had differing opinions in certain areas, it was agreed that this is an important area for statisticians to be involved in as we aim to clarify the objectives and applicability of clinical trials. Attendees at the PSI conference will be able to take advantage of two sessions on this topic on the morning of Tuesday May 12th, including statistical leaders from both FDA and MHRA as well as industry and academia. In addition the discussion group has had a letter to the editor accepted commenting on the Leuchs paper and is exploring a separate publication of its own.David Morgan (Ipsen)
Reference:
International Conference on Harmonisation (2014) Final Concept Paper E9(R1):
Addendum to Statistical Principles for Clinical Trials. Geneva, Switzerland.
http://www.ich.org/fileadmin/Public_Web_Site/ICH_Products/Guidelines/
Efficacy/E9/E9__R1__Final_Concept_Paper_October_23_2014.pdf - accessed 21
April 2015.
A-K Leuchs, J Zinserling, A Brandt, D Wirtz and N Benda (2015) Choosing
Appropriate Estimands in Clinical Trials. Therapeutic Innovation & Regulatory
Science
Appendix 1: List of meeting attendees and affiliations
Juan Abellan-Andres Gruenenthal
Soren Andersen Novo Nordisk
Frank Bretz Novartis
Chrissie Fletcher Amgen
Lesley France AstraZeneca
Andrew Garrett Quintiles
Ray Harris Eisai
Oliver Keene GlaxoSmithKline
Magnus Kjaer AstraZeneca
David Morgan Ipsen
Michael O’Kelly Quintiles
Alan Phillips Icon
James Roger Live Data
Our monthly webinar explores examples of innovative data visualisations relevant to our day to day work. Each month a new dataset is provided from a clinical trial or other relevant example, and participants are invited to submit a graphic that communicates interesting and relevant characteristics of the data.
Develop your non-technical skills by reading The Art of Explanation by Ros Atkins and joining the Sept-Dec 2025 book club. You will be invited to join facilitated discussions of the concepts and ideas and apply skills from the book in-between sessions.
Our monthly webinar will allow attendees to gain practical knowledge and skills in Open-Source coding and tools, with a focus on applications in the pharmaceutical industry. The sessions will provide starting points in a number of areas, correct any common misconceptions and provide valuable resources for further learning.
Date: Tuesday 9th September 2025 Join us to hear Sofia Weigle and Weng-Kee Wong present their recent work.
This course is aimed at biostatisticians with no or some pediatric drug development experience who are interested to further their understanding. We will give you an introduction to the pediatric drug development landscape. This will include identifying the key regulations and processes governing pediatric development, a discussion on the needs and challenges when conducting pediatric research and a focus on the ways to overcome these challenges from a statistical perspective.
This networking event is aimed at statisticians that are new to the pharmaceutical industry who wish to meet colleagues from different companies and backgrounds.
AI agents are large language models equipped with tools that can autonomously tackle challenging tasks. This talk will explore how generative AI agents can enable biomedical discovery.
The webinar is targeted at statisticians working in the pharmaceutical industry, and the objective is to 1) provide a basic understanding of IV methodology including how it relates to causal inference, and 2) present two inspirational pharma-relevant applications.
The Pre-Clinical Special Interest Group (SIG) Workshop 2025 will take place over two half-days on 7 - 8 October in Verona, Italy, bringing together experts from industry, academia, and regulatory institutions to discuss key challenges and innovations in pre-clinical research.
Four sessions will include ML foundation (including an introduction, data exploration for ML and dimensionality reduction and feature selection), Supervised learning (including support vector machines and model evaluation and interpretation), model optimization and unsupervised learning (including clustering) and advanced topics (including neural networks, deep learning and large language models).
The program will feature insightful sessions led by distinguished invited speakers, alongside a poster session showcasing the latest advancements in the field. Further details will be provided.
Date: Wednesday 5th November 2025 A careers talk about medical statistics and how it plays a crucial role in developing new medicines
Date: 19 November 2025 This event is aimed at students with an interest in the field of Medical Statistics, for example within pharmaceuticals, healthcare and/or medical research.
This networking event is aimed at statisticians that are new to the pharmaceutical industry who wish to meet colleagues from different companies and backgrounds.
As an Associate Director Biostatistics Early Development, you will be a key member of our biostatistics group, you will play a crucial role in the design, analysis, and interpretation of clinical trials for early development programs.
If you are passionate about biostatistics and real-world data, and are looking for an exciting opportunity to contribute to groundbreaking research, we encourage you to apply.
Are you passionate about making a difference in the world of healthcare? Novartis is seeking a dynamic and experienced professional to join our team in London at The Westworks.
As the Director of HTA Biostatistics & Medical Affairs, you will play a pivotal role in shaping the future of healthcare by providing strategic biostatistical leadership and expertise.
An exciting opportunity has arisen for a permanent Senior Medical Statistician & Statistical Programmer to join the UKCRC fully registered Derby Clinical Trials Support Unit (Derby CTSU).
As a Senior Principal Biostatistician, you will be responsible and accountable for all statistical work, both scientific and operational, for one or more assigned clinical trials
About cookies on this site
We use cookies to collect and analyse information on site performance and usage, to provide social media features and to enhance and customise content and advertisements.
About cookies on this site
Cookies used on the site are categorized and below you can read about each category and allow or deny some or all of them. When categories than have been previously allowed are disabled, all cookies assigned to that category will be removed from your browser. Additionally you can see a list of cookies assigned to each category and detailed information in the cookie declaration.
Necessary cookies
Some cookies are required to provide core functionality. The website won't function properly without these cookies and they are enabled by default and cannot be disabled.
CookieHub is a Consent Management Platform (CMP) which allows users to control storage and processing of personal information.
Cloudflare is a global network designed to make everything you connect to the Internet secure, private, fast, and reliable.
Microsoft Azure is a cloud computing platform offering a wide range of services, including virtual machines, databases, and AI tools.
AWS Elastic Load Balancing (ELB) automatically distributes incoming web traffic across multiple servers or services hosted on AWS.
Preferences
Preference cookies enables the web site to remember information to customize how the web site looks or behaves for each user. This may include storing selected currency, region, language or color theme.
Analytical cookies
Analytical cookies help us improve our website by collecting and reporting information on its usage.
Google Analytics is a web analytics service offered by Google that tracks and reports website traffic.
Vimeo, Inc. is an American video hosting, sharing, services provider, and broadcaster. Vimeo focuses on the delivery of high-definition video across a range of devices.
Cookies used on the site are categorized and below you can read about each category and allow or deny some or all of them. When categories than have been previously allowed are disabled, all cookies assigned to that category will be removed from your browser. Additionally you can see a list of cookies assigned to each category and detailed information in the cookie declaration.
Necessary cookies
Some cookies are required to provide core functionality. The website won't function properly without these cookies and they are enabled by default and cannot be disabled.
| Name | Hostname | Vendor | Expiry |
|---|---|---|---|
| ARRAffinity | .psiweb.org | Session | |
This cookie is set by websites run on the Windows Azure cloud platform. It is used for load balancing to make sure the visitor page requests are routed to the same server in any browsing session. | |||
| ARRAffinitySameSite | .psiweb.org | Session | |
Used to distribute traffic to the website on several servers in order to optimize response times. | |||
| __cf_bm | .vimeo.com | Cloudflare, Inc. | 1 hour |
The __cf_bm cookie supports Cloudflare Bot Management by managing incoming traffic that matches criteria associated with bots. The cookie does not collect any personal data, and any information collected is subject to one-way encryption. | |||
| _cfuvid | .vimeo.com | Cloudflare, Inc. | Session |
Used by Cloudflare WAF to distinguish individual users who share the same IP address and apply rate limits | |||
| __cf_bm | .glueup.com | Cloudflare, Inc. | 1 hour |
The __cf_bm cookie supports Cloudflare Bot Management by managing incoming traffic that matches criteria associated with bots. The cookie does not collect any personal data, and any information collected is subject to one-way encryption. | |||
| AWSALBTGCORS | psi.glueup.com | Amazon Web Services, Inc. | 7 days |
Used by Target Group-based load balancers for session stickiness. | |||
| AWSALBCORS | psi.glueup.com | Amazon Web Services, Inc. | 7 days |
Maintains session stickiness and secure routing between the user and backend servers through AWS load balancing. | |||
| PHPSESSID | psi.glueup.com | Session | |
Cookie generated by applications based on the PHP language. This is a general purpose identifier used to maintain user session variables. It is normally a random generated number, how it is used can be specific to the site, but a good example is maintaining a logged-in status for a user between pages. | |||
| History.store | Session | ||
| cookiehub | .psiweb.org | CookieHub | 365 days |
Used by CookieHub to store information about whether visitors have given or declined the use of cookie categories used on the site. | |||
Preferences
Preference cookies enables the web site to remember information to customize how the web site looks or behaves for each user. This may include storing selected currency, region, language or color theme.
| Name | Hostname | Vendor | Expiry |
|---|---|---|---|
| vuid | .vimeo.com | 400 days | |
These cookies are used by the Vimeo video player on websites. | |||
Analytical cookies
Analytical cookies help us improve our website by collecting and reporting information on its usage.
| Name | Hostname | Vendor | Expiry |
|---|---|---|---|
| _ga_ | .psiweb.org | 400 days | |
Contains a unique identifier used by Google Analytics 4 to determine that two distinct hits belong to the same user across browsing sessions. | |||
| _ga | .psiweb.org | 400 days | |
Contains a unique identifier used by Google Analytics to determine that two distinct hits belong to the same user across browsing sessions. | |||
