In October 2014 the ICH Steering Committee endorsed a final Concept Paper with the goal of developing new regulatory guidance, suggested to be an Addendum to ICH E9, which promotes harmonised standards on the choice of estimands in clinical trials and an agreed framework for planning, conducting and interpreting sensitivity analyses of clinical trial data. An Expert Working Group sponsored by ICH has been established
to further develop the concept and is due to report findings in December
2015. Frank Bretz and Chrissie Fletcher are the EFPIA representatives on
the ICH E9(R1) Expert Working Group.
As a first step in developing the response from the European industry to any guidance issued by the ICH E9(R1) Expert Working Group, PSI/EFSPI held a 1-day workshop on estimands and sensitivity analyses in February 2015. A list of meeting attendees with affiliations is given in Appendix 1. Most participants recognised that they had not been aware of the word “estimand” - still less actually used it – before the report “The prevention and treatment of missing data in clinical trials” was published by the National Research Council in 2010. So one important aspect of this area in the next few years will be to communicate and educate both statisticians and non-statisticians in the concepts, terminology and consequences of the choice of estimands for a clinical trial. The estimand framework helps to distinguish between the target of estimation (estimand) and the method of estimation (estimator).An estimand reflects what is to be estimated to address the scientific question of interest posed by a trial.
It will be important to develop and publicise some simple but informative examples which can motivate all involved in clinical trials to become more focussed on the choice of estimands. For example two relevant classes of estimand are the ‘de jure’ estimand (“efficacy”) and the ‘de facto’ estimand (“effectiveness”).
” The efficacy is an ideal treatment effect that could have been reached if all patients had fully adhered or, in other words, the expected effect a patient may reach if he or she takes the medication as directed. Because of the lack of perfect adherence in practice, the efficacy cannot be observed. In contrast, effectiveness comprises treatment effects that occur when full adherence to treatment is lacking. Besides the sole benefit of the treatment, effectiveness also takes into account safety and tolerability issues.” (Leuchs et al, 2015). The discussion group noted that the positioning of this new document as an addendum to ICH E9 may give the impression that it is of interest only or primarily to statisticians. Yet the “Statement of the Perceived Problem” makes it clear that all concerned with clinical trials need to give this area further thought – “Incorrect choice of estimand and unclear definitions for estimands lead to problems in relation to trial design, conduct and analysis and introduce potential for inconsistencies in inference and decision making.” Although the die is cast, some members of the PSI/EFSPI discussion group felt that attaching the addendum to ICH E8 on “General Considerations For Clinical Trials” might have involved a broader leadership in making the changes in practice required. In particular E8 would benefit from the addition of a robust definition of the objectives of a study.
The group would position the estimand as a more detailed statement of the objective, formulated to address the scientific question of interest posed by the trial. It would therefore specify:
• a population of interest, e.g. the population defined by the target Summary of Product Characteristics (not to be confused with the analysis population);
• an endpoint of interest, i.e. a measurement at a specific timepoint, and
• a measure of the relevant effect of the intervention, taking into account the potential confounding due to post-randomisation events such as non-compliance, use of rescue medication or death. It was noted that while a study design follows from the objectives and estimands, feasibility aspects may result in a feedback loop to refine them. It was felt that clearly defined estimands will be needed to make regulatory claims and that each estimand involved would need to be linked back to objectives and the study design.
It was recognised that different stakeholders may be interested in different estimands, in particular distinguishing between regulatory approval and Health Technology Assessment. However any assumption that the former are more interested in efficacy estimands and the latter in effectiveness estimands is over-simplistic, as there is some evidence of regulators not finding ‘de jure’ estimands useful.
The ICH E9(R1) Concept Paper focuses equally on sensitivity analyses, but the discussion group spent less time in this area. We agreed with the Concept Paper that there is no clear framework for planning and executing sensitivity analyses and that this can lead to inconsistency in regulatory decision-making. The recent paper by Leuchs et al (2015) gives a useful distinction between the goals of internal and external validity for sensitivity analysis and this framework may form a useful foundation for further work.
While the group had differing opinions in certain areas, it was agreed that this is an important area for statisticians to be involved in as we aim to clarify the objectives and applicability of clinical trials. Attendees at the PSI conference will be able to take advantage of two sessions on this topic on the morning of Tuesday May 12th, including statistical leaders from both FDA and MHRA as well as industry and academia. In addition the discussion group has had a letter to the editor accepted commenting on the Leuchs paper and is exploring a separate publication of its own.David Morgan (Ipsen)
Reference:
International Conference on Harmonisation (2014) Final Concept Paper E9(R1):
Addendum to Statistical Principles for Clinical Trials. Geneva, Switzerland.
http://www.ich.org/fileadmin/Public_Web_Site/ICH_Products/Guidelines/
Efficacy/E9/E9__R1__Final_Concept_Paper_October_23_2014.pdf - accessed 21
April 2015.
A-K Leuchs, J Zinserling, A Brandt, D Wirtz and N Benda (2015) Choosing
Appropriate Estimands in Clinical Trials. Therapeutic Innovation & Regulatory
Science
Appendix 1: List of meeting attendees and affiliations
Juan Abellan-Andres Gruenenthal
Soren Andersen Novo Nordisk
Frank Bretz Novartis
Chrissie Fletcher Amgen
Lesley France AstraZeneca
Andrew Garrett Quintiles
Ray Harris Eisai
Oliver Keene GlaxoSmithKline
Magnus Kjaer AstraZeneca
David Morgan Ipsen
Michael O’Kelly Quintiles
Alan Phillips Icon
James Roger Live Data
In October 2014 the ICH Steering Committee endorsed a final Concept Paper with the goal of developing new regulatory guidance, suggested to be an Addendum to ICH E9, which promotes harmonised standards on the choice of estimands in clinical trials and an agreed framework for planning, conducting and interpreting sensitivity analyses of clinical trial data. An Expert Working Group sponsored by ICH has been established
to further develop the concept and is due to report findings in December
2015. Frank Bretz and Chrissie Fletcher are the EFPIA representatives on
the ICH E9(R1) Expert Working Group.
As a first step in developing the response from the European industry to any guidance issued by the ICH E9(R1) Expert Working Group, PSI/EFSPI held a 1-day workshop on estimands and sensitivity analyses in February 2015. A list of meeting attendees with affiliations is given in Appendix 1. Most participants recognised that they had not been aware of the word “estimand” - still less actually used it – before the report “The prevention and treatment of missing data in clinical trials” was published by the National Research Council in 2010. So one important aspect of this area in the next few years will be to communicate and educate both statisticians and non-statisticians in the concepts, terminology and consequences of the choice of estimands for a clinical trial. The estimand framework helps to distinguish between the target of estimation (estimand) and the method of estimation (estimator).An estimand reflects what is to be estimated to address the scientific question of interest posed by a trial.
It will be important to develop and publicise some simple but informative examples which can motivate all involved in clinical trials to become more focussed on the choice of estimands. For example two relevant classes of estimand are the ‘de jure’ estimand (“efficacy”) and the ‘de facto’ estimand (“effectiveness”).
” The efficacy is an ideal treatment effect that could have been reached if all patients had fully adhered or, in other words, the expected effect a patient may reach if he or she takes the medication as directed. Because of the lack of perfect adherence in practice, the efficacy cannot be observed. In contrast, effectiveness comprises treatment effects that occur when full adherence to treatment is lacking. Besides the sole benefit of the treatment, effectiveness also takes into account safety and tolerability issues.” (Leuchs et al, 2015). The discussion group noted that the positioning of this new document as an addendum to ICH E9 may give the impression that it is of interest only or primarily to statisticians. Yet the “Statement of the Perceived Problem” makes it clear that all concerned with clinical trials need to give this area further thought – “Incorrect choice of estimand and unclear definitions for estimands lead to problems in relation to trial design, conduct and analysis and introduce potential for inconsistencies in inference and decision making.” Although the die is cast, some members of the PSI/EFSPI discussion group felt that attaching the addendum to ICH E8 on “General Considerations For Clinical Trials” might have involved a broader leadership in making the changes in practice required. In particular E8 would benefit from the addition of a robust definition of the objectives of a study.
The group would position the estimand as a more detailed statement of the objective, formulated to address the scientific question of interest posed by the trial. It would therefore specify:
• a population of interest, e.g. the population defined by the target Summary of Product Characteristics (not to be confused with the analysis population);
• an endpoint of interest, i.e. a measurement at a specific timepoint, and
• a measure of the relevant effect of the intervention, taking into account the potential confounding due to post-randomisation events such as non-compliance, use of rescue medication or death. It was noted that while a study design follows from the objectives and estimands, feasibility aspects may result in a feedback loop to refine them. It was felt that clearly defined estimands will be needed to make regulatory claims and that each estimand involved would need to be linked back to objectives and the study design.
It was recognised that different stakeholders may be interested in different estimands, in particular distinguishing between regulatory approval and Health Technology Assessment. However any assumption that the former are more interested in efficacy estimands and the latter in effectiveness estimands is over-simplistic, as there is some evidence of regulators not finding ‘de jure’ estimands useful.
The ICH E9(R1) Concept Paper focuses equally on sensitivity analyses, but the discussion group spent less time in this area. We agreed with the Concept Paper that there is no clear framework for planning and executing sensitivity analyses and that this can lead to inconsistency in regulatory decision-making. The recent paper by Leuchs et al (2015) gives a useful distinction between the goals of internal and external validity for sensitivity analysis and this framework may form a useful foundation for further work.
While the group had differing opinions in certain areas, it was agreed that this is an important area for statisticians to be involved in as we aim to clarify the objectives and applicability of clinical trials. Attendees at the PSI conference will be able to take advantage of two sessions on this topic on the morning of Tuesday May 12th, including statistical leaders from both FDA and MHRA as well as industry and academia. In addition the discussion group has had a letter to the editor accepted commenting on the Leuchs paper and is exploring a separate publication of its own.David Morgan (Ipsen)
Reference:
International Conference on Harmonisation (2014) Final Concept Paper E9(R1):
Addendum to Statistical Principles for Clinical Trials. Geneva, Switzerland.
http://www.ich.org/fileadmin/Public_Web_Site/ICH_Products/Guidelines/
Efficacy/E9/E9__R1__Final_Concept_Paper_October_23_2014.pdf - accessed 21
April 2015.
A-K Leuchs, J Zinserling, A Brandt, D Wirtz and N Benda (2015) Choosing
Appropriate Estimands in Clinical Trials. Therapeutic Innovation & Regulatory
Science
Appendix 1: List of meeting attendees and affiliations
Juan Abellan-Andres Gruenenthal
Soren Andersen Novo Nordisk
Frank Bretz Novartis
Chrissie Fletcher Amgen
Lesley France AstraZeneca
Andrew Garrett Quintiles
Ray Harris Eisai
Oliver Keene GlaxoSmithKline
Magnus Kjaer AstraZeneca
David Morgan Ipsen
Michael O’Kelly Quintiles
Alan Phillips Icon
James Roger Live Data
In October 2014 the ICH Steering Committee endorsed a final Concept Paper with the goal of developing new regulatory guidance, suggested to be an Addendum to ICH E9, which promotes harmonised standards on the choice of estimands in clinical trials and an agreed framework for planning, conducting and interpreting sensitivity analyses of clinical trial data. An Expert Working Group sponsored by ICH has been established
to further develop the concept and is due to report findings in December
2015. Frank Bretz and Chrissie Fletcher are the EFPIA representatives on
the ICH E9(R1) Expert Working Group.
As a first step in developing the response from the European industry to any guidance issued by the ICH E9(R1) Expert Working Group, PSI/EFSPI held a 1-day workshop on estimands and sensitivity analyses in February 2015. A list of meeting attendees with affiliations is given in Appendix 1. Most participants recognised that they had not been aware of the word “estimand” - still less actually used it – before the report “The prevention and treatment of missing data in clinical trials” was published by the National Research Council in 2010. So one important aspect of this area in the next few years will be to communicate and educate both statisticians and non-statisticians in the concepts, terminology and consequences of the choice of estimands for a clinical trial. The estimand framework helps to distinguish between the target of estimation (estimand) and the method of estimation (estimator).An estimand reflects what is to be estimated to address the scientific question of interest posed by a trial.
It will be important to develop and publicise some simple but informative examples which can motivate all involved in clinical trials to become more focussed on the choice of estimands. For example two relevant classes of estimand are the ‘de jure’ estimand (“efficacy”) and the ‘de facto’ estimand (“effectiveness”).
” The efficacy is an ideal treatment effect that could have been reached if all patients had fully adhered or, in other words, the expected effect a patient may reach if he or she takes the medication as directed. Because of the lack of perfect adherence in practice, the efficacy cannot be observed. In contrast, effectiveness comprises treatment effects that occur when full adherence to treatment is lacking. Besides the sole benefit of the treatment, effectiveness also takes into account safety and tolerability issues.” (Leuchs et al, 2015). The discussion group noted that the positioning of this new document as an addendum to ICH E9 may give the impression that it is of interest only or primarily to statisticians. Yet the “Statement of the Perceived Problem” makes it clear that all concerned with clinical trials need to give this area further thought – “Incorrect choice of estimand and unclear definitions for estimands lead to problems in relation to trial design, conduct and analysis and introduce potential for inconsistencies in inference and decision making.” Although the die is cast, some members of the PSI/EFSPI discussion group felt that attaching the addendum to ICH E8 on “General Considerations For Clinical Trials” might have involved a broader leadership in making the changes in practice required. In particular E8 would benefit from the addition of a robust definition of the objectives of a study.
The group would position the estimand as a more detailed statement of the objective, formulated to address the scientific question of interest posed by the trial. It would therefore specify:
• a population of interest, e.g. the population defined by the target Summary of Product Characteristics (not to be confused with the analysis population);
• an endpoint of interest, i.e. a measurement at a specific timepoint, and
• a measure of the relevant effect of the intervention, taking into account the potential confounding due to post-randomisation events such as non-compliance, use of rescue medication or death. It was noted that while a study design follows from the objectives and estimands, feasibility aspects may result in a feedback loop to refine them. It was felt that clearly defined estimands will be needed to make regulatory claims and that each estimand involved would need to be linked back to objectives and the study design.
It was recognised that different stakeholders may be interested in different estimands, in particular distinguishing between regulatory approval and Health Technology Assessment. However any assumption that the former are more interested in efficacy estimands and the latter in effectiveness estimands is over-simplistic, as there is some evidence of regulators not finding ‘de jure’ estimands useful.
The ICH E9(R1) Concept Paper focuses equally on sensitivity analyses, but the discussion group spent less time in this area. We agreed with the Concept Paper that there is no clear framework for planning and executing sensitivity analyses and that this can lead to inconsistency in regulatory decision-making. The recent paper by Leuchs et al (2015) gives a useful distinction between the goals of internal and external validity for sensitivity analysis and this framework may form a useful foundation for further work.
While the group had differing opinions in certain areas, it was agreed that this is an important area for statisticians to be involved in as we aim to clarify the objectives and applicability of clinical trials. Attendees at the PSI conference will be able to take advantage of two sessions on this topic on the morning of Tuesday May 12th, including statistical leaders from both FDA and MHRA as well as industry and academia. In addition the discussion group has had a letter to the editor accepted commenting on the Leuchs paper and is exploring a separate publication of its own.David Morgan (Ipsen)
Reference:
International Conference on Harmonisation (2014) Final Concept Paper E9(R1):
Addendum to Statistical Principles for Clinical Trials. Geneva, Switzerland.
http://www.ich.org/fileadmin/Public_Web_Site/ICH_Products/Guidelines/
Efficacy/E9/E9__R1__Final_Concept_Paper_October_23_2014.pdf - accessed 21
April 2015.
A-K Leuchs, J Zinserling, A Brandt, D Wirtz and N Benda (2015) Choosing
Appropriate Estimands in Clinical Trials. Therapeutic Innovation & Regulatory
Science
Appendix 1: List of meeting attendees and affiliations
Juan Abellan-Andres Gruenenthal
Soren Andersen Novo Nordisk
Frank Bretz Novartis
Chrissie Fletcher Amgen
Lesley France AstraZeneca
Andrew Garrett Quintiles
Ray Harris Eisai
Oliver Keene GlaxoSmithKline
Magnus Kjaer AstraZeneca
David Morgan Ipsen
Michael O’Kelly Quintiles
Alan Phillips Icon
James Roger Live Data
In October 2014 the ICH Steering Committee endorsed a final Concept Paper with the goal of developing new regulatory guidance, suggested to be an Addendum to ICH E9, which promotes harmonised standards on the choice of estimands in clinical trials and an agreed framework for planning, conducting and interpreting sensitivity analyses of clinical trial data. An Expert Working Group sponsored by ICH has been established
to further develop the concept and is due to report findings in December
2015. Frank Bretz and Chrissie Fletcher are the EFPIA representatives on
the ICH E9(R1) Expert Working Group.
As a first step in developing the response from the European industry to any guidance issued by the ICH E9(R1) Expert Working Group, PSI/EFSPI held a 1-day workshop on estimands and sensitivity analyses in February 2015. A list of meeting attendees with affiliations is given in Appendix 1. Most participants recognised that they had not been aware of the word “estimand” - still less actually used it – before the report “The prevention and treatment of missing data in clinical trials” was published by the National Research Council in 2010. So one important aspect of this area in the next few years will be to communicate and educate both statisticians and non-statisticians in the concepts, terminology and consequences of the choice of estimands for a clinical trial. The estimand framework helps to distinguish between the target of estimation (estimand) and the method of estimation (estimator).An estimand reflects what is to be estimated to address the scientific question of interest posed by a trial.
It will be important to develop and publicise some simple but informative examples which can motivate all involved in clinical trials to become more focussed on the choice of estimands. For example two relevant classes of estimand are the ‘de jure’ estimand (“efficacy”) and the ‘de facto’ estimand (“effectiveness”).
” The efficacy is an ideal treatment effect that could have been reached if all patients had fully adhered or, in other words, the expected effect a patient may reach if he or she takes the medication as directed. Because of the lack of perfect adherence in practice, the efficacy cannot be observed. In contrast, effectiveness comprises treatment effects that occur when full adherence to treatment is lacking. Besides the sole benefit of the treatment, effectiveness also takes into account safety and tolerability issues.” (Leuchs et al, 2015). The discussion group noted that the positioning of this new document as an addendum to ICH E9 may give the impression that it is of interest only or primarily to statisticians. Yet the “Statement of the Perceived Problem” makes it clear that all concerned with clinical trials need to give this area further thought – “Incorrect choice of estimand and unclear definitions for estimands lead to problems in relation to trial design, conduct and analysis and introduce potential for inconsistencies in inference and decision making.” Although the die is cast, some members of the PSI/EFSPI discussion group felt that attaching the addendum to ICH E8 on “General Considerations For Clinical Trials” might have involved a broader leadership in making the changes in practice required. In particular E8 would benefit from the addition of a robust definition of the objectives of a study.
The group would position the estimand as a more detailed statement of the objective, formulated to address the scientific question of interest posed by the trial. It would therefore specify:
• a population of interest, e.g. the population defined by the target Summary of Product Characteristics (not to be confused with the analysis population);
• an endpoint of interest, i.e. a measurement at a specific timepoint, and
• a measure of the relevant effect of the intervention, taking into account the potential confounding due to post-randomisation events such as non-compliance, use of rescue medication or death. It was noted that while a study design follows from the objectives and estimands, feasibility aspects may result in a feedback loop to refine them. It was felt that clearly defined estimands will be needed to make regulatory claims and that each estimand involved would need to be linked back to objectives and the study design.
It was recognised that different stakeholders may be interested in different estimands, in particular distinguishing between regulatory approval and Health Technology Assessment. However any assumption that the former are more interested in efficacy estimands and the latter in effectiveness estimands is over-simplistic, as there is some evidence of regulators not finding ‘de jure’ estimands useful.
The ICH E9(R1) Concept Paper focuses equally on sensitivity analyses, but the discussion group spent less time in this area. We agreed with the Concept Paper that there is no clear framework for planning and executing sensitivity analyses and that this can lead to inconsistency in regulatory decision-making. The recent paper by Leuchs et al (2015) gives a useful distinction between the goals of internal and external validity for sensitivity analysis and this framework may form a useful foundation for further work.
While the group had differing opinions in certain areas, it was agreed that this is an important area for statisticians to be involved in as we aim to clarify the objectives and applicability of clinical trials. Attendees at the PSI conference will be able to take advantage of two sessions on this topic on the morning of Tuesday May 12th, including statistical leaders from both FDA and MHRA as well as industry and academia. In addition the discussion group has had a letter to the editor accepted commenting on the Leuchs paper and is exploring a separate publication of its own.David Morgan (Ipsen)
Reference:
International Conference on Harmonisation (2014) Final Concept Paper E9(R1):
Addendum to Statistical Principles for Clinical Trials. Geneva, Switzerland.
http://www.ich.org/fileadmin/Public_Web_Site/ICH_Products/Guidelines/
Efficacy/E9/E9__R1__Final_Concept_Paper_October_23_2014.pdf - accessed 21
April 2015.
A-K Leuchs, J Zinserling, A Brandt, D Wirtz and N Benda (2015) Choosing
Appropriate Estimands in Clinical Trials. Therapeutic Innovation & Regulatory
Science
Appendix 1: List of meeting attendees and affiliations
Juan Abellan-Andres Gruenenthal
Soren Andersen Novo Nordisk
Frank Bretz Novartis
Chrissie Fletcher Amgen
Lesley France AstraZeneca
Andrew Garrett Quintiles
Ray Harris Eisai
Oliver Keene GlaxoSmithKline
Magnus Kjaer AstraZeneca
David Morgan Ipsen
Michael O’Kelly Quintiles
Alan Phillips Icon
James Roger Live Data
In October 2014 the ICH Steering Committee endorsed a final Concept Paper with the goal of developing new regulatory guidance, suggested to be an Addendum to ICH E9, which promotes harmonised standards on the choice of estimands in clinical trials and an agreed framework for planning, conducting and interpreting sensitivity analyses of clinical trial data. An Expert Working Group sponsored by ICH has been established
to further develop the concept and is due to report findings in December
2015. Frank Bretz and Chrissie Fletcher are the EFPIA representatives on
the ICH E9(R1) Expert Working Group.
As a first step in developing the response from the European industry to any guidance issued by the ICH E9(R1) Expert Working Group, PSI/EFSPI held a 1-day workshop on estimands and sensitivity analyses in February 2015. A list of meeting attendees with affiliations is given in Appendix 1. Most participants recognised that they had not been aware of the word “estimand” - still less actually used it – before the report “The prevention and treatment of missing data in clinical trials” was published by the National Research Council in 2010. So one important aspect of this area in the next few years will be to communicate and educate both statisticians and non-statisticians in the concepts, terminology and consequences of the choice of estimands for a clinical trial. The estimand framework helps to distinguish between the target of estimation (estimand) and the method of estimation (estimator).An estimand reflects what is to be estimated to address the scientific question of interest posed by a trial.
It will be important to develop and publicise some simple but informative examples which can motivate all involved in clinical trials to become more focussed on the choice of estimands. For example two relevant classes of estimand are the ‘de jure’ estimand (“efficacy”) and the ‘de facto’ estimand (“effectiveness”).
” The efficacy is an ideal treatment effect that could have been reached if all patients had fully adhered or, in other words, the expected effect a patient may reach if he or she takes the medication as directed. Because of the lack of perfect adherence in practice, the efficacy cannot be observed. In contrast, effectiveness comprises treatment effects that occur when full adherence to treatment is lacking. Besides the sole benefit of the treatment, effectiveness also takes into account safety and tolerability issues.” (Leuchs et al, 2015). The discussion group noted that the positioning of this new document as an addendum to ICH E9 may give the impression that it is of interest only or primarily to statisticians. Yet the “Statement of the Perceived Problem” makes it clear that all concerned with clinical trials need to give this area further thought – “Incorrect choice of estimand and unclear definitions for estimands lead to problems in relation to trial design, conduct and analysis and introduce potential for inconsistencies in inference and decision making.” Although the die is cast, some members of the PSI/EFSPI discussion group felt that attaching the addendum to ICH E8 on “General Considerations For Clinical Trials” might have involved a broader leadership in making the changes in practice required. In particular E8 would benefit from the addition of a robust definition of the objectives of a study.
The group would position the estimand as a more detailed statement of the objective, formulated to address the scientific question of interest posed by the trial. It would therefore specify:
• a population of interest, e.g. the population defined by the target Summary of Product Characteristics (not to be confused with the analysis population);
• an endpoint of interest, i.e. a measurement at a specific timepoint, and
• a measure of the relevant effect of the intervention, taking into account the potential confounding due to post-randomisation events such as non-compliance, use of rescue medication or death. It was noted that while a study design follows from the objectives and estimands, feasibility aspects may result in a feedback loop to refine them. It was felt that clearly defined estimands will be needed to make regulatory claims and that each estimand involved would need to be linked back to objectives and the study design.
It was recognised that different stakeholders may be interested in different estimands, in particular distinguishing between regulatory approval and Health Technology Assessment. However any assumption that the former are more interested in efficacy estimands and the latter in effectiveness estimands is over-simplistic, as there is some evidence of regulators not finding ‘de jure’ estimands useful.
The ICH E9(R1) Concept Paper focuses equally on sensitivity analyses, but the discussion group spent less time in this area. We agreed with the Concept Paper that there is no clear framework for planning and executing sensitivity analyses and that this can lead to inconsistency in regulatory decision-making. The recent paper by Leuchs et al (2015) gives a useful distinction between the goals of internal and external validity for sensitivity analysis and this framework may form a useful foundation for further work.
While the group had differing opinions in certain areas, it was agreed that this is an important area for statisticians to be involved in as we aim to clarify the objectives and applicability of clinical trials. Attendees at the PSI conference will be able to take advantage of two sessions on this topic on the morning of Tuesday May 12th, including statistical leaders from both FDA and MHRA as well as industry and academia. In addition the discussion group has had a letter to the editor accepted commenting on the Leuchs paper and is exploring a separate publication of its own.David Morgan (Ipsen)
Reference:
International Conference on Harmonisation (2014) Final Concept Paper E9(R1):
Addendum to Statistical Principles for Clinical Trials. Geneva, Switzerland.
http://www.ich.org/fileadmin/Public_Web_Site/ICH_Products/Guidelines/
Efficacy/E9/E9__R1__Final_Concept_Paper_October_23_2014.pdf - accessed 21
April 2015.
A-K Leuchs, J Zinserling, A Brandt, D Wirtz and N Benda (2015) Choosing
Appropriate Estimands in Clinical Trials. Therapeutic Innovation & Regulatory
Science
Appendix 1: List of meeting attendees and affiliations
Juan Abellan-Andres Gruenenthal
Soren Andersen Novo Nordisk
Frank Bretz Novartis
Chrissie Fletcher Amgen
Lesley France AstraZeneca
Andrew Garrett Quintiles
Ray Harris Eisai
Oliver Keene GlaxoSmithKline
Magnus Kjaer AstraZeneca
David Morgan Ipsen
Michael O’Kelly Quintiles
Alan Phillips Icon
James Roger Live Data
In October 2014 the ICH Steering Committee endorsed a final Concept Paper with the goal of developing new regulatory guidance, suggested to be an Addendum to ICH E9, which promotes harmonised standards on the choice of estimands in clinical trials and an agreed framework for planning, conducting and interpreting sensitivity analyses of clinical trial data. An Expert Working Group sponsored by ICH has been established
to further develop the concept and is due to report findings in December
2015. Frank Bretz and Chrissie Fletcher are the EFPIA representatives on
the ICH E9(R1) Expert Working Group.
As a first step in developing the response from the European industry to any guidance issued by the ICH E9(R1) Expert Working Group, PSI/EFSPI held a 1-day workshop on estimands and sensitivity analyses in February 2015. A list of meeting attendees with affiliations is given in Appendix 1. Most participants recognised that they had not been aware of the word “estimand” - still less actually used it – before the report “The prevention and treatment of missing data in clinical trials” was published by the National Research Council in 2010. So one important aspect of this area in the next few years will be to communicate and educate both statisticians and non-statisticians in the concepts, terminology and consequences of the choice of estimands for a clinical trial. The estimand framework helps to distinguish between the target of estimation (estimand) and the method of estimation (estimator).An estimand reflects what is to be estimated to address the scientific question of interest posed by a trial.
It will be important to develop and publicise some simple but informative examples which can motivate all involved in clinical trials to become more focussed on the choice of estimands. For example two relevant classes of estimand are the ‘de jure’ estimand (“efficacy”) and the ‘de facto’ estimand (“effectiveness”).
” The efficacy is an ideal treatment effect that could have been reached if all patients had fully adhered or, in other words, the expected effect a patient may reach if he or she takes the medication as directed. Because of the lack of perfect adherence in practice, the efficacy cannot be observed. In contrast, effectiveness comprises treatment effects that occur when full adherence to treatment is lacking. Besides the sole benefit of the treatment, effectiveness also takes into account safety and tolerability issues.” (Leuchs et al, 2015). The discussion group noted that the positioning of this new document as an addendum to ICH E9 may give the impression that it is of interest only or primarily to statisticians. Yet the “Statement of the Perceived Problem” makes it clear that all concerned with clinical trials need to give this area further thought – “Incorrect choice of estimand and unclear definitions for estimands lead to problems in relation to trial design, conduct and analysis and introduce potential for inconsistencies in inference and decision making.” Although the die is cast, some members of the PSI/EFSPI discussion group felt that attaching the addendum to ICH E8 on “General Considerations For Clinical Trials” might have involved a broader leadership in making the changes in practice required. In particular E8 would benefit from the addition of a robust definition of the objectives of a study.
The group would position the estimand as a more detailed statement of the objective, formulated to address the scientific question of interest posed by the trial. It would therefore specify:
• a population of interest, e.g. the population defined by the target Summary of Product Characteristics (not to be confused with the analysis population);
• an endpoint of interest, i.e. a measurement at a specific timepoint, and
• a measure of the relevant effect of the intervention, taking into account the potential confounding due to post-randomisation events such as non-compliance, use of rescue medication or death. It was noted that while a study design follows from the objectives and estimands, feasibility aspects may result in a feedback loop to refine them. It was felt that clearly defined estimands will be needed to make regulatory claims and that each estimand involved would need to be linked back to objectives and the study design.
It was recognised that different stakeholders may be interested in different estimands, in particular distinguishing between regulatory approval and Health Technology Assessment. However any assumption that the former are more interested in efficacy estimands and the latter in effectiveness estimands is over-simplistic, as there is some evidence of regulators not finding ‘de jure’ estimands useful.
The ICH E9(R1) Concept Paper focuses equally on sensitivity analyses, but the discussion group spent less time in this area. We agreed with the Concept Paper that there is no clear framework for planning and executing sensitivity analyses and that this can lead to inconsistency in regulatory decision-making. The recent paper by Leuchs et al (2015) gives a useful distinction between the goals of internal and external validity for sensitivity analysis and this framework may form a useful foundation for further work.
While the group had differing opinions in certain areas, it was agreed that this is an important area for statisticians to be involved in as we aim to clarify the objectives and applicability of clinical trials. Attendees at the PSI conference will be able to take advantage of two sessions on this topic on the morning of Tuesday May 12th, including statistical leaders from both FDA and MHRA as well as industry and academia. In addition the discussion group has had a letter to the editor accepted commenting on the Leuchs paper and is exploring a separate publication of its own.David Morgan (Ipsen)
Reference:
International Conference on Harmonisation (2014) Final Concept Paper E9(R1):
Addendum to Statistical Principles for Clinical Trials. Geneva, Switzerland.
http://www.ich.org/fileadmin/Public_Web_Site/ICH_Products/Guidelines/
Efficacy/E9/E9__R1__Final_Concept_Paper_October_23_2014.pdf - accessed 21
April 2015.
A-K Leuchs, J Zinserling, A Brandt, D Wirtz and N Benda (2015) Choosing
Appropriate Estimands in Clinical Trials. Therapeutic Innovation & Regulatory
Science
Appendix 1: List of meeting attendees and affiliations
Juan Abellan-Andres Gruenenthal
Soren Andersen Novo Nordisk
Frank Bretz Novartis
Chrissie Fletcher Amgen
Lesley France AstraZeneca
Andrew Garrett Quintiles
Ray Harris Eisai
Oliver Keene GlaxoSmithKline
Magnus Kjaer AstraZeneca
David Morgan Ipsen
Michael O’Kelly Quintiles
Alan Phillips Icon
James Roger Live Data
