Evaluation of the fragility of pivotal trials used to support FDA approval for plaque psoriasis

2020
Regulatory Medicine
Authors

Sophia Z. Shalhout

Romi Bloom

Lynn Drake

David M. Miller

Published

April 6, 2020

Doi

https://doi.org/10.1016/j.jaad.2020.04.057

Abstract
We evaluate the fragility of pivotal trials used to support FDA approval in Plaque Psoriasis.

Journal: Journal of the American Academcy of Dermatology. This is the author’s preprint. Please see the published version on the JAAD Website

Authors: Sophia Z. Shalhout PhD1, Romi Bloom, MD2, Lynn Drake MD2,3 and David M. Miller MD, PhD1,2

1Division of Hematology/Oncology, 2Department of Dermatology, 3Wellman Center for Photomedicine, Massachusetts General Hospital, Harvard Medical School, Boston, MA, USA.

Corresponding author: David Miller, MD, PhD, FAAD

Funding sources: None

Conflict of Interest Disclosure: Dr. Miller has received honoraria for participating on advisory boards for Pfizer, Merck, Regeneron and Sanofi Genzyme

Acknowledgements: We would like to thank Hang Lee, Ph.D. of the MGH-Harvard Catalyst Biostatistics Program for his assistance and guidance with the fragility index and the statistical analysis plan. We would also like to thank Melissa Reyes, MD, MPH, DTMH, LCDR, USPHS of the Division of Dermatology and Dental Products, FDA for her review and assistance. This article reflects the views of the authors and should not be construed to represent the views or policies of the FDA.

Keywords: Psoriasis, Biologics, pivotal trials, Psoriasis Area and Severity Index (PASI), Physician Global Assessment (PGA), fragility index, fragility quotient

Abbreviations: 21 CFR- Code of Federal Regulations Title 21, BLA- Biological License Application, FDA- U.S. Food and Drug Administration, FI- Fragility Index, FQ- Fragility Quotients, IL- Interleukin, MFI- Median Fragility Index, MFQ- Mean Fragility Quotient, NDA- New Drug Application, NME- New Molecular Entity, PASI- Psoriasis Area and Severity Index, PGA- Physician Global Assessment, RCT- Randomized controlled trials, SD- standard deviation, TNF- Tumor Necrosis Factor

ABSTRACT

Background: Over the last five years, there has been a rapid growth in the number of clinical trials used to support an FDA approval for systemic therapies with labeled indications for plaque psoriasis.
Objective: We aim to evaluate the fragility of clinical trial data used to support FDA-approval of therapies for psoriasis.
Methods: We reviewed primary endpoints of the efficacy pivotal trials of all systemic medications with a labeled indication for plaque psoriasis available from Drugs@FDA.
Results: 69 clinical trial primary endpoints met inclusion criteria and were assessed for robustness, yielding a median fragility index (MFI) of 72 and a median fragility quotient (MFQ) of 0.19.
Limitations: Efficacy and statistical analysis data for several approved medications were not available on the product label or on Drugs@FDA.
Conclusions: When compared to randomized controlled trials for FDA approval across various diseases, pivotal trials in psoriasis appear quite robust to changes in outcomes.

Capsule Summary:

  • Fragility index (FI) is a metric used to assess the robustness of clinical trial data. The FI is the minimum number of subjects whose outcome would have to change from an event to a non-event to alter the trial result from significant to non-significant.

  • Pivotal trials in psoriasis are quite robust to changes in outcomes. Additional consideration regarding trial design with more emphasis on conservation of resources in psoriasis is appropriate.

INTRODUCTION

Psoriasis is a widespread dermatologic condition affecting more than 8 million Americans. In recent years, there has been rapid growth in the number of agents approved by the U.S. Food and Drug Administration (FDA) for psoriasis (Figure 1). The first therapy approved for psoriasis was methotrexate in 1972. Currently, there are a total of twenty-eight distinct systemic therapies that have been FDA approved for psoriasis. Strikingly, 64% of these therapies were approved in the last five years alone (Figure 1). In part fueling the proliferation of approvals has been the discovery that specific effector molecules such as Tumor Necrosis Factor (TNF), Interleukin (IL)-17 and IL-23 are critical in the pathogenesis. Commensurate with the diverse molecular origins, there are 12 distinct therapeutic classes of medications approved for psoriasis (Figure 1).

Figure 1. Psoriasis Therapy Time Series. Time series demonstrating the exponential growth in the development of biologics within the past five years.



To demonstrate clinical benefit and support FDA approval to market a new drug, the proposed indication is typically evaluated by pre-specified, primary endpoint(s) in pivotal trials assessing efficacy. Primary efficacy endpoints are the basis for the design and success of a trial, which is determined by significance testing of proposed hypotheses. To assess the robustness of the randomized controlled trials (RCTs) used in FDA approved therapies for psoriasis, we used the fragility index (FI), an established metric for evaluating the statistical fragility of clinical trial data. The FI demonstrates the ease for which statistically significant results are lost with alterations in the numbers of the outcomes.1 To calculate the fragility index of RCTs, the results of a trial are arranged in a two by two contingency table (Supplementary Figure 1).

Supplementary Figure 1: Two by Two Contingency Table demonstrating how the Fragility Indices are Calculated for each primary endpoint of RCT Fragility indices are calculated by arranging the results of a trial in a two by two contingency table. The total number of subjects, N, in the trial arm is maintained constant throughout each iteration of single step event status modifications. An event is added to the group with the smaller number of events (a + 1), while subtracting a nonevent (b - 1) to maintain the total number of subjects (a +b). If the new p value produced by a Fisher exact test does not equal or exceed 0.05 (ie. the alpha is pre-specified, for example in RCTs with co-primary endpoints, an alpha of 0.025 may be pre-specified), then another round of these modifications continues. If after several iterations, the new p value produced equals or exceed 0.05, the number of events added to reach this p value is the fragility index. The fragility index is the smallest integer x where the trial result’s significance is changed.



The total number of subjects in the trial arm is maintained constant throughout each iteration of single step event status modifications. An event is added to the group with the smaller number of events, while subtracting a nonevent. If the new p value produced by a Fisher exact test does not equal or exceed 0.05, then another round of these modifications continues. If the new p value produced equals or exceeds 0.05, the number of events added to reach this p value is the fragility index. If the p value is still not 0.05 or greater, iterations continue until the first instance where the Fisher exact test equals 0.05 or greater. For example, an RCT with an FI of 3 indicates that only 3 subjects in that study would need to change from an event to a non-event to alter the trial result from significant to non-significant. We provide an in-depth analysis of the robustness of pivotal trials used to support the FDA approvals of systemic therapies in psoriasis.

METHODS

To evaluate the evidence used to support a labeled indication in psoriasis, this analysis includes New Drug Application (NDA) or Biological License Application (BLA) medications that are indexed on the FDA website (https://www.accessdata.fda.gov/scripts/cder/daf/index.cfm). The selection criteria used in these analyses can be found in Supplementary Figure 2. To date, twenty-eight therapies have been approved by the FDA for the treatment of plaque-psoriasis. Four of these therapies were not included in our analysis since the clinical trial data for the NDA/BLA was not available for review. Ten biosimilars were not included in our analysis of primary endpoints, and fragility due to the fundamental differences in the approval process for biosimilars. A total of 33 efficacy trials were used to support an FDA approval in psoriasis by the remaining 14 therapies included in our analysis. We analyzed the pre-defined primary endpoints that were used to support the therapies’ first FDA approved label with a psoriasis indication that met inclusion criteria for fragility index assessment (ie dichotomous outcome, two-parallel design). Sixty-nine primary or co-primary endpoints met this inclusion criteria. Fragility indices (FI) were calculated using the package ‘fragilityindex’ and figures were generated with the package ‘ggplot2’, using the R programming language, version 3.6.1 (R Foundation for Statistical Computing). The confidence level for the FI calculation was based on the alpha-level used in pivotal trials indicated on the product label. For co-primary endpoints, the confidence level was adjusted according to the alpha splitting performed by the FDA. Fragility quotients (FQ) were calculated by dividing the fragility index by the number of subjects in that trial. Raw data for FI and FQ calculations can be viewed at (https://www.themillerlab.io/project/regulatory-medicine/).

Supplementary Figure 2: Selection Criteria for Therapies with an FDA Approval in Psoriasis Studies and Primary/Co-Primary Endpoints used in Analysis Twenty-eight therapies have been approved by the FDA for the treatment of plaque-psoriasis. A total of 33 efficacy trials were used to support an FDA approval in psoriasis by the remaining 14 therapies that had clinical data available for review and were not biosimilars. We analyzed the pre-defined primary endpoints that were used to support the therapies’ first FDA approved label with a psoriasis indication that met inclusion criteria for fragility index assessment (ie dichotomous outcome, two-parallel design). Sixty-nine primary or co-primary endpoints met this inclusion criteria.



RESULTS

The primary efficacy endpoints of pivotal trials used to support an FDA approval in psoriasis have been evaluated exclusively with the use of two instruments: PASI (Psoriasis Area and Severity Index) and PGA (Physician Global Assessment). Each trial was designed by comparing the efficacy of the proposed therapeutic versus a placebo, with only one application incorporating an additional active comparator arm as a primary endpoint that was included on the label: the BLA for brodalumab incorporated ustekinumab as a co-primary. Of note, other sponsors’ development programs included active biologic comparator arms but were pre-specified as secondary or exploratory endpoints and were not included in this analysis. All submitted applications had at least two pivotal trials and several used three pivotal trials to support their efficacy labeling claims in psoriasis for a total of 33 trials (Figure 2). The mean number of subjects enrolled in the arms pre-specified for primary and co-primary endpoint analysis in the labeled pivotal trials was 1421 +/- 575 (mean ± SD). However, this average subject number underestimates the mean of the total number of subjects enrolled in pivotal trials since subjects are also enrolled in other arms pre-specified for dose escalating studies and/or secondary endpoint analysis.

Figure 2. Total Number of Subjects in the Primary and Co-Primary Endpoint Trial Arms. This figure depicts the cumulative number of subjects used in pivotal trials that were evaluated by the FDA for an approval in psoriasis. These include only the trial arms pre-specified for primary endpoint analysis in the labeled pivotal trials; subjects enrolled in other arms of the pivotal trials pre-specified for dose escalating studies or secondary endpoint analysis are excluded in this total.


The FI was used to evaluate the robustness of the results reported in each primary endpoint hypothesis test. Due to the fundamental differences in the approval process for biosimilars, only data from non-biosimilars were included in the analysis of primary endpoints. The median FI (MFI) of pivotal trials in psoriasis was 72 (Figure 3).

Figure 3. Fragility Index of Pivotal Trial Primary Endpoints in Plaque Psoriasis. The fragility index was calculated for the primary or co-primary endpoints from 14 FDA-approved therapies in psoriasis with available data. 69 endpoints met FI inclusion criteria for analysis. The Median fragility index (MFI) was 72.



The fragility quotient (FQ) is a statistical summary used to adjust for the potential effects on FI from variations in study subject number size. To normalize for sample size differences across trials, the FQ was evaluated for each of the clinical trial primary endpoint results in psoriasis, yielding a median FQ (MFQ) of 0.19 (Figure 4).

Figure 4. Fragility Quotient of Pivotal Trial Primary Endpoints in Plaque Psoriasis. The fragility quotient was calculated for the 69 primary or co-primary endpoints that met FI inclusion criteria for analysis. The FQ was determined by dividing the FI by the total subjects enrolled in the primary or co-primary endpoint arms. The Median fragility quotient (MFQ) was 0.19.



Many of the 14 psoriasis systemic therapies also carry an FDA approval for the treatment of non-plaque psoriasis indications as well, including rheumatoid arthritis, Crohn’s disease, and psoriatic arthritis. In comparison to the MFI of 72 in psoriasis, the MFI of pivotal trial primary endpoints used for the FDA approval of non-plaque psoriasis indications was only 15, approximately 5-fold less than the MFI for trials in psoriasis (Figure 5). The MFQ for the non-plaque psoriasis trials was 0.077, approximately 2.5-fold less than the MFQ for trials in psoriasis (Supplementary Table 1).

Figure 5. Fragility Index of Pivotal Trial Primary Endpoints in Non-Plaque Psoriasis Indications. Eight of the biologic therapies with indications in psoriasis also carry an FDA-approval for non-plaque psoriasis indications (e.g. rheumatoid arthritis, Crohn’s disease, etc.). The Fragility Index of the non-plaque psoriasis pivotal trial primary endpoints used for FDA approval and that met the selection criteria for FI analysis are graphed. The upper panel is color coded by drug name and the bottom panel is the corresponding FI data color coded by indication. The MFI is 15.



DISCUSSION

There has been a rapid proliferation of FDA-approved medications with labeled indications for psoriasis over the last decade, with particular growth over the last five years. The reasons are likely multifactorial, involving advances in the development of biosimilars and perhaps, most importantly, improvement in our understanding of the pathophysiology of psoriasis that has led to the development of effective targeted treatments. We evaluated the statistical persuasiveness of the efficacy pivotal trial endpoints used in the FDA drug approval process to establish clinical benefit in therapies with an indication for plaque psoriasis.

In their seminal paper, Walsh et al.1 evaluated 399 RCTs across several diseases and found the MFI to be 8. Since that publication, the FI of trials in numerous medical conditions has been assessed. We performed a literature search and review of previous articles where the fragility index of various diseases and clinical trials in medicine were evaluated2-5. No articles assessing the fragility index in any psoriasis trials were found in the literature search. Our meta-analysis included forty-nine previous reports encompassing 3,632 trials revealing an overall low MFI of 3 (Supplementary Table 2-3). This is in sharp contrast to an MFI of 72 in the psoriasis pivotal trials reported herein. A caveat in this comparison is that we restricted our analysis to psoriasis efficacy pivotal trials used to support FDA-approved labeled claims, whereas many trials in our meta-analysis of the FI in the literature focus on earlier exploratory RCTs. Given that later phase trials benefit from information gained from earlier phase investigations, earlier studies may be more fragile. That said, a recent evaluation of the robustness of pivotal trials used to support FDA- approvals in oncology found the MFI to be 2.5 However, oncological indications appropriately have different standards for approval.

Supplemental Table 2. Summary of the Total Articles, and Trials assessed in the meta-analysis of the Literature Review of Articles Assessing Fragility Index of Clinical Trials in Medicine


Reference Year Single Trial or Meta Analysis? Clinical Research Discipline Did all the assessed Trials have a Pharmacological intervention? Number of Trials MFI
Del Paggio et al^1^ 2019 Meta Analysis Oncology Yes 17 2.0
Aves, T. et al^2^ 2019 Meta Analysis epinephrine and cardiac arrest Yes 15 0.0
Berti, A. et al^3^ 2018 Meta Analysis Giant Cell Arteritis Yes 10 4.5
Wayant, C. et al^4^ 2019 Meta Analysis antiretroviral agents in HIV-^1^-infected adults and adolescents. Yes 39 5.0
Van Howe, R. S. et al^5^ 2019 Meta Analysis antiretroviral agents in HIV-^1^ adults Yes 3 5.0
Sato K et al^6^ 2019 Meta Analysis cryptogenic stroke Yes 6 0.0
Edwards E et al^7^ 2018 Meta Analysis Venous thromboembolism Yes 21 5.0
Ridgeon, E. E. et al^8^ 2016 Meta Analysis RCTs in Acute Care No 56 2.0
Walsh, M.^9^ 2014 Meta Analysis RCTs/ general No 399 8.0
Atal, I.^10^ 2019 Meta Analysis RCTs/general; statistically significant No 400 12.0
Atal, I.^10^ 2019 Meta Analysis RCTs/general; statistically insignificant No 506 7.0
Mazzinari, G. et al^11^ 2018 Meta Analysis Anesthesiology; high ranking medical journal RCTS No 35 4.0
Mazzinari, G. et al^11^ 2018 Meta Analysis top ^15^ Anesthesiology journals No 104 3.0
Goerke, K. et al^12^ 2019 Meta Analysis Anesthesiology No 131 3.0
Grolleau, F.et al^13^ 2019 Meta Analysis Anesthesiology No 166 3.0
Meyer, C. et al^14^ 2019 Meta Analysis ST-Elevation Myocardial Infarction No 42 10.0
Docherty, K. F.et al^15^ 2017 Meta Analysis Chronic heart failure No 20 26.0
Agrawal, R. et al^16^ 2019 Meta Analysis cirrhosis No 21 1.0
Ruzbarsky, J. J. et al^17^ 2019 Meta Analysis clavicular fractures No 15 2.0
Chase Kruse, B. et al^18^ 2017 Meta Analysis diabetes No 19 16.0
Russell, M. J. et al^19^ 2018 Single Trial fluid bolus in pediatric septic shock No 1 1.5
Ruzbarsky, J. J. et al^20^ 2019 Meta Analysis shoulder and elbow surgery No 30 4.0
Brown, J. et al^21^ 2019 Meta Analysis Emergency Medicine No 180 4.0
Mazzinari, G. et al^22^ 2019 Single Trial Glidescope Laryngoscopy No 1 1.0
Meyer, C. et al^23^ 2019 Meta Analysis dyspepsia, H pylori No 52 3.0
Noel, C. W. et al^24^ 2018 Meta Analysis head and neck surgery No 27 1.0
Ruzbarsky, J. J. et al^25^ 2019 Meta Analysis hand surgery No 5 3.0
Rickard, M. et al^26^ 2019 Meta Analysis hydronephrosis No 130 2.0
Shen, Y. et al^27^ 2019 Meta Analysis intracranial hemorrhage No 98 5.0
Bowers, A. et al^28^ 2018 Meta Analysis laparoscopic and endoscopic surgery No 54 2.5
Niforatos, J. D.et al^29^ 2019 Meta Analysis high impact journal RCTs No 17 5.0
Shochet, L. R. et al^30^ 2017 Meta Analysis nephrology No 107 3.0
Pedziwiatr, M. et al^31^ 2019 Meta Analysis nutrition No 37 1.0
Checketts, J. X. et al^32^ 2018 Meta Analysis orthopedic surgery No 72 2.0
Skinner, M. et al^33^ 2019 Meta Analysis otolaryngology No 69 3.0
Vargas, M. et al^34^ 2019 Meta Analysis oxygen therapy in critically ill No 19 0.0
Hallet, J. et al^35^ 2015 Meta Analysis pancreatic fistula No 4 3.5
Tignanelli, C. J. et al^36^ 2019 Meta Analysis patient care No 14 3.0
Matics, T. J. et al^37^ 2019 Meta Analysis pediatric clinical care No 43 2.0
Khormaee, S. et al^38^ 2018 Meta Analysis pediatric orthopedic No 17 3.0
Matics, T. J.et al^39^ 2017 Meta Analysis pediatric No 17 7.0
Bertaggia, L. et al^40^ 2019 Meta Analysis peri operative No 46 2.0
Chin, B. et al^41^ 2019 Meta Analysis plastic surgery No 90 1.0
Evaniew, N. et al^42^ 2015 Meta Analysis spine surgery No 40 2.0
Khan, M. et al^43^ 2017 Meta Analysis sports medicine No 48 2.0
Parisien, R. L. et al^44^ 2019 Meta Analysis sports surgery No 102 5.0
Topcuoglu, M. A. et al^45^ 2019 Meta Analysis Ischemic stroke No 25 7.0
Parisien, R. L. et al^46^ 2019 Meta Analysis trauma research No 198 5.0
Narayan, V. M. et al^47^ 2018 Meta Analysis urology No 41 3.0
Gnech, M.et al^48^ 2019 Meta Analysis vesicoureteral reflux No 22 5.0
Schelly, D. et al^49^ 2019 Single Trial well elderly II No 1 2.0

Supplemental Table 3. Articles Included in the Fragility Index Analysis of Clinical Trials in Medicine



Interestingly, when we restricted a separate analysis of the MFI and MFQ to the labeled pivotal trials used for FDA-approval in other non-dermatological inflammatory conditions of the same psoriasis medications, using the exact same strict inclusion criteria, the MFI was 5-fold lower than in psoriasis trials and the MFQ was 2.5-fold lower. Although comparisons to other fields are limited by salient differences in patient population, disease incidence and severity, there may be some utility in the juxtaposition. That notwithstanding, an evaluation of primary endpoints of efficacy pivotal trials for FDA approvals in atopic dermatitis (dupilumab, MFI of 41), hidradenitis suppurativa (adalimumab, MFI of 20.5) and pemphigus vulgaris (rituximab, MFI of 16) revealed substantively lower MFIs compared with psoriasis (Supplementary Table 4).

Supplemental Table 4. Fragility Index and Quotient for Biologics with Other Dermatological Indications


Admittedly, cross-disciplinary comparisons must be made cautiously, but these comparisons reveal the robustness of the clinical trials in psoriasis are not simply explained due to the advantages of later phase trials, the criteria needed for FDA approval, or the selection criteria for analysis. Thus, when compared to RCTs for FDA approvals across various diseases, pivotal trials in psoriasis appear quite robust to changes in outcomes.

There is no current agreement on what is the most appropriate FI; consequently, interpretation of its meaning is prone to subjectivity. Furthermore, there are valid criticisms regarding the use of FI and FQ in evaluating RCTs, especially when low FIs are used to criticize the strength of trial conclusions.6,7 Acuna et al. argue that in an effort to minimize risk to patients and optimize resource utilization, RCTs are purposefully planned to enroll the least number of subjects required to detect a minimal meaningful clinical benefit.7 Therefore, it is not surprising, and maybe even appropriate, that many RCTs demonstrate a degree of fragility. However, the RCTs used in efficacy pivotal trials for indications in psoriasis are exceptionally robust.

While there is no specific standard level of robustness for pivotal trials, a post hoc power analysis of the pivotal trials in psoriasis revealed a median power of 100% for the primary endpoints (Supplementary Table 1). Indeed, assuming the exact same event rate in the psoriasis pivotal trials but reducing the number of subjects by 50%, still results in a robust MFI of 33 for all the development programs, with none losing the ability to reject the null. One must consider that in addition to efficacy, long-term safety is a critical consideration in the number of subjects enrolled in trials. Nevertheless, in the era of rising drug costs and therapeutics with generally acceptable risk profiles and statistically persuasive efficacy, perhaps additional consideration regarding trial design with more emphasis on conservation of resources in psoriasis is appropriate. For example, to conserve resources in pivotal clinical trials in psoriasis, the use of co-primary endpoints in psoriasis may be re-visited. There are numerous clinical measures used to evaluate psoriasis in routine practice and in clinical trials, though no one superior tool has emerged.8 Puzenat et al. performed a systemic review of clinical studies in psoriasis and concluded that the PASI instrument was the most complete and most extensively validated score.9 As a result of its performance, improvements in PASI have been used as a primary endpoint by every non-biosimilar application in our analysis since its inception with the exception of one. Nevertheless, due to limitations in assessment of certain aspects of the disease (e.g. cases of minimal body surface area involvement or involvement of acral skin), PGA has been routinely incorporated as a co-primary endpoint in drug development programs, including every approved application since 2015. Although important in possibly minimizing the limitations of any one assessment tool, the use of co-primary endpoints affects clinical trial design, as alpha splitting to conserve a family-wise error rate can necessitate an inflation in sample size. However, the notion that PASI and PGA are sufficiently complementary to justify mandated use as co-primary endpoints, has recently been challenged. After analyzing 30 randomized controlled trials in moderate-to-severe psoriasis, Robinson et al. concluded that for agents which produced a 75% reduction in PASI scores in at least 25% of patients, PASI and PGA had substantial redundancy.10 Indeed, in the 9 applications from our analysis that used both PASI 75 and PGA as co-primary endpoints in their pivotal trials, PASI 75 and PGA were significantly correlated (Pearson Correlation 0.75, p = 0.00002, Supplementary Figure 3 ).

Supplemental Figure 3. Correlation of Co-Primary Endpoints Per Study Several therapies used co-primary endpoints of PASI 75 and PGA in the pivotal efficacy trials used to support an FDA approval. The percent of subjects that met the PASI 75 and PGA co-primary endpoints are graphed and demonstrate a correlation of 0.75.



The use of co-primary endpoints has implications on the cost of drug development programs and patient risk, as it affects the number of subjects needed to establish statistical significance, and thus affects the number of patients exposed to unproven and potentially harmful investigational agents. Indeed, Robinson et al. argue that due to the divergence of PASI and PGA at lower therapeutic effectiveness, co-primary endpoints may be more appropriate for early phase clinical studies.10 In contrast, with efficacy pivotal trials, where more clarity regarding efficacy has already been established in earlier parts of a drug development program, consideration for a single primary endpoint with PASI may be warranted to conserve costs and resources.

Furthermore, current legislative statutes do not have an active comparator efficacy requirement; thus, sponsors do not need to provide evidence that the investigational agent is superior to currently available therapy. Therefore, recent incorporation of active comparators into trial design is not mandated by statute but increases cost and resources necessary for trials. However, this incorporation of active comparator arms may reflect commercial strategic factors such as insurance formulary preference or fulfillment of a global development program. In addition, the inclusion of active comparator arms may be ethically favored when designed to compare the new therapy against the best available standard of care.

To provide “substantial evidence” of clinical benefit, the null hypothesis of a pivotal trial’s primary endpoint must be rejected. To do so, there must be statistical persuasiveness that the effect seen is not due to chance. The probability of rejecting the null hypothesis is related to the power of a study, which is influenced by the effect size of the intervention, the decided upon significance level and the sample size. Our analysis demonstrates that the efficacy pivotal trials for systemic therapies with labeled indications for psoriasis appear to be substantially robust compared to those for other indications with an FDA approval. In the past, the FI has generally been used to indicate that most reported trials with statistical significance would lose this significance with a small change in the number of outcomes.1-6,11-13 Here, we show the opposite for the pivotal trials in psoriasis which are far less fragile, and perhaps warrant consideration in trial design to reduce costs.

REFERENCES

  1. Walsh M, Srinathan SK, McAuley DF, et al. The statistical significance of randomized controlled trial results is frequently fragile: a case for a Fragility Index. J Clin Epidemiol. 2014;67(6):622-628.
  2. Van Howe RS. The Fragility Index in HIV/AIDS Trials. J Gen Intern Med. 2019.
  3. Wayant C, Meyer C, Gupton R, Som M, Baker D, Vassar M. The Fragility Index in a Cohort of HIV/AIDS Randomized Controlled Trials. J Gen Intern Med. 2019;34(7):1236-1243.
  4. Berti A, Cornec D, Medina Inojosa JR, Matteson EL, Murad MH. Treatments for giant cell arteritis: Meta-analysis and assessment of estimates reliability using the fragility index. Semin Arthritis Rheum. 2018;48(1):77-82.
  5. Del Paggio JC, Tannock IF. The fragility of phase 3 trials supporting FDA-approved anticancer medicines: a retrospective analysis. Lancet Oncol. 2019;20(8):1065-1069.
  6. Niforatos JD, Zheutlin AR, Chaitoff A, Pescatore RM. The Fragility Index of Practice Changing Clinical Trials is Low and Highly Correlated with P-Values. J Clin Epidemiol. 2019.
  7. Acuna SA, Sue-Chue-Lam C, Dossa F. The Fragility Index-P Values Reimagined, Flaws and All. JAMA Surg. 2019;154(7):674.
  8. Spuls PI, Lecluse LL, Poulsen ML, Bos JD, Stern RS, Nijsten T. How good are clinical severity and outcome measures for psoriasis?: quantitative evaluation in a systematic review. J Invest Dermatol. 2010;130(4):933-943.
  9. Puzenat E, Bronsard V, Prey S, et al. What are the best outcome measures for assessing plaque psoriasis severity? A systematic review of the literature. J Eur Acad Dermatol Venereol. 2010;24 Suppl 2:10-16.
  10. Robinson A, Kardos M, Kimball AB. Physician Global Assessment (PGA) and Psoriasis Area and Severity Index (PASI): why do both? A systematic analysis of randomized controlled trials of biologic agents for moderate to severe plaque psoriasis. J Am Acad Dermatol. 2012;66(3):369-375.
  11. Atal I, Porcher R, Boutron I, Ravaud P. The statistical significance of meta-analyses is frequently fragile: definition of a fragility index for meta-analyses. J Clin Epidemiol. 2019;111:32-40.
  12. Meyer C, Bowers A, Tritz D, et al. The fragility of randomized trial outcomes underlying management of dyspepsia and Helicobacter pylori infections. Int J Evid Based Healthc. 2019.
  13. Vargas M, Buonanno P, Iacovazzo C, Servillo G. Epinephrine for out of hospital cardiac arrest: A systematic review and meta-analysis of randomized controlled trials. Resuscitation. 2019;136:54-60.

Supplemental Methods.

Here we supply the data on the Fragility Index and Fragility quotients that can be found in the manuscript: Fragility Index and Fragility Quotient Calculations

REFERENCES FOR SUPPLEMENTARY TABLE 3 LITERATURE REVIEW:
1. Del Paggio JC, Tannock IF. The fragility of phase 3 trials supporting FDA-approved anticancer medicines: a retrospective analysis. Lancet Oncol 2019;20:1065-9.
2. Aves T, Chopra A, Patel M, Lin S. Epinephrine for Out-of-Hospital Cardiac Arrest: An Updated Systematic Review and Meta-Analysis. Crit Care Med 2019.
3. Berti A, Cornec D, Medina Inojosa JR, Matteson EL, Murad MH. Treatments for giant cell arteritis: Meta-analysis and assessment of estimates reliability using the fragility index. Semin Arthritis Rheum 2018;48:77-82.
4. Wayant C, Meyer C, Gupton R, Som M, Baker D, Vassar M. The Fragility Index in a Cohort of HIV/AIDS Randomized Controlled Trials. J Gen Intern Med 2019;34:1236-43.
5. Van Howe RS. The Fragility Index in HIV/AIDS Trials. J Gen Intern Med 2019.
6. Sato K, Toda T, Iwata A. Fragility Index in Randomized Controlled Trials of Ischemic Stroke. J Stroke Cerebrovasc Dis 2019;28:1290-4.
7. Edwards E, Wayant C, Besas J, Chronister J, Vassar M. How Fragile Are Clinical Trial Outcomes That Support the CHEST Clinical Practice Guidelines for VTE? Chest 2018;154:512-20.
8. Ridgeon EE, Young PJ, Bellomo R, Mucchetti M, Lembo R, Landoni G. The Fragility Index in Multicenter Randomized Controlled Critical Care Trials. Crit Care Med 2016;44:1278-84.
9. Walsh M, Srinathan SK, McAuley DF, et al. The statistical significance of randomized controlled trial results is frequently fragile: a case for a Fragility Index. J Clin Epidemiol 2014;67:622-8.
10. Atal I, Porcher R, Boutron I, Ravaud P. The statistical significance of meta-analyses is frequently fragile: definition of a fragility index for meta-analyses. J Clin Epidemiol 2019;111:32-40.
11. Mazzinari G, Ball L, Serpa Neto A, et al. The fragility of statistically significant findings in randomised controlled anaesthesiology trials: systematic review of the medical literature. Br J Anaesth 2018;120:935-41.
12. Goerke K, Parke M, Horn J, et al. Are results from randomized trials in anesthesiology robust or fragile? An analysis using the fragility index. Int J Evid Based Healthc 2019.
13. Grolleau F, Collins GS, Smarandache A, et al. The Fragility and Reliability of Conclusions of Anesthesia and Critical Care Randomized Trials With Statistically Significant Findings: A Systematic Review. Crit Care Med 2019;47:456-62.
14. Meyer C, Bowers A, Jaiswal D, et al. An analysis of randomized controlled trials underpinning ST-elevation myocardial infarction management guidelines. Am J Emerg Med 2019;37:2229-38.
15. Docherty KF, Campbell RT, Jhund PS, Petrie MC, McMurray JJV. How robust are clinical trials in heart failure? Eur Heart J 2017;38:338-45.
16. Agrawal R, Majeed M, Attar BM, et al. Reexamining the data used in the 2012 guidelines of the American Association for the Study of Liver Diseases for the management of adult patients with ascites due to cirrhosis. Ann Gastroenterol 2019;32:642-9.
17. Ruzbarsky JJ, Khormaee S, Rauck RC, Warren RF. Fragility of randomized clinical trials of treatment of clavicular fractures. J Shoulder Elbow Surg 2019;28:415-22.
18. Chase Kruse B, Matt Vassar B. Unbreakable? An analysis of the fragility of randomized trials that support diabetes treatment guidelines. Diabetes Res Clin Pract 2017;134:91-105.
19. Russell MJ, Kanthimathinathan HK. Is There an Optimum Duration of Fluid Bolus in Pediatric Septic Shock? A Critical Appraisal of “Fluid Bolus Over 15-20 Versus 5-10 Minutes Each in the First Hour of Resuscitation in Children With Septic Shock: A Randomized Controlled Trial” by Sankar et al (Pediatr Crit Care Med 2017; 18:e435-e445). Pediatr Crit Care Med 2018;19:369-71.
20. Ruzbarsky JJ, Rauck RC, Manzi J, Khormaee S, Jivanelli B, Warren RF. The fragility of findings of randomized controlled trials in shoulder and elbow surgery. J Shoulder Elbow Surg 2019;28:2409-17.
21. Brown J, Lane A, Cooper C, Vassar M. The Results of Randomized Controlled Trials in Emergency Medicine Are Frequently Fragile. Ann Emerg Med 2019;73:565-76.
22. Mazzinari G, Rovira L, Henao L, et al. Effect of Dynamic Versus Stylet-Guided Intubation on First-Attempt Success in Difficult Airways Undergoing Glidescope Laryngoscopy: A Randomized Controlled Trial. Anesth Analg 2019;128:1264-71.
23. Meyer C, Bowers A, Tritz D, et al. The fragility of randomized trial outcomes underlying management of dyspepsia and Helicobacter pylori infections. Int J Evid Based Healthc 2019.
24. Noel CW, McMullen C, Yao C, et al. The fragility of statistically significant findings from randomized trials in head and neck surgery. Laryngoscope 2018;128:2094-100.
25. Ruzbarsky JJ, Khormaee S, Daluiski A. The Fragility Index in Hand Surgery Randomized Controlled Trials. J Hand Surg Am 2019;44:698 e1- e7.
26. Rickard M, Lorenzo AJ, Hannick JH, Blais AS, Koyle MA, Bagli DJ. Over-reliance on P Values in Urology: Fragility of Findings in the Hydronephrosis Literature Calls for Systematic Reporting of Robustness Indicators. Urology 2019;133:204-10.
27. Shen Y, Cheng X, Zhang W. The fragility of randomized controlled trials in intracranial hemorrhage. Neurosurg Rev 2019;42:9-14.
28. Bowers A, Meyer C, Tritz D, et al. Assessing quality of randomized trials supporting guidelines for laparoscopic and endoscopic surgery. J Surg Res 2018;224:233-9.
29. Niforatos JD, Zheutlin AR, Chaitoff A, Pescatore RM. The fragility index of practice changing clinical trials is low and highly correlated with P-values. J Clin Epidemiol 2019.
30. Shochet LR, Kerr PG, Polkinghorne KR. The fragility of significant results underscores the need of larger randomized controlled trials in nephrology. Kidney Int 2017;92:1469-75.
31. Pedziwiatr M, Mizera M, Wysocki M, et al. The fragility of statistically significant results from clinical nutrition randomized controlled trials. Clin Nutr 2019
32. Checketts JX, Scott JT, Meyer C, Horn J, Jones J, Vassar M. The Robustness of Trials That Guide Evidence-Based Orthopaedic Surgery. J Bone Joint Surg Am 2018;100:e85.
33. Skinner M, Tritz D, Farahani C, Ross A, Hamilton T, Vassar M. The fragility of statistically significant results in otolaryngology randomized trials. Am J Otolaryngol 2019;40:61-6.
34. Vargas M, Servillo G. Liberal versus conservative oxygen therapy in critically ill patients: using the fragility index to determine robust results. Crit Care 2019;23:132.
35. Hallet J, Zih FS, Deobald RG, et al. The impact of pancreaticojejunostomy versus pancreaticogastrostomy reconstruction on pancreatic fistula after pancreaticoduodenectomy: meta-analysis of randomized controlled trials. HPB (Oxford) 2015;17:113-22.
36. Tignanelli CJ, Napolitano LM. The Fragility Index in Randomized Clinical Trials as a Means of Optimizing Patient Care. JAMA Surg 2019;154:74-9.
37. Matics TJ, Khan N, Jani P, Kane JM. The Fragility of Statistically Significant Findings in Pediatric Critical Care Randomized Controlled Trials. Pediatr Crit Care Med 2019;20:e258-e62.
38. Khormaee S, Choe J, Ruzbarsky JJ, et al. The Fragility of Statistically Significant Results in Pediatric Orthopaedic Randomized Controlled Trials as Quantified by the Fragility Index: A Systematic Review. J Pediatr Orthop 2018;38:e418-e23.
39. Matics TJ, Khan N, Jani P, Kane JM. The Fragility Index in a Cohort of Pediatric Randomized Controlled Trials. J Clin Med 2017;6.
40. Bertaggia L, Baiardo Redaelli M, Lembo R, et al. The Fragility Index in peri-operative randomised trials that reported significant mortality effects in adults. Anaesthesia 2019;74:1057-60.
41. Chin B, Copeland A, Gallo L, et al. The Fragility of Statistically Significant Randomized Controlled Trials in Plastic Surgery. Plast Reconstr Surg 2019;144:1238-45.
42. Evaniew N, Files C, Smith C, et al. The fragility of statistically significant findings from randomized trials in spine surgery: a systematic survey. Spine J 2015;15:2188-97.
43. Khan M, Evaniew N, Gichuru M, et al. The Fragility of Statistically Significant Findings From Randomized Trials in Sports Surgery: A Systematic Survey. Am J Sports Med 2017;45:2164-70.
44. Parisien RL, Trofa DP, Dashe J, et al. Statistical Fragility and the Role of P Values in the Sports Medicine Literature. J Am Acad Orthop Surg 2019;27:e324-e9.
45. Topcuoglu MA, Arsava EM. The Fragility Index in Randomized Controlled Trials for Patent Foramen Ovale Closure in Cryptogenic Stroke. J Stroke Cerebrovasc Dis 2019;28:1636-9.
46. Parisien RL, Dashe J, Cronin PK, Bhandari M, Tornetta P, 3rd. Statistical Significance in Trauma Research: Too Unstable to Trust? J Orthop Trauma 2019;33:e466-e70.
47. Narayan VM, Gandhi S, Chrouser K, Evaniew N, Dahm P. The fragility of statistically significant findings from randomised controlled trials in the urological literature. BJU Int 2018;122:160-6.
48. Gnech M, Lovatt CA, McGrath M, et al. Quality of reporting and fragility index for randomized controlled trials in the vesicoureteral reflux literature: where do we stand? J Pediatr Urol 2019;15:204-12.
49. Schelly D, Ohl A. Examining Clinical Meaningfulness in Randomized Controlled Trials: Revisiting the Well Elderly II. Am J Occup Ther 2019;73:7301205120p1-p13.