META-ANALYSIS TO ESTIMATE EXPECTABLE DROPOUT RATES IN RANDOMIZED CONTROLLED CLINICAL TRIALS ON AGE-RELATED MACULAR DEGENERATION-SUPPORTING FOR OPTIMIZED SAMPLE SIZE CALCULATION

Stephanie Knippschild, Frank Krummenauer and Christine Baulig. Institute for Medical Biometry and Epidemiology, Faculty of Health, Witten/Herdecke University, Witten Germany. ...................................................................................................................... Manuscript Info Abstract ......................... ........................................................................ Manuscript History


ISSN: 2320-5407
Int. J. Adv. Res. 4 (12), 1906-1922 1907 inappropriate designs, unrepresentative samples, small samples, incorrect methods of analysis and faulty interpretation has often been observed in publications (Altman 1994). Today, these deficiencies can still be observed in publications and the high relevanz of invalid and useless, published data is as topical today as it was 20 years ago. In 2014 the Lancet published an article series on the growing research waste to highlight this problem. Ioannidis reported that "information obtained" from publications "might not be useful or important, and statistical precision or power is often too low or used in a misleading way" (Ioannidis et al., 2014). He traced this problem back to poor trial protocols and designs among other reasons.
The focus of this paper is on an important part within the designing phase of a clinical trial: the calculation of an appropriate sample size. Neither too few nor too many patients should be included within a clinical trial for ethical or economic reasons. The sample size should therefore be big enough to achieve valid (i.e. statistically significant) results referring to the primary endpoint but small enough to avoid ethical problems. This balancing act can only be achieved with a realistic sample size calculation which also includes the specification of a realistic dropout rate. This rate presents an increase of the net number of trial participants necessary for statistical analysis; the resulting increased number of trial participants "to be expected for recruitment" therefore takes dropout patterns during the observation period of a trial into account. Its pre-specification seeks to ensure the availability of the net number of patients by nominal "over-recruitment". Accounting for dropouts is also recommended by the ICH guidelines E6 "Guideline For Good Clinical Practice" (Agency 2002), E9 "Statistical Principles for Clinical Trials" (Agency 1998) and by the CONSORT Statement (Moher et al. 2010). The importance of this factor is frequently described (Julious 2010, Schumacher M. 2008) in literature, but unfortunately only little information is available on dropouts in clinical trials (Baulig et al., 2016a, Baulig et al., 2016b, Knippschild et al., 2014. In particular for studies of the treatment for age-related macular degeneration, there is no data accessible concerning withdrawals so far. Consequently, the sample size has to be calculated by using information from a pilot study or has to be specified on the basis of published trials providing this information. This limited information, however, may be crucially biased (although unintended) as being drawn from only a single or very few trial reports and lacking representativity. At the same time, there is a high risk of incorrect assumptions, which may impact the quality of study results as well as ethical aspects of planning.

Objectives:
The aim of this meta-analysis was to estimate meta-dropout rates in RCTs on the treatment of AMD for the typical follow-up periods of 12 and 24 months in order to enable optimized trail planning for this high volume research segment. These meta-dropout rates may serve to support further investigations in research regarding both from an ethical and economical perspective as well as facilitate transparent reporting.

Methods:
The primary endpoint of the meta-analysis was the total dropout rate in RCTs on AMD treatment after a 12-month individual follow-up. A key secondary endpoint was the corresponding total dropout rate after a 24-month followup.

Search strategy and inclusion criteria:
In contrast to clinical outcome measurements, the dropout rate in medical trials represents a methodological endpoint. To achieve a maximum of sensitivity in data collection, this analysis was therefore based on a full text handsearch in six pre-selected journals. The journals were consented by statistical and ophthalmological trialists: 1908 Data extraction and data management: First one author screened suitable publications by reviewing all titles and abstracts of each study. All clinical trials which address the treatment of AMD were documented and stored for further processing. The results for the primary and secondary key endpoint and validation of all RCTs were extracted independently by two parallel readers as recommended by the Cochrane Institute (Higgins JPT). Data collection included trial specifications such as title of the study, authors, journal and year of publication and raw data such as number of patients randomized into the study and number of patients not available for final analysis / missing after a follow-up period of 12 and 24 months respectively. In addition, dropout rates for subgroups were determined, whereby subgroups were defined according to the actual patterns. This classification was based on the randomization sequences and addressed different therapy options in AMD RCTs:  "treatment group": application of inhibitors for vascular endothelial growth factor (VEGF), dietary supplements, photodynamic, medical, biogenetic, radiation and surgical/laser therapy,  "placebo group": application of a sham treatment as sham injection, sham medication etc.  "observation group": patients were only observed and did not receive any treatment.
Furthermore, the two independent readers evaluated information about trial design and report determinants (evidence of cooperation partners such as methodological core facilities or evidence of industrial partners), the methodological quality of the study (randomization, masking level) and patients' health status (multimorbidity besides the AMD indication e.g. diabetes etc.).
To evaluate the previously estimated meta-dropout rates in the subgroups and to achieve a maximum sensitivity for these results, the difference in dropout rates between the "active-therapy group (only VEGF therapy) in comparison to the "placebo group" after a study duration of 12 months (primary end point) was examined. Therefore, all trials were extracted, which offer exactly this study design from the dataset to maintain the randomization sequenz and to evaluate an inter-trial wise effect-estimator.
Documentation of the raw data was carried out with the software Excel ® (Office 2010 release for Windows ® ).

Statistical Analysis:
All analyses were carried out with the software Comprehensive Meta Analysis (CMA ® ) release 2.2.064, Biostat, Englewood 2011 (M 2011). Based on the reported number of patients randomized and of patients available at the 12-month follow-up evaluation of the trial, the total dropout rate was estimated by means of an exact 95% confidence interval based on the binomial distribution. For meta estimation of the total 12-month dropout rate a random effects model assumption was made and the DerSimonian-Laird estimator was used to estimate τ 2 ; the metadropout rate was then again presented by means of its 95% confidence interval and the underlying total number of patients at recall. Statistical heterogeneity was explored by means of forest plots as well as I² statistics (iterative Paule-Mandel method to estimate the between-study variance); an I² value above 75% was considered to indicate a high heterogeneity. The software "R" (R Core Team (2013): A language and environment for statistical computing. R Foundation for Statistical Computing, Vienna, Austria) (R Core Team 2013) was used to generate the graphics. To illustrate a possible publication bias for the present meta-analysis, funnel plots were created by plotting the dropout rates (in logit scale) in relation to the respective trial size (in standard error scale). It can be expected that the dropout profile from the included publications shifts towards clinical trials with low or moderate dropout rates, which generates an asymmetry as studies with high dropout rates are published lesslikely. In this case the Duval and Tweedie trim and fill method was applied to adjust the dropout rates for such underreporting by again using the random effects model to derive conservative meta-dropout rate estimates.

Assessment of risk of bias in individual studies:
We noticed that the evaluation of a putative risk of bias basically depends on a transparent description within the publication. Only details listed in the publication can finally be documented and analysed. These may not necessarily contain a secure reference to a correctly conducted clinical trial of a high quality, but give an indication which authors prepare their publications in accordance with the CONSORT Statement which suggests a standard performance in study publication. Provided with this information, we assessed the risk of bias of the included clinical trials in accordance with the Cochrane Handbook (Julian PT Higgins 2011) by verifying transparent reporting of the following items: concealment of randomization and blinding (patients, healthcare providers, data collectors and/or outcome assessors), statistical sample size calculation, availability of a CONSORT flow chart in 1909 the trial report (Moher et al. 2010), whether this trial was stopped earlier or not and whether it was a multicenter trial.

Results:
Study Selection: Screening of publication titles and abstracts in the six journals specified above identified 1,062 study reports on treatment of age-related macular degeneration ( Figure 1 For the subgroup analysis, nine clinical trial reports (1,160 study participants) were available for the estimation of dropout rates in the "placebo group", four (559 study participants) for the estimation of dropout rates in the "observation group" and 44 e.g. 63 study arms (14,748 study participants) for estimation of dropout rates in the "treatment group" after a 12-month study duration. Within the "treatment group" the following therapy arms were considered: medical therapy (one study arm), surgical/Laser treatment (four study arms), radiation therapy (four study arms), VEGF therapy (23 study arms) and PDT treatment -also in addition with cortisone (11 study arms), combination of VEGF in addition with PDT (one study arm), biogenetic therapy (one study arm) and treatment with dietary supplements (3 study arms). Also study arms with patients receiving a standard therapy in addition with sham were numbered among the "treatment subgroup" (6 study arms). After a follow-up of 24 months, seven (701 study participants) clinical trial reports were included for the analysis of the estimation of dropout rates in the "placebo group", five (7,412 study participants) for the estimation of dropout rates in the "observation group" and 21 e.g. 29 study arms (2,173 study participants) for the estimation of dropout rates in the "treatment group". Within the "treatment group" for the estimation of the dropout rate after 24 months study duration the following therapy arms were considered: surgical/Laser treatment (4 study arms), radiation therapy (2 study arms), VEGF therapy (7 study arms) and PDT treatment -also in addition with cortisone (7 study arms), combination of VEGF in addition with PDT (2 study arms), treatment with dietary supplements (3 study arms) and study arms with patients receiving a standard therapy in addition with sham (3 study arms).

Primary endpoint:
Considering all exclusion criteria, 45 study reports (16, (Boyer et al. 2009). Random effects model estimation revealed a total dropout rate of 9.2 % [95% CI7.1%-11.9%], but was due to significant heterogeneity (I² = 96.3%; τ 2 = 0.7). The associated forest plot is shown in figure 2 and indicated evidence of asymmetric reporting. By adjusting the dropout rate using the Duval and Tweedie trim and fill method, ten publications had to be imputed, resulting in an adjusted 12-month dropout rate of 11.6% [95% CI 9.1%-14.6%].

Secondary endpoint:
Considering the 22 study reports (7,845 participants) included after 24-month follow-up the number of participants varied from N=50 (Luke et al. 2009 figure 3 displays the dropout rates for all studies included in this analysis. According to a considerable heterogeneity (I 2 = 96.8%, τ 2 = 0.6), the adjustment by using the Duval and Tweedie trim and fill method imputed three publications and estimated an adjusted dropout rate of 21.6% [95% CI 20.6%-22.6%].

Subgroup analysis:
For the analysis of subgroups, 14,748 participants in the "treatment group", 1,160 participants in the "placebo group", and "only" 559 patients in the "observation group" were included after a 12-month study duration; 7,412 /2,173 and 701 participants were available, respectively, up to a 24-month study duration.  Rosenfeld et al. 2006) (605 participants in the "placebo group" and 1,503 participants in the "VEGFtreatment group") were included in the analysis. Random effects model estimation revealed a rate difference for dropouts of 4.0% [95% CI -2.0% -10.0%] after a 12-months study duration, but was due to significant heterogeneity (I 2 =74.6%, τ 2 = 0.0021).

Risk of bias / study quality:
The risk of bias for the included studies was assessed as shown in table 2. Only one publication was identified as reporting 100% of the previously specified quality indicators for the RCT reports and for this reason was included in this analysis as a study report of maximum quality. Furthermore, 28 publications (54%) were included which reported less than 50% of the factors. Only 20 (39%) of 52 publications reported an implemented randomization scheme and clearly described an appropriate concealment strategy. A blinded study design was reported in 39 (75%) of 52 publications. In 25 clinical trials (48%) participants were blinded, healthcare providers in 19% of the analyzed trials (10 studies), and outcome assessors in 46% (24 clinical trials). The data collectors were unaware of the applied therapy in 54% (28 studies) of the analyzed trials. The number of patients and accordingly the dropout rate were documented via flowchart in 35% (18 studies) and a sample size calculation was reported in 50% (26 clinical trials) of the analyzed publications. 36 (69%) of the studies were multicenter trials.

Summary of main results:
The dropout rate is an important factor with respect to study planning and analysis of clinical trials in view of ethically and statistically relevant results and the aim to avoid "research waste". Despite this importance there are very few recommendations regarding this issue especially in terms of the number of dropouts to be expected in multifarious indications. For this reason, this meta-analysis sought to quantify the expectable dropout rate in clinical trials on age-related macular degeneration. It was found to be 9.2% [95% CI 7.1%-11.9%] during a study duration of 12 months and rose to 19.0% [95% CI 14.2%-25.0%], depending on the study duration after 24 months. This important information of rather low dropouts was not easy to obtain without thorough research. In view of the severity of this disease and the possibility of stopping its development, a low dropout rate may have been expected, but an increase of a moderate amount of withdrawals has to be assumed with regard to the age of AMD study participants and the age-related comorbidity patterns. From this point of view, a dropout rate below 10% might already surprise.

Limitation:
Finding clinical trials (RCTs) of high quality is essential for the outcome of meta-analyses. This requires a complete search and the identification of possibly unbiased publications. The decision on the optimal process (handsearch or electronic searches) for data collection is a key step of analyses. Hopewell et al. investigated the question whether handsearch identifies more randomized controlled trials than electronic searching (Hopewell S 2008). They found that handsearching alone will miss a small proportion of studies and that a combination of handsearching and electronic searching is the most comprehensive approach in identifying reports of randomized trials". In contrast to other investigations, we addressed a methodological end point ("dropout rate"), which is not accessible by means of 1911 standard tools such as the MeSH term approach in PubMed ® . This was the main reason why we favored a handsearch. To lower the selection bias of included journals, two experts were asked to consent the journals', regarding high impact as outlets for RCTs in age-related macular degeneration. For all that, we were aware that a small percentage of all possible journals were represented within our data collection due to the selected years of 2004-2013.
Another limitation of our analysis arises from heterogeneity in the reported data. A considerable heterogeneity was as curtained both for the primary and the secondary endpoint. A Q =1,204.239, I 2 =96.346 and τ 2 =0.723 was shown for the study duration of 12 months, and a Q =660.960, I 2 =96.823 τ 2 =0.639 after a 24-month study duration. The trim and fill method by Duval and Tweedie was applied to explain this high heterogeneity and to adjust the effect estimates. The funnel plots (figures 4 and 5) reveal that most of the included studies have many participants and relatively low dropout rates. The expected symmetry shifts to the top and left side of the diagram. For this reason, it may be assumed that there is a lack of publications due to missing publications of small trials on the one hand and to trials with (too) high dropout rates on the other hand. In case of high dropout rates, studies possibly were not published due to problems during analysis caused by the reduced sample size or to appropriate stopping rules in the statistical analysis plan of these trials. "The trim and fill method uses an iterative procedure to remove the most extreme small studies from one side of the funnel plot, re-computing the effect size at each iteration until the funnel plot is symmetric concerning the (new) effect size". "In theory, this will yield an unbiased estimate for the effect size" (Borenstein M 2009). However, the trim and fill approach does not correspond to its usual interpretation of bias reduction in the recent setting, but rather provides a conservative rate adjustment. The resulting rate estimates have to be reconsidered from the practical perspective:  In the recent setting, trim and fill accounts for possible unpublished trials with "too large" dropout rates, since such trials would hardly be considered valid and representative.  The adjusted rate estimate accounting for such trials might introduce a different kind of bias leading to an unethical overestimation of gross sample sizes.
Application of the Duval and Tweedie trim and fill method resulted in an increased adjusted dropout rate of 11.6 % [95% CI 9.1 -14.6 %]. In comparison to the primary calculated rate of 9.2 % [95% CI 7.1 -11.9%], both point estimates are of comparable order ("expect about 10% dropouts for a 12-month observational period!"). In consideration of the upper confidence limits, an expectable rate up to 15% has to be taken into account. Hence, the authors recommend the unadjusted estimate sample size calculation of RCTs in age-related macular degeneration.
Stratification for study groups reveals a higher dropout rate of 12% [95% CI 9.0%-16.5%] for the "placebo group" and 13% [95% CI 8.3%-19.5%] for the "observation group", as opposed to the "treatment group" with 8% [95% CI 6.4%-11.1%]. These results obviously show a difference in dropout rates between subgroups in this investigation, which might lead to the assumption that some patients become aware of their treatment allocation and withdraw participation. A distortion within the results is possible, caused by the small number of cases within the "observation-and placebo group". Note that, the analysis is limited as these subgroup meta estimators were evaluated not "intra-trial" but "inter-trial" wise. This means that any study arms of each RCT were used for the analysis of subgroups, as assigned to the respective subgroups. To gain more information about the estimated dropout rate in between the subgroups -the authors performed an additional meta-analysis of the risk difference between therapy groups. For this, only those studies were included which had at least one VEGF-therapy arm and one placebo arm. However, only three studies (Gillies et al. 2004, Rosenfeld et al. 2006 were identified that met these criteria -all three studies included a VEGF therapy among therapy groups. These three RCTs included 1,503 study participants in VEGF and 605 in the placebo group after 12 months. In order to reveal the intra-trial dropout rate-differences the risk-difference was chosen as target parameter for this special evaluation. The 12 months dropout rate was estimated to be 4.0% [95% CI -2.0% -10.0%] higher in the "placebo group" than in the" (VEGF-) treatment group" (Baulig et al. 2016b). This result was very surprising since a higher difference in dropouts between these groups was expected. The rationale behind this result may be the maximum attention paid to patients during treatment with VEGF inhibitors as well as placebo VEGF injections. This result may be biased due to the very small number of studies available for this analysis, but further research is limited to the ethical aspect of such RCTs.

1912
Review and comparison of current literature: Three research groups already published results for dropout rates in clinical trials. Gehling et al. analyzed the implication of withdrawals of opioid studies in patients with chronic pain due to osteoarthritis by a high dropout rate (Gehling et al., 2011). Therefore, they calculated dropout rates and odds ratios according to different risk factors such as adverse events and lack of analgesic efficacy. In this investigation Gehling et al. found a dropout rate of 38% in the placebo group. Independent of the reasons for dropouts, the study group found an increased risk for dropouts (OR=1.32) in the treatment group compared to the placebo group (Gehling et al. 2011). Given the intention and also the study design, this meta-analysis is not directly comparable to the investigation from Gehling et al. In summary, our analysis showed different results for overall dropout rates and group comparison (Baulig et al. 2016b). At this point, the number of dropouts to be expected in various medical disciplines appears to depend on the indication.
The second study was published by Makatsori et al. (Makatsori et al., 2014) who examined an overall dropout rate and dropout profiles in sublingual allergen immunotherapy. Their publication considered several characteristics: size of the study, duration, allergen preparation, age of participants, number of study sites, type of respiratory disease being treated and the treatment schedule employed. In addition, the authors examined whether dropout rates differ between the active and the placebo-treated group. As a result, Makatsori found an overall dropout rate of 14% in the placebo group after a study duration of one year, and 16% in the active treatment group (Makatsori et al. 2014), which is higher than the dropout rate found in our investigation. However, the outcome from Makatsori showed only minor differences in dropout rates between study groups in comparison to our results.
A meta-analysis performed by our research group investigated the influence of comorbidities to drop out rates in cataract surgery (

Study quality / Risk of bias:
As mentioned in the Methods section, the assessment of study quality is a characteristic which may be addressed only with a high risk of false statement based on the documentation. In those cases where the defined aspects to assess study quality were not evident in a publication, this is not necessarily an indication of low trial quality.
Primarily it would indicate poor documentation and low transparency. Therefore we cannot guarantee that this assessment is absolutely objective. Based on our documentation and analysis, we found less than 50% of the predefined factors documented in 46% of the included studies (24 out of 52) and a documentation rate of under 30% in 35% (18 out of 52) of included publications. At first glance, these findings might suggest a moderate study quality. In addition to these findings, the funnel plots show that a big part of included studies hold many participants, which may indicate a standard conduction with relatively high quality in processes of these trials. In summary, the authors were not able to rule out a poor study quality underlying some trials which were used in this analysis and should therefore be taken into account when results are interpreted.

Conclusion:
This investigation sought to quantify the expectable dropout rates in RCTs on age-related macular degeneration for the sake of optimized clinical trial planning both from a methodological and an ethical perspective. These results show that sample size calculations should account for dropouts of at least 10% during a 12-month follow-up and of at least 20% during a 24-month follow-up period. In addition, it was observed, that placebo patients showed an only moderately increased dropout rate compared to active treatment (placebo 12%, active treatment 8% and observation 13%).        Table 1: Dropout rates and number of included studies for analyses, stratified by subgroups ("placebo", "active therapy" and "observation" group) and study duration. (CI=Confidence Interval, N=Number of included studies).