Bias-Variation Dilemma Challenges Clinical Trials: Inherent Limitations of Randomized Controlled Trials and Meta-Analyses Comparing Hernia Therapies

Abstract

Purpose:Evaluation of hernia therapies according to the current rules of Evidence Based Medicine is widely reduced to results of RCTs or meta-analyses. RCTs have been accepted as a most important tool to confirm a superior effect of an intervention. Unfortunately, in hernia surgery, comparisons of RCTs and correspondingly their use in meta-analyses, are not, surprisingly often, able to confirm any significant impact of a specific procedure due to intrinsic restrictions in a multi-causal setting with its web of influences. Methods:Based on our own experiences of clinical studies in surgery, the present article outlines several situations, with their respective reasons, which argue the severe limitations of RCTs and meta-analysis to define an optimum treatment. Results:Meta-analyses accumulate the variations of each trial, which then may cover any clear causal relationship. RCTs usually are dealing with subgroups of standard patients thus excluding the majority of our patients. Low statistical power of current cohort sizes restricts the analysis of subgroups or of effects with low incidences. Simple comparisons of means frequently are hampered by nonlinear relationships to outcome. The relevance of a specific variable is difficult to separate from other influences. The limited surveillance period of studies ignores a delayed change in outcome. Randomization cannot guarantee a standardized patient’s condition. All the arguments have to be considered as a crucial and fundamental consequence of the bias-variance dilemma or principle of uncertainty in medicine, and underline the many limitations of RCTs to evaluate any specific impact of hernia therapies on e.g. infection, pain or recurrence. Conclusions: Many surgical issues cannot be and should not be investigated by RCTs, in particular, if a marked patients’ heterogeneity has to be considered or the low incidences of the outcome readout cannot be addressed with sufficient statistical power without getting lost in the variation mire. Registries with their non-restricted data-acquisition should be regarded as reliable alternatives for postoperative outcome quality surveillance studies.