European Urology

At a number of panel discussions at scientific meetings over recent months, urologists have debated whether partial nephrectomy for renal cortical tumors confers a survival benefit over radical nephrectomy. Proponents of both views put forward intelligent, valid points but almost always came short of a conclusion. What has emerged from these discussions is an important question: Can data from retrospective analyses, with their inherent potential for selection bias, provide stronger evidence than that obtained from a randomized controlled trial closed prior to meeting the target accrual numbers?

Randomized controlled clinical trials (RCTs) are powerful tools, generating the highest level of evidence to determine whether a cause-and-effect relationship exists between a health care intervention and a given outcome. However, the relevance and exactitude of an RCT's findings are intimately tied to the rigor with which it is designed, executed, and analyzed.

Undoubtedly, surgical RCTs are difficult to conduct. The challenges include:

• • Poor accrual, which leads to underpowered studies
• • Refusal of randomization by patients because they may already have chosen their surgeon for his or her qualifications or have a preference for a given approach
• • Blinding of surgeons, patients, and, to a lesser degree, the party assessing the measured outcome, which can be impractical
• • The difficulty of controlling for the disparities in skill and experience between surgeons and centers.

Consolidated Standards of Reporting Trials (CONSORT) guidelines [1] and [2], issued in 1996 and most recently updated in 2010, provide a framework for trialists to publish their results with transparency, providing the critical reader with answers about why the study was initiated, how the research was conducted, and how the results were analyzed. The CONSORT guidelines include a checklist of items that should be included in clinical trial reports, ranging from how participants were allocated to interventions to the generalizability or external validity of the trial findings.

A critical assessment of the quality of reporting of RCTs in urologic literature between 1996 to 2004 showed that less than half of the 152 RCTs assessed conformed to the CONSORT guidelines with regard to key methodological criteria, such as justification of the sample size, a single primary outcome, effect-size estimates for secondary outcome variables, and a description of the implementation of randomization [3].

In this issue of European Urology, Porpiglia and colleagues report the results of a single-surgeon surgical RCT comparing the recovery of continence at 3 mo between two groups, 60 patients each, treated by laparoscopic radical prostatectomy with or without robotic assistance [4]. The authors conclude that robot-assisted laparoscopic radical prostatectomy provided better functional results with regard to recovery of potency and continence.

The report performs relatively well on the CONSORT checklist but has considerable deficiencies, such as the lack of effect-size estimates for secondary end points and underpowered subset analyses (eg, the finding of potency is based on a comparison of 35 patients in each arm, with only 14 vs 11 men undergoing bilateral intrafascial preservation in the robotic and laparoscopic groups, respectively). Along with the failure to report 95% confidence intervals, such inconsistencies lead to imprecision in their conclusions.

Porpiglia and colleagues, however, are to be congratulated for conducting an excellent single-surgeon quality-assurance trial. Although the concept that an outcome or performance can often be improved simply by measuring the outcome and providing feedback is well recognized [5], [6], and [7], Porpiglia et al. took this concept to another level of sophistication. Initially, a surgeon suspects from personal experience that there is a 25% difference in recovery of urinary continence at 3 mo favoring robotic over laparoscopic prostatectomy. Then the surgeon endeavors to scientifically test this hypothesis through an RCT comparing the two approaches. Unfortunately, the authors did not elaborate on the implications of these results for their practice.

The generalizability of this trial's findings is limited. Often, studies comparing the outcome of two surgical techniques are merely a comparison of surgeons’ performances, unless conducted on a larger scale and designed specifically to account for such variables. Several studies have shown a wide variability of complication rate, and functional and oncologic outcomes among surgeons performing radical prostatectomy [8] and [9]. The magnitude of the differences between surgeons is so important that its impact on a comparative study outcome would dwarf that of any potential difference associated with the surgical approach.

When a trial is designed to randomize patients to receive one of two surgical approaches, all performed by the same surgeon, the misconception is that it compares the surgical approaches, while, in fact, it compares only the performance of the surgeon with one or the other technique.

Conflicts of interest

The author has nothing to disclose.