Article type
Year
Abstract
Background: Systematic reviewers of new surgical procedures are often forced to rely on low level evidence because randomised controlled trials (RCTs) of new surgical procedures are relatively rare. As a result, reviewers may be faced with the decision of whether to include conference abstracts of surgical RCTs in systematic reviews or only include evidence which has been fully published, including lower level evidence.
Objectives: This study compared conference abstracts of surgical RCTs to their subsequent full publication to determine the level of agreement between the publications.
Methods: The conference proceedings of three surgical specialty conferences (urology, orthopaedics, gastroendoscopy) were handsearched for abstracts of randomised controlled trials. The Cochrane Central Register of Controlled Trials, MEDLINE and EMBASE (1998-Nov 2003) were searched to identify subsequent full publications. Data regarding study design, characteristics and numbers of participants, outcomes reported and conclusions were extracted by two reviewers and checked by a third. Any disagreements were resolved by discussion. The level of concordance between abstracts and full publications was summarised into a percentage score for key outcomes.
Results: Of 1843 abstracts, only 92 were reports of RCTs and of these only 45 (2.4%) were surgical. 39/45 (87%) conference abstracts had been published in full in the five years since presentation at a conference (average time to publication 22.3 months, range 2 66). Two non-English language studies were excluded leaving 37 abstracts and full publications for analysis. The quality of reporting in abstracts was poorer than in full publications, due, at least in part, to space restrictions; however, some aspects of reporting were poor in both abstracts and full publications. Method of allocation concealment was only reported in 16 (43%) full publications and was not reported in any abstract. Method of blinding was not reported in 31 (84%) abstracts and 23 (62%) full publications. More participants were randomised in full publications (median 81) than in abstracts (median 60) with 7 abstracts reporting interim results. In terms of participants analysed, only one abstract and 3 full publications clearly stated use of intention-to-treat, with 2 abstracts and 19 full publications providing reasons for losses to follow-up. Overall, 24% of abstracts and full publications reported the same numbers of participants randomised and analysed. Due to these differences in numbers of participants included, results reported were only the same for 45% (15/33) of abstracts and full publications; however, the direction of results was the same in 70% (23/ 33) (Concordance could not be determined in four studies due to lack of detail in abstracts). The primary conclusion was the same in 32/37 (87%) studies.
Conclusions: While it is difficult to judge the methodological quality of abstracts, this problem also exists in many full reports of surgical RCTs. As the direction of results and the conclusions drawn rarely differed, abstracts of surgical RCTs can be included in systematic reviews, but caution must be exercised. Reviewers should attempt to contact authors for additional study details, and abstracts included in meta-analysis should be clearly identified and sensitivity analysis performed to allow readers to judge the validity of the findings.
Objectives: This study compared conference abstracts of surgical RCTs to their subsequent full publication to determine the level of agreement between the publications.
Methods: The conference proceedings of three surgical specialty conferences (urology, orthopaedics, gastroendoscopy) were handsearched for abstracts of randomised controlled trials. The Cochrane Central Register of Controlled Trials, MEDLINE and EMBASE (1998-Nov 2003) were searched to identify subsequent full publications. Data regarding study design, characteristics and numbers of participants, outcomes reported and conclusions were extracted by two reviewers and checked by a third. Any disagreements were resolved by discussion. The level of concordance between abstracts and full publications was summarised into a percentage score for key outcomes.
Results: Of 1843 abstracts, only 92 were reports of RCTs and of these only 45 (2.4%) were surgical. 39/45 (87%) conference abstracts had been published in full in the five years since presentation at a conference (average time to publication 22.3 months, range 2 66). Two non-English language studies were excluded leaving 37 abstracts and full publications for analysis. The quality of reporting in abstracts was poorer than in full publications, due, at least in part, to space restrictions; however, some aspects of reporting were poor in both abstracts and full publications. Method of allocation concealment was only reported in 16 (43%) full publications and was not reported in any abstract. Method of blinding was not reported in 31 (84%) abstracts and 23 (62%) full publications. More participants were randomised in full publications (median 81) than in abstracts (median 60) with 7 abstracts reporting interim results. In terms of participants analysed, only one abstract and 3 full publications clearly stated use of intention-to-treat, with 2 abstracts and 19 full publications providing reasons for losses to follow-up. Overall, 24% of abstracts and full publications reported the same numbers of participants randomised and analysed. Due to these differences in numbers of participants included, results reported were only the same for 45% (15/33) of abstracts and full publications; however, the direction of results was the same in 70% (23/ 33) (Concordance could not be determined in four studies due to lack of detail in abstracts). The primary conclusion was the same in 32/37 (87%) studies.
Conclusions: While it is difficult to judge the methodological quality of abstracts, this problem also exists in many full reports of surgical RCTs. As the direction of results and the conclusions drawn rarely differed, abstracts of surgical RCTs can be included in systematic reviews, but caution must be exercised. Reviewers should attempt to contact authors for additional study details, and abstracts included in meta-analysis should be clearly identified and sensitivity analysis performed to allow readers to judge the validity of the findings.