Article type
Year
Abstract
Introduction: A concern for reviewers in evaluating the merits of a study is whether bad reporting sometimes causes a "good" study to gamer a low score, while, although thought to be less common, a well-reported "bad" study might receive a high score.
Objectives: 1) To ascertain whether missing information is simply not reported, or not done. 2) To evaluate whether querying authors about missing data will change the quality score of a trial. 3) To assess if reporting is getting better over time. 4) To better understand what assumptions can be made about missing data.
Methods: Forty-seven studies meeting the inclusion criteria in a systematic review of acupuncture for chronic pain were evaluated for missing or ambiguous reporting of data about method of group allocation and blinding of outcomes assessor (to assess for double blinding). Four studies were determined to contain adequate information; 43 did not. Letters were sent to first authors of these 43 trials requesting the missing data.
Results: Twenty-one responses (49%) were received. 10 letters were returned to sender with no further contact details. Failure to respond was not associated with year of study. Prior to contacting these authors, the reviewers had scored (using (he Jadad Scale) 15 studies (32%) as high quality and 32 studies (68%) as low quality. The information supplied by the authors changed the quality rating on seven studies (33% of respondents); 14 (67%) remained the same.
Unchanged Low to lower Low to high High to higher
14 2 3 2
Of the 7 respondents queried as to whether outcomes assessor was blinded, 5 reported no. Of the 20 respondents queried on method of allocation, 2 reported alternate assignment even though authors put "randomization" in the paper. All 5 positive changes resulted from confirmation of adequate concealment of allocation
Discussion: If the information has not been reported, it probably has not been done. Chi-square analysis indicates that reporting is not getting better over time on key evidence of concealment or outcomes assessor (p>.05). Standardized reporting could help decrease ambiguity about the important variables involved in evaluating quality of randomized controlled trials. Until that occurs, contacting authors may be the best approach available. However, one must weigh the potential gains of obtaining missing information against the reporting bias it might introduce.
Objectives: 1) To ascertain whether missing information is simply not reported, or not done. 2) To evaluate whether querying authors about missing data will change the quality score of a trial. 3) To assess if reporting is getting better over time. 4) To better understand what assumptions can be made about missing data.
Methods: Forty-seven studies meeting the inclusion criteria in a systematic review of acupuncture for chronic pain were evaluated for missing or ambiguous reporting of data about method of group allocation and blinding of outcomes assessor (to assess for double blinding). Four studies were determined to contain adequate information; 43 did not. Letters were sent to first authors of these 43 trials requesting the missing data.
Results: Twenty-one responses (49%) were received. 10 letters were returned to sender with no further contact details. Failure to respond was not associated with year of study. Prior to contacting these authors, the reviewers had scored (using (he Jadad Scale) 15 studies (32%) as high quality and 32 studies (68%) as low quality. The information supplied by the authors changed the quality rating on seven studies (33% of respondents); 14 (67%) remained the same.
Unchanged Low to lower Low to high High to higher
14 2 3 2
Of the 7 respondents queried as to whether outcomes assessor was blinded, 5 reported no. Of the 20 respondents queried on method of allocation, 2 reported alternate assignment even though authors put "randomization" in the paper. All 5 positive changes resulted from confirmation of adequate concealment of allocation
Discussion: If the information has not been reported, it probably has not been done. Chi-square analysis indicates that reporting is not getting better over time on key evidence of concealment or outcomes assessor (p>.05). Standardized reporting could help decrease ambiguity about the important variables involved in evaluating quality of randomized controlled trials. Until that occurs, contacting authors may be the best approach available. However, one must weigh the potential gains of obtaining missing information against the reporting bias it might introduce.