Background: Systematic reviewers need to compare the design and results from multiple clinical trials that are often highly heterogeneous. This is a cognitively demanding task that has traditionally required the manual extraction and manipulation of data from journal articles. Computer-based visualization techniques from the field of human-computer interaction could help provide cognitive support for exploring heterogeneous trials, but use of these techniques requires that trial information be available in machine-understandable form. In previous work, we designed a machine-understandable "trial bank" called RCT Bank. We were able to capture a wide diversity of randomized clinical trials (RCTs) from different clinical domains into RCT Bank, and we showed how trial-bank reporting can be integrated with traditional journal publishing. We now describe the design and evaluation of an interactive visualization tool called CTeXplorer to help systematic reviewers explore complex design differences among RCTs. We use as our sample the RCTs included in a Cochrane review on prevention of mother-to-child transmission (MTCT) of HIV, a set of trials that are particularly heterogeneous in design.
Objectives: To design and test a visualization tool to help systematic reviewers better and more quickly understand design differences among RCTs on MTCT prevention.
Methods: We retrieved reports of 12 RCTs on MTCT prevention published between 1994 and 2006. We abstracted and stored the following into RCT Bank: 1) eligibility criteria and subject characteristics, 2) intervention details, 3) outcomes and results, and 4) methodological features. We defined "use cases" that described detailed information needs of systematic reviewers and trialists. We iteratively created and tested views for effective cognitive support, and programmed these views so that they are generalizable to trials from any clinical domain. We conducted qualitative exploratory testing with three target users to assess usability and to brainstorm on tool design. The users (an HIV trialist, a general internist/systematic reviewer, and a Cochrane editor) used CTeXplorer to complete a set of pilot-tested tasks and a questionnaire during a one-on-one session that was videotaped with tracked face and voice streams. The users also participated in a focus group with the design team. Data were analyzed using standard usability evaluation techniques.
Results: CTeXplorer organizes information from multiple trials into three frames on the screen: eligibility rules across the top, intervention details on the bottom left, and study outcomes on the bottom right. For example, users can see at a glance relative trial sizes, which drugs were tested in which trials and when during pregnancy and labor in the mother and the infant, and the timepoints at which maternal and infant safety and infant HIV infection were assessed. Users can manipulate information in one frame with automatic updating in the other frames. The users found the presentation and organization of information intuitive and easy to learn. Comments included: "I am very impressed and see practical applications" "I felt surprisingly comfortable" and "This tool can revolutionize how we look at things." Suggested improvements included visualizing trial results closely with trial design; ability to rank and annotate trials on trial quality; ability to save views; exporting data to other tools (especially Excel); and highlighting hypotheses not tested by the trials.
Conclusions: We have shown that computable trial information can drive an effective tool for visualizing heterogeneity in RCT designs. Target users were able to easily learn and use CTeXplorer to achieve a quick cognitive overview of a very heterogeneous group of RCTs on MTCT prevention. In addition to adding results, future work includes incorporating ontologies and reasoning heuristics to develop a more clinically sophisticated general RCT visualization tool.