Article type
Year
Abstract
Background: In April 2019 Cochrane launched its Screen4Me workflow. A service available to Cochrane author teams that comprises three components: crowdsourcing, via Cochrane Crowd, machine learning through the use of Cochrane’s RCT classifiers and existing data – already known studies that have previously been classified by the Crowd. The aim of S4M is to help reduce the number of studies to assess for potential inclusion.
Objectives: This analysis sought to provide quantitative data regarding the use of the new workflow including: number of times S4M has been used, average mean reductions in size of search results set for each component of S4M, as well as average time taken. We also plan to perform a qualitative analysis to better understand how Screen4Me is being used by two main user groups: ‘the implementers’, and ‘the workers’.
Methods: For the quantitative analysis we downloaded the latest Screen4Me usage data from the CRS in March 2020. We collected data on: number of uses of S4M for specific Cochrane reviews, type of review (new or update), overall percentage reduction in number of search results, individual percentage reduction for each of the three components. For the qualitative analysis we will survey the Cochrane Information Specialists community and the Cochrane Crowd community. We will aim to get understand how easy or not CISs have found using Screen4Me and what recommendations for improvement they have, and for the Cochrane Crowd we will seek to ascertain satisfaction with participating in Screen4Me tasks in terms of task difficulty and task rewards.
Results: Screen4Me has been used 75 times by Cochrane Information Specialists across 15 Cochrane review groups. 60 uses of it have involved the known assessment and RCT classifier components alone, while 15 have also used the Cochrane Crowd. Overall mean reduction in search results sets post S4M was 54%. Average time taken where all three components were used was two weeks. We will present the results of the qualitative surveys at the Colloquium.
Conclusions: Screen4Me can reduce the number of search results for author teams to assess by a significant amount and is a robust and easy to use workflow available to Cochrane review groups.
Patient or healthcare consumer involvement: Anyone can join Cochrane Crowd and play a role in helping to identify studies for inclusion within Cochrane systematic reviews via Screen4Me.
Objectives: This analysis sought to provide quantitative data regarding the use of the new workflow including: number of times S4M has been used, average mean reductions in size of search results set for each component of S4M, as well as average time taken. We also plan to perform a qualitative analysis to better understand how Screen4Me is being used by two main user groups: ‘the implementers’, and ‘the workers’.
Methods: For the quantitative analysis we downloaded the latest Screen4Me usage data from the CRS in March 2020. We collected data on: number of uses of S4M for specific Cochrane reviews, type of review (new or update), overall percentage reduction in number of search results, individual percentage reduction for each of the three components. For the qualitative analysis we will survey the Cochrane Information Specialists community and the Cochrane Crowd community. We will aim to get understand how easy or not CISs have found using Screen4Me and what recommendations for improvement they have, and for the Cochrane Crowd we will seek to ascertain satisfaction with participating in Screen4Me tasks in terms of task difficulty and task rewards.
Results: Screen4Me has been used 75 times by Cochrane Information Specialists across 15 Cochrane review groups. 60 uses of it have involved the known assessment and RCT classifier components alone, while 15 have also used the Cochrane Crowd. Overall mean reduction in search results sets post S4M was 54%. Average time taken where all three components were used was two weeks. We will present the results of the qualitative surveys at the Colloquium.
Conclusions: Screen4Me can reduce the number of search results for author teams to assess by a significant amount and is a robust and easy to use workflow available to Cochrane review groups.
Patient or healthcare consumer involvement: Anyone can join Cochrane Crowd and play a role in helping to identify studies for inclusion within Cochrane systematic reviews via Screen4Me.