Background:

Comparative-effectiveness research has the potential to improve the evidence needed by patients and clinicians to make individualised choices among healthcare options. New evidence must be grounded in good science and methodology standards provide the basis for scientific integrity. Development of standards can benefit from stakeholder involvement. The Patient-Centered Outcomes Research Institute (PCORI) developed methodology standards with the goal of encouraging research and trustworthy information that helps patients and clinicians make decisions. These standards outline the minimal requirements for scientifically valid patient-centred outcomes research. A public comment process was used in the initial development of standards in 2012 and revision in 2016.

Objectives:

To assess the extent to which public comments influenced the original and revised methodology standards; describe the changes made to the standards in response to public comments; and, summarise the past experience and future plans for continuing to solicit and respond to public comments as part of developing methodology standards.

Methods:

This is a qualitative assessment of the public comments and resulting changes in standards. All
comments were organised by topic, analysed across all topics, and categorised by content and functional themes. We compared drafts to revised standards to assess the impact the comments had on the final version.

Results:

In 2 cycles, PCORI received 140 submissions with over 1500 individual comments. The 1st cycle contained more comments about the purpose and development of the standards, while the 2nd cycle focused on new standards, clarification of meaning, and simplifying wording. In both cycles, comments included suggestions for additional standards. Comments led to the addition of standards and revision of standards and text in every topic. Approximately 25% of the revisions were substantive, while 75% were to improve clarity.

Conclusion:

Public comments are effective for engaging stakeholders in creating standards for research. Use of public comments is expected to increase the influence and utility of these standards.