Background: There is a consistent body of literature documenting the problem of publication bias (the publication of study results based on the direction and statistical significance of study findings). However, most of this evidence is limited to self-report from authors and quantitative studies of publication bias that focus only on the direction of results. Little is known about the actual editorial process itself, or the process through which editors decide which articles to send out for review, how peer reviewers are selected for articles, how peer reviewer comments are taken into consideration, and what criteria or standards are actually used by editors when evaluating articles.

Objective: To describe the editorial processes at three leading biomedical journals, identify the factors that influence editors' decisions to publish manuscripts, and identify sources of systematic bias in the editorial decision-making process.

Methods: To examine the editorial process, we observed editorial practices and conducted semi-structured interviews with editors at three leading biomedical journals. The questions fell within four domains: 1) describing the editorial process whereby articles are considered, reviewed, and accepted or rejected, 2) describing the peer review process, 3) identifying factors that influence editors' decision-making process, and 4) describing editor characteristics at each journal. We examined the transcripts qualitatively to identify recurring themes within the domains.

Results: Editorial policies and procedures varied greatly among the three journals. Editorial committee meetings were often held weekly or biweekly, but varied in size (4-15 editors), purpose (general commentary and review of peer-reviewed manuscripts, statistical review, decision of final acceptance or rejection, appeals on rejected papers, or problematic issues such as research ethics and publication misconduct), and composition (all editors, senior editors only, a select group of editors, chairperson, the presence or absence of an ombudsman or visitors).

Journals used either closed peer review (reviewer identities concealed from author, n=2) or open (identities known between reviewer and author, n=1). Most manuscripts received between 2 and 4 reviews. Peer reviewers were selected by 1) the editors' personal knowledge and familiarity from past experience, 2) expertise (from a large database of previous reviewers cross-referenced by name and specialty), 3) references listed in the manuscript, and 4) suggested reviewers by authors.

The majority of submitted manuscripts were outright rejected (50-80%) by one or two editors prior to external peer review based on the following factors: 1) topic lacked novelty, 2) topic not suited for the journal's readership, 3) weak methodological quality, and/or 4) lacked impact to change current clinical practice.

Characteristics of the editors were 1) male (overall = 60%, chief editor = 100%), 2) editorial experience (range 2 to 36 years), and 3) a wide-range of medical disciplines, expertise, and interests.

Conclusions: Editorial practices and procedures vary widely among biomedical journals. A number of factors that significantly influence an editor's decision to accept or reject the majority of submitted manuscripts often occur before external peer review.

Acknowledgements: The authors thank the journal editors for their participation in this project. This study was funded by the National Institutes of Health.