Introduction: Despite of general availability of the scientific information, the treatment standards differ sometimes considerably among major centers. These differences in standards can be explained either by differences in the selection of published results or in the interpretation of published results or in both of it.

Objective: To examine to which extent differences in the interpretation of published results influence the definition of standards.

Methods: 42 German Cancer Centers were invited to submit their standards for three tumors. The submitted standards were re-distributed to another center to be checked for necessary changes. Next, the literature should be submitted which supports the requested changes. The requested changes plus the supportive literature was again re-distributed to a third center in order to assess the general scientific quality of the submitted literature on an scale of 0-100 and to decide how well the the enclosed literature supported the requested changes. Finally, a structured instrument (38 yes/no questions) was designed to describe the "strength of evidence of clinical trials (SECT)".

Results: Of the 15 submitted and re-distributed standards 4 were returned without suggested changes, 7 with suggested changes supported by references and 2 not supported by references. More than 50% of the colleagues felt that the requested changes were only partially or not at all supported by the references. The conventional assessment of the scientific quality of the references was quantified and varied extemely. The structured assessment (SECT) resulted in much more homogenous results.

Discussion: The assessment of the quality of reported trials should be standardized. The results will improve in three dimensions: in the quality of standards, in the quality of the scientific support of standards, and in the agreement on these standards.