General

 A study reported in 1983 comparing peer rating of medical articles with citation counts (taken from Science Citation Index for the 5 years following the articles’ date of publication), involving 279 “first order papers” (all 1974 research papers abstracted in the 1975 or 1976 volumes of the Yearbook of Cancer, highest peer rating), 276 “second order papers” (random sample of the 1974 research papers that were only listed in the 1975 or 1976 volumes of the Yearbook of Cancer, next highest peer rating), and 315 “average” papers (no special peer rating) published in 1974 on the subject of cancer research randomly selected from Biological Abstracts, showed that the more highly regarded articles tended to be cited more than other articles to a statistically significant degree (significant at the .005 level). Specifically, the average percentage of papers cited per year was as follows:

                 first-order group                    73.7% of papers

                second-order group               72.7% of papers

                average group                        55.7% of papers

while the average number of citations per paper for the 5-year period was as follows:

                 first-order group                        30.59 citations

                 second-order group                  24.60 citations   

                 average group                           11.17 citations              (Source)

        Ibid. . . . showed thatcomparison of citation ratings for the 3 groups using only 1974 and 1975 citations (citations made during these 2 years took place before the Yearbook of Cancer could have been a factor in calling attention to the articles) indicated as well that the more highly regarded articles were cited more than other articles to a statistically significant degree (significance level at the .001 level). This suggests that the quality of the article itself was the factor leading to the generally higher number of citations.                   (Source)

 Academic

 A study reported in 1978 comparing peer ratings of graduate programs (Roose-Andersen ratings of graduate faculty in 10 scientific fields) with bibliometric ratings of those programs (number of papers published and “quality” of papers based on the papers’ citation ratings), involving analysis of 127,000 papers from 450 journals in 10 fields published during the period 1965-73 and programs in 115 universities, showed thatthe correlations between peer and bibliometric ratings were as follows (fields included—biochemistry, chemistry, developmental biology, mathematics, microbiology, pharmacology, physics, physiology, psychology, and zoology):

                 between peer ratings and total number of papers published in the programs, the rank correlations ranged from .635 to .898;

                 between peer ratings and “quality” of papers published in the programs, the rank correlations ranged from .275 to .834;

                 between peer ratings and total influence (product of total number of papers published and “quality” of papers), the rank correlations ranged from .647 to .910.

(All correlations Spearman rank correlations.)               (Source)

 A study reported in 1980 comparing the subjective rating by science faculty of the importance of the average article in a group of scientific journals compared to the rating of importance of the average article in those journals based on a citation count (faculty survey size: not given; responding: 298 or 25%, involving 56 journals in 10 scientific fields) showed thatthe agreement between subjective and citation-based rating of journals was quite good in most fields. Specifically, the average overall correlation between the subjective and citation rating was r = .78, with the correlations in the 7 most strongly correlated fields ranging from r = .76 (psychology) to r = .96 (geoscience). In the 3 least correlated fields the correlations ranged from r = .37 (electrical engineering) to r = .69 (physics).                         (Source)

 A study reported in 1983 investigating the relationship between the scholarliness of academic papers and their impact by comparing the number of references in each of 110 papers (taken from the American Sociological Review and the American Journal of Sociology during the years 1972-73) to the number of citations listed for each of the papers in Social Science Citation Index, 1972-81, showed thatthere was a very modest but statistically significant relationship between scholarliness and impact. Specifically, the correlation coefficient (partial gamma coefficient) was .26 and significant at the .03 level.                (Source)

Dr. David Kohl

 "Libraries in the digital age are experiencing the most profound transformation since ancient Mesopotamian scribes first began gathering and organizing cuneiform tablets."

Go to top