In a recent editorial in the Journal of Cell Biology the executive editors of The Journal of Cell Biology and The Journal of Experimental Medicine joined with the executive director of the Rockefeller University Press to test the validity of data associated with journal impact factors. For those unfamiliar with the calculation it is a measure of the number of cites to recent articles compared to the number of recent articles for a given period. In other words, the impact factor for a given year measures the average number of times a paper published in the previous two years was cited during the year in question.
For example, the journal impact factor for the journal Nature is reported to be 26.681 calculated from the following data:
Cites in 2006 to articles published in:
2005 = 25820
2004 = 26022
Sum = 51842
Number of articles published in:
2005 = 1065
2004 = 878
Sum = 1943
Calculation = Cites to recent articles / Number of recent articles = 51842 / 1943 = 26.681
The authors of the editorial tried to replicate the journal impact factors from data supported by Thomson Scientific (owners of the Web of Science) and were unable to report the same results. This puts into question the data supplied by Thomson Scientific and the overall validity of journal impact factors. One wonders if journal impact factors should be given the importance that they are within the scientific community given the fact that they might be calculated from suspect data.