The Journal of Cell Biology has a scathing editorial on the reliability (and utility, actually) of impact factors.
This database appeared to have been assembled in an ad hoc manner to create a facsimile of the published data that might appease us. It did not.
And the killer?
Just as scientists would not accept the findings in a scientific paper without seeing the primary data, so should they not rely on Thomson Scientific's impact factor, which is based on hidden data [...] we hope that people will begin to develop their own metrics for assessing scientific quality rather than rely on an ill-defined and manifestly unscientific number.
There is a somewhat unsatisfactory response by Thomson, that completely misses several points, not least those of transparency and reproducibility.
We all knew that impact factors were crap anyway. Now we just have to convince the funding bodies.
HT to Alex.
Bootnote: JCB are pretty serious about this, actually. They have a link to usage statistics at the bottom of each article, which I think is reasonably new.
Comments
We all knew that impact factors were crap anyway. Now we just have to convince the funding bodies.
To me, this is one of the great mysteries of science. Why do scientists submit themselves to assessment using patently-flawed and useless metrics? The answer seems to be "because we have to be assessed somehow and this is the accepted way, even though everyone knows that it's rubbish".
It's just not good enough - we wouldn't accept these standards in our own work or that of others. We should be forming an angry mob with pitchforks and torches at the door of whoever is responsible, instead of this meek acceptance.
Anyway, we can hope that a few more articles like this will get a few people thinking about the issue and start a groundswell.
Posted by: Neil | January 8, 2008 04:21 PM
Hey, first of all congrats on getting into OpenLab 2007, and thanks for stopping by my blog.
Re the Impact Factor thing, check out this new free site, based on data from Scopus. I read about it in last week's issue of Nature. The JCB/Thomson brawl is mentioned there, too.
Posted by: Steve Matheson | January 11, 2008 12:33 PM
... and yet I use them weekly. Until someone comes up with another measure that is not subjective, my clients will continue to want to know the Thomson ISIS impact factor of a target journal. Do we have any ideas on a better method of ranking the penetration of journal articles that does not involve gut feel?
Don't forget that this 'science' you speak of features multiple approaches before an conclusion is considered canonical. TISI IF's have no peer - no credible challenge to their authority. Until we can come up with a better way of doing this, the the TISI IF's will continue to dominate discussions on the worthiness of journals.
Please try to ignore the fact that Thomson is one of the largest communication companies on God's given Earth ...
Posted by: Nige | January 13, 2008 08:50 AM