The Tyranny of Citations

Comparing Citations Across Disciplines When using citations as any kind of measure of quality, it is important to recognize the huge differences between disciplines. For example, a very highly cited social scientist (say, one of Harvard’s best professors) might have a lifetime citation score of around 3,000–4,000, whereas a top molecular biologist could have a score of over 15,000–20,000. The discrepancies in citation levels across disciplines are demonstrated in the number of new cited references that appear in ISI every week. The sciences generate approximately 350,000 new cited references weekly, the social sciences 50,000, and the humanities 15,000. Bibliometric indicators have been used more consistently across the sciences than in the humanities and social sciences. Such use is most evident in the natural and life sciences. These disciplines publish more journal articles and have a higher prevalence of coauthorship. In the social sciences, it is now quite common for there to be up to three authors attributed to an article, but any more is unusual—whereas in the sciences, coauthors can easily extend to the tens or twenties.

the field of anaesthesia.Some of these loopholes have been closed.Nevertheless, journal impact factors should still be applied with some caution.

Comparing Citations Across Disciplines
When using citations as any kind of measure of quality, it is important to recognize the huge differences between disciplines.For example, a very highly cited social scientist (say, one of Harvard's best professors) might have a lifetime citation score of around 3,000-4,000, whereas a top molecular biologist could have a score of over 15,000-20,000.The discrepancies in citation levels across disciplines are demonstrated in the number of new cited references that appear in ISI every week.The sciences generate approximately 350,000 new cited references weekly, the social sciences 50,000, and the humanities 15,000.
Bibliometric indicators have been used more consistently across the sciences than in the humanities and social sciences.Such use is most evident in the natural and life sciences.These disciplines publish more journal articles and have a higher prevalence of coauthorship.In the social sciences, it is now quite common for there to be up to three authors attributed to an article, but any more is unusual-whereas in the sciences, coauthors can easily extend to the tens or twenties.
Another issue that skews disciplinary comparison is the publication rhythm and turnaround times of journals.Some medical disciplines have weekly journals; in history, the journals are often quarterly.In the discipline of economics it can take up to two years from the time an article has been accepted to the date of publication.In the arts and humanities, writing articles for journals is much less common.These disciplines tend more toward publishing monographs.

Conclusion
One of the most interesting, though possibly unsurprising, outcomes associated with the heightened awareness of citations is the extent to which they are being used to create league tables of top scientists.In 2005, Jorge Hirsch developed an hindex, essentially a method of counting citations, which he uses to identify and rank the most-cited physicists.For some scholars, counting one's own citations has by repute become almost obsessional.I personally know of a physicist who checks his numbers every single morning.
It could be argued that this level of citations awareness is somewhat unhealthy and overly competitive.Then again, maybe it is inevitable in a world that celebrates those who are first-to identify a fact or explain a phenomenon.It is worth mentioning at this point that for those who would like to improve their citation levels there is a very quick and easy method.Ensure that you place all your academic papers (and others) onto your website with live links to the full text.This will not only generate a few more citations for you, but also, and more importantly, it will get your work out to other scholars and generally disseminate your ideas more widely.This must be good for science.

Philip G. Altbach
Philip G. Altbach is Monan professor of higher education and director of the Center for International Higher Education at Boston College.See also "The Place of Citations in Today's Academy," by Amanda Goodall, in this issue-for a related perspective.
T he analysis of citations-examining what scholars and sci- entists publish for the purpose of assessing their productivity, impact, or prestige-has become a cottage industry in higher education.This approach has been taken to extremes both for the assessment of individuals and of the productivity and influence of entire universities or even academic systems.Pioneered in the 1950s in the United States, bibliometrics was invented as a tool for tracing research ideas, the progress of science, and the impact of scientific work.Developed for the hard sciences, it was expanded to the social sciences and humanities.
Citation analysis, relying mostly on the databases of the Institute for Scientific Information (ISI), is used worldwide.Increasingly sophisticated bibliometric methodologies permit ever more fine-grained analysis of the articles included in the ISI corpus of publications.The basic idea of bibliometrics is to examine the impact of scientific and scholarly work, not to measure quality.The somewhat questionable assumption is that if an article is widely cited, it has an impact, and also is of high quality.Quantity of publications is not the main criterion.A researcher may have one widely cited article and be considered influential, while another scholar with many uncited works is seen as less useful.
Bibliometrics plays a role in the sociology of science, revealing how research ideas are communicated, and how scientific discovery takes place.It can help to analyze how some ideas become accepted and others discarded.It can point to the most widely cited ideas and individuals, but the correlation between quality and citations is less clear.

international higher education citations and rankings
For some scholars, counting one's own citations has by repute become almost obsessional.I personally know of a physicist who checks his numbers every single morning.

An American Orientation
The bibliometric system was invented to serve American science and scholarship.Although the citation system is now used by an international audience, it remains largely American in focus and orientation.It is exclusively in English-due in part to the predominance of scientific journals in English and in part because American scholars communicate exclusively in English.Researchers have noted that Americans largely cite the work of other Americans in US-based journals, while scholars in other parts of the world are more international in their research perspectives.American insularity further distorts the citation system in terms of both language and nationality.
The American orientation is not surprising.The United States dominates the world's R&D budget-around half of the world's R&D funds are still spent in the United States, although other countries are catching up, and a large percentage of the world's research universities are located in the United States.In the 2005 Times Higher Education Supplement ranking, 31 of the world's top 100 (research-focused) universities were located in the United States.A large proportion of internationally circulated scientific journals are edited in the United States, because of the size and strength of the American academic market, the predominance of English, and the overall productivity of the academic system.This high US profile enhances the academic and methodological norms of American academe in most scientific fields.While the hard sciences are probably less prone to an American orientation and are by their nature less insular, the social sciences and some other fields often demand that authors conform to the largely American methodological norms and orientations of journals in those fields.

A Small Subuniverse
The journals included in the databases used for citation analysis are a tiny subset of the total number of scientific journals worldwide.They are, for the most part, the mainstream English-medium journals in the disciplines.The ISI was established to examine the sciences, and it is not surprising that the hard sciences are overrepresented and the social sciences and humanities less prominent.Further, scientists tend to cite more material, thus boosting the numbers of citations of scientific articles and presumably their impact.As Amanda Goodall points out, the sciences produce some 350,000 new, cited references weekly, while the social sciences generate 50,000 and the humanities 15,000.This means that universities with strength in the hard sciences are deemed more influential and are seen to have a greater impact-as are individuals who work in these fields.The biomedical fields are especially overrepresented because of the numbers of citations that they generate.All of this means that individuals and institutions in developing countries, where there is less strength in the hard sciences and less ability to build expensive laboratories and other facilities, are at a significant disadvantage.

The Use, and Misuse, of Citations
It is important to remember that the citation system was invented mainly to understand how scientific discoveries and innovations are communicated and how research functions.It was not, initially, seen as a tool for the evaluation of individual scientists or entire universities or academic systems.The citation system is useful for tracking how scientific ideas in certain disciplines are circulated among researchers at top universities in the industrialized countries, as well as how ideas and individual scientists use and communicate research findings.
A system invented for quite limited functions is used to fulfill purposes for which it was not intended.Citation analysis purports to measure the productivity of scholars and scientists-although productivity is defined narrowly in terms of published scientific articles cited in journals and, to some extent, books and other sources available to the ISI database.Scientists who are widely cited are deemed to be more productive and influential and presumably rewarded for their work.Hiring authorities, promotion committees, and salary-review officials use citations as a central part of the evaluation process.This approach overemphasizes the work of scientists-those with access to publishing in the key journals and those with the resources to do cutting-edge research in an increasingly expensive academic environment.Another problem is the overemphasis of academics in the hard sciences rather than those in the social sciences and, especially, the humanities.
Academics in many countries are urged, or even forced, to publish their work in journals that are part of a citation system-the major English-language journals published in the United States and a few other countries.This forces them into the norms and paradigms of these journals and may well keep them from conducting research and analysis of topics directly relevant to their own countries.
Citation analysis, along with other measures, is used prominently to assess the quality of departments and universities around the world and is also employed to rank institutions and systems.This practice, too, creates significant distortions.Again, the developing countries and small industrialized nations that do not use English as the language of higher education are at a disadvantage.Universities strong in the sciences have an advantage in the rankings, as are those where faculty members publish in journals within the citation systems.
international higher education citations and rankings 4 Academics in many countries are urged, or even forced, to publish their work in journals that are part of a citation system.

Conclusion
This article criticizes the unsophisticated use of citation analysis for the evaluation of individuals, departments, institutions, and systems.The misuse of citation analysis distorts the original reasons for creating bibliometric systems.Inappropriately stretching bibliometrics is grossly unfair to those being evaluated and ranked.The "have-nots" in the world scientific system are put at a major disadvantage.Creative research in universities around the world is downplayed because of the control of the narrow paradigms of the citation analysis system.This system overemphasizes work written in English.The hard sciences are given too much attention, and the system is particularly hard on the humanities.Scholarship that might be published in "nonacademic" outlets, including books and popular journals, is ignored.Evaluators and rankers need to go back to the drawing boards to think about a reliable system that can accurately measure the scientific and scholarly work of individuals and institutions.The unwieldy and inappropriate use of citation analysis and bibliometrics for evaluation and ranking does not serve higher education well-and it entrenches existing inequalities.

Tarla Shah
Tarla Shah is managing editor of Higher Education Digest and on the staff of the Centre for Higher Education Research and Information, at the Open University.Address: Centre for Higher Education Research and Information,Open University,44 Bedford Row,London WC1R 4LL,UK. Email: T.Shah@open.ac.uk.T his report provides comparable cross-national data on affordability and accessibility of higher education.It shows how different countries perform on a spectrum of indicators of affordability and accessibility and allows nations to see how well they are doing relative to other countries around the world.The report also assigns different rankings to countries' efforts in making education accessible and affordable.
Including both costs and resources, the report uses the following sets of indicators to look at affordability: costs as a fraction of ability to pay support as a fraction of ability to pay, support as a fraction of costs, and cost minus support as a fraction of ability to pay.The indicators used for accessibility are participation rates; attainment rates; the educational equity index (the quantification of educational inequality by measuring the degree to which students from high-socioeconomic-status backgrounds-as measured by paternal education levels-are overrepresented in higher education; and gender parity index. The affordability section of the report looks at data on afford-ability of higher education in 15 countries.The report compares countries on six different measures of affordability (as a percentage of ability to pay): education costs, total costs, net costs, net cost after tax expenditure, out-of-pocket costs and out-of-pocket costs, after tax expenditures.These taken together provide the following weighted overall affordability ranking: (1) Sweden, (2) Finland, (3) the Netherlands, (4) Belgium (Flemish Community), ( 5) Ireland, ( 6) Belgium (French Community), (7) Austria, (8) Germany, (9) France, (10) Italy, (11) Canada, (12) Australia, (13) United States, ( 14) United Kingdom, (15) New Zealand, and ( 16) Japan.
The analytical findings of the study on comparative affordability reveal a number of trends.Sweden is the most affordable country because of its combination of low educational costs, generous grants, and high take-up of loans.Finland and the Netherlands also do well because of low to middle educational costs, generous grants, and reasonable but limited loan programs.Because of limited student aid programs, the rest of continental Europe fares only moderately well despite low educational costs.The United Kingdom and New Zealand are near the bottom of the ranking because of high costs and low national incomes.
The findings on comparable accessibility suggest that the Netherlands and Finland have high participation rates and good or excellent gender parity scores.Finland's high score is largely due to its very high participation rates.The Netherlands gets the top spot because of its excellence in education equity and gender parity.The United Kingdom, the United States, Canada, Australia, and Ireland cluster in the mid-to-high zone of the rankings, which demonstrates striking evidence of policy congruence across a shared linguistic zone.Germany, Belgium, and Austria fare well in terms of gender parity index, but are at or near the bottom of the other three accessibility measures.None has a particularly high participation or attainment rate, and all of them have student bodies that are elite relative to the national make-up.
Overall, the report concludes that Finland and the Netherlands are the "undisputed success stories" of the survey in terms of both accessibility and affordability.Both have large student bodies, high attainment rates, extensive grant pro-