Scientists are more than their bibliometrics

By Helena Lucente

Metrics and other quantitative data assessments of effectiveness are ubiquitous throughout science. However, a common concern of researchers is the overreliance of committees and administrators on metrics that do not accurately portray all of the positive effects an individual can have on research and universities. A recent article in Nature highlights how the University of Utrecht has adopted a new set of criteria to address this problem. Instead of evaluating faculty on publications and citations, they are evaluating their scientific, education and outreach contributions to consider all the ways an applicant may be affecting the university community.

Through the efforts of early faculty and students, the University of Utrecht has developed a descriptive portfolio that asks about the applicants’ scientific and societal contributions to the institution along with a personal statement of who they are and their future plans. The new system gives reviewers a qualitative perspective of the full breadth of what faculty members have achieved instead of solely quantitative, and potentially misleading, metrics.

The work from the University of Utrecht comes as the hypercompetitive research environment, coupled with perverse publication incentives, has heightened suspicion of the veracity of scientific results. Furthermore, there is a growing concern among scientists that rewarding scientists for the quantity of publications, rather than the quality, is motivating them to publish a greater number of low-impact, low-quality, results. Are scientists adjusting their methods to comply with the pressures of an environment that requires a high output by publishing less rigorous science? While the data are far from conclusive, the changes at the University of Utrecht could foster a system that values community engagement and encourages high-risk and thoughtful research.

A reliance on bibliometrics also influences an early investigator’s ability to obtain funding. Rescuing Biomedical Research and the National Institutes of Health are promoting models that invest more in the creative and innovative research of young scientists. The example set forth by the University of Utrecht to evaluate both the quantitative and qualitative accomplishments of investigators is an impressive model and one worth watching as the research enterprise searches for methods to account for the full contributions of scientists to society.

—

Helena Lucente wants to bridge the gap between science and society through improving science advocacy, policy, education and communication. She is pursuing her Ph.D. in Oncological Sciences and M.S. in Clinical Investigation at the University of Utah. She can be reached on Twitter at @HelenaLucente1 and via email at Helena.Lucente@hci.utah.edu.

The RBR Writing Program is intended to help graduate students and postdocs receive policy writing experience. For more information, contact RBR Director Chris Pickett.