An RBR Writing Program post by Eve Granatosky
In today’s hypercompetitive environment, properly evaluating the productivity and success of researchers is critical for tenure and promotion decisions, grant funding decisions and many other aspects of research and career advancement. Some basic metrics, such as a researcher’s number of peer-reviewed publications or amount of grant funding, fail to adequately capture the value of research. Applying new, more inclusive metrics to evaluate researchers could help evaluate a scientist’s productivity irrespective of accumulated resources or support. More specific metrics could also bridge the communication gap between academics and other scientific stakeholders and ensure that all relevant perspectives on research are considered.
A report from the Association of American Medical Colleges and RAND Europe summarizes a new set of metrics for assessing the value of research at medical schools and teaching hospitals. This initiative was a three-stage process. It began with a comprehensive review of existing metrics used in different contexts and by different audiences. The second stage brought together basic scientists, clinical researchers and public health experts to discuss which metrics were most relevant to them. The third stage solicited input from nonscientists on the metrics identified in the previous stages. Finally, the report summarized the results of a pilot study that applied the newly derived metrics to evaluate researchers at the University of Wisconsin-Madison.
A detailed list of 100 metrics was published alongside the report. One new set of metrics seeks to measure how scientific networks are established and grown, and how science is disseminated throughout a community. These types of metrics appealed to the largest number of academic and non-academic stakeholders in the AAMC/Rand study. Evaluations that use these metrics could potentially measure how well research results are translated into community, and larger societal, benefits.
While developing these new metrics is important, the accuracy and usefulness of the metrics is not known. The pilot study at the University of Wisconsin-Madison suggested that the participants had increased their number of citations compared to the world average, expanded their collaborations both on- and off-campus, and improved their research and communication skills. Although these results are impressive, similar studies will need to be done at other campuses to determine how well the metrics are truly providing a more refined picture of a university’s productivity.
A sustainable biomedical research enterprise relies on recruiting and retaining a diverse and talented cadre of scientists. Creating more illustrative, widely understandable metrics can improve communicating the value of research to scientists and nonscientists alike. In turn, the implementation of more accurate metrics to evaluate a researcher’s productivity and success could help ensure that the best researchers are accurately identified, encouraged and supported and provided with opportunities to contribute to the overall prosperity of biomedical research in the United States.
—
Eve Granatosky is passionate about promoting effective science communication and encouraging career development for graduate students. She is currently completing her Ph.D. in Biochemistry at the University of Notre Dame and is the co-founder of the Science Policy Initiative at Notre Dame. Follow Eve on Twitter (@granatosky) or contact her by email (egranato@nd.edu).
The RBR Writing Program is intended to help graduate students and postdocs receive policy writing experience. For more information, contact RBR Director Chris Pickett.
Leave a Reply