Modern biomedical research labs require grants, typically from the National Institutes of Health, to function. An increase in the number of scientists seeking NIH R01 funding combined with low R01 success rates has meant that most grant applications go through multiple rounds of revision and resubmission before being funded. This time spent on writing and revising grants takes time away from other pursuits such as conducting research, service, mentoring and teaching.
To better understand the factors affecting R01 applications and resubmissions, I examined the trends in the types of R01 grants submitted to the NIH. Applicants for R01s can submit a Type 1 New application or, if they currently hold an R01, a Type 2 Renewal application. Type 1 and 2 R01 awards are similar in length and size, but success rates for Type 2 applications have consistently been above 35 percent for the past 15 years, while success rates for Type 1 awards have typically been between 15 and 20 percent.
To begin, I calculated the number of Type 1 and Type 2 awards made each year from 1985 to 2018 (Fig. 1A; data downloaded from NIH ExPORTER). Type 1 awards increased at times that mapped loosely to increases in federal funding for the NIH: the budget doubling between 1997 and 2003, the ARRA funding boost of 2009 and 2010 and the recent increases in the NIH budget starting around 2015.
The number of Type 2 awards increased and decreased similar to Type 1 awards with two exceptions. Type 2 awards did not increase during the budget doubling period, nor have they changed in response to the funding increases starting in 2015 (Fig. 1A). Rather, aside from a brief spike from 2009 ARRA funding, the number of Type 2 awards has largely declined since 2003. This is despite Type 2 success rates remaining steady since the mid-2000s. For the success rate to remain steady while Type 2 awards decline, Type 2 applications must also be in decline. Some of these data have been previously presented on the NIH’s Open Mike blog.
Fig. 1: Changes in Type 1 and Type 2 NIH R01 awards. (A) Type 1 (blue) and Type 2 (yellow) awards made between 1985 and 2018. Ratio of Type 1 to Type 2 (B) awards and (C) applications. Red arrows indicate years in which NIH resubmission policy changed.
I next determined the ratio of Type 1 to Type 2 awards annually over the time frame. From 1985 to 1996, the NIH awarded about 1.2 Type 1 awards for every Type 2 award (Fig. 1B). The ratio increased between 1996 and 2000, and remained steady around 1.7 Type 1 awards for every Type 2 award until 2009. In 2009, the ratio began to increase again, and in 2014 the increase in the ratio became much steeper so that in 2018, the NIH was awarding 4.4 Type 1 R01s for every Type 2 award.
Information on grant applications is not typically part of the public data provided by the NIH, but it has published data on Type 1 and Type 2 R01 applications between 1997 and 2018. The ratio of Type 1 to Type 2 applications increased in the late 90s, was relatively flat between 2000 and 2008, increased again starting around 2009, and the increase became steeper in 2014 (Fig. 1C). In 2018, there were 9.8 Type 1 R01 applications for every Type 2 application. These data are consistent with the Type 1 to Type 2 award ratio (Fig. 1B).
The changes in the Type 1 to Type 2 award and application ratio were consistent across the institutes and centers. The ten largest ICs accounted for over 80 percent of Type 1 and Type 2 R01s made between 1997 and 2018. The ratio of Type 1 to Type 2 awards made by each of these ten ICs increased from 1997 to 2000, 2009 to 2014 and 2014 to 2018. The ratio during the 2000 to 2009 period, when it was largely flat for the whole NIH, saw relatively small changes in these ten ICs (Table 1).
Table 1:
One potential explanation for these changes is that 1996, 2009 and 2014 marked an influx of new PIs, who did not have an R01 to renew, thereby prompting an increase in Type 1 awards made relative to Type 2 awards. However, removing first-time R01 awardees from the dataset and reanalyzing the ratio with only established PIs showed similar changes to that for all R01 awardees (Fig. 2).
Fig. 2: Ratio of Type 1 to Type 2 R01 awards for only established PIs. Red arrows as in Fig. 1.
Another potential explanation is that changes in NIH grant resubmission policy affected the Type 1 to Type 2 award ratio. The first submission of a Type 1 or Type 2 award is known as an A0. If the A0 is unfunded, the PI can file a resubmission, termed an A1. If a PI has a current R01 nearing the end of its funding cycle and wishes to continue this work, a Type 2 A0 Renewal application can be submitted. However, as is stated by the NIH:
“If your renewal [Type 2 A0] and subsequent resubmission [A1] of renewal application are not funded, you must use the ‘new’ [Type 1] application type to compete for additional funding and continuity with your previous award will not be retained.”
Therefore, PIs are limited to two attempts at a Type 2 award before they must submit a Type 1 application, which they can now submit an unlimited number of (Fig. 3).
Fig. 3: Flow chart of NIH grant application submission and resubmission.
The inflection points in the Type 1 to Type 2 award ratio correspond to changes in the NIH resubmission policy (Fig. 1B, C). Prior to 1996, grant applicants could submit unlimited resubmissions to their Type 1 or Type 2 application. The 1996 policy (NOT-96-161) limited applicants to A0, A1 and A2 submissions. In 2009, the NIH eliminated the A2 resubmission and mandated that a PI could not immediately recycle their unfunded Type 2 A1 as a Type 1 A0 submission. Rather, the application had to be “fundamentally revised to qualify as new” (NOT-OD-09-003). In 2014, the NIH retained the no-A2 policy, but relaxed the restriction on the relationship between unfunded Type 2 and subsequent Type 1 submissions (NOT-OD-14-074), leading to what the NIH has deemed virtual A2 submissions.
If changes to the resubmission policy upset the Type 1 to Type 2 equilibrium, then a new equilibrium would be established only after all Type 1 awards have had a chance to be renewed. Because R01s last four to five years, then it should take four to five years for equilibrium to be reestablished. The change in the Type 1 to Type 2 ratio starting in 1996 took until 2000 to become stable, consistent with this prediction. The Type 1 to Type 2 ratio increase starting in 2009 should have ended around 2014, but the NIH changed the resubmission policy again in 2014. After the 2014 change, this model predicts the ratio should become steady after 2019. It will likely take until 2020 or 2021 to adequately test the prediction here, provided the NIH does not again change resubmission policy.
The reason for limiting the number of resubmissions in 1996 and 2009 were to fund “high quality applications earlier, with fewer resubmissions.” The reason for the 2014 change was due to the 2009 policy possibly disproportionately harming new investigators. Eliminating the A2 in 2009 did cause more A0s to be funded, but it is not clear whether changes in resubmission policy achieved the goals of funding applications earlier with fewer submissions. Whether under the 1996, 2009 or 2014 resubmission policies, was a PI more likely to follow an unfunded A1/A2 with a new A0 application or give up any chance of funding? The increase in Type 1 awards and applications relative to Type 2 awards and applications could represent, in part, an increase in the number of applications that are recycled from unfunded A1/A2 submissions. The acceleration after 2014 could reflect the relative ease of recycling an unfunded A1 for resubmission as an A0 compared to resubmission under the 2009 policy.
The NIH recently analyzed the effects of the 2014 resubmission policy on the submission of “virtual A2s”—A0 submissions that were derived from an unfunded A1. The analysis showed that roughly 25 percent of unfunded A1s were resubmitted as virtual A2s and the success rate for virtual A2s was similar to true A0s. However, the analysis was limited to A1 applications unfunded in fiscal 2014 and resubmitted as an A0/virtual A2 through mid-2016. Through the lens of the data presented here, the virtual A2 analysis was done on a population in the midst of an equilibrium shift, not at steady state. The NIH should consider repeating their virtual A2 analysis by following unfunded/resubmitted grants for a longer timeframe.
The data presented here suggest that NIH resubmission policy has a direct effect on whether New or Renewal applications are submitted and funded. The burst in Type 1 applications and awards coinciding with resubmission policy changes suggest a larger than appreciated fraction of the Type 1 application pool is recycled unfunded applications. If the goal of changing resubmission policy is to fund applications at earlier stages with fewer resubmissions, these data question whether these policies are meeting those goals.
A separate but related aspect that is the anecdotal evidence that some PIs view Type 2 submissions as not worth their time because there is a sense the bar for demonstrating sufficient progress during the previous grant period to warrant continued funding is unreasonably high. The perception of how study sections view Type 2 applications relative to Type 1, along with changes in resubmission policy that apparently promote an increase in Type 1 applications and awards, may be the driving forces behind the decline of Type 2 applications and awards.