2011 Discovery Grants Competition Aftermath

Anomalies in the results of the 2011 NSERC Discovery Grants competition provoked a flurry of activity nearly one year ago. My blog post from April last year reported on surprising results for several of my colleagues at Toronto. An email flurry among Canadian mathematicians culminated in a late April public statement which was eventually signed by 336 Canadian researchers, including 35 Fellows of the Royal Society and 27 Canada Research Chairs. In late May, a majority of the Evaluation Group for Section 1508 released a public letter to NSERC President Suzanne Fortier reporting on a “lack of fairness”, a “lack of transparency” and that their “confidence in the program, as currently administered, is regrettably shaken”. President Fortier gave a presentation to the Canadian Mathematical Community at the Summer CMS Meeting. President Fortier’s slides have not been released but there are notes and rebuttals by Walter Craig. Claims of grade inflation have also been analyzed.

Toronto Math Appeals of 2011 Results

Last May, seven Toronto mathematicians submitted appeals of their 2011 results. As far as I can determine, NSERC does not provide statistics on the number of appeals but seven math appeals was unprecedented from Toronto. Appeals are normally resolved within two to three months. As of late September, the decisions on the appeals were not yet announced. Toronto Math made email and telephone inquiries asking that decisions be made soon since successful appellants would need time to prepare new proposals in advance of the November 1 submission deadline.

Email correspondence revealed that Toronto’s appeals were handled differently than those from other universities. In particular, NSERC’s Isabelle Blain informed me that Toronto’s appeals were processed through a “pilot” program involving two or more appeals advisers. Three of the appeals decisions were announced in late September and the remaining four appeals were announced on October 19, 2011: 2 of 7 appeals were granted. The good news for the successful applicants was bittersweet since it set in motion a rushed effort to prepare proposals for the 2012 competition. Unfortunately, the reports from the appeals advisers for one of the successful appellants could not be consulted while preparing the 2012 proposals since they were not provided by NSERC until after the submission deadline. Toronto appeals were rejected in circumstances when one or more of the appeals advisers advocated for granting the appeal. Some Toronto appellants perceive the “pilot” appeals process to have been unfair. Conversations along these lines continue….

Repercussions: Uncertainty, Frugality and HQP

In the wake of the 2011 anomalies, many Toronto mathematicians who face renewal of their NSERC Grant in 2012, 2013 and even in 2014 are hesitant to commit funds toward hiring postdocs or toward training graduate students. Uncertainty has provoked frugality. Running a successful, even world leading, research program is no longer sufficient to justify an NSERC Discovery Grant. The new evaluation metrics, introduced by NSERC in 2009, require evidence that the principal investigator is successful at training highly qualified personnel (HQP). Taking frugal actions now will limit HQP production over the next few years resulting in a lower or zero grant next time: a vicious cycle which obstructs research advancement and training of young investigators. With the goals of breaking this cycle and maintaining the research vibrancy postdocs bring, the Department of Mathematics at Toronto has raised its funding support level (by 33%) for postdoctoral hires.

Looking Forward: Continued Instability

The 2012 Discovery Grants Competition results will be announced in the next week or two. Hearsay and anecdotal reports about the evaluation process have been reassuring. The Evaluation Group received recommendations jointly authored by the Long Range Plan Committee Chair Nancy Reid and the Math-NSERC Liaison Committee Chair Walter Craig. Similar recommendations were made in a December letter by a group of physicists to President Fortier. The Long Range Plan should help heal the rifts between the mathematics and statistics communities of Canada when it is released later this year. There are reasons to be optimistic in the short term.

In the intermediate term, there is likely to be continued instability with the peer review system at NSERC. Despite the 2011 anomalies for math and the ensuing kerfuffle, receiving similar complaints and suggestions for improvements by physicists, and faced with public criticisms by astronomers, chemists and engineers, NSERC staff insists that the new system is working well. The new evaluation system, the “conference model” with its “bins” and HQP touchstone, was introduced in 2009 and will be the subject of a five year review in 2014. Whether led by current or future NSERC staff, or by an energized scientific community, I anticipate substantial changes, and therefore instabilities, to the peer review process at NSERC in 2014 or 2015.

Shrinking Funds for Basic Research

Beyond the fairness of the evaluation process, there is the issue of shrinking federal investment in basic research. According to NSERC's 2010-2011 tables, expenditures on Discovery have been flat to decreasing when viewed in constant 2000 dollars over the past decade (data taken from Table 1). In contrast, the total NSERC budget has grown with the expansion occurring in programs aimed at commercialization rather than basic research.

 

Meanwhile, strategic recruitment leveraged by the CRC and CERC programs has increased the number of faculty submitting Discovery Grant applications (data taken from Table 27). Over the same time period, the number of successful applications has decreased (data taken from Table 27).

 

Table 50 shows that mathematicians do not really benefit from programs outside of the Discovery Grants. Mathematics, the “poetry of logical ideas”, is profoundly relevant to various industries. Canada’s research mathematicians need to find ways to share their expertise and leverage their Discovery Grant funds the way other disciplines do.