Working Scientist podcast: Inside the NIH grant-review process
Elizabeth Pier tells Julie Gould about her research into agreement levels among reviewers evaluating the same NIH grant applications.
Julie Gould and Elizabeth Pier discuss how the US National Institutes of Health grant review process works.
In this first episode of a six-part weekly series about funding, Julie Gould outlines the US National Institutes of Health (NIH) grant-review process and the extent to which reviewers evaluating the same applications agree or disagree. Is the current system the best way, she asks Elizabeth Pier, lead author of a March 2018 paper published in Proceedings of the National Academy of Sciences, Low agreement among reviewers evaluating the same NIH grant applications.
This episode concludes with a slot sponsored by the European Research Council. Jean-Pierre Bourguignon, its president, outlines the organization's role and remit as a grant funder.
Julie Gould and Elizabeth Pier discuss how the US National Institutes of Health grant review process works.
Happy 2019! I hope you’ve all managed to take some time to celebrate. As it’s a new year and new years often come with a makeover in one form or another, the Nature Careers team decided to give the podcast a makeover. As well as a new name, we’ve also got a new format. So, instead of our monthly episodes, we’re going to be producing more episodes in 2019 and grouping them together into different series, featuring six weekly episodes followed by a short break. So, here’s series one – funding – and as an added extra, each episode in this series will end with a ten-minute sponsored slot from the European Research Council. So, without any further ado, let’s go....
Hello, I’m Julie Gould and this is Working Scientist, a Nature Careers podcast.
Grant funding plays such an overwhelming role in the career of an academic scientist, and the funders are all too aware of it. Now, I know that all researchers spend many sleepless nights and cups of coffee writing grant proposals, so when I first started doing the research for this series, I wanted to find the best experts to give you the best tips on how to write the best grant proposals to make things a little bit easier for you.
But then I came across a research paper that made me stop and reflect. In March 2018, Elizabeth Pier – who was then a PhD student at the University of Wisconsin, Madison, in the Educational Psychology department – published a paper as part of her thesis in PNAS.
The paper was entitled "Low agreement among reviewers evaluating the same NIH grant applications." When I first read this title, I thought, "Wait a minute, I thought the idea of the whole funding process was that the top proposals were being funded, the ones where everyone in the peer review system agreed that these were the best ideas supported by the best researchers to do the work."
But clearly, this title shows that there’s something else going on in the background, so I wanted to find out more. The research was funded by the National Institutes of Health, which commissioned an independent study to examine the potential for bias to enter into the peer review process.
The overarching goal of the whole project was to look for evidence of gender or racial bias, based on the characteristics of the PI or the application, and where in the process these biases might enter.
Now, this particular piece of research from Pier is just one of the studies. It recreated a peer review panel to see how these meetings unfold and how they affect the decision-making process. Using previously accepted NIH project proposals, Pier explored this as part of her research. But before we go any further, it’s worth me outlining some of the basic steps of how the NIH proposal review system works.
Now, just so you’re clear, these steps are the bare bones and they miss out a lot of the details, but they should give a flavour of what happens once you hit submit.
So, the NIH uses a two-stage review process.
In the first stage, between two and five reviewers individually evaluate each grant application, and they rate them using the NIH’s nine-point scale, with one for exceptional and nine for poor. They also record what they feel are the application’s strengths and weaknesses.
The reviewers will then meet for what’s called a study section meeting to discuss their preliminary ratings. The discussion only looks at the top half of all the applications they have evaluated. The study section members then collectively assign a final rating, and this is averaged into a final priority score.
So, that’s stage one. Then in the second stage, members of the NIH advisory councils use this priority score and the written critiques from the reviewers to make funding recommendations to the Director of the NIH institute or centre that awards the funding.
So, given all that, I spoke to Elizabeth Pier who now works as a research manager at Education Analytics to find out more about her research.
In this particular study, I was really interested in looking at the degree of agreement between different reviewers and what is even happening before the reviewers come together and how are reviewers going about scoring these applications based on their assessments. So, another way of putting that is: Are the reviewers agreeing not only on the score that they assign, but are they also identifying similar strengths and weaknesses in the critiques that they write prior to the meeting, and also what’s the relationship between that numeric score and the written evaluation?
So, there were some sobering results...
We found that numerically speaking, there really was no agreement between the different individual reviewers in the score that they assigned to the proposals. We also found that when we were looking at the relationship between the strengths and weaknesses, written proposal, and the score that was assigned, we did see a relationship between the number of weaknesses that a reviewer would identify in their critique and the score that the reviewer assigned, but that relationship between the weaknesses and the score doesn’t hold up between different reviewers.
Another way of saying this is that the individual reviewers were really consistent – the more weaknesses they identified in the proposal, the lower the score awarded. But unfortunately, it appeared that each reviewer had a different idea of what a weakness is and what score that meant the proposal would ultimately be given. So, what this means is…
...we can’t really compare the evaluations of different reviewers and the degree of disagreement that we see in the scores seems to be a reflection of a different sense of calibration in what constitutes a bad score versus a good score.
The reviewers do come together for a meeting to discuss the papers based on the initial reviews, and in the meetings that Elizabeth Pier recreated…
As you would predict and as people told us based on their intuition participating in these kinds of meetings, the range of scores does get smaller after discussions, so there’s a degree of consensus building within individual peer review panels, but the agreement between different panels actually got wider after discussion, and we had a unique opportunity here because we had four different panels that were evaluating the same applications.
So in practice each application is only evaluated by one study section but for the purposes of this study we exploited that we had these four different groups looking at the same proposals.
And so, in the process of building consensus within a given panel, different panels actually went further apart.
So really the outcome that you’re coming to is that it’s potentially better that these reviewers don’t meet?
Our studies haven’t indicated any value or benefit in the sense of improving the consistency or reliability of the process.
But what about the variability in the quality of the proposal being discussed, doesn’t that make a difference?
But we had to ask people to donate their applications and the summary statement that they received to us and the donations that we received just happened to be funded. And so, we tried to say that above a certain quality threshold, our results suggest that it’s essentially a random process and the meeting doesn’t seem to remove that randomness.
However, I will say a caveat to that caveat is that the applications that get discussed in the meeting have already gone through triage, so only the top 50% of applications based on their preliminary score even get discussed in the meeting. So, what we are talking about is given that top 50% of proposals, after you’ve already excluded the ones that really have no chance of being funded initially, there really is a lot of randomness, but even more so, there’s already randomness such that the applications that have been weeded out so to speak and don’t get the opportunity to be discussed in the meeting might actually have a lot of merits. #
Had it been assigned to a different panel with different reviewer it very well could have gone on to be discussed.
So, what you’re saying really is that luck plays a very large role in whether or not your research gets funded.
Yes, that is what our results suggest. Above a certain degree, if you have a relatively competitive application, there aren’t any major issues that would immediately disqualify it to any kind of representative effort in the field, there’s a great deal of randomness and luck that we find in determining who does and does not get funding.
So, what does that mean for all those people who are spending all these hours and hours and hours and hours and hours on getting their funding applications organised and sorted and written up? I mean my heart goes out to them...
Yes, my heart does as well. I mean it’s one of the reasons I studied this for my dissertation because it’s incredibly important for individual careers and also incredibly important for the progress of science, right?
We want to make sure that the most deserving ideas are getting rewarded and funded and that it’s not just picking out of a hat.
So, I mean there are a couple of pieces of silver lining. I think that we see evidence, especially as grants get resubmitted, that being responsive to reviewers’ critiques can play a strong role in conveying to reviewers improvement over time, and so there is something to be said for if you get rejected or you don’t get funded, having some tenacity and resubmitting that application and doing everything you can to address the reviewers’ critiques and feedback.
It can make a potential difference. I also think that it’s important for folks not to take it personally. As academics, we definitely are used to rejection and used to plenty of times when we think we have really great ideas and reviewers of manuscripts or of grant applications don’t seem to agree with us, so I would encourage people to take a little bit of solace in that it’s not necessarily a reflection of the quality of the ideas but it’s more kind of a feature of the process.
How would you suggest then that the process is improved?
There should be some assessments of whether what some scholars have called a "modified lottery system" could do.
So, the idea being that there’s some initial screening process that experts do conduct to make sure, like I said, they’re kind of weeding out any really problematic proposals, things that are just wildly out of left field in terms of being feasible to complete given the budget or things like that.
And then after that kind of initial screen then it really is just a random selection. And the reason I think that would be an improvement is because if the process is already random above a certain quality threshold, which our study suggests it is, we might as well save the money and the time involved to convene thousands of people and spend millions of dollars to have these meetings if the outcome is essentially the same as a random process.
Now, we’ll touch on the idea of a lottery-style funding system later on in the series, but what we can say now is that change is going to be slow – it always is in academia.
Is there anything that can be done in the meantime, before this lottery style system or something completely different is created?
Starting to accept the fact that it’s not a completely objective process, that humans are fallible, they are subjective, and when you’re asking experts to make very complex judgements about the potential likelihood of success of a project, that’s a really difficult decision that’s going to bring in a lot of heuristics and biases that go into their decision making.
My final question to Elizabeth was what advice have you got for anyone who’s currently writing a grant proposal to the NIH.
One piece of advice, which is probably pretty obvious but I will say is backed by our findings, is that weaknesses are much more predictive of the score that reviewers will assign, rather than strengths. So, what that means is minimise as many weaknesses as you can.
So, after all of that, I’m intrigued. What do you think? How would you feel about a more lottery-style funding system? Please send in your thoughts to the Nature Careers team which you can do via Twitter, Facebook and LinkedIn. And over the next two episodes, I’ll be speaking to different experts on how to minimise the number of weaknesses within your funding application, in the hopes – fingers crossed – that you’ll have a bit more success and a bit more luck.
Now, that’s all for this section of our Working Scientist podcast. We now have a slot sponsored by and featuring the work of the European Research Council. Thanks for listening. I’m Julie Gould.
So, my name is Jean-Pierre Bourguignon and my title is President of the European Research Council, which of course is supported by the European Union as through the European Commission.
I’m a French mathematician, I should say. I spent most of my career in the CNRS (Centre National de la Recherche Scientifique).
My field was differential geometry, but did a lot of work actually at the boundary of theoretical physics, general relativity and Dirac operators and these kind of topics, but still always as a mathematician.
The European Research Council is actually an interesting story. It was created in 2007, so it’s now 11 years of age, and it was a long process. Myself, the first time I heard about the possibility of having an ERC was 1995, and it was a long effort by the scientific community, and step after step we had to convince people in the Commission, people in the European Council – namely the countries – that they should support such a project, but still it has a lot of very specific characteristics, particularly the power which has been given to its scientific council is considerable.
It really was an innovation and the council has the responsibility of deciding on how to spend the money and how to do the evaluation. This is unique in the setting of the European Commission, that a group of 22 scientists are given such a responsibility and of course as President of the European Research Council, I have some very specific ones, which is to confirm the list of people who are granted and really guarantee the quality of the work done.
The mission of the ERC was really to make Europe more attractive, to be a place where science can develop really in the most ambitious way and to push the ambition, particularly of young people, upward, that is to make them independent early enough and to take their vision on board.
You know, we are at the stage of giving 1100 research grants this year, which is of course a very significant amount of money. The budget is now really over €2 billion per year, and we are covering all fields of science – that is physical sciences, engineering, including maths, computer science, and so on, life sciences, social sciences and humanities.
If you want to know what you are doing, you need to talk and meet and discuss with the people you are funding and so I do travel a lot, particularly in Europe, to meet the people we call our grantees – the people who get the grants from the ERC – and this part of my job is really extremely worthwhile and extremely rewarding because the selection process is a very tough one – the typical success rate at this moment is 13%.
And it means that people have all proposed very ambitious projects that are conditioned to be successful at the ERC, high risk again, we need to encourage the panels who are selecting people to really accept to take risks. And that’s one thing I hear regularly from grantees, telling me, ‘I submitted a very similar project to my national agency but then I was not funded, it was considered too risky, then I submitted to the ERC, and then the ERC funds me,’ so it makes a big difference.
Another component which is very important in our strategy is the fact that the clause, which has been put in place by the European Research Council.
Really, we have three categories at the moment for the individual grants, which is the starting grants, consolidator grants and advanced grants, and it means timed to PhD.
So, starting grants – to 2 to 7 years, consolidator grants – 7 to 12 years, and advanced grants – there is no condition, it just means people who are already confirmed, and while doing that it means that in the end we are dedicating typically two thirds of our budget to the younger people, people who are typically below 40 years of age.
Very often people get the belief that really if you are not from one of the leading research institutions in Europe, then you have no chance. This is not the case. I mean the institution which is your host institution is not part of the evaluation, which is really the key for the evaluation is the project.
You have to show that you have thought of what kind of resources will be needed, and you describe them, but this is not the institution as such which is very important.
So, it means that in particular in terms of the support we give, part of it could be also buying expensive equipment if you need them and if it’s not available in your institution.
So, we consider the project, not just as helping the people, but helping the people also to set up the environment which will make it possible to get the project through.
So, this is sometimes one misconception that people have, that they get the feeling that if they are coming from a smaller place, they have no chance.
The number of institutions the ERC has been signing with is close to 800 now, so of course it’s quite a significant number of institutions based in Europe and of course some of the leading ones got more grants than others, but definitely even small institutions have been very successful at the ERC.
One of the key things that the ERC is doing is empowering researchers. This is something very, very important for us, and a very good example for this is one of the specific characteristics of the ERC programme which is called portability, but the host institution is not part of the selection criteria – it’s just here to make sure that there is a legal body that is able to receive the contract and it gives a lot of power to the researchers.
And one of the typical powers, what I mentioned, portability, which means that the researcher can change the host institution if he or she feels that it’s not given the proper treatment, or maybe they could give other personal reasons to do that.
This is the whole philosophy behind it – we really want the researchers in the driver’s seat at the level of the council but also at the level of how they run their contract, and of course there is an institution behind it because you want to be sure that there is a legal basis for this, but we really want the researchers to be able to do their research in the best possible conditions.
The map I have in front of me, which is the map of the world, has on top of it one of our mottos which is ‘Open to the world.’
One of the conditions to be funded is that you have to spend at least 50% of your time in Europe, but you can be from any country, and we want to be sure that Europe is the leader to tackle some of the most challenging scientific problems.
At the very beginning, we could notice that the percentage of women who were applying to the ERC was less than the percentage of women in the scientific community, and we felt this was definitely not adequate.
Also, we had for the ones who applied, the success rate was definitely lower than the success rate for men. Through very sustained efforts, and identifying the issue from the very beginning, I think we made very significant progress.
So, first of all, the percentage of women applying to the ERC has been steadily growing.
We are now basically at the level where the percentage of women applying to the ERC is very similar to the level of the percentage of women in the age group of the different goals we have.
So, from that point of view, I think we really achieved something which means that there’s no some kind of resistance or reluctance of women to apply, so this is one step.
And then, of course very important but I think the two are linked – the fact that in recent years, the ERC women have been on average more successful than men.
It’s a very slight difference but since we started the situation shows the opposite.
We are very pleased that all the efforts we made, particularly to tackle complicit bias or various other things have been more or less successful.
I’ve been visiting many, many countries in Europe, in particular countries in the "EU13," because I feel you need to understand the real situation people are exposed to in the various countries, and they are the ones who joined Europe the most recently, and most of them located in the eastern part of Europe and it’s very important to realise that actually that situation can be quite different from one country to the next.
It has to do with teaching load, it has to do with the power structures in the institutes, it has to do of course with the support which is available to people, so for me it was very, very important to meet the researchers because that’s for me the key point.
Also, to meet the authorities in these countries and to understand in which environment they are operating because I think that’s the very best way.
We, at the level of the European Research Council, have also introduced some help, in particular by encouraging the various countries, could be also regions, to support possibilities of researchers in typically underrepresented regions (it could be EU13 countries) – to really give them some possibility of spending some time with the support of their countries or their region, in ERC teams, so that they understand what it takes to submit a proposal, but also to understand better, to really also test their ideas with other people so that then they have a much better idea what it takes to submit a proposal and therefore they are better prepared personally, not just intellectually, to really submit a proposal in good conditions because they have seen what a difference it makes and also what also kind of effort you have to put in if you want to be successful.
Originally posted on Nature Careers - 04 January 2019 - https://www.nature.com/articles/d41586-019-00016-0
- Career Advice
From design to lift-off: blasting experiments into outer space
On 25 July, an experiment that Charles Cockell had spent years planning blasted into space. A SpaceX rocket blasted off from Florida, heading to the International Space Station (ISS). It carried 18 bioreactors, each the size of a deck of cards, that would be used to study whether bacteria could mine useful minerals on the Moon, Mars or asteroids. Getting the experiment off the ground (literally) was “one of the most exciting things I’ve experienced”, says Cockell, an astrobiologist at the University of Edinburgh, UK. But the process of getting from proposal to lift-off can be long and involved: Cockell’s biomining experiment was more than 11 years in the making. There are various routes to the ISS, but most go through one of the five space agencies — those in Canada, Japan, Russia and the United States, and the 11-nation European Space Agency (ESA) — that support the space station. These agencies periodically release calls for research proposals that can help to meet their space-science goals. GETTING SCIENCE INTO SPACE Each national space agency involved in the International Space Station (ISS) issues periodic calls for proposals for experiments that can be conducted on the station. More information can be found at the links here. NASA Calls for proposals are listed on the NASA Solicitation and Proposal Integrated Review and Evaluation System. The ISS US National Lab issues calls for proposals, but also accepts independent submissions at any time. Canadian Space Agency The Canadian agency’s research opportunities are listed on its website. European Space Agency (ESA) ESA’s ISS research programme is part of the European Programme for Life and Physical Sciences in Space. Japan Aerospace Exploration Agency The Japanese agency supports research using its Kibo module on the ISS. Roscosmos Calls for proposals are listed on the Russian space agency’s website, in Russian. Commercial opportunities The space station is opening up to more commercial research, which can allow faster and cheaper access to space for experiments that have commercial applications. Proposals are still handled by NASA and ESA. Calls for proposals are listed on the NASA Solicitation and Proposal Integrated Review and Evaluation System. The ISS US National Lab issues calls for proposals, but also accepts independent submissions at any time. Canadian Space Agency The Canadian agency’s research opportunities are listed on its website. European Space Agency (ESA) ESA’s ISS research programme is part of the European Programme for Life and Physical Sciences in Space. Japan Aerospace Exploration Agency The Japanese agency supports research using its Kibo module on the ISS. Roscosmos Calls for proposals are listed on the Russian space agency’s website, in Russian. Commercial opportunities The space station is opening up to more commercial research, which can allow faster and cheaper access to space for experiments that have commercial applications. Proposals are still handled by NASA and ESA. “Most of our research has an eye toward enabling exploration, but it can also have terrestrial benefits,” says Craig Kundrot, director of NASA’s Division of Space Life and Physical Sciences Research and Application. The US portion of the ISS has been designated a national laboratory, meaning that it’s available for any research that would benefit from access to space, even if the project is not aimed at advancing space exploration. Cockell’s experiment is intended to study how future Moon or Mars missions could use bacteria to extract materials from rocks in space, including minerals and metals for construction, water for rocket fuel, and soil. Around 600 experiments are conducted each year on the station, which has extremely limited space, power and time allocated to astronauts to work on experiments. The payload specialists for research on the ISS — those who plan what to send and when — have to juggle all of these factors as they decide which experiments can go when, and how they will all fit together. “They’re playing tetris on a daily basis,” says Kundrot. Getting chosen by a space agency is just the start of the process. Then comes the tricky business of designing an experiment that can be packed into a rocket, blasted to the station and conducted by an astronaut who has hundreds of other responsibilities each day. It must also survive re-entry into the atmosphere and recovery, sometimes from hard-to-reach places such as the middle of the Pacific Ocean. Luckily, there are teams of engineering contractors and payload specialists that take the lead on making the experiment function within the limits on mass, size and power consumption. “As a scientist, you’re not responsible for getting it to work. There’s a whole team who support the technical aspects,” says Monica Driscoll, a molecular biologist at Rutgers University in Piscataway, New Jersey. She was involved in a project running from December 2018 to January 2019 that used the nematode worm Caenorhabditis elegans to study the neurological effects of space flight. Depending on the experiment, the engineering challenge can be relatively simple or fiendishly complicated. Physical-science endeavours tend to take longer to plan because they need bespoke equipment, whereas those in the biological sciences can often repurpose gear that worked in other projects. Driscoll had the benefit of drawing on the experience of previous work on C. elegans in space, but Cockell’s bioreactors had to be designed from scratch — part of the reason for the long timeline. On the experimental-design side, the challenge is to keep the project as simple as possible while getting useful results. Something as basic as using a freezer, which wouldn’t require a second thought in a lab on Earth, can add another layer of complexity. The freezer on the ISS, which is designed to operate in microgravity, has a smaller storage capacity than a freezer in a domestic refrigerator, limiting the number of experiments that can use it. It has a waiting list, says Kundrot. Plus, you then have to work out how to keep the samples frozen during re-entry and recovery. That’s why Cockell’s team decided to forgo freezing its samples and instead opted to keep them in cooling packs for the return to Earth. “Even simple things are complicated,” says Cockell. “If you want an astronaut to turn a knob, it has to be in their schedule.” The experiment then undergoes several dry runs on Earth. One to verify that the science will work — for example, that Cockell’s bacteria were able to grow in the bioreactors — and then another, more detailed run-through of the operation of the experiment as if it was actually on the station. Once it has passed those tests to the satisfaction of the researchers, the space agency and the engineering contractors, the experiment goes to a launch site, such as the Kennedy Space Center in Merritt Island, Florida, and is loaded onto a rocket. That, says Cockell, is when the excitement of what you are doing really hits home — as your work is readied to leave the planet. “The station is not actually that far away — just 400 kilometres up, less than the distance from Edinburgh to London,” says Cockell. “But it’s the frontier of physical difficulty.” Cockell’s experiment is now back on Earth, and he and his team are getting started on analysing the results. But they are already planning to go back to space. In two years, they will conduct another version of the experiment, this time bypassing the space agencies and using commercial contractors — Kayser Space in Didcot, UK, and its sister company Kayser Italia near Livorno, Italy. For £170,000 (US$207,000), the researchers have bought access to the ISS for their experiments, without having to rely on the ESA peer reviewers to approve their work. Cockell says that it’s “just like buying a plane ticket to do research in another country”. Opportunities like this are opening up space research in a way that would not have been conceivable just a few years ago, he says. “It’s exciting for future generations. Within the next few years, this is something that a lot more people will be able to do.” Originally posted on nature.com on 11th October 2019 - https://www.nature.com/articles/d41586-019-03097-z
- Career Advice
How to get a funded internship in industry
As part of a broad mission to prepare science students for careers outside academia, the US National Science Foundation (NSF) has expanded a funding initiative to support master’s and PhD students for six-month internships in companies, government laboratories and non-profit organizations. The INTERN supplemental-funding opportunity, launched last year for select departments within the NSF, will now be open to almost every graduate student supported by an NSF grant, says Prakash Balan, programme director in the NSF’s Directorate for Engineering. At a time when available US industry positions far outnumber job openings in US academia, the internships can give students real-world training for their futures, he adds. “Opportunities like this give students exposure and experience at a time when it matters most,” he says. The programme also fits in with the agency’s overall agenda. “NSF has a long-term vision to foster the growth of a competitive and diverse workforce,” Balan says. “We want to advance the science and innovation skills of the nation at large.” The expanded programme provides up to US$55,000 to support a student for six months. The sum is meant to cover travel, tuition, stipends, materials and other expenses. The student’s supervisor can use up to $2,500 of the award to visit the site hosting the internship and co-mentor the student. The NSF has pledged to support up to 200 students in each of the fiscal years 2019 and 2020, although Balan says that more awards could be provided if the demand is great enough. To be eligible, students must have completed at least one year of their master’s or PhD programmes. Jennifer Weller, programme director of the NSF’s Directorate for Biological Sciences, says that she expects to receive many applications from biology students. “I’ve already had 20 phone calls asking for more details,” she says. Weller explains that she worked for five years in industry (at the biotechnology company PE/Applied Biosystems) before eventually returning to academia. Fostering the flow of talent and ideas between academia and industry should be a top priority for the agency, she says. Interest in the INTERN initiative goes both ways. The programme began after corporations contacted the NSF asking for help finding student interns, Balan says. Those requests encouraged the agency to think how best to connect graduate students with industry. To apply for an INTERN award, students must provide a letter from their supervisor and from the prospective company or other host organization, and must make a convincing case that the experience would help them to achieve their overall training and career goals. “It’s not something that can be pulled together with a casual contact,” Balan says. “The host organization and the university have to put their minds together to create something very powerful for the student.” In another outreach effort, the NSF is seeking submissions for its 2026 Idea Machine, a competition to elicit big ideas for future research projects. The agency is looking for ambitious visions within the fields of science, technology, engineering and mathematics. “We’re crowdsourcing the best and brightest ideas so we know what the community’s thinking,” Balan says. The agency plans to award two to four winners $26,000 each, with the possibility that the winning entries would trigger long-term NSF investments. The competition, which is open to members of the general public as well as to scientists, will accept suggestions up to 26 October. Originally published on Nature Careers on 11 September 2018 - https://www.nature.com/articles/d41586-018-06641-5
- Sponsored Content Article
Working Scientist podcast: How to beat research funding's boom and bust cycle
Julie Gould and Michael Teitelbaum discuss the highs and lows of funding cycles and how to survive them as an early career researcher. Your browser does not support the audio element. In the penultimate episode of this six-part series on grants and funding, Julie Gould asks how early career researchers can develop their careers in the face of funding's "boom and bust" cycle and the short-termism it engenders. Governments are swayed by political uncertainty and technological developments, argues Michael Teitelbaum, author of Falling Behind? Boom, Bust, and the Global Race for Scientific Talent. In the US, for example, space research funding dramatically increased after Soviet Russia launched the Sputnik 1 satellite in 1957, ending after the 1969 moon landing. Similar booms followed in the 1970s, 80s, and 90s, says Teitelbaum, a Wertheim Fellow in the Labor and Worklife Program at Harvard Law School and senior advisor to the Alfred P. Sloan Foundation in New York. But he argues that they are unsustainable and can have a negative impact on the careers of junior scientists and their research. Will Brexit trigger a funding downturn, and if so, for how long? Watch this space, says Teitelbaum. Sponsored content: European Research Council (ERC) Retired Portuguese Navy Captain Joaquim Alves, a principal investigator at the Centre for the History of Science and Technology, University of Lisbon, leads the European Research Council project MEDEA-CHART, dedicated to the study of medieval and early modern nautical charts. He describes his career and the support he has received from the ERC. TRANSCRIPT Julie Gould and Michael Teitelbaum discuss the highs and lows of funding cycles and how to survive them as an early career researcher. Julie Gould Hello, I’m Julie Gould and this is Working Scientist, a Nature Careers podcast. Welcome to the fifth and penultimate episode of our series on funding. In the previous episode, we looked at a recent major upheaval in the UK science funding environment, with the creation of UK Research and Innovation. This time, we’re looking at some of the processes that determine how funding decisions are and have been made in the past, and what impact that these decisions can have on careers in scientific research. But before we go on, don’t forget that at the end of this Working Scientist podcast, we’ve got a ten-minute sponsored slot from the European Research Council. Right, so funding – how do governments decide where to put their money? Professor Michael Teitelbaum, a demographer at the Labor and Worklife Program at Harvard Law School, has studied how funding has been allocated in the US since the world wars, and he’s found that funding comes in cycles, and he calls them "alarm/boom/bust" cycles, and I asked Michael to give us a quick, simple introduction into what these cycles are. Michael Teitelbaum Government funding for basic research often runs in cycles. Politicians and governments decide that there needs to be more funding for basic research and they often will raise the funding quite rapidly to show a significant effect, but then are unable to sustain that rate of increase. Sometimes the funding even declines subsequently. So, you get a cycle of boom followed by bust, over a period of perhaps a decade. My conclusion is that this is quite unhealthy for basic research, which is a quintessentially long-term kind of activity involving long study periods to become fully professional, followed by long careers in basic research. If the funding increases sharply and then doesn’t continue to increase or declines, that is very destabilising for both basic research itself and for career prospects in basic research. Julie Gould And why do you think the governments react in such a way by actually putting quite considerable sums of money towards whatever basic research they’re aiming to fund? Michael Teitelbaum It’s not universal, but it’s common that governments are convinced by industry or by academic institutions that they have been funding basic research insufficiently, and they tend to over-respond to that kind of representation by increasing funding at levels that cannot be sustained over the longer term. Julie Gould Why would you say that these cycles are destructive towards the careers of researchers? Michael Teitelbaum Well, the problem is that basic research and careers in basic research are fundamentally long-term propositions, and this kind of funding which is for a period of years and then disappears is destabilising to a system that requires many years of graduate and advanced study and research to become a professional in basic research. And research projects that take many years to develop, you can’t really achieve a great deal in basic research in only a few years, and if you study for 8-10 years or more to become a research scientist, you might find yourself, with these short cycles of funding, you might find yourself finishing your studies just in time to face a very poor career situation in those fields. Julie Gould In his book called Falling Behind?: Boom, Bust, and the Global Race for Scientific Talent, Michael explored some of these "alarm/boom/bust" cycles in the US from the past century. Now one of the examples he uses in the book is the shock of the successful 1957 Soviet Union launch of the first satellite, Sputnik 1. Michael Teitelbaum This led to what I would consider to be a near political panic among leaders of the US government, especially people such as Lyndon Johnson who was then majority leader in the US Senate, and led to an enormous increase in funding for space and rocketry and controls for catching up with the Soviet Union in space. That cycle ended with the success of John F Kennedy’s promise to successfully land humans on the Moon and return them to Earth safely by the end of the 1960s. When that spectacular achievement was achieved, the political system tended to lose interest in the massive funding for the space programme and there was a bust. The third cycle in the 1980s was stimulated by then President Reagan’s so-called Strategic Defense Initiative - critics called it the Star Wars Initiative - which led to massive funding, but only short-term for that initiative. And then the final two cycles that I identify in the book were different in the sense that they weren’t military, they weren’t strategic in that sense. The first was the internet, the boom resulting from the internet becoming a commercial activity rather than a research or academic activity and the expansion therefore of the internet and other kinds of booms in the 1990s. Again, that was in the private sector not in the government sector. And finally, overlapping that was a decision by the US Congress and the presidential leadership of both parties to double funding over a five-year period for the National Institutes of Health. A massive increase for five years, averaging about 14% per year that then was followed by flat funding for subsequent years. Julie Gould So, what cycle are we in at the moment? Michael Teitelbaum One of the characteristics of a cycle like this is you don’t know it’s a cycle until it finishes, so we can’t be sure at this point that we’re in an ‘alarm/boost/bust’ cycle. We could just be in an alarm and boom cycle without a bust to follow – we will have to come back and talk in five years to see if there is a bust that ensues at the end. But the current boom situation is in information technology, in social media, in fields that are largely created by industry and particularly by firms in Silicon Valley and in the Seattle area, led by Intel and Microsoft in particular. In terms of their lobbying, they argue they cannot find the skilled personnel they need to remain competitive internationally, that there’s a shortage of skilled personnel in these fields. It’s not a new claim. It’s been a claim that was common in all of these other booms and busts over the previous half-century. But their goal is not to encourage a funding boom from the federal government for their fields because they are in the commercial sector and they’re profit-seeking firms. What they’re looking for – and they’ve been successful in their lobbying efforts – is large-scale access to temporary workers coming from low-wage countries, largely via visas with hot names like H1B and L1 and so on. They’ve been quite successful with getting these short-term, temporary workers – large numbers of them in the hundreds of thousands – claiming that otherwise they would not be able to continue to be competitive internationally. And then there’s also parallel lobbying from higher education groups. Their goals are indeed to increase research grant funding because it’s a very substantial source of revenue for them, but also to continue to have easy access to large numbers of international graduate students who pay full tuition. Julie Gould How can early career researchers keep track of these cycles and see and feel what’s happening and learn to navigate them? Michael Teitelbaum I think the key words would be pay attention and be flexible. If you’re an early career researcher or aspiring to be a researcher in one of these fields, you need to keep track of what we are discussing here in terms of increased funding from government sources or decreased funding, increased numbers of temporary visas or decreased numbers of temporary visas. All of these things will have some impact over time on your personal experience. So, you need to pay attention, for example, to the trajectories of key science funding agencies. I would say a way to do that is to pay attention to reports from credible publications that do report in an objective way on what is happening in the politics, if you will, of funding and of temporary visas. You would have to pay attention to the budget requests of key agencies and assess whether those requests are likely, if they are responded to positively, are they likely to be sustainable over the longer term, or are they likely to be short-term pulses of funding, which would be destabilising. And then those who are already doing research and are funded by government agencies need to be cautious in responding to requests for proposals that seem to be short-term pulses of funding or boom-type funding. They need to build a portfolio, I would say, of different funding sources, rather than depend on a particular source that seems to be flush with money at the moment but may not be in the future. In other words, the same kind of advice that any investment advisor would give to a client – that they should diversify their commitments and thereby reduce their exposure to risk in the future. Julie Gould Speaking of the future, the impact that political systems have on scientific funding and thinking back on the previous episode with James Wilsdon on the UK scientific funding environment, I asked Michael what he thought might happen - or not - with Brexit - or not. Michael Teitelbaum If that were to happen – I know there’s a great deal of concern in the UK among academic institutions in terms of whether they would be able to apply what has become quite a large amount of basic research funding from the European Union – I think that’s all up in the air now so I don’t think we can make any forecasts or projections about what will happen, but it’s an issue that I think should be watched. If I were a young scientist engaged in pursuing a career in basic research in the UK, I would be paying a lot of attention to this. Julie Gould Okay, well let’s chat again in five years’ time. Michael Teitelbaum Laughs. I don’t think we need five years for that one, that’s probably two years, but it’s not now – we can’t do it now. Julie Gould So, what does this all mean? Well, the long and short of it is we don’t know what’s going to happen in the future, but what I think we can say is that the funding environment at the moment is a difficult one to navigate, so the more skills and tools amassed for writing grant proposals will be vital for survival in the scientific workforce. In the final episode of this series, we’ll hear more about some alternative ways of distributing scientific funding that may alleviate some of the pressures that researchers face in the current, very competitive climate. Now, that’s all for this section of our Working Scientist podcast. We now have a slot sponsored by and featuring the work of the European Research Council. Joaquim Alves Gaspar tells of his work in cartography and with the European Research Council project MEDEA-CHART. Thanks for listening. I’m Julie Gould. Joaquim Alves Gaspar My name is Joaquim Alves Gaspar. I was born in Lisbon, Portugal 69 years ago. I joined the Portuguese Navy when I was 19, and I served for about 40 years. In 2006, that is 12 years ago, I started a PhD programme on the geometric analysis and numerical modelling of old nautical charts, which I completed in 2010. In my thesis, I have proposed and tested a series of cartometric methods, that means geometrical methods of analysis and numerical modelling, aimed at a better understanding of how old charts were constructed and used at sea. As soon as I got the degree, I was invited to become a member of a research centre in the Faculty of Sciences at the University of Lisbon, where I am now and where I have been working for eight years, first as a postdoctoral researcher and now, after winning the grant, as a principal investigator. Most of what I know about the technical and the scientific methods related to the history of nautical cartography, I learned it from the Navy. I am not only referring to the theoretical background which people can study from the books, but also to the actual experience of contacting a ship at sea, and using nautical charts for the planning and the execution of navigation. It was this knowledge and this experience that gave me the capacity to fully understand old charts, not only as historical artefacts, but images of the world, which is a traditional approach, but also and mostly as instruments to navigate. This is something that a traditional historian of cartography is not prepared to do. By looking into those charts with the eyes of a cartographer and of a navigator and with the assistance of the analytical and modelling tools that I have developed, I could establish a meaningful connection between the methods of chart construction in all kinds, of course, as described in the historical sources and the practice of navigation. This development has opened new and promising lands of research. That is what my ERC project is about. I applied to and I won a starting grant in the section S6 – that is the history of the human past. It was at that time the first ever Portuguese proposal to be accepted in that particular section. It was the first ever grant that was considered to a project on the history of cartography and also, as far as I know, no one is using these kinds of techniques to study old maps. The total amount of the grant is about €1.2 million, to be applied during five years. The funding will be mostly used to pay the six grantees now working with us to cover travel expenses and to buy some equipment. We have a team of eight members: the PI (myself), a retired Navy officer, a senior researcher who is a physicist who converted to the history of science and he is now the head of the department of history and philosophy science, a postdoctoral researcher who is also a physicist by education, three PhD students, a junior computer expert who is developing our information systems and a project manager and she is a neuroscientist by education. Of these, only one of the PhD students is an historian by education. This tells us something about what I have called the multidisciplinary nature of my project. The general objective of the project, as stated in my proposal, is to solve a series of questions which have, should I say, eluded historians of cartography for a very long time, pertaining to the birth, the technical evolution and the use of nautical charts during the Middle Ages and also the early modern period. For example, we want to clarify when, how, why and where the first nautical charts were constructed. This is a very popular subject among the international community of historians of cartography. Not only we have been very successful in bringing many of them to the discussion, but also significant progress has been made in the last year. For example, it is now consensual among us that the oldest nautical charts were constructed using navigational information collected by the pilot at sea. Certain distortions affecting the old charts were caused by the use of magnetic compasses to navigate, which as you know, don’t point exactly to the geographical north. The difference is the so-called declination, magnetic declination. The novelty in my project is that we intend to provide good answers to those questions by using what we call a multidisciplinary approach including a novelty of techniques of geometrical analysis, numerical modelling, carbon-14 dating and multispectral analysis of the old parchments, which will complement, of course, the traditional methods of historical research. So far, one and a half years after the project started, the results are promising. Aim the highest possible and don’t just give it a try – do it using everything you’ve got. Don’t be humble. ERC grants are intended to be given to the very best researchers proposing the best projects. If you are confident that you have an excellent idea, one that will make the panel members raise out of their chairs, and that you are the right person to make it work, then don’t be shy. Go for it. However, having made the decision of proceeding to the next stage, you will now need a great deal of humbleness to be able to create the best possible proposal. The reason is that you will have to engage into an extremely competitive process with highly competent and motivated people. In other words, you will have to work hard and be professional. It took me a full year to write the proposal, despite my experience and background. Let me elaborate a bit on this. You know you have a wonderful idea, otherwise you wouldn’t have engaged in the process. The job now will be to organise each idea into a meaningful and visible project, and of course, to convince the evaluation panel that you are the best possible person to make it work. Don’t leave anything to fortune or chance, so that you won’t blame yourself for not taking into account all the variables. That’s all I have to advise. One of the unwritten goals of the project is to pass the message. I won’t live forever and I want my methods and my techniques to be passed and to be used again by other people, and Portugal is the best place because I also want to give a push to the research on the subject of Portugal. Originally posted on Nature Careers - 01 February 2019 - https://www.nature.com/articles/d41586-019-00403-7