Working Scientist podcast: How to beat research funding's boom and bust cycle
Julie Gould and Michael Teitelbaum discuss the highs and lows of funding cycles and how to survive them as an early career researcher.
In the penultimate episode of this six-part series on grants and funding, Julie Gould asks how early career researchers can develop their careers in the face of funding's "boom and bust" cycle and the short-termism it engenders.
Governments are swayed by political uncertainty and technological developments, argues Michael Teitelbaum, author of Falling Behind? Boom, Bust, and the Global Race for Scientific Talent.
In the US, for example, space research funding dramatically increased after Soviet Russia launched the Sputnik 1 satellite in 1957, ending after the 1969 moon landing.
Similar booms followed in the 1970s, 80s, and 90s, says Teitelbaum, a Wertheim Fellow in the Labor and Worklife Program at Harvard Law School and senior advisor to the Alfred P. Sloan Foundation in New York.
But he argues that they are unsustainable and can have a negative impact on the careers of junior scientists and their research. Will Brexit trigger a funding downturn, and if so, for how long? Watch this space, says Teitelbaum.
Sponsored content: European Research Council (ERC)
Retired Portuguese Navy Captain Joaquim Alves, a principal investigator at the Centre for the History of Science and Technology, University of Lisbon, leads the European Research Council project MEDEA-CHART, dedicated to the study of medieval and early modern nautical charts. He describes his career and the support he has received from the ERC.
Julie Gould and Michael Teitelbaum discuss the highs and lows of funding cycles and how to survive them as an early career researcher.
Hello, I’m Julie Gould and this is Working Scientist, a Nature Careers podcast. Welcome to the fifth and penultimate episode of our series on funding. In the previous episode, we looked at a recent major upheaval in the UK science funding environment, with the creation of UK Research and Innovation.
This time, we’re looking at some of the processes that determine how funding decisions are and have been made in the past, and what impact that these decisions can have on careers in scientific research. But before we go on, don’t forget that at the end of this Working Scientist podcast, we’ve got a ten-minute sponsored slot from the European Research Council.
Right, so funding – how do governments decide where to put their money? Professor Michael Teitelbaum, a demographer at the Labor and Worklife Program at Harvard Law School, has studied how funding has been allocated in the US since the world wars, and he’s found that funding comes in cycles, and he calls them "alarm/boom/bust" cycles, and I asked Michael to give us a quick, simple introduction into what these cycles are.
Government funding for basic research often runs in cycles. Politicians and governments decide that there needs to be more funding for basic research and they often will raise the funding quite rapidly to show a significant effect, but then are unable to sustain that rate of increase. Sometimes the funding even declines subsequently. So, you get a cycle of boom followed by bust, over a period of perhaps a decade. My conclusion is that this is quite unhealthy for basic research, which is a quintessentially long-term kind of activity involving long study periods to become fully professional, followed by long careers in basic research. If the funding increases sharply and then doesn’t continue to increase or declines, that is very destabilising for both basic research itself and for career prospects in basic research.
And why do you think the governments react in such a way by actually putting quite considerable sums of money towards whatever basic research they’re aiming to fund?
It’s not universal, but it’s common that governments are convinced by industry or by academic institutions that they have been funding basic research insufficiently, and they tend to over-respond to that kind of representation by increasing funding at levels that cannot be sustained over the longer term.
Why would you say that these cycles are destructive towards the careers of researchers?
Well, the problem is that basic research and careers in basic research are fundamentally long-term propositions, and this kind of funding which is for a period of years and then disappears is destabilising to a system that requires many years of graduate and advanced study and research to become a professional in basic research. And research projects that take many years to develop, you can’t really achieve a great deal in basic research in only a few years, and if you study for 8-10 years or more to become a research scientist, you might find yourself, with these short cycles of funding, you might find yourself finishing your studies just in time to face a very poor career situation in those fields.
In his book called Falling Behind?: Boom, Bust, and the Global Race for Scientific Talent, Michael explored some of these "alarm/boom/bust" cycles in the US from the past century. Now one of the examples he uses in the book is the shock of the successful 1957 Soviet Union launch of the first satellite, Sputnik 1.
This led to what I would consider to be a near political panic among leaders of the US government, especially people such as Lyndon Johnson who was then majority leader in the US Senate, and led to an enormous increase in funding for space and rocketry and controls for catching up with the Soviet Union in space. That cycle ended with the success of John F Kennedy’s promise to successfully land humans on the Moon and return them to Earth safely by the end of the 1960s. When that spectacular achievement was achieved, the political system tended to lose interest in the massive funding for the space programme and there was a bust.
The third cycle in the 1980s was stimulated by then President Reagan’s so-called Strategic Defense Initiative - critics called it the Star Wars Initiative - which led to massive funding, but only short-term for that initiative. And then the final two cycles that I identify in the book were different in the sense that they weren’t military, they weren’t strategic in that sense. The first was the internet, the boom resulting from the internet becoming a commercial activity rather than a research or academic activity and the expansion therefore of the internet and other kinds of booms in the 1990s. Again, that was in the private sector not in the government sector. And finally, overlapping that was a decision by the US Congress and the presidential leadership of both parties to double funding over a five-year period for the National Institutes of Health. A massive increase for five years, averaging about 14% per year that then was followed by flat funding for subsequent years.
So, what cycle are we in at the moment?
One of the characteristics of a cycle like this is you don’t know it’s a cycle until it finishes, so we can’t be sure at this point that we’re in an ‘alarm/boost/bust’ cycle. We could just be in an alarm and boom cycle without a bust to follow – we will have to come back and talk in five years to see if there is a bust that ensues at the end. But the current boom situation is in information technology, in social media, in fields that are largely created by industry and particularly by firms in Silicon Valley and in the Seattle area, led by Intel and Microsoft in particular. In terms of their lobbying, they argue they cannot find the skilled personnel they need to remain competitive internationally, that there’s a shortage of skilled personnel in these fields. It’s not a new claim. It’s been a claim that was common in all of these other booms and busts over the previous half-century. But their goal is not to encourage a funding boom from the federal government for their fields because they are in the commercial sector and they’re profit-seeking firms. What they’re looking for – and they’ve been successful in their lobbying efforts – is large-scale access to temporary workers coming from low-wage countries, largely via visas with hot names like H1B and L1 and so on. They’ve been quite successful with getting these short-term, temporary workers – large numbers of them in the hundreds of thousands – claiming that otherwise they would not be able to continue to be competitive internationally. And then there’s also parallel lobbying from higher education groups. Their goals are indeed to increase research grant funding because it’s a very substantial source of revenue for them, but also to continue to have easy access to large numbers of international graduate students who pay full tuition.
How can early career researchers keep track of these cycles and see and feel what’s happening and learn to navigate them?
I think the key words would be pay attention and be flexible. If you’re an early career researcher or aspiring to be a researcher in one of these fields, you need to keep track of what we are discussing here in terms of increased funding from government sources or decreased funding, increased numbers of temporary visas or decreased numbers of temporary visas. All of these things will have some impact over time on your personal experience. So, you need to pay attention, for example, to the trajectories of key science funding agencies. I would say a way to do that is to pay attention to reports from credible publications that do report in an objective way on what is happening in the politics, if you will, of funding and of temporary visas. You would have to pay attention to the budget requests of key agencies and assess whether those requests are likely, if they are responded to positively, are they likely to be sustainable over the longer term, or are they likely to be short-term pulses of funding, which would be destabilising. And then those who are already doing research and are funded by government agencies need to be cautious in responding to requests for proposals that seem to be short-term pulses of funding or boom-type funding. They need to build a portfolio, I would say, of different funding sources, rather than depend on a particular source that seems to be flush with money at the moment but may not be in the future. In other words, the same kind of advice that any investment advisor would give to a client – that they should diversify their commitments and thereby reduce their exposure to risk in the future.
Speaking of the future, the impact that political systems have on scientific funding and thinking back on the previous episode with James Wilsdon on the UK scientific funding environment, I asked Michael what he thought might happen - or not - with Brexit - or not.
If that were to happen – I know there’s a great deal of concern in the UK among academic institutions in terms of whether they would be able to apply what has become quite a large amount of basic research funding from the European Union – I think that’s all up in the air now so I don’t think we can make any forecasts or projections about what will happen, but it’s an issue that I think should be watched. If I were a young scientist engaged in pursuing a career in basic research in the UK, I would be paying a lot of attention to this.
Okay, well let’s chat again in five years’ time.
Laughs. I don’t think we need five years for that one, that’s probably two years, but it’s not now – we can’t do it now.
So, what does this all mean? Well, the long and short of it is we don’t know what’s going to happen in the future, but what I think we can say is that the funding environment at the moment is a difficult one to navigate, so the more skills and tools amassed for writing grant proposals will be vital for survival in the scientific workforce. In the final episode of this series, we’ll hear more about some alternative ways of distributing scientific funding that may alleviate some of the pressures that researchers face in the current, very competitive climate. Now, that’s all for this section of our Working Scientist podcast. We now have a slot sponsored by and featuring the work of the European Research Council. Joaquim Alves Gaspar tells of his work in cartography and with the European Research Council project MEDEA-CHART. Thanks for listening. I’m Julie Gould.
Joaquim Alves Gaspar
My name is Joaquim Alves Gaspar. I was born in Lisbon, Portugal 69 years ago. I joined the Portuguese Navy when I was 19, and I served for about 40 years. In 2006, that is 12 years ago, I started a PhD programme on the geometric analysis and numerical modelling of old nautical charts, which I completed in 2010. In my thesis, I have proposed and tested a series of cartometric methods, that means geometrical methods of analysis and numerical modelling, aimed at a better understanding of how old charts were constructed and used at sea.
As soon as I got the degree, I was invited to become a member of a research centre in the Faculty of Sciences at the University of Lisbon, where I am now and where I have been working for eight years, first as a postdoctoral researcher and now, after winning the grant, as a principal investigator.
Most of what I know about the technical and the scientific methods related to the history of nautical cartography, I learned it from the Navy. I am not only referring to the theoretical background which people can study from the books, but also to the actual experience of contacting a ship at sea, and using nautical charts for the planning and the execution of navigation. It was this knowledge and this experience that gave me the capacity to fully understand old charts, not only as historical artefacts, but images of the world, which is a traditional approach, but also and mostly as instruments to navigate. This is something that a traditional historian of cartography is not prepared to do. By looking into those charts with the eyes of a cartographer and of a navigator and with the assistance of the analytical and modelling tools that I have developed, I could establish a meaningful connection between the methods of chart construction in all kinds, of course, as described in the historical sources and the practice of navigation. This development has opened new and promising lands of research. That is what my ERC project is about.
I applied to and I won a starting grant in the section S6 – that is the history of the human past. It was at that time the first ever Portuguese proposal to be accepted in that particular section. It was the first ever grant that was considered to a project on the history of cartography and also, as far as I know, no one is using these kinds of techniques to study old maps. The total amount of the grant is about €1.2 million, to be applied during five years. The funding will be mostly used to pay the six grantees now working with us to cover travel expenses and to buy some equipment. We have a team of eight members: the PI (myself), a retired Navy officer, a senior researcher who is a physicist who converted to the history of science and he is now the head of the department of history and philosophy science, a postdoctoral researcher who is also a physicist by education, three PhD students, a junior computer expert who is developing our information systems and a project manager and she is a neuroscientist by education. Of these, only one of the PhD students is an historian by education. This tells us something about what I have called the multidisciplinary nature of my project.
The general objective of the project, as stated in my proposal, is to solve a series of questions which have, should I say, eluded historians of cartography for a very long time, pertaining to the birth, the technical evolution and the use of nautical charts during the Middle Ages and also the early modern period. For example, we want to clarify when, how, why and where the first nautical charts were constructed. This is a very popular subject among the international community of historians of cartography. Not only we have been very successful in bringing many of them to the discussion, but also significant progress has been made in the last year. For example, it is now consensual among us that the oldest nautical charts were constructed using navigational information collected by the pilot at sea. Certain distortions affecting the old charts were caused by the use of magnetic compasses to navigate, which as you know, don’t point exactly to the geographical north. The difference is the so-called declination, magnetic declination. The novelty in my project is that we intend to provide good answers to those questions by using what we call a multidisciplinary approach including a novelty of techniques of geometrical analysis, numerical modelling, carbon-14 dating and multispectral analysis of the old parchments, which will complement, of course, the traditional methods of historical research. So far, one and a half years after the project started, the results are promising.
Aim the highest possible and don’t just give it a try – do it using everything you’ve got. Don’t be humble. ERC grants are intended to be given to the very best researchers proposing the best projects. If you are confident that you have an excellent idea, one that will make the panel members raise out of their chairs, and that you are the right person to make it work, then don’t be shy. Go for it. However, having made the decision of proceeding to the next stage, you will now need a great deal of humbleness to be able to create the best possible proposal. The reason is that you will have to engage into an extremely competitive process with highly competent and motivated people. In other words, you will have to work hard and be professional. It took me a full year to write the proposal, despite my experience and background. Let me elaborate a bit on this. You know you have a wonderful idea, otherwise you wouldn’t have engaged in the process. The job now will be to organise each idea into a meaningful and visible project, and of course, to convince the evaluation panel that you are the best possible person to make it work. Don’t leave anything to fortune or chance, so that you won’t blame yourself for not taking into account all the variables. That’s all I have to advise.
One of the unwritten goals of the project is to pass the message. I won’t live forever and I want my methods and my techniques to be passed and to be used again by other people, and Portugal is the best place because I also want to give a push to the research on the subject of Portugal.
Originally posted on Nature Careers - 01 February 2019 - https://www.nature.com/articles/d41586-019-00403-7
- Career Advice
How to deliver sound science in resource-poor regions
A well-equipped laboratory stocked with reagents and supplied with uninterrupted electricity and unlimited water might seem like a basic requirement for conducting research. But scientists who work in regions that have limited resources or that are riven by conflict cannot take such amenities for granted. They must perpetually seek scarce grants, publish their own journals, form their own scientific societies and — crucially — draw on their deep reserves of resilience. Nature asked five such researchers how they run productive labs in the face of electricity shortages, border-checkpoint closures, poor Internet connections and other challenges. Marlo Mendoza: Engage with stakeholders Forestry researcher, University of the Philippines, Los Baños For the past 13 years, I have been profiling the contamination of the Marilao, Meycauayan and Obando River System (MMORS), which was on the ‘Dirty 30’ list of the most polluted places in the world in 2007, according to the non-profit organization Pure Earth. There are many polluting industries upstream, including the largest lead smelter in the Philippines, gold smelters, jewellery workshops and tanneries. Downstream are fish farms. We found elevated levels of heavy metals in the water, in the sediments and in fish, especially shellfish, which are sold in the local markets (M. E. T. Mendoza et al. J. Nat. Stud. 11, 1–18; 2012). At least 100,000 people in the municipalities of Marilao, Meycauayan and Obando, and in the metropolitan Manila area, are eating contaminated fish. There are no toxicologists in the area who can accurately diagnose illnesses connected with heavy-metal ingestion. So when we looked at medical records, there were no entries for heavy-metal poisoning. If we cannot prove that these metals are causing harm to people, it’s very difficult to convince policymakers and local executives to take action. We have no local laboratories that can analyse heavy metals found in fish, or in water or blood samples. Local officials, the governor and some of the mayors were really antagonistic because the fishing industry is a major source of income for these municipalities. I have been very careful, even from the outset, to always update the mayors on our projects, and I am accompanied by local and regional government representatives whenever I do my monitoring activities. I do nothing without their consent and am very transparent in my work. One of my strategies was to build a network of stakeholders — including national agencies such as the Bureau of Fisheries and Aquatic Resources and the Department of Environment and Natural Resources — that share my concerns. I also built a good rapport with people who live in the region. There are several associations for fishers and leather-makers in these areas, and we work with them and include them in consultations and meetings about water-quality management. Our project helped to have the area declared as a legally designated water-quality management area. That’s why we’re able to continue our work. We used funding from Pure Earth to do regular longitudinal sampling in sections of the river system, including of sediment, water, fish and other aquatic life. There’s a problem collecting data and samples, because it is costly and the national and local governments have limited funds. There is also no single repository of data with which monitoring can be more effectively planned and analysed. Our monitoring results were included in a Pure Earth database that was shared with other stakeholders, including regional environmental-management offices and local government units. In turn, this encouraged those agencies to conduct studies to complement our work and to share their data. So I was able to get money from the Asian Development Bank, Green Cross Switzerland and the Hong Kong Shanghai Banking Corporation, as well as a small amount from the Coca-Cola Company, to conduct environmental monitoring — including assessment of heavy metals in selected aquatic organisms. Emmanuel I. Unuabonah: Use available resources Materials chemist, Redeemer’s University, Osun State, Nigeria Potable water is a challenge for us here in Africa and across the world: around 1.8 billion people worldwide get their drinking water from a source that is polluted with faeces. As part of our work, we are developing hybrid clay composites to adsorb enteric bacteria, such as Escherichia coli, Salmonella species and Vibrio cholerae, from water. We also use composites made from readily available materials such as kaolinite clay, papaya seeds and plantain peels to extract heavy metals from water. We are not funded by the government. On average, for close to 100 days a year, we have no electricity. We have an alternative utility on campus, so when the power goes off at the national grid during work hours, the generator comes on. If we get lucky with timings, we are guaranteed 36 hours of uninterrupted power to run experiments. But when the generator isn’t running and the grid power has gone off, we just have to wait. Sometimes I use my salary to fund my research and to keep our students. Then I have to struggle to write international grants. I’m so grateful to The World Academy of Sciences in Trieste, Italy; the last grant it gave in 2014 (for US$63,230) took care of stipends, school fees and research expenses for the students, and we used part of it to buy equipment. A colleague at the University of Edinburgh, UK, sent us a $600 bacteria-testing kit last year, but we can’t use it now because a related microscope part was damaged by a power surge. We have a lot of wonderful ideas, wonderful theses just hanging about the shelves, but nobody’s utilizing them. Some young scientists developed cheap electrical power systems from electronic waste materials, but they don’t have the money to develop them further. Nigeria has a thriving oil industry, but the government’s Petroleum Trust Development Fund uses oil-industry proceeds mostly to fund scholarships for Nigerian students abroad, and spends very little on scientific research. Kalulu Muzele Taba: Aim for the possible Organic chemist, University of Kinshasa, Democratic Republic of the Congo Our research seeks to solve problems that have societal impacts, such as malaria, which is endemic in Kinshasa, the capital of the Democratic Republic of the Congo (DRC). People in the poorest areas of Kinshasa are growing about 55 different plants, including citronella and papaya, to try to treat the symptoms of the disease. We thought, why not investigate these plants? We tested eight of the most-used plants and showed that extracts and metabolites had considerable antimalarial activities. We have a small booklet in French and in the local language, Lingala, that we send to people to explain how to use these plants more effectively. We are also studying plant extracts that can be used to treat antibiotic-resistant Mycobacterium tuberculosis (K. B. M. Jose et al. Med. Clin. Rev. 4, 5; 2018), using a grant from The World Academy of Sciences. We don’t get money from the state. In our lab, we don’t have equipment. We don’t have money to buy solvents. Water is available between 5 a.m. and 7 a.m., so we have a container that collects water at night, and during the day we have a pump. For electricity it’s much harder. In the middle of the day, it can come and go many times, and you hope it won’t go while you’re working. We used to have a small generator as a backup for computers, but it’s broken. We don’t fold our hands and cry and say that things will get better. We do whatever we can. Most of the time we buy our own reagents and solvents with our salaries. We try to motivate our master’s and PhD students by finding a way to collaborate with the outside, writing to foreign labs to see whether our students can get overseas fellowships. One student, Joséphine Ntumba, went abroad three times, to the Catholic University of Louvain in Louvain-la-Neuve, Belgium. She has completed her PhD and teaches at the University of Kinshasa. I did my PhD at Northwestern University in Evanston, Illinois, and then went to the Max Planck Institute for Coal Research in Mülheim an der Ruhr, Germany. It was hard to come back. It was not only material, but mental too. I knew that some things would be impossible, but I feel that I have to contribute and inspire young people in science. For the past five or six years, I have been the editor-in-chief of the journal Congo Sciences, which I co-founded. We started it because we wanted to bring visibility to research done in the country. The journal was financed for some time by the Academy of Research and Higher Education (ARES) in Brussels. The academy has stopped funding it now, but we are still publishing the journal. For the past ten years, I’ve been trying to create an academy of sciences for the DRC, similar to the American Association for the Advancement of Science in Washington DC. We have to try to get scientists together and to speak as one voice, and then perhaps the state can start understanding that financing research at the university is important. These are some of the things that make me feel happy that I came back home. Maybe I lost a lot as a scientist, but as a Congolese, I hope I can do something for my place, and for the world. To scientists working in comparable circumstances, I would say that although it’s hard, it’s not impossible. Know that you should find maybe not the best solution, but the least-worst one. GGateway students on the training scheme in information technology funded by the Basque government.Credit: Mohammed Safia Rasha Abu-Safieh: Choose the positive Computer engineer and co-founder of GGateway, Gaza Strip I co-founded GGateway, a social-enterprise company in the Gaza Strip that provides outsourcing services for information and communications technology (ICT) around the world. We offer training and employment to recent university graduates in Gaza with ICT degrees. Our main goal with GGateway is to help people to have a source of income. With the shortage of clean water, the polluted sea and the blockade imposed by Israel and Egypt since 2007, living conditions here are dire. We came up with the idea of GGateway in 2012. In November 2013 we launched a pilot, and in February 2015 we got the green light from the Korea International Cooperation Agency for a $1.3-million grant to fund our plan. That was one of the biggest, happiest things that ever happened to us. We were running a pilot project for the United Nations Relief and Works Agency for Palestine Refugees in the Near East (UNRWA), but in July and August 2014 we had the 50 days of conflict with Israel. That was difficult: the bombing went on all day, there was no electricity and we had limited access to water and food. There was no safe zone, no safe area. So we had to stop our first project. Two days after a ceasefire ended the conflict, we wrote up another eight new concepts for different projects, and the UNRWA agreed to operate three. We could either be positive or cry all day. We chose the positive side and to move on. The cables that we use for all our technical networks are on the list of items that Israel does not allow to enter Gaza. With the support of the UNRWA, it took us nearly four months to bring them in from Israel. Without the UNRWA, it would have taken us a year or more. We also use a generator because most days we get electricity for two to four hours. The overall unemployment rate among graduates in Gaza with ICT degrees is 70%. Among female ICT graduates, it’s 92%. We applied for grants that focus on vulnerable women, and got one from the Basque government in Spain to train 60 female graduates for jobs. We also got a grant from the US-Middle East Partnership Initiative to empower and train 300 ICT graduates to become professional freelancers, and won a $3-million grant in June from the World Bank to train students and software engineers. We have contracts with UN Women and UN Habitat. We are able to travel outside Gaza only two to three times a year, maximum, and sometimes not at all; we need to apply for an exit permit from Israel with support of the UNRWA. Sometimes we are accepted, sometimes rejected; no reason is given. We often lose opportunities if we are registered for a conference, for example, or for training. When we see what we are doing — that it’s changing people’s lives, despite all of the difficulties around us — it makes us feel good. Elizabeth Tilley: Focus on small but crucial changes Sanitation economics researcher, University of Malawi, The Polytechnic, Blantyre I came to Malawi in 2015 after 9 years as a project officer and PhD student at the Swiss Federal Institute of Aquatic Science and Technology in Dübendorf. I had worked on sanitation projects in Nepal, South Africa, Tanzania and Nicaragua. Most of my work in Malawi now is teaching and supervising master’s and PhD students on such projects as making fuel briquettes out of dried faecal sludge. We work on ‘shit-flow diagrams’ — trying to map and understand where excreta is being generated and how much of it is being treated. We have a very bad Internet connection, and it’s a barrier to downloading files or making Skype calls. We don’t have subscriptions to journals. We have 30 computers for 4,000 students. Paper and photocopying are very expensive. We go days without water to even flush the toilets. At the university, we don’t have toilet paper, so I bring my own each day and I keep a secret bottle of soap. The research agenda in Malawi is driven by big donors from the global north, including national governments such as Norway and Japan, and private donors and non-governmental organizations. Very little funding goes to African researchers for work on topics that they’ve identified themselves. The fact that northern countries offer funding opportunities to those in the global south is an excellent form of development. But some proposals call for the participation of a southern partner with no requirements for the division of funding. This means that the southern partner is sometimes given a limited budget for limited work that has limited impact. I would encourage northern researchers to think about doing sabbaticals in African universities. It gives the southern researcher a chance to focus on publications or research, to be exposed to new ideas and methods, and to connect with a broader network, and the northern researcher can learn how things operate in the south and appreciate what works well at home. When you start to think about how crushing the whole system is, you can go crazy. I had a student who just wrote to tell me that he got into a master’s programme in the United States, and to thank me for the reference letter. He’s so excited, and that’s the kind of thing I can hold on to for a couple of months. First submitted on Nature Careers on 24 July 2018 - https://www.nature.com/articles/d41586-018-05768-9
- Sponsored Content Article
Working Scientist podcast: Inside the NIH grant-review process
Julie Gould and Elizabeth Pier discuss how the US National Institutes of Health grant review process works. Your browser does not support the audio element. In this first episode of a six-part weekly series about funding, Julie Gould outlines the US National Institutes of Health (NIH) grant-review process and the extent to which reviewers evaluating the same applications agree or disagree. Is the current system the best way, she asks Elizabeth Pier, lead author of a March 2018 paper published in Proceedings of the National Academy of Sciences, Low agreement among reviewers evaluating the same NIH grant applications. Paid content This episode concludes with a slot sponsored by the European Research Council. Jean-Pierre Bourguignon, its president, outlines the organization's role and remit as a grant funder. TRANSCRIPT Julie Gould and Elizabeth Pier discuss how the US National Institutes of Health grant review process works. Julie Gould: Happy 2019! I hope you’ve all managed to take some time to celebrate. As it’s a new year and new years often come with a makeover in one form or another, the Nature Careers team decided to give the podcast a makeover. As well as a new name, we’ve also got a new format. So, instead of our monthly episodes, we’re going to be producing more episodes in 2019 and grouping them together into different series, featuring six weekly episodes followed by a short break. So, here’s series one – funding – and as an added extra, each episode in this series will end with a ten-minute sponsored slot from the European Research Council. So, without any further ado, let’s go.... (Theme music) Hello, I’m Julie Gould and this is Working Scientist, a Nature Careers podcast. Grant funding plays such an overwhelming role in the career of an academic scientist, and the funders are all too aware of it. Now, I know that all researchers spend many sleepless nights and cups of coffee writing grant proposals, so when I first started doing the research for this series, I wanted to find the best experts to give you the best tips on how to write the best grant proposals to make things a little bit easier for you. But then I came across a research paper that made me stop and reflect. In March 2018, Elizabeth Pier – who was then a PhD student at the University of Wisconsin, Madison, in the Educational Psychology department – published a paper as part of her thesis in PNAS. The paper was entitled "Low agreement among reviewers evaluating the same NIH grant applications." When I first read this title, I thought, "Wait a minute, I thought the idea of the whole funding process was that the top proposals were being funded, the ones where everyone in the peer review system agreed that these were the best ideas supported by the best researchers to do the work." But clearly, this title shows that there’s something else going on in the background, so I wanted to find out more. The research was funded by the National Institutes of Health, which commissioned an independent study to examine the potential for bias to enter into the peer review process. The overarching goal of the whole project was to look for evidence of gender or racial bias, based on the characteristics of the PI or the application, and where in the process these biases might enter. Now, this particular piece of research from Pier is just one of the studies. It recreated a peer review panel to see how these meetings unfold and how they affect the decision-making process. Using previously accepted NIH project proposals, Pier explored this as part of her research. But before we go any further, it’s worth me outlining some of the basic steps of how the NIH proposal review system works. Now, just so you’re clear, these steps are the bare bones and they miss out a lot of the details, but they should give a flavour of what happens once you hit submit. So, the NIH uses a two-stage review process. In the first stage, between two and five reviewers individually evaluate each grant application, and they rate them using the NIH’s nine-point scale, with one for exceptional and nine for poor. They also record what they feel are the application’s strengths and weaknesses. The reviewers will then meet for what’s called a study section meeting to discuss their preliminary ratings. The discussion only looks at the top half of all the applications they have evaluated. The study section members then collectively assign a final rating, and this is averaged into a final priority score. So, that’s stage one. Then in the second stage, members of the NIH advisory councils use this priority score and the written critiques from the reviewers to make funding recommendations to the Director of the NIH institute or centre that awards the funding. So, given all that, I spoke to Elizabeth Pier who now works as a research manager at Education Analytics to find out more about her research. Elizabeth Pier: In this particular study, I was really interested in looking at the degree of agreement between different reviewers and what is even happening before the reviewers come together and how are reviewers going about scoring these applications based on their assessments. So, another way of putting that is: Are the reviewers agreeing not only on the score that they assign, but are they also identifying similar strengths and weaknesses in the critiques that they write prior to the meeting, and also what’s the relationship between that numeric score and the written evaluation? Julie Gould: So, there were some sobering results... Elizabeth Pier: We found that numerically speaking, there really was no agreement between the different individual reviewers in the score that they assigned to the proposals. We also found that when we were looking at the relationship between the strengths and weaknesses, written proposal, and the score that was assigned, we did see a relationship between the number of weaknesses that a reviewer would identify in their critique and the score that the reviewer assigned, but that relationship between the weaknesses and the score doesn’t hold up between different reviewers. Julie Gould: Another way of saying this is that the individual reviewers were really consistent – the more weaknesses they identified in the proposal, the lower the score awarded. But unfortunately, it appeared that each reviewer had a different idea of what a weakness is and what score that meant the proposal would ultimately be given. So, what this means is… Elizabeth Pier: ...we can’t really compare the evaluations of different reviewers and the degree of disagreement that we see in the scores seems to be a reflection of a different sense of calibration in what constitutes a bad score versus a good score. Julie Gould: The reviewers do come together for a meeting to discuss the papers based on the initial reviews, and in the meetings that Elizabeth Pier recreated… Elizabeth Pier: As you would predict and as people told us based on their intuition participating in these kinds of meetings, the range of scores does get smaller after discussions, so there’s a degree of consensus building within individual peer review panels, but the agreement between different panels actually got wider after discussion, and we had a unique opportunity here because we had four different panels that were evaluating the same applications. So in practice each application is only evaluated by one study section but for the purposes of this study we exploited that we had these four different groups looking at the same proposals. And so, in the process of building consensus within a given panel, different panels actually went further apart. Julie Gould: So really the outcome that you’re coming to is that it’s potentially better that these reviewers don’t meet? Elizabeth Pier: Our studies haven’t indicated any value or benefit in the sense of improving the consistency or reliability of the process. Julie Gould: But what about the variability in the quality of the proposal being discussed, doesn’t that make a difference? Elizabeth Pier: But we had to ask people to donate their applications and the summary statement that they received to us and the donations that we received just happened to be funded. And so, we tried to say that above a certain quality threshold, our results suggest that it’s essentially a random process and the meeting doesn’t seem to remove that randomness. However, I will say a caveat to that caveat is that the applications that get discussed in the meeting have already gone through triage, so only the top 50% of applications based on their preliminary score even get discussed in the meeting. So, what we are talking about is given that top 50% of proposals, after you’ve already excluded the ones that really have no chance of being funded initially, there really is a lot of randomness, but even more so, there’s already randomness such that the applications that have been weeded out so to speak and don’t get the opportunity to be discussed in the meeting might actually have a lot of merits. # Had it been assigned to a different panel with different reviewer it very well could have gone on to be discussed. Julie Gould: So, what you’re saying really is that luck plays a very large role in whether or not your research gets funded. Elizabeth Pier: Yes, that is what our results suggest. Above a certain degree, if you have a relatively competitive application, there aren’t any major issues that would immediately disqualify it to any kind of representative effort in the field, there’s a great deal of randomness and luck that we find in determining who does and does not get funding. Julie Gould: So, what does that mean for all those people who are spending all these hours and hours and hours and hours and hours on getting their funding applications organised and sorted and written up? I mean my heart goes out to them... Elizabeth Pier: Yes, my heart does as well. I mean it’s one of the reasons I studied this for my dissertation because it’s incredibly important for individual careers and also incredibly important for the progress of science, right? We want to make sure that the most deserving ideas are getting rewarded and funded and that it’s not just picking out of a hat. So, I mean there are a couple of pieces of silver lining. I think that we see evidence, especially as grants get resubmitted, that being responsive to reviewers’ critiques can play a strong role in conveying to reviewers improvement over time, and so there is something to be said for if you get rejected or you don’t get funded, having some tenacity and resubmitting that application and doing everything you can to address the reviewers’ critiques and feedback. It can make a potential difference. I also think that it’s important for folks not to take it personally. As academics, we definitely are used to rejection and used to plenty of times when we think we have really great ideas and reviewers of manuscripts or of grant applications don’t seem to agree with us, so I would encourage people to take a little bit of solace in that it’s not necessarily a reflection of the quality of the ideas but it’s more kind of a feature of the process. Julie Gould: How would you suggest then that the process is improved? Elizabeth Pier: There should be some assessments of whether what some scholars have called a "modified lottery system" could do. So, the idea being that there’s some initial screening process that experts do conduct to make sure, like I said, they’re kind of weeding out any really problematic proposals, things that are just wildly out of left field in terms of being feasible to complete given the budget or things like that. And then after that kind of initial screen then it really is just a random selection. And the reason I think that would be an improvement is because if the process is already random above a certain quality threshold, which our study suggests it is, we might as well save the money and the time involved to convene thousands of people and spend millions of dollars to have these meetings if the outcome is essentially the same as a random process. Julie Gould: Now, we’ll touch on the idea of a lottery-style funding system later on in the series, but what we can say now is that change is going to be slow – it always is in academia. Is there anything that can be done in the meantime, before this lottery style system or something completely different is created? Elizabeth Pier: Starting to accept the fact that it’s not a completely objective process, that humans are fallible, they are subjective, and when you’re asking experts to make very complex judgements about the potential likelihood of success of a project, that’s a really difficult decision that’s going to bring in a lot of heuristics and biases that go into their decision making. Julie Gould: My final question to Elizabeth was what advice have you got for anyone who’s currently writing a grant proposal to the NIH. Elizabeth Pier: One piece of advice, which is probably pretty obvious but I will say is backed by our findings, is that weaknesses are much more predictive of the score that reviewers will assign, rather than strengths. So, what that means is minimise as many weaknesses as you can. Julie Gould: So, after all of that, I’m intrigued. What do you think? How would you feel about a more lottery-style funding system? Please send in your thoughts to the Nature Careers team which you can do via Twitter, Facebook and LinkedIn. And over the next two episodes, I’ll be speaking to different experts on how to minimise the number of weaknesses within your funding application, in the hopes – fingers crossed – that you’ll have a bit more success and a bit more luck. Now, that’s all for this section of our Working Scientist podcast. We now have a slot sponsored by and featuring the work of the European Research Council. Thanks for listening. I’m Julie Gould. (Theme music) Jean-Pierre Bourguignon: So, my name is Jean-Pierre Bourguignon and my title is President of the European Research Council, which of course is supported by the European Union as through the European Commission. I’m a French mathematician, I should say. I spent most of my career in the CNRS (Centre National de la Recherche Scientifique). My field was differential geometry, but did a lot of work actually at the boundary of theoretical physics, general relativity and Dirac operators and these kind of topics, but still always as a mathematician. The European Research Council is actually an interesting story. It was created in 2007, so it’s now 11 years of age, and it was a long process. Myself, the first time I heard about the possibility of having an ERC was 1995, and it was a long effort by the scientific community, and step after step we had to convince people in the Commission, people in the European Council – namely the countries – that they should support such a project, but still it has a lot of very specific characteristics, particularly the power which has been given to its scientific council is considerable. It really was an innovation and the council has the responsibility of deciding on how to spend the money and how to do the evaluation. This is unique in the setting of the European Commission, that a group of 22 scientists are given such a responsibility and of course as President of the European Research Council, I have some very specific ones, which is to confirm the list of people who are granted and really guarantee the quality of the work done. The mission of the ERC was really to make Europe more attractive, to be a place where science can develop really in the most ambitious way and to push the ambition, particularly of young people, upward, that is to make them independent early enough and to take their vision on board. You know, we are at the stage of giving 1100 research grants this year, which is of course a very significant amount of money. The budget is now really over €2 billion per year, and we are covering all fields of science – that is physical sciences, engineering, including maths, computer science, and so on, life sciences, social sciences and humanities. If you want to know what you are doing, you need to talk and meet and discuss with the people you are funding and so I do travel a lot, particularly in Europe, to meet the people we call our grantees – the people who get the grants from the ERC – and this part of my job is really extremely worthwhile and extremely rewarding because the selection process is a very tough one – the typical success rate at this moment is 13%. And it means that people have all proposed very ambitious projects that are conditioned to be successful at the ERC, high risk again, we need to encourage the panels who are selecting people to really accept to take risks. And that’s one thing I hear regularly from grantees, telling me, ‘I submitted a very similar project to my national agency but then I was not funded, it was considered too risky, then I submitted to the ERC, and then the ERC funds me,’ so it makes a big difference. Another component which is very important in our strategy is the fact that the clause, which has been put in place by the European Research Council. Really, we have three categories at the moment for the individual grants, which is the starting grants, consolidator grants and advanced grants, and it means timed to PhD. So, starting grants – to 2 to 7 years, consolidator grants – 7 to 12 years, and advanced grants – there is no condition, it just means people who are already confirmed, and while doing that it means that in the end we are dedicating typically two thirds of our budget to the younger people, people who are typically below 40 years of age. Very often people get the belief that really if you are not from one of the leading research institutions in Europe, then you have no chance. This is not the case. I mean the institution which is your host institution is not part of the evaluation, which is really the key for the evaluation is the project. You have to show that you have thought of what kind of resources will be needed, and you describe them, but this is not the institution as such which is very important. So, it means that in particular in terms of the support we give, part of it could be also buying expensive equipment if you need them and if it’s not available in your institution. So, we consider the project, not just as helping the people, but helping the people also to set up the environment which will make it possible to get the project through. So, this is sometimes one misconception that people have, that they get the feeling that if they are coming from a smaller place, they have no chance. The number of institutions the ERC has been signing with is close to 800 now, so of course it’s quite a significant number of institutions based in Europe and of course some of the leading ones got more grants than others, but definitely even small institutions have been very successful at the ERC. One of the key things that the ERC is doing is empowering researchers. This is something very, very important for us, and a very good example for this is one of the specific characteristics of the ERC programme which is called portability, but the host institution is not part of the selection criteria – it’s just here to make sure that there is a legal body that is able to receive the contract and it gives a lot of power to the researchers. And one of the typical powers, what I mentioned, portability, which means that the researcher can change the host institution if he or she feels that it’s not given the proper treatment, or maybe they could give other personal reasons to do that. This is the whole philosophy behind it – we really want the researchers in the driver’s seat at the level of the council but also at the level of how they run their contract, and of course there is an institution behind it because you want to be sure that there is a legal basis for this, but we really want the researchers to be able to do their research in the best possible conditions. The map I have in front of me, which is the map of the world, has on top of it one of our mottos which is ‘Open to the world.’ One of the conditions to be funded is that you have to spend at least 50% of your time in Europe, but you can be from any country, and we want to be sure that Europe is the leader to tackle some of the most challenging scientific problems. At the very beginning, we could notice that the percentage of women who were applying to the ERC was less than the percentage of women in the scientific community, and we felt this was definitely not adequate. Also, we had for the ones who applied, the success rate was definitely lower than the success rate for men. Through very sustained efforts, and identifying the issue from the very beginning, I think we made very significant progress. So, first of all, the percentage of women applying to the ERC has been steadily growing. We are now basically at the level where the percentage of women applying to the ERC is very similar to the level of the percentage of women in the age group of the different goals we have. So, from that point of view, I think we really achieved something which means that there’s no some kind of resistance or reluctance of women to apply, so this is one step. And then, of course very important but I think the two are linked – the fact that in recent years, the ERC women have been on average more successful than men. It’s a very slight difference but since we started the situation shows the opposite. We are very pleased that all the efforts we made, particularly to tackle complicit bias or various other things have been more or less successful. I’ve been visiting many, many countries in Europe, in particular countries in the "EU13," because I feel you need to understand the real situation people are exposed to in the various countries, and they are the ones who joined Europe the most recently, and most of them located in the eastern part of Europe and it’s very important to realise that actually that situation can be quite different from one country to the next. It has to do with teaching load, it has to do with the power structures in the institutes, it has to do of course with the support which is available to people, so for me it was very, very important to meet the researchers because that’s for me the key point. Also, to meet the authorities in these countries and to understand in which environment they are operating because I think that’s the very best way. We, at the level of the European Research Council, have also introduced some help, in particular by encouraging the various countries, could be also regions, to support possibilities of researchers in typically underrepresented regions (it could be EU13 countries) – to really give them some possibility of spending some time with the support of their countries or their region, in ERC teams, so that they understand what it takes to submit a proposal, but also to understand better, to really also test their ideas with other people so that then they have a much better idea what it takes to submit a proposal and therefore they are better prepared personally, not just intellectually, to really submit a proposal in good conditions because they have seen what a difference it makes and also what also kind of effort you have to put in if you want to be successful. Originally posted on Nature Careers - 04 January 2019 - https://www.nature.com/articles/d41586-019-00016-0
- Career Advice
How to get a funded internship in industry
As part of a broad mission to prepare science students for careers outside academia, the US National Science Foundation (NSF) has expanded a funding initiative to support master’s and PhD students for six-month internships in companies, government laboratories and non-profit organizations. The INTERN supplemental-funding opportunity, launched last year for select departments within the NSF, will now be open to almost every graduate student supported by an NSF grant, says Prakash Balan, programme director in the NSF’s Directorate for Engineering. At a time when available US industry positions far outnumber job openings in US academia, the internships can give students real-world training for their futures, he adds. “Opportunities like this give students exposure and experience at a time when it matters most,” he says. The programme also fits in with the agency’s overall agenda. “NSF has a long-term vision to foster the growth of a competitive and diverse workforce,” Balan says. “We want to advance the science and innovation skills of the nation at large.” The expanded programme provides up to US$55,000 to support a student for six months. The sum is meant to cover travel, tuition, stipends, materials and other expenses. The student’s supervisor can use up to $2,500 of the award to visit the site hosting the internship and co-mentor the student. The NSF has pledged to support up to 200 students in each of the fiscal years 2019 and 2020, although Balan says that more awards could be provided if the demand is great enough. To be eligible, students must have completed at least one year of their master’s or PhD programmes. Jennifer Weller, programme director of the NSF’s Directorate for Biological Sciences, says that she expects to receive many applications from biology students. “I’ve already had 20 phone calls asking for more details,” she says. Weller explains that she worked for five years in industry (at the biotechnology company PE/Applied Biosystems) before eventually returning to academia. Fostering the flow of talent and ideas between academia and industry should be a top priority for the agency, she says. Interest in the INTERN initiative goes both ways. The programme began after corporations contacted the NSF asking for help finding student interns, Balan says. Those requests encouraged the agency to think how best to connect graduate students with industry. To apply for an INTERN award, students must provide a letter from their supervisor and from the prospective company or other host organization, and must make a convincing case that the experience would help them to achieve their overall training and career goals. “It’s not something that can be pulled together with a casual contact,” Balan says. “The host organization and the university have to put their minds together to create something very powerful for the student.” In another outreach effort, the NSF is seeking submissions for its 2026 Idea Machine, a competition to elicit big ideas for future research projects. The agency is looking for ambitious visions within the fields of science, technology, engineering and mathematics. “We’re crowdsourcing the best and brightest ideas so we know what the community’s thinking,” Balan says. The agency plans to award two to four winners $26,000 each, with the possibility that the winning entries would trigger long-term NSF investments. The competition, which is open to members of the general public as well as to scientists, will accept suggestions up to 26 October. Originally published on Nature Careers on 11 September 2018 - https://www.nature.com/articles/d41586-018-06641-5