after many years of research they found the solution

Scientific discovery is an ongoing process that takes time, observation, data collection and analysis, patience and more. At the NPRCs, our recent COVID-19 research is an example of the ongoing basic science process — how current research builds on previous discoveries and how discoveries help improve human health. This article from the National Institutes of Health (NIH) explains why basic science, such as the NPRCs conduct, is important and how taking time, as long as it takes, is a necessary part of scientific discovery.

Discoveries in Basic Science: A Perfectly Imperfect Process

Have you ever wondered why science takes so long? Maybe you haven’t thought about it much. But waiting around to hear more about COVID-19 may have you frustrated with the process.

Science can be slow and unpredictable. Each research advance builds on past discoveries, often in unexpected ways. It can take many years to build up enough basic knowledge to apply what scientists learn to improve human health.

“You really can’t understand how a disease occurs if you don’t understand how the basic biological processes work in the first place,” says Dr. Jon Lorsch, director of NIH’s National Institute of General Medical Sciences. “And of course, if you don’t understand how the underlying processes work, you don’t have any hope of actually fixing them and curing those diseases.”

Basic research asks fundamental questions about how life works. Scientists study cells, genes, proteins, and other building blocks of life. What they find can lead to better ways to predict, prevent, diagnose, and treat disease.

How Basic Research Works

When scientists are interested in a topic, they first read previous studies to find out what’s known. This lets them figure out what questions still need to be asked.

Using what they learn, scientists design new experiments to answer important unresolved questions. They collect and analyze data, and evaluate what the findings might mean.

The type of experiment depends on the question and the field of science. A lot of what we know about basic biology so far has come from studying organisms other than people.

“If one wants to delve into the intricate details of how cells work or how the molecules inside the cells work together to make processes happen, it can be very difficult to study them in humans,” Lorsch explains. “But you can study them in a less complicated life form.”

These are called research organisms. The basic biology of these organisms can be similar to ours, and much is already known about their genetic makeup. They can include yeast, fruit flies, worms, zebrafish, and mice.

Computers can also help answer basic science questions. “You can use computers to look for patterns and to try to understand how the different data you’ve collected can fit together,” Lorsch says.

But computer models have limits. They often rely on what’s already known about a process or disease. So it’s important that the models include the most up-to-date information. Scientists usually have more confidence in predictions when different computer models come up with similar answers.

This is true for other types of studies, too. One study usually only uncovers a piece of a much larger puzzle. It takes a lot of data from many different scientists to start piecing the puzzle together.

Building Together

Science is a collective effort. Researchers often work together and communicate with each other regularly. They chat with other scientists about their work, both in their lab and beyond. They present their findings at national and international conferences. Networking with their peers lets them get feedback from other experts while doing their research.

Once they’ve collected enough evidence to support their idea, researchers go through a more formal peer-review process. They write a paper summarizing their study and try to get it published in a scientific journal. After they submit their study to a journal, editors review it and decide whether to send it to other scientists for peer review.

“Peer review keeps us all informed of each other’s work, makes sure we’re staying on the cutting-edge with our techniques, and maintains a level of integrity and honesty in science,” says Dr. Windy Boyd, a senior science editor who oversees the peer-review process at NIH’s scientific journal of environmental health research and news.

Different experts evaluate the quality of the research. They look at the methods and how the results were gathered.

“Peer reviewers can all be looking at slightly different parts of the work,” Boyd explains. “One reviewer might be an expert in one specific method, where another reviewer might be more of an expert in the type of study design, and someone else might be more focused on the disease itself.”

Peer reviewers may see problems with the experiments or think different experiments are needed. They might offer new ways to interpret the data. They can also reject the paper because of poor quality, a lack of new information, or other reasons. But if the research passes this peer review process, the study is published.

Just because a study is published doesn’t mean its interpretation of the data is “right.” Other studies may support a different hypothesis.

Scientists work to develop different explanations, or models, for the various findings. They usually favor the model that can explain the most data that’s available.

“At some point, the weight of the evidence from different research groups points strongly to an answer being the most likely,” Lorsch explains. “You should be able to use that model to make predictions that are testable, which further strengthens the likelihood that that answer is the correct one.”

An Ever-Changing Process

Science is always a work in progress. It takes many studies to figure out the “most accurate” model—which doesn’t mean the “right” model.

It’s a self-correcting process. Sometimes experiments can give different results when they’re repeated. Other times, when the results are combined with later studies, the current model no longer can explain all the data and needs to be updated.

“Science is constantly evolving; new tools are being discovered,” Boyd says. “So our understanding can also change over time as we use these different tools.”

Science looks at a question from many different angles with many different techniques. Stories you may see or read about a new study may not explain how it fits into the bigger picture.

“It can seem like, at times, studies contradict each other,” Boyd explains. “But the studies could have different designs and often ask different questions.”

The details of how studies are different aren’t always explained in stories in the media. Only over time does enough evidence accumulate to point toward an explanation of all the different findings on a topic.

“The storybook version of science is that the scientist is doing something, and there’s this eureka moment where everything is revealed,” Lorsch says. “But that’s really not how it happens. Everything is done one increment at a time.”

You are leaving the NPRC.org website

• About NPRCs

How research can lead to some extraordinary (and very unlikely) discoveries

Professor of Enzymology, University of Dundee

Disclosure statement

Philip Cohen owns shares in GlaxoSmithKline and AstraZeneca and collaborates with GlaxoSmithKline, Merck-Serono, Boehringer Ingelheim, Janssen Pharmaceutica and Ono pharmaceuticals

View all partners

When it comes to research, you never really know in which fantastic directions it might take you. A discovery in one research area can, many years later, change the world in another.

In the mid-to-late 1970s, I worked with my team here in Dundee, doing a great deal of research around insulin and diabetes. We made a number of significant breakthroughs in our understanding of how insulin works .

This was fundamental research at the molecular level, examining the processes of proteins and enzymes, how they work in the body and the results that can arise . This kind of work does not necessarily start off being focused on a particular disease. We were interested in processes or reactions in cells – their links to disease or certain conditions often come later.

During the course of this work, we identified an enzyme called GSK3 , which plays a key role in regulating the conversion of blood glucose into glycogen, its storage form in the tissues.

When insulin is secreted from the pancreas into the blood, it acts on liver and muscle to switch GSK3 off, and so accelerates the conversion of glucose into glycogen in these tissues. Therefore, it was initially hoped that drugs might be developed that could “switch off” GSK3 activity and therefore be beneficial for the treatment of type 2 diabetes .

But further, fascinating discoveries were made. Subsequent research in many laboratories revealed that GSK3 had many other functions in the body, including in the attachment of phosphate to a protein in the brain called “Tau” .

When abnormally high levels of phosphate become attached to Tau, they cause it to aggregate and form deposits in the brain called “tangles” – one of the hallmarks of Alzheimer’s disease . These findings led to renewed interest in developing drugs that switch off GSK3 in the hope that they would benefit Alzheimer’s patients.

A number of pharmaceutical and biotechnology companies took up this challenge, and a drug called Tideglusib was developed by the Spanish biotechnology company Noscira and entered clinical trials for the treatment of Alzheimers and progressive supranuclear palsy , another neurodegenerative disease of the brain. This drug passed Phase I clinical trials, indicating that it could be used safely in human patients, and further trials of this drug in larger numbers of patients are now progressing.

From diabetes to dentistry

At the same time, it also emerged that GSK3 is a key component of a biochemical pathway which leads to the destruction of certain proteins associated with early responses to tissue damage.

This same pathway is activated when teeth are damaged and, in a remarkable development, Paul Sharpe and his colleagues at King’s College London applied low doses of Tideglusib to biodegradable collagen sponges, which were then inserted into tooth cavities. They found that the sponges degraded with time and were replaced by new dentine, the main supporting structure of the tooth.

This could transform the way we treat teeth cavities, making man-made fillings a thing of the past. Since collagen sponges are already available commercially and approved clinically, and Tideglusib has also passed safety tests, there is a real opportunity now to get this treatment quickly into dental clinics.

The results of this study, published in Scientific Reports, received worldwide media attention. As Sharpe commented:

The simplicity of our approach makes it ideal as a clinical dental product for the natural treatment of large cavities, by providing both pulp protection and restoring dentine.

One of the fascinations of carrying out fundamental research is that one can never predict what it will eventually lead to and how the discoveries may be used to benefit human health – and wealth.

When we discovered GSK3 in the late 1970s, the idea that it might revolutionise dentistry would have sounded like science fiction. But here we are, with a line that stretches all the way back to a laboratory in Dundee and leads towards the treatment of neurogenerative diseases such as Alzheimer’s. And who knows, GSK3 inhibitors might yet turn out to be useful for the treatment of diabetes after all.

This is an excellent illustration of how it can take years or even decades before the results of fundamental research can be fully exploited. It also shows, once again, the critical importance of long-term support from government and other funders of basic research. Without such support, we’ll never know what we’re missing.

• Research funding
• fundamental research

Faculty of Law - Academic Appointment Opportunities

Operations Manager

Senior Education Technologist

Audience Development Coordinator (fixed-term maternity cover)

Lecturer (Hindi-Urdu)

• Skip to main content
• Keyboard shortcuts for audio player

• LISTEN & FOLLOW
• Apple Podcasts
• Google Podcasts
• Amazon Music
• Amazon Alexa

Your support helps make our show possible and unlocks access to our sponsor-free feed.

Experiments That Keep Going And Going And Going

Nell Greenfieldboyce

William Beal, standing at center, started a long-term study on seed germination in 1879. He buried 20 bottles with seeds in them for later researchers to unearth and plant. Michigan State University hide caption

William Beal, standing at center, started a long-term study on seed germination in 1879. He buried 20 bottles with seeds in them for later researchers to unearth and plant.

A biologist who has been watching a dozen bottles of bacteria evolve for nearly a quarter of a century is hoping he can find someone to keep his lab experiment going long after he dies.

Meanwhile, just by coincidence, a botanist who works across campus is carefully tending an experiment that started before he was born, all the way back in 1879.

These two researchers, both at Michigan State University in East Lansing, represent different sides of an unusual phenomenon in science: experiments that outlive the people who started them.

Most researchers design studies to churn out results as quickly as possible. But because nature can work on vast time scales, some questions can take longer to answer than any one scientist's career.

Richard Lenski began his evolution experiment in 1988 with a simple question: Does evolution always lead to the same end point? If he started with 12 identical flasks, full of identical bacteria, would they all change over time in the same way? Or would random mutations send each bottle's population spinning off in a different direction?

Richard Lenski examines the growth of bacteria on a plate on Jan. 12. He began an evolution experiment in 1988 with 12 identical flasks of bacteria to see if the populations would change over time in the same way. G.L. Kohuth/Michigan State University hide caption

Richard Lenski examines the growth of bacteria on a plate on Jan. 12. He began an evolution experiment in 1988 with 12 identical flasks of bacteria to see if the populations would change over time in the same way.

"This was an experiment that was intended to be a long-term experiment, although I had no idea that it would be multiple decades," says Lenski, an evolutionary biologist. "It does just keep producing new and interesting results, so it doesn't seem to be near the end of its life span."

Every day, someone in his lab has to do the brief, tedious chore of feeding and caring for the bacteria. On day number 8,449, Lenski reached into an incubator and pulled out his old friends.

These E. coli bacteria reproduce so rapidly that, in one day, they speed through seven generations — creating the equivalent of their great-great-great-great grandchildren and letting Lenski watch their evolution in real time.

Lenski brought the flasks over to a lab bench and reached for his glasses. "When I started this experiment, I didn't need reading glasses," he notes, "and now looking at things close-up is always more work than it used to be."

For the first decade of his experiment, the bacteria in each flask mostly changed in similar ways. For example, they all were producing larger cells.

Then things got kind of boring for a while because the changes started coming more slowly. Lenski had other projects going on in his lab, and figured that maybe he'd learned all he could from this one.

"And so I was sort of thinking, 'OK, maybe it's time to stop the experiment,' " he says, recalling that he asked a few colleagues what they thought of that idea. "And they basically said, 'Nope, you can't stop it, it's gone on too long.' "

So he stuck with it. And a few years later, in 2003, something happened. The liquid in one flask looked strange. "This flask was considerably more cloudy," says Lenski. "I was suspicious that we had a contaminant."

It turns out that the bacteria in that one flask had actually changed in a dramatic way. After 30,000 generations, they had suddenly gained the ability to consume citrate, a chemical that had always been in the flasks — but that was never intended to be a food, since laboratory E. coli normally can't eat it.

What's more, Lenski was able to trace exactly how that new trait emerged. Over the years, he's been freezing samples of his bacteria, so he was able to go back and track every little genetic change that's taken place through the generations, using technologies that didn't even exist when he first started this study.

Lenski is now convinced that this study should keep going far into the future, to see what else might evolve. He'd like to see this experiment go on not just for 50,000 bacterial generations but 50,000 human generations, to "really get some very hard numbers on the process of evolution."

The fact that Lenski won't be around to see those hard numbers doesn't seem to bother him.

"My wife and I were very fortunate that one of our daughters had a baby about 20 months ago. And that really changes one's perception of time even more than a long-term experiment," Lenski says.

He notes that people tend to conflate the universe with their own existence, "but having children, grandchildren and so on ... you really just come to grips with the vast span of time that is available. And we only get to occupy a tiny portion of it."

Lenski, who is 56 years old, thinks he'll watch his bacteria for about another decade. Then he'll have to find someone to inherit this project. It's not a particularly expensive or difficult study — so he just needs to find someone younger who has a lab and is willing to carry his vision forward.

"They might be in their, you know, early- or mid-30s or something like that," Lenski says, "and then they can decide whether they want to do it for just the next five or 10 years or whether they want to continue it for another 30 years and perhaps pass it on to somebody who hasn't even been born yet."

Is it really possible to keep an experiment going like that? The answer is undoubtedly yes, as Lenski learned years ago when he heard of another long-term study happening on campus.

"Here I was, proud of myself for what was at that time maybe a 15-year experiment, discovering that it wasn't even the oldest experiment on campus — that there was another one up around 100 years, or even past that," recalls Lenski.

Seeds Buried Long, Long Ago

That experiment is currently cared for by Frank Telewski, who runs Michigan State University's botanical garden. The garden is named after botanist William J. Beal, and he started a long-term study on seed germination all the way back in 1879.

Beal was inspired by local farmers who had been asking him this question: If we weed our fields year after year, will we ever reach a point where the weeds just don't come back?

"Well, that was a very interesting question," says Telewski, because it wasn't at all clear how long seeds might remain viable in the soil. "We know that seeds can remain dormant for a long period of time, and Professor Beal's key question was, 'How long?' "

Bottles like this 90-year-old one were filled with seeds and sand, then buried by William Beal. Researchers periodically unearth a bottle and plant the seeds to see if they grow. Kurt Stepnitz/Michigan State University hide caption

Bottles like this 90-year-old one were filled with seeds and sand, then buried by William Beal. Researchers periodically unearth a bottle and plant the seeds to see if they grow.

So Beal put a precise quantity of seeds from different species into 20 sand-filled bottles and stashed them underground. The original plan was to dig up one bottle every five years and see what would grow.

"Clearly, burying 20 bottles and only taking one out every five years, the plan was to go beyond Professor Beal's career, let alone Professor Beal's life," says Telewski.

The only writings from Beal about his experiment are dry scientific reports, but Telewski assumes it meant a lot to him.

"He had to be passionate about it," says Telewski. "You don't do something like this, you know, with that long-term desire, without being passionate."

Beal opened six bottles before he retired. Then he passed it on to a colleague, Henry Darlington. Eventually it was taken over by others, including Robert Bandurski and Jan Zeevaart, who passed it on to Telewski.

The experiment has lasted longer than Beal ever intended because the caretakers extended it. They first decided to open a bottle only once every decade, and now, once every two decades.

Telewski dug up his first bottle 12 years ago. He did it at night, with a flashlight, trying not to draw any attention to the secret burial spot. He says it was exciting to think back and remember that the last person to see the seed was Beal, 120 years ago. "For me that holds a level of significance, that holds a level of fascination, charm," says Telewski.

And he says the mysteries of long-term seed viability remain scientifically interesting. Only two plant species sprouted from the last Beal bottle. Telewski can't wait to dig up the next bottle, in 2020.

Will that be the year that nothing germinates, wonders Telewski, or "will something that hasn't germinated in 30, 40 years all of a sudden appear?"

This kind of inherited experiment is unusual, says Telewski, but in another way, the whole of science is one big long-term effort. Every time researchers record a careful observation, or stash a specimen in a museum, they make it possible for some unknown person of the future to pick up where they left off.

"And isn't that wonderful that somebody, somewhere, thought forward enough to say, 'Let's hold onto this, let's keep this experiment going, let's design this experiment to go on and see where it takes us,' " says Telewski.

Telewski already has someone in mind to inherit the Beal study when he retires. "There's one particular person I've been speaking with, and I think she's going to be very excited to pick it up," he says.

If all goes as planned, he thinks the experiment will probably outlive her, too.

Long-Term Science Experiments

Some research studies don't yield quick results, and scientists design experiments that continue for years, if not decades. Below is a sampling of some long-term projects, many of which continue to this day. (Mouse over the bars for more information about each study.)

Credits: Adam Cole, Alyson Hurt and Andrew Prince / NPR

• Skip to primary navigation
• Skip to main content
• Skip to primary sidebar
• Skip to footer

Understanding Science

How science REALLY works...

• Understanding Science 101
• Misconceptions
• Accepted theories may be modified or overturned as new evidence and perspective emerges.
• Scientists are likely to accept a new or modified theory if it explains everything the old theory did and more.
• The process of theory change may take time and involve controversy, but eventually the scientific explanation that is more accurate will be accepted.

Misconception:  Scientific ideas are absolute and unchanging.

Misconception:  Because scientific ideas are tentative and subject to change, they can’t be trusted.

Correction:  Accepted scientific ideas are well-supported and reliable, but could be revised if warranted by the evidence.  Read more about it.

Even theories change

Accepted theories are the best explanations available so far for how the world works. They have been thoroughly tested , are supported by multiple lines of evidence , and have proved useful in generating explanations and opening up new areas for research. However, science is always a work in progress, and even theories change. How? We’ll look at some over-arching theories in physics as examples:

• Classical mechanics In the 1600s, building on the ideas of others, Isaac Newton constructed a theory (sometimes called classical mechanics or Newtonian mechanics) that, with a simple set of mathematical equations, could explain the movement of objects both in space and on Earth. This single explanation helped us understand both how a thrown baseball travels and how the planets orbit the sun. The theory was powerful, useful, and has proven itself time and time again in studies. Yet it wasn’t perfect…
• Special relativity Classical mechanics was one-upped by Albert Einstein’s theory of special relativity. In contrast to the assumptions of classical mechanics, special relativity postulated that as one’s frame of reference (i.e., where you are and how you are moving) changes, so too do measurements of space and time. So, for example, a person speeding away from Earth in a spacecraft will perceive the distance of the spacecraft’s travel and the elapsed time of the trip to be different than would a person sitting at Cape Canaveral. Special relativity was preferred because it explained more phenomena: it accounted for what was known about the movement of large objects (from baseballs to planets) and helped explain new observations relating to electricity and magnetism.
• General relativity Even special relativity was superseded by another theory. General relativity helped explain everything that special relativity did, as well as our observations of gravitational forces.
• Our next theory… General relativity has been enormously successful and has generated unique expectations that were later borne out in observations, but it too seems up for a change. For example, general relativity doesn’t mesh with what we know about the interactions between extremely tiny particles (which the theory of quantum mechanics addresses). Will physicists develop a new theory that simultaneously helps us understand the interactions between the very large and the very small? Time will tell, but they are certainly working on it!

All the theories described above worked — that is, they generated accurate expectations, were supported by evidence , opened up new avenues of research, and offered satisfying explanations. Classical mechanics, by the way, is still what engineers use to design airplanes and bridges, since it is so accurate in explaining how large (i.e., macroscopic) and slow (i.e., substantially slower than light) objects interact. Nevertheless, the theories described above did change. How? A well-supported theory may be accepted by scientists, even if the theory has some problems. In fact, few theories fit our observations of the world perfectly. There is usually some anomalous observation that doesn’t seem to fit with our current understanding. Scientists assume that by working to understand such anomalies, they’ll either disentangle them to see how they fit with the current theory or they’ll make progress towards a new theory. And eventually that does happen: a new or modified theory is proposed that explains everything that the old theory explained plus other observations that didn’t quite fit with the old theory. When that new or modified theory is proposed to the scientific community, scientists come to understand the new theory, see why it is a superior explanation to the old theory, and eventually, accept the new theory – though this process can take many years.

SCIENTIFIC CONTROVERSY: TRUE OR FALSE?

Here, we’ve discussed true scientific controversy — a debate within the scientific community over which scientific idea is more accurate and should be used as the basis of future research. True scientific controversy involves competing scientific ideas that are evaluated according to the standards of science — i.e., fitting the evidence, generating accurate expectations, offering satisfying explanations, inspiring research, etc. However, occasionally, special interest groups try to misrepresent a non-scientific idea, which meets none of these standards, as inspiring scientific controversy. To learn to identify these false controversies, visit:

• What controversy: Is a controversy misrepresented or blown out of proportion? , one of the tips in our Science Toolkit.
• Science in action

For an example of how evolutionary theory changed to account for a new idea, check out the story of Lynn Margulis,  Cells within cells: An extraordinary claim with extraordinary evidence .

Science at multiple levels

Summing up the process

Subscribe to our newsletter

• The science flowchart
• Science stories
• Grade-level teaching guides
• Teaching resource database
• Journaling tool

Identifying problems and solutions in scientific text

• Open access
• Published: 06 April 2018
• Volume 116 , pages 1367–1382, ( 2018 )

Cite this article

You have full access to this open access article

• Kevin Heffernan 1 &
• Simone Teufel 1  

10k Accesses

34 Citations

2 Altmetric

Explore all metrics

Research is often described as a problem-solving activity, and as a result, descriptions of problems and solutions are an essential part of the scientific discourse used to describe research activity. We present an automatic classifier that, given a phrase that may or may not be a description of a scientific problem or a solution, makes a binary decision about problemhood and solutionhood of that phrase. We recast the problem as a supervised machine learning problem, define a set of 15 features correlated with the target categories and use several machine learning algorithms on this task. We also create our own corpus of 2000 positive and negative examples of problems and solutions. We find that we can distinguish problems from non-problems with an accuracy of 82.3%, and solutions from non-solutions with an accuracy of 79.7%. Our three most helpful features for the task are syntactic information (POS tags), document and word embeddings.

Similar content being viewed by others

Pattern-Based Acquisition of Scientific Entities from Scholarly Article Titles

Towards Computation of Novel Ideas from Corpora of Scientific Text

Ensemble Labeling Towards Scientific Information Extraction (ELSIE)

Avoid common mistakes on your manuscript.

Introduction

Problem solving is generally regarded as the most important cognitive activity in everyday and professional contexts (Jonassen 2000 ). Many studies on formalising the cognitive process behind problem-solving exist, for instance (Chandrasekaran 1983 ). Jordan ( 1980 ) argues that we all share knowledge of the thought/action problem-solution process involved in real life, and so our writings will often reflect this order. There is general agreement amongst theorists that state that the nature of the research process can be viewed as a problem-solving activity (Strübing 2007 ; Van Dijk 1980 ; Hutchins 1977 ; Grimes 1975 ).

One of the best-documented problem-solving patterns was established by Winter ( 1968 ). Winter analysed thousands of examples of technical texts, and noted that these texts can largely be described in terms of a four-part pattern consisting of Situation, Problem, Solution and Evaluation. This is very similar to the pattern described by Van Dijk ( 1980 ), which consists of Introduction-Theory, Problem-Experiment-Comment and Conclusion. The difference is that in Winter’s view, a solution only becomes a solution after it has been evaluated positively. Hoey changes Winter’s pattern by introducing the concept of Response in place of Solution (Hoey 2001 ). This seems to describe the situation in science better, where evaluation is mandatory for research solutions to be accepted by the community. In Hoey’s pattern, the Situation (which is generally treated as optional) provides background information; the Problem describes an issue which requires attention; the Response provides a way to deal with the issue, and the Evaluation assesses how effective the response is.

An example of this pattern in the context of the Goldilocks story can be seen in Fig.  1 . In this text, there is a preamble providing the setting of the story (i.e. Goldilocks is lost in the woods), which is called the Situation in Hoey’s system. A Problem in encountered when Goldilocks becomes hungry. Her first Response is to try the porridge in big bear’s bowl, but she gives this a negative Evaluation (“too hot!”) and so the pattern returns to the Problem. This continues in a cyclic fashion until the Problem is finally resolved by Goldilocks giving a particular Response a positive Evaluation of baby bear’s porridge (“it’s just right”).

Reproduced with permission from Hoey ( 2001 )

Example of problem-solving pattern when applied to the Goldilocks story.

It would be attractive to detect problem and solution statements automatically in text. This holds true both from a theoretical and a practical viewpoint. Theoretically, we know that sentiment detection is related to problem-solving activity, because of the perception that “bad” situations are transformed into “better” ones via problem-solving. The exact mechanism of how this can be detected would advance the state of the art in text understanding. In terms of linguistic realisation, problem and solution statements come in many variants and reformulations, often in the form of positive or negated statements about the conditions, results and causes of problem–solution pairs. Detecting and interpreting those would give us a reasonably objective manner to test a system’s understanding capacity. Practically, being able to detect any mention of a problem is a first step towards detecting a paper’s specific research goal. Being able to do this has been a goal for scientific information retrieval for some time, and if successful, it would improve the effectiveness of scientific search immensely. Detecting problem and solution statements of papers would also enable us to compare similar papers and eventually even lead to automatic generation of review articles in a field.

There has been some computational effort on the task of identifying problem-solving patterns in text. However, most of the prior work has not gone beyond the usage of keyword analysis and some simple contextual examination of the pattern. Flowerdew ( 2008 ) presents a corpus-based analysis of lexio-grammatical patterns for problem and solution clauses using articles from professional and student reports. Problem and solution keywords were used to search their corpora, and each occurrence was analysed to determine grammatical usage of the keyword. More interestingly, the causal category associated with each keyword in their context was also analysed. For example, Reason–Result or Means-Purpose were common causal categories found to be associated with problem keywords.

The goal of the work by Scott ( 2001 ) was to determine words which are semantically similar to problem and solution, and to determine how these words are used to signal problem-solution patterns. However, their corpus-based analysis used articles from the Guardian newspaper. Since the domain of newspaper text is very different from that of scientific text, we decided not to consider those keywords associated with problem-solving patterns for use in our work.

Instead of a keyword-based approach, Charles ( 2011 ) used discourse markers to examine how the problem-solution pattern was signalled in text. In particular, they examined how adverbials associated with a result such as “thus, therefore, then, hence” are used to signal a problem-solving pattern.

Problem solving also has been studied in the framework of discourse theories such as Rhetorical Structure Theory (Mann and Thompson 1988 ) and Argumentative Zoning (Teufel et al. 2000 ). Problem- and solutionhood constitute two of the original 23 relations in RST (Mann and Thompson 1988 ). While we concentrate solely on this aspect, RST is a general theory of discourse structure which covers many intentional and informational relations. The relationship to Argumentative Zoning is more complicated. The status of certain statements as problem or solutions is one important dimension in the definitions of AZ categories. AZ additionally models dimensions other than problem-solution hood (such as who a scientific idea belongs to, or which intention the authors might have had in stating a particular negative or positive statement). When forming categories, AZ combines aspects of these dimensions, and “flattens” them out into only 7 categories. In AZ it is crucial who it is that experiences the problems or contributes a solution. For instance, the definition of category “CONTRAST” includes statements that some research runs into problems, but only if that research is previous work (i.e., not if it is the work contributed in the paper itself). Similarly, “BASIS” includes statements of successful problem-solving activities, but only if they are achieved by previous work that the current paper bases itself on. Our definition is simpler in that we are interested only in problem solution structure, not in the other dimensions covered in AZ. Our definition is also more far-reaching than AZ, in that we are interested in all problems mentioned in the text, no matter whose problems they are. Problem-solution recognition can therefore be seen as one aspect of AZ which can be independently modelled as a “service task”. This means that good problem solution structure recognition should theoretically improve AZ recognition.

In this work, we approach the task of identifying problem-solving patterns in scientific text. We choose to use the model of problem-solving described by Hoey ( 2001 ). This pattern comprises four parts: Situation, Problem, Response and Evaluation. The Situation element is considered optional to the pattern, and so our focus centres on the core pattern elements.

Goal statement and task

Many surface features in the text offer themselves up as potential signals for detecting problem-solving patterns in text. However, since Situation is an optional element, we decided to focus on either Problem or Response and Evaluation as signals of the pattern. Moreover, we decide to look for each type in isolation. Our reasons for this are as follows: It is quite rare for an author to introduce a problem without resolving it using some sort of response, and so this is a good starting point in identifying the pattern. There are exceptions to this, as authors will sometimes introduce a problem and then leave it to future work, but overall there should be enough signal in the Problem element to make our method of looking for it in isolation worthwhile. The second signal we look for is the use of Response and Evaluation within the same sentence. Similar to Problem elements, we hypothesise that this formulation is well enough signalled externally to help us in detecting the pattern. For example, consider the following Response and Evaluation: “One solution is to use smoothing”. In this statement, the author is explicitly stating that smoothing is a solution to a problem which must have been mentioned in a prior statement. In scientific text, we often observe that solutions implicitly contain both Response and Evaluation (positive) elements. Therefore, due to these reasons there should be sufficient external signals for the two pattern elements we concentrate on here.

When attempting to find Problem elements in text, we run into the issue that the word “problem” actually has at least two word senses that need to be distinguished. There is a word sense of “problem” that means something which must be undertaken (i.e. task), while another sense is the core sense of the word, something that is problematic and negative. Only the latter sense is aligned with our sense of problemhood. This is because the simple description of a task does not predispose problemhood, just a wish to perform some act. Consider the following examples, where the non-desired word sense is being used:

“Das and Petrov (2011) also consider the problem of unsupervised bilingual POS induction”. (Chen et al. 2011 ).

“In this paper, we describe advances on the problem of NER in Arabic Wikipedia”. (Mohit et al. 2012 ).

Here, the author explicitly states that the phrases in orange are problems, they align with our definition of research tasks and not with what we call here ‘problematic problems’. We will now give some examples from our corpus for the desired, core word sense:

“The major limitation of supervised approaches is that they require annotations for example sentences.” (Poon and Domingos 2009 ).

“To solve the problem of high dimensionality we use clustering to group the words present in the corpus into much smaller number of clusters”. (Saha et al. 2008 ).

When creating our corpus of positive and negative examples, we took care to select only problem strings that satisfy our definition of problemhood; “ Corpus creation ” section will explain how we did that.

Corpus creation

Our new corpus is a subset of the latest version of the ACL anthology released in March, 2016 Footnote 1 which contains 22,878 articles in the form of PDFs and OCRed text. Footnote 2

The 2016 version was also parsed using ParsCit (Councill et al. 2008 ). ParsCit recognises not only document structure, but also bibliography lists as well as references within running text. A random subset of 2500 papers was collected covering the entire ACL timeline. In order to disregard non-article publications such as introductions to conference proceedings or letters to the editor, only documents containing abstracts were considered. The corpus was preprocessed using tokenisation, lemmatisation and dependency parsing with the Rasp Parser (Briscoe et al. 2006 ).

Definition of ground truth

Our goal was to define a ground truth for problem and solution strings, while covering as wide a range as possible of syntactic variations in which such strings naturally occur. We also want this ground truth to cover phenomena of problem and solution status which are applicable whether or not the problem or solution status is explicitly mentioned in the text.

To simplify the task, we only consider here problem and solution descriptions that are at most one sentence long. In reality, of course, many problem descriptions and solution descriptions go beyond single sentence, and require for instance an entire paragraph. However, we also know that short summaries of problems and solutions are very prevalent in science, and also that these tend to occur in the most prominent places in a paper. This is because scientists are trained to express their contribution and the obstacles possibly hindering their success, in an informative, succinct manner. That is the reason why we can afford to only look for shorter problem and solution descriptions, ignoring those that cross sentence boundaries.

Example of our extraction method for problems using dependencies. (Color figure online)

To define our ground truth, we examined the parsed dependencies and looked for a target word (“problem/solution”) in subject position, and then chose its syntactic argument as our candidate problem or solution phrase. To increase the variation, i.e., to find as many different-worded problem and solution descriptions as possible, we additionally used semantically similar words (near-synonyms) of the target words “problem” or “solution” for the search. Semantic similarity was defined as cosine in a deep learning distributional vector space, trained using Word2Vec (Mikolov et al. 2013 ) on 18,753,472 sentences from a biomedical corpus based on all full-text Pubmed articles (McKeown et al. 2016 ). From the 200 words which were semantically closest to “problem”, we manually selected 28 clear synonyms. These are listed in Table  1 . From the 200 semantically closest words to “solution” we similarly chose 19 (Table  2 ). Of the sentences matching our dependency search, a subset of problem and solution candidate sentences were randomly selected.

An example of this is shown in Fig.  2 . Here, the target word “drawback” is in subject position (highlighted in red), and its clausal argument (ccomp) is “(that) it achieves low performance” (highlighted in purple). Examples of other arguments we searched for included copula constructions and direct/indirect objects.

If more than one candidate was found in a sentence, one was chosen at random. Non-grammatical sentences were excluded; these might appear in the corpus as a result of its source being OCRed text.

800 candidates phrases expressing problems and solutions were automatically extracted (1600 total) and then independently checked for correctness by two annotators (the two authors of this paper). Both authors found the task simple and straightforward. Correctness was defined by two criteria:

The sentence must unambiguously and clearly state the phrase’s status as either a problem or a solution. For problems, the guidelines state that the phrase has to represent one of the following:

An unexplained phenomenon or a problematic state in science; or

A research question; or

An artifact that does not fulfil its stated specification.

For solutions, the phrase had to represent a response to a problem with a positive evaluation. Implicit solutions were also allowed.

The phrase must not lexically give away its status as problem or solution phrase.

The second criterion saves us from machine learning cues that are too obvious. If for instance, the phrase itself contained the words “lack of” or “problematic” or “drawback”, our manual check rejected it, because it would be too easy for the machine learner to learn such cues, at the expense of many other, more generally occurring cues.

Sampling of negative examples

We next needed to find negative examples for both cases. We wanted them not to stand out on the surface as negative examples, so we chose them so as to mimic the obvious characteristics of the positive examples as closely as possible. We call the negative examples ‘non-problems’ and ‘non-solutions’ respectively. We wanted the only differences between problems and non-problems to be of a semantic nature, nothing that could be read off on the surface. We therefore sampled a population of phrases that obey the same statistical distribution as our problem and solution strings while making sure they really are negative examples. We started from sentences not containing any problem/solution words (i.e. those used as target words). From each such sentence, we at random selected one syntactic subtree contained in it. From these, we randomly selected a subset of negative examples of problems and solutions that satisfy the following conditions:

The distribution of the head POS tags of the negative strings should perfectly match the head POS tags Footnote 3 of the positive strings. This has the purpose of achieving the same proportion of surface syntactic constructions as observed in the positive cases.

The average lengths of the negative strings must be within a tolerance of the average length of their respective positive candidates e.g., non-solutions must have an average length very similar (i.e. \(+/-\) small tolerance) to solutions. We chose a tolerance value of 3 characters.

Again, a human quality check was performed on non-problems and non-solutions. For each candidate non-problem statement, the candidate was accepted if it did not contain a phenomenon, a problematic state, a research question or a non-functioning artefact. If the string expressed a research task, without explicit statement that there was anything problematic about it (i.e., the ‘wrong’ sense of “problem”, as described above), it was allowed as a non-problem. A clause was confirmed as a non-solution if the string did not represent both a response and positive evaluation.

If the annotator found that the sentence had been slightly mis-parsed, but did contain a candidate, they were allowed to move the boundaries for the candidate clause. This resulted in cleaner text, e.g., in the frequent case of coordination, when non-relevant constituents could be removed.

From the set of sentences which passed the quality-test for both independent assessors, 500 instances of positive and negative problems/solutions were randomly chosen (i.e. 2000 instances in total). When checking for correctness we found that most of the automatically extracted phrases which did not pass the quality test for problem-/solution-hood were either due to obvious learning cues or instances where the sense of problem-hood used is relating to tasks (cf. “ Goal statement and task ” section).

Experimental design

In our experiments, we used three classifiers, namely Naïve Bayes, Logistic Regression and a Support Vector Machine. For all classifiers an implementation from the WEKA machine learning library (Hall et al. 2009 ) was chosen. Given that our dataset is small, tenfold cross-validation was used instead of a held out test set. All significance tests were conducted using the (two-tailed) Sign Test (Siegel 1956 ).

Linguistic correlates of problem- and solution-hood

We first define a set of features without taking the phrase’s context into account. This will tell us about the disambiguation ability of the problem/solution description’s semantics alone. In particular, we cut out the rest of the sentence other than the phrase and never use it for classification. This is done for similar reasons to excluding certain ‘give-away’ phrases inside the phrases themselves (as explained above). As the phrases were found using templates, we know that the machine learner would simply pick up on the semantics of the template, which always contains a synonym of “problem” or “solution”, thus drowning out the more hidden features hopefully inherent in the semantics of the phrases themselves. If we allowed the machine learner to use these stronger features, it would suffer in its ability to generalise to the real task.

ngrams Bags of words are traditionally successfully used for classification tasks in NLP, so we included bags of words (lemmas) within the candidate phrases as one of our features (and treat it as a baseline later on). We also include bigrams and trigrams as multi-word combinations can be indicative of problems and solutions e.g., “combinatorial explosion”.

Polarity Our second feature concerns the polarity of each word in the candidate strings. Consider the following example of a problem taken from our dataset: “very conservative approaches to exact and partial string matches overgenerate badly”. In this sentence, words such as “badly” will be associated with negative polarity, therefore being useful in determining problem-hood. Similarly, solutions will often be associated with a positive sentiment e.g. “smoothing is a good way to overcome data sparsity” . To do this, we perform word sense disambiguation of each word using the Lesk algorithm (Lesk 1986 ). The polarity of the resulting synset in SentiWordNet (Baccianella et al. 2010 ) was then looked up and used as a feature.

Syntax Next, a set of syntactic features were defined by using the presence of POS tags in each candidate. This feature could be helpful in finding syntactic patterns in problems and solutions. We were careful not to base the model directly on the head POS tag and the length of each candidate phrase, as these are defining characteristics used for determining the non-problem and non-solution candidate set.

Negation Negation is an important property that can often greatly affect the polarity of a phrase. For example, a phrase containing a keyword pertinent to solution-hood may be a good indicator but with the presence of negation may flip the polarity to problem-hood e.g., “this can’t work as a solution”. Therefore, presence of negation is determined.

Exemplification and contrast Problems and solutions are often found to be coupled with examples as they allow the author to elucidate their point. For instance, consider the following solution: “Once the translations are generated, an obvious solution is to pick the most fluent alternative, e.g., using an n-gram language model”. (Madnani et al. 2012 ). To acknowledge this, we check for presence of exemplification. In addition to examples, problems in particular are often found when contrast is signalled by the author (e.g. “however, “but”), therefore we also check for presence of contrast in the problem and non-problem candidates only.

Discourse Problems and solutions have also been found to have a correlation with discourse properties. For example, problem-solving patterns often occur in the background sections of a paper. The rationale behind this is that the author is conventionally asked to objectively criticise other work in the background (e.g. describing research gaps which motivate the current paper). To take this in account, we examine the context of each string and capture the section header under which it is contained (e.g. Introduction, Future work). In addition, problems and solutions are often found following the Situation element in the problem-solving pattern (cf. “ Introduction ” section). This preamble setting up the problem or solution means that these elements are likely not to be found occurring at the beginning of a section (i.e. it will usually take some sort of introduction to detail how something is problematic and why a solution is needed). Therefore we record the distance from the candidate string to the nearest section header.

Subcategorisation and adverbials Solutions often involve an activity (e.g. a task). We also model the subcategorisation properties of the verbs involved. Our intuition was that since problematic situations are often described as non-actions, then these are more likely to be intransitive. Conversely solutions are often actions and are likely to have at least one argument. This feature was calculated by running the C&C parser (Curran et al. 2007 ) on each sentence. C&C is a supertagger and parser that has access to subcategorisation information. Solutions are also associated with resultative adverbial modification (e.g. “thus, therefore, consequently”) as it expresses the solutionhood relation between the problem and the solution. It has been seen to occur frequently in problem-solving patterns, as studied by Charles ( 2011 ). Therefore, we check for presence of resultative adverbial modification in the solution and non-solution candidate only.

Embeddings We also wanted to add more information using word embeddings. This was done in two different ways. Firstly, we created a Doc2Vec model (Le and Mikolov 2014 ), which was trained on \(\sim \,19\)  million sentences from scientific text (no overlap with our data set). An embedding was created for each candidate sentence. Secondly, word embeddings were calculated using the Word2Vec model (cf. “ Corpus creation ” section). For each candidate head, the full word embedding was included as a feature. Lastly, when creating our polarity feature we query SentiWordNet using synsets assigned by the Lesk algorithm. However, not all words are assigned a sense by Lesk, so we need to take care when that happens. In those cases, the distributional semantic similarity of the word is compared to two words with a known polarity, namely “poor” and “excellent”. These particular words have traditionally been consistently good indicators of polarity status in many studies (Turney 2002 ; Mullen and Collier 2004 ). Semantic similarity was defined as cosine similarity on the embeddings of the Word2Vec model (cf. “ Corpus creation ” section).

Modality Responses to problems in scientific writing often express possibility and necessity, and so have a close connection with modality. Modality can be broken into three main categories, as described by Kratzer ( 1991 ), namely epistemic (possibility), deontic (permission / request / wish) and dynamic (expressing ability).

Problems have a strong relationship to modality within scientific writing. Often, this is due to a tactic called “hedging” (Medlock and Briscoe 2007 ) where the author uses speculative language, often using Epistemic modality, in an attempt to make either noncommital or vague statements. This has the effect of allowing the author to distance themselves from the statement, and is often employed when discussing negative or problematic topics. Consider the following example of Epistemic modality from Nakov and Hearst ( 2008 ): “A potential drawback is that it might not work well for low-frequency words”.

To take this linguistic correlate into account as a feature, we replicated a modality classifier as described by (Ruppenhofer and Rehbein 2012 ). More sophisticated modality classifiers have been recently introduced, for instance using a wide range of features and convolutional neural networks, e.g, (Zhou et al. 2015 ; Marasović and Frank 2016 ). However, we wanted to check the effect of a simpler method of modality classification on the final outcome first before investing heavily into their implementation. We trained three classifiers using the subset of features which Ruppenhofer et al. reported as performing best, and evaluated them on the gold standard dataset provided by the authors Footnote 4 . The results of the are shown in Table  3 . The dataset contains annotations of English modal verbs on the 535 documents of the first MPQA corpus release (Wiebe et al. 2005 ).

Logistic Regression performed best overall and so this model was chosen for our upcoming experiments. With regards to the optative and concessive modal categories, they can be seen to perform extremely poorly, with the optative category receiving a null score across all three classifiers. This is due to a limitation in the dataset, which is unbalanced and contains very few instances of these two categories. This unbalanced data also is the reason behind our decision of reporting results in terms of recall, precision and f-measure in Table  3 .

The modality classifier was then retrained on the entirety of the dataset used by Ruppenhofer and Rehbein ( 2012 ) using the best performing model from training (Logistic Regression). This new model was then used in the upcoming experiment to predict modality labels for each instance in our dataset.

As can be seen from Table  4 , we are able to achieve good results for distinguishing a problematic statement from non-problematic one. The bag-of-words baseline achieves a very good performance of 71.0% for the Logistic Regression classifier, showing that there is enough signal in the candidate phrases alone to distinguish them much better than random chance.

Taking a look at Table  5 , which shows the information gain for the top lemmas,

we can see that the top lemmas are indeed indicative of problemhood (e.g. “limit”,“explosion”). Bigrams achieved good performance on their own (as did negation and discourse) but unfortunately performance deteriorated when using trigrams, particularly with the SVM and LR. The subcategorisation feature was the worst performing feature in isolation. Upon taking a closer look at our data, we saw that our hypothesis that intransitive verbs are commonly used in problematic statements was true, with over 30% of our problems (153) using them. However, due to our sampling method for the negative cases we also picked up many intransitive verbs (163). This explains the almost random chance performance (i.e.  50%) given that the distribution of intransitive verbs amongst the positive and negative candidates was almost even.

The modality feature was the most expensive to produce, but also didn’t perform very well is isolation. This surprising result may be partly due to a data sparsity issue

where only a small portion (169) of our instances contained modal verbs. The breakdown of how many types of modal senses which occurred is displayed in Table  6 . The most dominant modal sense was epistemic. This is a good indicator of problemhood (e.g. hedging, cf. “ Linguistic correlates of problem- and solution-hood ” section) but if the accumulation of additional data was possible, we think that this feature may have the potential to be much more valuable in determining problemhood. Another reason for the performance may be domain dependence of the classifier since it was trained on text from different domains (e.g. news). Additionally, modality has also shown to be helpful in determining contextual polarity (Wilson et al. 2005 ) and argumentation (Becker et al. 2016 ), so using the output from this modality classifier may also prove useful for further feature engineering taking this into account in future work.

Polarity managed to perform well but not as good as we hoped. However, this feature also suffers from a sparsity issue resulting from cases where the Lesk algorithm (Lesk 1986 ) is not able to resolve the synset of the syntactic head.

Knowledge of syntax provides a big improvement with a significant increase over the baseline results from two of the classifiers.

Examining this in greater detail, POS tags with high information gain mostly included tags from open classes (i.e. VB-, JJ-, NN- and RB-). These tags are often more associated with determining polarity status than tags such as prepositions and conjunctions (i.e. adverbs and adjectives are more likely to be describing something with a non-neutral viewpoint).

The embeddings from Doc2Vec allowed us to obtain another significant increase in performance (72.9% with Naïve Bayes) over the baseline and polarity using Word2Vec provided the best individual feature result (77.2% with SVM).

Combining all features together, each classifier managed to achieve a significant result over the baseline with the best result coming from the SVM (81.8%). Problems were also better classified than non-problems as shown in the confusion matrix in Table  7 . The addition of the Word2Vec vectors may be seen as a form of smoothing in cases where previous linguistic features had a sparsity issue i.e., instead of a NULL entry, the embeddings provide some sort of value for each candidate. Particularly wrt. the polarity feature, cases where Lesk was unable to resolve a synset meant that a ZERO entry was added to the vector supplied to the machine learner. Amongst the possible combinations, the best subset of features was found by combining all features with the exception of bigrams, trigrams, subcategorisation and modality. This subset of features managed to improve results in both the Naïve Bayes and SVM classifiers with the highest overall result coming from the SVM (82.3%).

The results for disambiguation of solutions from non-solutions can be seen in Table  8 . The bag-of-words baseline performs much better than random, with the performance being quite high with regard to the SVM (this result was also higher than any of the baseline performances from the problem classifiers). As shown in Table  9 , the top ranked lemmas from the best performing model (using information gain) included “use” and “method”. These lemmas are very indicative of solutionhood and so give some insight into the high baseline returned from the machine learners. Subcategorisation and the result adverbials were the two worst performing features. However, the low performance for subcategorisation is due to the sampling of the non-solutions (the same reason for the low performance of the problem transitivity feature). When fitting the POS-tag distribution for the negative samples, we noticed that over 80% of the head POS-tags were verbs (much higher than the problem heads). The most frequent verb type being the infinite form.

This is not surprising given that a very common formulation to describe a solution is to use the infinitive “TO” since it often describes a task e.g., “One solution is to find the singletons and remove them”. Therefore, since the head POS tags of the non-solutions had to match this high distribution of infinitive verbs present in the solution, the subcategorisation feature is not particularly discriminatory. Polarity, negation, exemplification and syntactic features were slightly more discriminate and provided comparable results. However, similar to the problem experiment, the embeddings from Word2Vec and Doc2Vec proved to be the best features, with polarity using Word2Vec providing the best individual result (73.4% with SVM).

Combining all features together managed to improve over each feature in isolation and beat the baseline using all three classifiers. Furthermore, when looking at the confusion matrix in Table  10 the solutions were classified more accurately than the non-solutions. The best subset of features was found by combining all features without adverbial of result, bigrams, exemplification, negation, polarity and subcategorisation. The best result using this subset of features was achieved by the SVM with 79.7%. It managed to greatly improve upon the baseline but was just shy of achieving statistical significance ( \(p=0.057\) ).

In this work, we have presented new supervised classifiers for the task of identifying problem and solution statements in scientific text. We have also introduced a new corpus for this task and used it for evaluating our classifiers. Great care was taken in constructing the corpus by ensuring that the negative and positive samples were closely matched in terms of syntactic shape. If we had simply selected random subtrees for negative samples without regard for any syntactic similarity with our positive samples, the machine learner may have found easy signals such as sentence length. Additionally, since we did not allow the machine learner to see the surroundings of the candidate string within the sentence, this made our task even harder. Our performance on the corpus shows promise for this task, and proves that there are strong signals for determining both the problem and solution parts of the problem-solving pattern independently.

With regard to classifying problems from non-problems, features such as the POS tag, document and word embeddings provide the best features, with polarity using the Word2Vec embeddings achieving the highest feature performance. The best overall result was achieved using an SVM with a subset of features (82.3%). Classifying solutions from non-solutions also performs well using the embedding features, with the best feature also being polarity using the Word2Vec embeddings, and the highest result also coming from the SVM with a feature subset (79.7%).

In future work, we plan to link problem and solution statements which were found independently during our corpus creation. Given that our classifiers were trained on data solely from the ACL anthology, we also hope to investigate the domain specificity of our classifiers and see how well they can generalise to domains other than ACL (e.g. bioinformatics). Since we took great care at removing the knowledge our classifiers have of the explicit statements of problem and solution (i.e. the classifiers were trained only on the syntactic argument of the explicit statement of problem-/solution-hood), our classifiers should in principle be in a good position to generalise, i.e., find implicit statements too. In future work, we will measure to which degree this is the case.

To facilitate further research on this topic, all code and data used in our experiments can be found here: www.cl.cam.ac.uk/~kh562/identifying-problems-and-solutions.html

http://acl-arc.comp.nus.edu.sg/ .

The corpus comprises 3,391,198 sentences, 71,149,169 words and 451,996,332 characters.

The head POS tags were found using a modification of the Collins’ Head Finder. This modified algorithm addresses some of the limitations of the head finding heuristics described by Collins ( 2003 ) and can be found here: http://nlp.stanford.edu/nlp/javadoc/javanlp/edu/stanford/nlp/trees/ModCollinsHeadFinder.html .

https://www.uni-hildesheim.de/ruppenhofer/data/modalia_release1.0.tgz.

Baccianella, S., Esuli, A., & Sebastiani, F. (2010). Sentiwordnet 3.0: An enhanced lexical resource for sentiment analysis and opinion mining. LREC , 10 , 2200–2204.

Google Scholar  

Becker, M., Palmer, A., & Frank, A. (2016). Clause types and modality in argumentative microtexts. In Workshop on foundations of the language of argumentation (in conjunction with COMMA) .

Briscoe, T., Carroll, J., & Watson, R. (2006). The second release of the rasp system. In Proceedings of the COLING/ACL on interactive presentation sessions, association for computational linguistics pp. 77–80.

Chandrasekaran, B. (1983). Towards a taxonomy of problem solving types. AI Magazine , 4 (1), 9.

Charles, M. (2011). Adverbials of result: Phraseology and functions in the problem-solution pattern. Journal of English for Academic Purposes , 10 (1), 47–60.

Article   Google Scholar  

Chen, D., Dyer, C., Cohen, S. B., & Smith, N. A. (2011). Unsupervised bilingual pos tagging with markov random fields. In Proceedings of the first workshop on unsupervised learning in NLP, association for computational linguistics pp. 64–71.

Collins, M. (2003). Head-driven statistical models for natural language parsing. Computational Linguistics , 29 (4), 589–637.

Article   MathSciNet   MATH   Google Scholar  

Councill, I. G., Giles, C. L., & Kan, M. Y. (2008). Parscit: An open-source CRF reference string parsing package. In LREC .

Curran, J. R., Clark, S., & Bos, J. (2007). Linguistically motivated large-scale NLP with C&C and boxer. In Proceedings of the 45th annual meeting of the ACL on interactive poster and demonstration sessions, association for computational linguistics pp. 33–36.

Flowerdew, L. (2008). Corpus-based analyses of the problem-solution pattern: A phraseological approach (Vol. 29). Amsterdam: John Benjamins Publishing.

Book   Google Scholar  

Grimes, J. E. (1975). The thread of discourse (Vol. 207). Berlin: Walter de Gruyter.

Hall, M., Frank, E., Holmes, G., Pfahringer, B., Reutemann, P., & Witten, I. H. (2009). The weka data mining software: An update. ACM SIGKDD Explorations Newsletter , 11 (1), 10–18.

Hoey, M. (2001). Textual interaction: An introduction to written discourse analysis . Portland: Psychology Press.

Hutchins, J. (1977). On the structure of scientific texts. UEA Papers in Linguistics , 5 (3), 18–39.

Jonassen, D. H. (2000). Toward a design theory of problem solving. Educational Technology Research and Development , 48 (4), 63–85.

Jordan, M. P. (1980). Short texts to explain problem-solution structures-and vice versa. Instructional Science , 9 (3), 221–252.

Kratzer, A. (1991). Modality. In von Stechow & Wunderlich (Eds.), Semantics: An international handbook of contemporary research .

Le, Q. V., & Mikolov, T. (2014). Distributed representations of sentences and documents. ICML , 14 , 1188–1196.

Lesk, M. (1986). Automatic sense disambiguation using machine readable dictionaries: How to tell a pine cone from an ice cream cone. In Proceedings of the 5th annual international conference on Systems documentation, ACM (pp. 24–26).

Madnani, N., Tetreault, J., & Chodorow, M. (2012). Exploring grammatical error correction with not-so-crummy machine translation. In Proceedings of the seventh workshop on building educational applications using NLP, association for computational linguistics pp. 44–53.

Mann, W. C., & Thompson, S. A. (1988). Rhetorical structure theory: Toward a functional theory of text organization. Text-Interdisciplinary Journal for the Study of Discourse , 8 (3), 243–281.

Marasović, A., & Frank, A. (2016). Multilingual modal sense classification using a convolutional neural network. In Proceedings of the 1st Workshop on Representation Learning for NLP .

McKeown, K., Daume, H., Chaturvedi, S., Paparrizos, J., Thadani, K., Barrio, P., et al. (2016). Predicting the impact of scientific concepts using full-text features. Journal of the Association for Information Science and Technology , 67 , 2684–2696.

Medlock, B., & Briscoe, T. (2007). Weakly supervised learning for hedge classification in scientific literature. ACL, Citeseer , 2007 , 992–999.

Mikolov, T., Sutskever, I., Chen, K., Corrado, G. S., & Dean, J. (2013). Distributed representations of words and phrases and their compositionality. In Advances in neural information processing systems (pp. 3111–3119).

Mohit, B., Schneider, N., Bhowmick, R., Oflazer, K., & Smith, N. A. (2012). Recall-oriented learning of named entities in arabic wikipedia. In Proceedings of the 13th conference of the European chapter of the association for computational linguistics, association for computational linguistics (pp. 162–173).

Mullen, T., & Collier, N. (2004). Sentiment analysis using support vector machines with diverse information sources. EMNLP , 4 , 412–418.

Nakov, P., Hearst, M. A. (2008). Solving relational similarity problems using the web as a corpus. In: ACL (pp. 452–460).

Poon, H., & Domingos, P. (2009). Unsupervised semantic parsing. In Proceedings of the 2009 conference on empirical methods in natural language processing: Volume 1-association for computational linguistics (pp. 1–10).

Ruppenhofer, J., & Rehbein, I. (2012). Yes we can!? Annotating the senses of English modal verbs. In Proceedings of the 8th international conference on language resources and evaluation (LREC), Citeseer (pp. 24–26).

Saha, S. K., Mitra, P., & Sarkar, S. (2008). Word clustering and word selection based feature reduction for maxent based hindi ner. In ACL (pp. 488–495).

Scott, M. (2001). Mapping key words to problem and solution. In Patterns of text: In honour of Michael Hoey Benjamins, Amsterdam (pp. 109–127).

Siegel, S. (1956). Nonparametric statistics for the behavioral sciences . New York: McGraw-hill.

MATH   Google Scholar  

Strübing, J. (2007). Research as pragmatic problem-solving: The pragmatist roots of empirically-grounded theorizing. In The Sage handbook of grounded theory (pp. 580–602).

Teufel, S., et al. (2000). Argumentative zoning: Information extraction from scientific text. PhD Thesis, Citeseer .

Turney, P. D. (2002). Thumbs up or thumbs down?: Semantic orientation applied to unsupervised classification of reviews. In Proceedings of the 40th annual meeting on association for computational linguistics, association for computational linguistics (pp. 417–424).

Van Dijk, T. A. (1980). Text and context explorations in the semantics and pragmatics of discourse . London: Longman.

Wiebe, J., Wilson, T., & Cardie, C. (2005). Annotating expressions of opinions and emotions in language. Language Resources and Evaluation , 39 (2), 165–210.

Wilson, T., Wiebe, J., & Hoffmann, P. (2005). Recognizing contextual polarity in phrase-level sentiment analysis. In Proceedings of the conference on human language technology and empirical methods in natural language processing, association for computational linguistics (pp. 347–354).

Winter, E. O. (1968). Some aspects of cohesion. In Sentence and clause in scientific English . University College London.

Zhou, M., Frank, A., Friedrich, A., & Palmer, A. (2015). Semantically enriched models for modal sense classification. In Workshop on linking models of lexical, sentential and discourse-level semantics (LSDSem) (p. 44).

Download references

Acknowledgements

The first author has been supported by an EPSRC studentship (Award Ref: 1641528). We thank the reviewers for their helpful comments.

Author information

Authors and affiliations.

Department of Computer Science and Technology, University of Cambridge, 15 JJ Thomson Avenue, Cambridge, CB3 0FD, UK

Kevin Heffernan & Simone Teufel

You can also search for this author in PubMed   Google Scholar

Corresponding author

Correspondence to Kevin Heffernan .

Rights and permissions

Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License ( http://creativecommons.org/licenses/by/4.0/ ), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made.

Reprints and permissions

About this article

Heffernan, K., Teufel, S. Identifying problems and solutions in scientific text. Scientometrics 116 , 1367–1382 (2018). https://doi.org/10.1007/s11192-018-2718-6

Download citation

Received : 08 October 2017

Published : 06 April 2018

Issue Date : August 2018

DOI : https://doi.org/10.1007/s11192-018-2718-6

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Problem-solving patterns
• Machine learning
• Find a journal
• Publish with us
• Track your research

• Newsletters

Site search

• Israel-Hamas war
• 2024 election
• Solar eclipse
• Supreme Court
• All explainers
• Future Perfect

Filed under:

Science has been in a “replication crisis” for a decade. Have we learned anything?

Bad papers are still published. But some other things might be getting better.

Share this story

• Share this on Facebook
• Share this on Twitter
• Share this on Reddit
• Share All sharing options

Share All sharing options for: Science has been in a “replication crisis” for a decade. Have we learned anything?

Much ink has been spilled over the “replication crisis” in the last decade and a half, including here at Vox . Researchers have discovered, over and over, that lots of findings in fields like psychology, sociology, medicine, and economics don’t hold up when other researchers try to replicate them.

This conversation was fueled in part by John Ioannidis’s 2005 article “ Why Most Published Research Findings Are False” and by the controversy around a 2011 paper that used then-standard statistical methods to find that people have precognition . But since then, many researchers have explored the replication crisis from different angles. Why are research findings so often unreliable? Is the problem just that we test for “statistical significance” — the likelihood that similarly strong results could have occurred by chance — in a nuance-free way? Is it that null results (that is, when a study finds no detectable effects) are ignored while positive ones make it into journals?

A recent write-up by Alvaro de Menard, a participant in the Defense Advanced Research Project’s Agency’s (DARPA) replication markets project (more on this below), makes the case for a more depressing view: The processes that lead to unreliable research findings are routine, well understood, predictable, and in principle pretty easy to avoid. And yet, he argues, we’re still not improving the quality and rigor of social science research.

While other researchers I spoke with pushed back on parts of Menard’s pessimistic take, they do agree on something: a decade of talking about the replication crisis hasn’t translated into a scientific process that’s much less vulnerable to it. Bad science is still frequently published, including in top journals — and that needs to change.

Most papers fail to replicate for totally predictable reasons

Let’s take a step back and explain what people mean when they refer to the “replication crisis” in scientific research.

When research papers are published, they describe their methodology, so other researchers can copy it (or vary it) and build on the original research. When another research team tries to conduct a study based on the original to see if they find the same result, that’s an attempted replication. (Often the focus is not just on doing the exact same thing, but approaching the same question with a larger sample and preregistered design.) If they find the same result, that’s a successful replication, and evidence that the original researchers were on to something. But when the attempted replication finds different or no results, that often suggests that the original research finding was spurious.

In an attempt to test just how rigorous scientific research is, some researchers have undertaken the task of replicating research that’s been published in a whole range of fields. And as more and more of those attempted replications have come back, the results have been striking — it is not uncommon to find that many, many published studies cannot be replicated.

One 2015 attempt to reproduce 100 psychology studies was able to replicate only 39 of them. A big international effort in 2018 to reproduce prominent studies found that 14 of the 28 replicated, and an attempt to replicate studies from top journals Nature and Science found that 13 of the 21 results looked at could be reproduced.

The replication crisis has led a few researchers to ask: Is there a way to guess if a paper will replicate? A growing body of research has found that guessing which papers will hold up and which won’t is often just a matter of looking at the same simple, straightforward factors.

A 2019 paper by Adam Altmejd, Anna Dreber, and others identifies some simple factors that are highly predictive: Did the study have a reasonable sample size? Did the researchers squeeze out a result barely below the significance threshold of p = 0.05? (A paper can often claim a “significant” result if this “p” threshold is met, and many use various statistical tricks to push their paper across that line.) Did the study find an effect across the whole study population, or an “interaction effect” (such as an effect only in a smaller segment of the population) that is much less likely to replicate?

Menard argues that the problem is not so complicated. “Predicting replication is easy,” he said. “There’s no need for a deep dive into the statistical methodology or a rigorous examination of the data, no need to scrutinize esoteric theories for subtle errors — these papers have obvious, surface-level problems.”

A 2018 study published in Nature had scientists place bets on which of a pool of social science studies would replicate. They found that the predictions by scientists in this betting market were highly accurate at estimating which papers would replicate.

“These results suggest something systematic about papers that fail to replicate,” study co-author Anna Dreber argued after the study was released.

Additional research has established that you don’t even need to poll experts in a field to guess which of its studies will hold up to scrutiny. A study published in August had participants read psychology papers and predict whether they would replicate. “Laypeople without a professional background in the social sciences are able to predict the replicability of social-science studies with above-chance accuracy,” the study concluded, “on the basis of nothing more than simple verbal study descriptions.”

The laypeople were not as accurate in their predictions as the scientists in the Nature study, but the fact they were still able to predict many failed replications suggests that many of them have flaws that even a layperson can notice.

Bad science can still be published in prestigious journals and be widely cited

Publication of a peer-reviewed paper is not the final step of the scientific process. After a paper is published, other research might cite it — spreading any misconceptions or errors in the original paper. But research has established that scientists have good instincts for whether a paper will replicate or not. So, do scientists avoid citing papers that are unlikely to replicate?

This striking chart from a 2020 study by Yang Yang, Wu Youyou, and Brian Uzzi at Northwestern University illustrates their finding that actually, there is no correlation at all between whether a study will replicate and how often it is cited. “Failed papers circulate through the literature as quickly as replicating papers,” they argue.

Looking at a sample of studies from 2009 to 2017 that have since been subject to attempted replications, the researchers find that studies have about the same number of citations regardless of whether they replicated.

If scientists are pretty good at predicting whether a paper replicates, how can it be the case that they are as likely to cite a bad paper as a good one? Menard theorizes that many scientists don’t thoroughly check — or even read — papers once published, expecting that if they’re peer-reviewed, they’re fine. Bad papers are published by a peer-review process that is not adequate to catch them — and once they’re published, they are not penalized for being bad papers.

The debate over whether we’re making any progress

Here at Vox, we’ve written about how the replication crisis can guide us to do better science . And yet blatantly shoddy work is still being published in peer-reviewed journals despite errors that a layperson can see.

In many cases, journals effectively aren’t held accountable for bad papers — many, like The Lancet , have retained their prestige even after a long string of embarrassing public incidents where they published research that turned out fraudulent or nonsensical. (The Lancet said recently that, after a study on Covid-19 and hydroxychloroquine this spring was retracted after questions were raised about the data source, the journal would change its data-sharing practices. )

Even outright frauds often take a very long time to be repudiated, with some universities and journals dragging their feet and declining to investigate widespread misconduct .

That’s discouraging and infuriating. It suggests that the replication crisis isn’t one specific methodological reevaluation, but a symptom of a scientific system that needs rethinking on many levels. We can’t just teach scientists how to write better papers. We also need to change the fact that those better papers aren’t cited more often than bad papers; that bad papers are almost never retracted even when their errors are visible to lay readers; and that there are no consequences for bad research.

In some ways, the culture of academia actively selects for bad research. Pressure to publish lots of papers favors those who can put them together quickly — and one way to be quick is to be willing to cut corners. “Over time, the most successful people will be those who can best exploit the system,” Paul Smaldino, a cognitive science professor at the University of California Merced, told my colleague Brian Resnick.

So we have a system whose incentives keep pushing bad research even as we understand more about what makes for good research.

The world needs more wonder

The Unexplainable newsletter guides you through the most fascinating, unanswered questions in science — and the mind-bending ways scientists are trying to answer them. Sign up today .

Researchers working on the replication crisis are more divided, though, on the question of whether the last decade of work on the replication crisis has left us better equipped to fight these problems — or left us in the same place where we started.

“The future is bright,” concludes Altmejd and Dreber’s 2019 paper about how to predict replications. “There will be rapid accumulation of more replication data, more outlets for publishing replications, new statistical techniques, and—most importantly—enthusiasm for improving replicability among funding agencies, scientists, and journals. An exciting replicability ‘upgrade’ in science, while perhaps overdue, is taking place.”

Menard, by contrast, argues that this optimism has not been borne out — none of our improved understanding of the replication crisis leads to more papers being published that actually replicate. The project that he’s a part of — an effort to design a better model to predict which papers replicate run by DARPA in the Defense Department — has not seen papers grow any more likely to replicate over time .

“I frequently encounter the notion that after the replication crisis hit there was some sort of great improvement in the social sciences, that people wouldn’t even dream of publishing studies based on 23 undergraduates any more ... In reality there has been no discernible improvement,” he writes.

Researchers who are more optimistic point to other metrics of progress. It’s true that papers that fail replication are still extremely common, and that the peer-review process hasn’t improved in a way that catches these errors. But other elements of the error-correction process are getting better.

“Journals now retract about 1,500 articles annually — a nearly 40-fold increase over 2000, and a dramatic change even if you account for the roughly doubling or tripling of papers published per year,” Ivan Oransky at Retraction Watch argues. “Journals have improved,” reporting more details on retracted papers and improving their process for retractions.

Other changes in common scientific practices seem to be helping too. For example, preregistrations — announcing how you’ll conduct your analysis before you do the study — lead to more null results being published .

“I don’t think the influence [of public conversations about the replication crisis on scientific practice] has been zero,” statistician Andrew Gelman at Columbia University told me. “This crisis has influenced my own research practices, and I assume it’s influenced many others as well. And it’s my general impression that journals such as Psychological Science and PNAS don’t publish as much junk as they used to.”

There’s some reassurance in that. But until those improvements translate to a higher percentage of papers replicating and a difference in citations for good papers versus bad papers, it’s a small victory. And it’s a small victory that has been hard-won. After tons of resources spent demonstrating the scope of the problem, fighting for more retractions, teaching better statistical methods, and trying to drag fraud into the open, papers still don’t replicate as much as researchers would hope, and bad papers are still widely cited — suggesting a big part of the problem still hasn’t been touched.

We need a more sophisticated understanding of the replication crisis, not as a moment of realization after which we were able to move forward with higher standards, but as an ongoing rot in the scientific process that a decade of work hasn’t quite fixed.

Our scientific institutions are valuable, as are the tools they’ve built to help us understand the world. There’s no cause for hopelessness here, even if some frustration is thoroughly justified. Science needs saving, sure — but science is very much worth saving.

Will you support Vox today?

We believe that everyone deserves to understand the world that they live in. That kind of knowledge helps create better citizens, neighbors, friends, parents, and stewards of this planet. Producing deeply researched, explanatory journalism takes resources. You can support this mission by making a financial gift to Vox today. Will you join us?

We accept credit card, Apple Pay, and Google Pay. You can also contribute via

Next Up In Future Perfect

Sign up for the newsletter today, explained.

Understand the world with a daily explainer plus the most compelling stories of the day.

Thanks for signing up!

Check your inbox for a welcome email.

Oops. Something went wrong. Please enter a valid email and try again.

Life is hard. Can philosophy help?

Don’t sneer at white rural voters — or delude yourself about their politics

You probably shouldn’t panic about measles — yet

Biden’s student loan forgiveness plan, explained

A hack nearly gained access to millions of computers. Here’s what we should learn from this.

College enrollment is up. The financial aid mess could bring it crashing down.

March 1, 2014

Why Your First Idea Can Blind You to a Better One

While we are working through a problem, the brain’s tendency to stick with familiar ideas can literally blind us to superior solutions

By Merim Bilalić & Peter McLeod

Danny Schwartz

I n a classic 1942 experiment, American psychologist Abraham Luchins asked volunteers to do some basic math by picturing water jugs in their mind. Given three empty containers, for example, each with a different capacity—21, 127 and three units of water—the participants had to figure out how to transfer liquid between the containers to measure out precisely 100 units. They could fill and empty each jug as many times as they wanted, but they had to fill the vessels to their limits. The solution was to first fill the second jug to its capacity of 127 units, then empty it into the first to remove 21 units, leaving 106, and finally to fill the third jug twice to subtract six units for a remainder of 100. Luchins presented his volunteers with several more problems that could be solved with essentially the same three steps; they made quick work of them. Yet when he gave them a problem with a simpler and faster solution than the previous tasks, they failed to see it.

This time Luchins asked the participants to measure out 20 units of water using containers that could hold 23, 49 and three liquid units. The solution is obvious, right? Simply fill the first jug and empty it into the third one: 23 – 3 = 20. Yet many people in Luchins’s experiment persistently tried to solve the easier problem the old way, emptying the second container into the first and then into the third twice: 49 – 23 – 3 – 3 = 20. And when Luchins gave them a problem that had a two-step solution but could not be solved using the three-step method to which the volunteers had become accustomed, they gave up, saying it was impossible.

The water jug experiment is one of the most famous examples of the Einstellung effect: the human brain’s dogged tendency to stick with a familiar solution to a problem—the one that first comes to mind—and to ignore alternatives. Often this type of thinking is a useful heuristic. Once you have hit on a successful method to, say, peel garlic, there is no point in trying an array of different techniques every time you need a new clove. The trouble with this cognitive shortcut, however, is that it sometimes prevents people from seeing more efficient or appropriate solutions than the ones they already know.

On supporting science journalism

If you're enjoying this article, consider supporting our award-winning journalism by subscribing . By purchasing a subscription you are helping to ensure the future of impactful stories about the discoveries and ideas shaping our world today.

Building on Luchins’s early work, psychologists replicated the Einstellung effect in many different laboratory studies with both novices and experts exercising a range of mental abilities, but exactly how and why it happened was never clear. About 15 years ago, by recording the eye movements of highly skilled chess players, we solved the mystery. It turns out that people under the influence of this cognitive shortcut literally do not see certain details in their environment that could provide them with a more effective solution. Research also suggests that many different cognitive biases discovered by psychologists over the years—those in the courtroom and the hospital, for instance—are in fact variations of the Einstellung effect.

Back to Square One

Since at least the early 1990s, psychologists have studied the Einstellung effect by recruiting chess players of varying skill levels, from amateur to grand master. In such experiments, researchers have presented players with specific arrangements of chess pieces on virtual chessboards and asked them to achieve a checkmate in as few moves as possible. Our own studies, for instance, provided expert chess players with scenarios in which they could accomplish a checkmate using a well-known sequence called smothered mate. In this five-step maneuver, the queen is sacrificed to draw one of the opponent’s pieces onto a square to block off the king’s escape route. The players also had the option to checkmate the king in just three moves with a much less familiar sequence. As in Luchins’s water jug studies, most of the players failed to find the more efficient solution.

During some of these studies, we asked the players what was going through their mind. They said they had found the smothered mate solution and insisted they were searching for a shorter one, to no avail. But the verbal reports offered no insight into why they could not find the swifter solution. In 2007 we decided to try something a little more objective: tracking eye movements with an infrared camera. Which part of the board people looked at and how long they looked at different areas would unequivocally tell us which aspects of the problem they were noticing and ignoring.

In this experiment, we followed the gaze of five expert chess players as they examined a board that could be solved either with the longer smothered mate maneuver or with the shorter three-move sequence. After an average of 37 seconds, all the players insisted that the smothered mate was the speediest possible way to corner the king. When we presented them with a board that could be solved only with the three-sequence move, however, they found it with no problem. And when we told the players that this same swift checkmate had been possible in the previous chessboard, they were shocked. “No, it is impossible,” one player exclaimed. “It is a different problem; it must be. I would have noticed such a simple solution.” Clearly, the mere possibility of the smothered mate move was stubbornly masking alternative solutions. In fact, the Einstellung effect was powerful enough to temporarily lower expert chess masters to the level of much weaker players.

The infrared camera revealed that even when the players said they were looking for a faster solution—and indeed believed they were doing so—they did not actually shift their gaze away from the squares they had already identified as part of the smothered mate move. In contrast, when presented with the one-solution chessboard, players initially looked at the squares and pieces important for the smothered mate and, once they realized it would not work, directed their attention toward other squares and soon hit on the shorter solution.

Basis for Bias

In 2013 Heather Sheridan, now at the University of Albany, and Eyal M. Reingold of the University of Toronto published studies that corroborate and complement our eye-tracking experiments. They presented 17 novice and 17 expert chess players with two different situations. In one scenario, a familiar checkmate maneuver such as the smothered mate was advantageous but second best to a distinct and less obvious solution. In the second situation, the more familiar sequence would be a clear blunder. As in our experiments, once amateurs and master chess players locked onto the helpful familiar maneuver, their eyes rarely drifted to squares that would clue them in to the better solution. When the well-known sequence was obviously a mistake, however, all the experts (and most of the novices) detected the alternative.

Credit: George Retseck

The Einstellung effect is by no means limited to controlled experiments in the lab or even to mentally challenging games such as chess. Rather it is the basis for many cognitive biases. English philosopher, scientist and essayist Francis Bacon was especially eloquent about one of the most common forms of cognitive bias in his 1620 book Novum Organum: “The human understanding when it has once adopted an opinion … draws all things else to support and agree with it. And though there be a greater number and weight of instances to be found on the other side, yet these it either neglects or despises, or else by some distinction sets aside and rejects…. Men … mark the events where they are fulfilled, but where they fail, though this happen much oftener, neglect and pass them by. But with far more subtlety does this mischief insinuate itself into philosophy and the sciences, in which the first conclusion colours and brings into conformity with itself all that comes after.”

In the 1960s English psychologist Peter Wason gave this particular bias a name: “confirmation bias.” In controlled experiments, he demonstrated that even when people attempt to test theories in an objective way, they tend to seek evidence that confirms their ideas and to ignore anything that contradicts them.

In The Mismeasure of Man , for example, Stephen Jay Gould of Harvard University reanalyzed data cited by researchers trying to estimate the relative intelligence of different racial groups, social classes and sexes by measuring the volumes of their skulls or weighing their brains, on the assumption that intelligence was correlated with brain size. Gould uncovered massive data distortion. On discovering that French brains were on average smaller than their German counterparts, French neurologist Paul Broca explained away the discrepancy as a result of the difference in average body size between citizens of the two nations. After all, he could not accept that the French were less intelligent than the Germans. Yet when he found that women’s brains were smaller than those in men’s noggins, he did not apply the same correction for body size, because he did not have any problem with the idea that women were less intelligent than men.

Somewhat surprisingly, Gould concluded that Broca and others like him were not as reprehensible as we might think. “In most cases discussed in this book we can be fairly certain that biases … were unknowingly influential and that scientists believed they were pursuing unsullied truth,” Gould wrote. In other words, just as we observed in our chess experiments, comfortably familiar ideas blinded Broca and his contemporaries to the errors in their reasoning. Here is the real danger of the Einstellung effect. We may believe that we are thinking in an open-minded way, completely unaware that our brain is selectively directing attention away from aspects of our environment that could inspire new thoughts. Any data that do not fit the solution or theory we are already clinging to are ignored or discarded.

The surreptitious nature of confirmation bias has unfortunate consequences in everyday life, as documented in studies on decision-making among doctors and juries. In a review of errors in medical thought, physician Jerome Groopman noted that in most cases of misdiagnosis, “the doctors didn’t stumble because of their ignorance of clinical facts; rather, they missed diagnoses because they fell into cognitive traps.” When doctors inherit a patient from another doctor, for example, the first clinician’s diagnosis can block the second from seeing important and contradictory details of the patient’s health that might change the diagnosis. It is easier to just accept the diagnosis—the “solution”—that is already in front of them than to rethink the entire situation. Similarly, radiologists examining chest x-rays often fixate on the first abnormality they find and fail to notice further signs of illness that should be obvious, such as a swelling that could indicate cancer. If those secondary details are presented alone, however, radiologists see them right away.

Related studies have revealed that jurors begin to decide whether someone is innocent or guilty long before all the evidence has been presented. In addition, their initial impressions of the defendant change how they weigh subsequent evidence and even their memory of evidence they saw before. Likewise, if an interviewer finds a candidate to be physically attractive, he or she will automatically perceive that person’s intelligence and personality in a more positive light, and vice versa. These biases, too, are driven by the Einstellung effect. It is easier to make a decision about someone if one maintains a consistent view of that person rather than sorting through contradictory evidence.

Can we learn to resist the Einstellung effect? Perhaps. In our chess experiments and the follow-up experiments by Sheridan and Reingold, some exceptionally skilled experts, such as grand masters, did in fact spot the less obvious optimal solution even when a slower but more familiar sequence of moves was possible. This suggests that the more expertise someone has in their field—whether chess, science or medicine—the more immune they are to cognitive bias.

But no one is completely impervious; even the grand masters failed when we made the situation tricky enough. Actively remembering that you are susceptible to the Einstellung effect is another way to counteract it. When considering the evidence on, say, the relative contribution of human-made and naturally occurring greenhouse gases to global temperature, remember that if you already think you know the answer, you will not judge the evidence objectively. Instead you will notice evidence that supports the opinion you already hold, evaluate it as stronger than it really is and find it more memorable than evidence that does not support your view.

We must try to learn to accept our errors if we sincerely want to improve our ideas. English naturalist Charles Darwin came up with a remarkably simple and effective technique to do just this. “I had … during many years, followed a golden rule, namely, that whenever a published fact, a new observation or thought came across me, which was opposed by my general results, to make a memorandum of it without fail and at once,” he wrote. “For I had found by experience that such facts and thoughts were far more apt to escape from memory than favourable ones.”

The Hershey-Chase Experiments (1952), by Alfred Hershey and Martha Chase

In 1951 and 1952, Alfred Hershey and Martha Chase conducted a series of experiments at the Carnegie Institute of Washington in Cold Spring Harbor, New York, that verified genes were made of deoxyribonucleic acid, or DNA. Hershey and Chase performed their experiments, later named the Hershey-Chase experiments, on viruses that infect bacteria, also called bacteriophages. The experiments followed decades of scientists’ skepticism about whether genetic material was composed of protein or DNA. The most well-known Hershey-Chase experiment, called the Waring Blender experiment, provided concrete evidence that genes were made of DNA. The Hershey-Chase experiments settled the long-standing debate about the composition of genes, thereby allowing scientists to investigate the molecular mechanisms by which genes function in organisms.

In the early twentieth century, scientists debated whether genes were made of DNA or protein. Genes control how organisms grow and develop and are the material basis for organisms’ ability to inherit traits like eye color or fur color from their parents. By 1900, scientists had identified the complete chemical composition, or building blocks, of DNA. They had also verified that all cells contained DNA, though DNA’s function remained ambiguous. Up until the 1940s, some scientists accepted the idea that genes were not made of DNA. Instead, those scientists supported the idea that DNA was a molecule that maintained cell structure. Scientists supported that idea in part because of a hypothesis called the tetranucleotide hypothesis. Phoebus Levene, a researcher at the Rockefeller Institute for Medical Research in New York City, New York, proposed the tetranucleotide hypothesis for DNA in 1933. According to Levene and other proponents of the hypothesis, DNA consisted of repeating sets of four different building blocks, called nucleotides. Some scientists concluded that a repeating sequence of nucleotides in DNA limited potential for variability. Those scientists considered variability necessary for DNA to function as genetic material. In other words, genes needed to have the capacity for enough variation to account for the different traits scientists observe in organisms. Conversely, scientists found that proteins had many more building blocks and therefore more possible arrangements than DNA. From that, some scientists claimed that genes must have been made of protein, not DNA.

The Hershey-Chase experiments were not the first studies to oppose the prevailing theory in the early 1900s that genetic material was composed of proteins. In 1944, nearly a decade before Hershey and Chase’s work, scientists published sound evidence that genes were made of DNA rather than protein. Starting in 1935, Oswald Avery, another researcher at the Rockefeller Institute, with his research associates Colin MacLeod and Maclyn McCarty, performed experiments that showed that DNA facilitated a genetic phenomenon in bacteria called bacterial transformation. Bacterial transformation is the process by which a bacterium can get and use new genetic material from its surroundings. During bacterial transformation, a non-disease-causing bacterium can transform into disease-causing bacteria if the non-disease-causing bacteria is exposed to a disease-causing bacteria. Transformation can occur even if the disease-causing-strain is dead, implying that bacterial transformation happens when the non-disease-causing bacteria inherits genetic material from the disease-causing bacteria. Avery and his colleagues found that the inherited factor that caused bacterial transformation contained DNA. However, Avery’s group did not discount the possibility that some non-DNA component in their sample caused bacterial transformation, rather than the DNA itself. Because of that, many scientists maintained the idea that proteins must govern the genetic phenomenon of bacterial transformation.

Starting in 1951, Alfred Hershey and Martha Chase conducted a series of experiments, later called the Hershey-Chase experiments, that verified the findings of Avery and his colleagues. Hershey was a researcher who studied viruses at the Carnegie Institution of Washington in Cold Spring Harbor, New York. He studied viruses that infect bacteria, also called bacteriophages, or phages. Chase became Hershey’s research technician in 1950.

In their experiments, Hershey and Chase analyzed what happened when phages infect bacteria. By the 1950s, scientists had evidence for how phages infected bacteria. They found that when phages infect a host bacterium, the phages first attach themselves to the outside of the bacterium. Then, a piece of the phage enters the bacterium and subsequently replicates itself inside the cell. After many replications, the phage causes the bacterium to lyse, or burst, thereby killing the host bacteria. Scientists classified the replicating piece as genetic material. Scientists also found that phages contained two classes of biological molecules: DNA and protein. Hershey and Chase sought to determine if the replicating piece of phages that entered bacteria during infection, the genetic parts, were solely DNA.

To perform their experiments, Hershey and Chase utilized a technique called radioactive isotope labeling. Chemical elements can exist in different structural forms called isotopes. Isotopes of the same element are nearly identical, but scientists can distinguish between them by experimental means. One way to differentiate between chemical elements with different isotopes is by analyzing their radiation. Some isotopes are less stable than others and give off radioactive signals that scientists can detect. Hershey and Chase marked phages by incorporating radioactive isotopes of phosphorus and sulfur in those phages. They allowed some phages to replicate by infecting bacteria, specifically Escherichia coli , or E. Coli , that scientists had grown in radioactive sulfur. The researchers let other phages infect and replicate in E. Coli that scientists had grown in radioactive phosphorus. DNA contains phosphorus, but not sulfur, whereas protein contains sulfur, but not phosphorus. Therefore, when Hershey and Chase marked phages with radioactive isotopes of those elements, they placed separate, distinguishable tags on the protein and DNA parts of the phages.

The first Hershey-Chase experiment aimed to confirm previous experimental findings that the DNA and protein components of phages were separable. In 1950, Thomas Anderson at the University of Pennsylvania in Philadelphia, Pennsylvania, showed that phages consisted of a protein shell, or coat, with DNA inside the shell. Anderson found that the phages could release their DNA and leave behind what he called a protein ghost. Hershey and Chase replicated Anderson’s experimental results using their radioactive isotope labeling method. Hershey and Chase were able to separate the phages into radioactive sulfur-containing protein ghosts and radioactive phosphorus-containing DNA. They found that the radioactive sulfur protein ghosts could attach to bacterial membranes while the radioactive phosphorus DNA could not. Hershey and Chase also tested if enzymes, molecules that facilitate chemical reactions in cells, could degrade DNA. They found that enzymes did not degrade the DNA of intact phages, but did degrade the DNA of separated phages. Those results indicated that in the intact phages, the protein coat surrounded the DNA and protected the DNA from degradation.

In another Hershey-Chase experiment, Hershey and Chase showed that when certain phages infected E. Coli , the phages injected their DNA into the host bacterium. In 1951, Roger Herriot at Johns Hopkins University in Baltimore, Maryland, demonstrated that after phages infected bacteria, their protein ghosts remained attached to the outside of the bacterial cells while their DNA was released elsewhere. Hershey and Chase aimed to show where the phage DNA went when it exited the protein coat and entered the bacteria. The researchers allowed radioactive phosphorus-labeled phages to attach to bacterial cell membranes in a liquid solution and infect the bacteria. Using a centrifuge, Hershey and Chase rapidly spun the samples to separate the bacterial cells from the surrounding solution. After centrifugation, they found that most of the radioactive phosphorus was detected in the cells rather than in the surrounding solution, meaning that the phage DNA must have entered the cells when the phages infected the bacteria.

The most well-known Hershey-Chase experiment was the final experiment, also called the Waring Blender experiment, through which Hershey and Chase showed that phages only injected their DNA into host bacteria, and that the DNA served as the replicating genetic element of phages. In the previous experiment, Hershey and Chase found evidence that phages injected their DNA into host bacteria. In the Waring Blender experiment, the scientists found that the phages did not inject any part of their protein coats in the host bacteria and the protein coats remained outside the bacteria, adhered to the bacterial membranes. For their experiment, Hershey and Chase prepared two samples of infected E. Coli . They infected one sample with radioactive phosphorus-labeled phages, and the other sample with radioactive sulfur-labeled phages. Then, they stirred each sample in a Waring Blender, which was a conventional kitchen blender. They used a blender because centrifuges spun too fast and would destroy the bacterial cells. The shearing forces of the blender removed the phage particles that adhered to the bacterial membranes, but preserved the integrity of the cells and most of the phage material that entered the cell. In the phosphorus-labeled sample that marked DNA but not protein, the blender removed forty percent of the labeled particles. In the sulfur-labeled sample that marked protein but not DNA, the blender removed eighty percent of the labeled particles. Those results indicated that the blender removed much more of the protein parts of the phage than the DNA parts, suggesting that the protein likely remained adhered to the outside of the cell during infection. Since the protein remained outside the cell, it could not be the replicating genetic material.

The Waring Blender only removed eighty percent of the radioactive sulfur-labeled phage, so Hershey and Chase could not account for twenty percent of the phage protein material. To show that the missing twenty percent of the phage protein did not enter the bacterial cells and replicate, the researchers infected E. Coli with radioactive sulfur-labeled phage again so that only the protein parts of the phage were labeled. They prepared two samples. For one sample, Hershey and Chase stirred the cells in the blender to remove the phage particles adhered to the outer bacterial membrane. After stirring, they allowed the phages to cause the cells to lyse, releasing newly replicated phages. For the second sample, Hershey and Chase did not stir the cells in the blender and measured the resulting replicated phages after the bacterial cells lysed. In the blender-stirred sample, less than one percent of the replicated phages contained the radioactive sulfur label. However, in the sample that Hershey and Chase did not stir in the blender, almost ten percent of the phages contained the radioactive sulfur label. The blender maintains any phage material that entered the bacterial cell. If protein was genetic material that entered the cell and replicated, then Hershey and Chase would have found more sulfur-labeled protein in the sample they stirred with the blender. The sample that they did not stir had more of the sulfur-labeled protein because the protein coats remained on the outside of the cell. Hershey and Chase concluded that protein was not genetic material, and that DNA was genetic material.

Unlike Avery’s experiments on bacterial transformations, the Hershey-Chase experiments were more widely and immediately accepted among scientists. The Hershey-Chase experiments mostly ended scientists’ suspicions that genes were made of protein rather than DNA. However, historians have questioned the conclusiveness of the Hershey-Chase experiments. In all the Waring Blender experiments, some protein and DNA material remained unaccounted for. Even in the final experiment, when Hershey and Chase allowed the bacterial cells to lyse after stirring in the blender, the scientists still recovered a small amount of protein, implying that some protein entered the cells during infection. Furthermore, the amount of contaminating protein in the Hershey-Chase Experiments exceeded the amount of contaminating protein that Avery’s group found in their experiments.

Historians of science have studied why scientists more readily accepted the Hershey-Chase experiments than Avery’s experiments. Science historian Frederic Lawrence Holmes writes that scientists more readily accepted the results of the Hershey-Chase experiments because Hershey communicated directly with skeptical scientists. Hershey sent letters to his colleagues in which he detailed the experimental findings of the Hershey-Chase experiments. Another historian of science, Michel Morange, writes that the Hershey-Chase experiments were performed at a time when scientists were ready to accept that genetic material could be DNA. Avery’s group conducted their experiments when the tetranucleotide hypothesis was popular and few scientists held that genes contained DNA. According to Morange, because Hershey and Chase conducted their experiments years later, scientists had gathered more experimental evidence and were willing to seriously consider that genes contained DNA.

In 1953, James Watson and Francis Crick, two scientists at the University of Cambridge in Cambridge, England, modeled the three-dimensional structure of DNA and demonstrated how DNA might function as genetic material. In 1969, Hershey shared the Nobel Prize in Physiology or Medicine with two other scientists, Max Delbrück and Salvador Luria, partly for his work on the Hershey-Chase experiments.

• Avery, Oswald, Colin MacLeod, and Maclyn McCarty. "Studies on the Chemical Nature of the Substance Inducing Transformation of Pneumococcal Types." The Journal of Experimental Medicine 79 (1944): 137–58.
• Fry, Michael. “Chapter 4 – Hershey and Chase Clinched the role of DNA as Genetic Material: Phage Studies Propelled the Birth of Molecular Biology.” In Landmark Experiments in Molecular Biology , 111–42. Cambridge: Academic Press, 2016.
• Hershey, Alfred D., and Martha Chase. “Independent Functions of Viral Protein and Nucleic Acid in Growth of Bacteriophage” The Journal of General Physiology 36 (1952): 39–56.
• Holmes, Frederic L. Meselson, Stahl, and the Replication of DNA: A History of “The Most Beautiful Experiment in Biology.” New Haven and London: Yale University Press, 2001.
• Hopson, Janet L., and Norman K. Wessells. Essentials of Biology . New York: McGraw-Hill, 1990.
• Judson, Horace Freeland. The Eighth Day of Creation . Cold Spring Harbor: Cold Spring Harbor Laboratory Press, 1996.
• Morange, Michel. A History of Molecular Biology . Cambridge and London: Harvard University Press, 1998.
• Olby, Robert Cecil. The Path to the Double Helix: The Discovery of DNA . Seattle: University of Washington Press, 1974.
• Stahl, Franklin W., and Alfred D. Hershey. We Can Sleep Later: Alfred D. Hershey and the Origins of Molecular Biology . Woodbury: Cold Spring Harbor Laboratory Press, 2000.

How to cite

Articles rights and graphics.

Copyright Arizona Board of Regents Licensed as Creative Commons Attribution-NonCommercial-Share Alike 3.0 Unported (CC BY-NC-SA 3.0)  

Last modified

Share this page.

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

• View all journals
• Explore content
• About the journal
• Publish with us
• Sign up for alerts
• NATURE PODCAST
• 03 March 2021

COVID, 2020 and a year of lost research

• Benjamin Thompson &
• Nick Petrić Howe

You can also search for this author in PubMed   Google Scholar

Hear the latest science news, with Nick Petrić Howe and Benjamin Thompson.

In this episode:

00:48 The pandemic's unequal toll on researchers

Although 2020 saw a huge uptick in the numbers of research papers submitted, these increases were not evenly distributed among male and female scientists. We look at how this could widen existing disparities in science, and damage future career prospects.

Editorial: COVID is amplifying the inadequacy of research-evaluation processes

09:18 Research Highlights

How a parasite can make viral infections more deadly, and the first known space hurricane.

Research Highlight: Intestinal worms throw open the door to dangerous viruses

Research Highlight: The first known space hurricane pours electron ‘rain’

11:36 Energy without oxygen

Millions of years ago, a microscopic protist swallowed a bacterium and gained the ability to breathe nitrate. This relationship partially replaced the cell's mitochondria and allowed it to produce abundant energy without oxygen. This week, researchers describe how this newly discovered symbiosis works.

Research Article: Graf et al.

News and Views: A microbial marriage reminiscent of mitochondrial evolution

19:22 Briefing Chat

We discuss some highlights from the Nature Briefing. This time, the weakening of the Gulf Stream, and a new satellite to monitor deforestation in the Amazon.

The Guardian: Atlantic Ocean circulation at weakest in a millennium, say scientists

Science: Brazil’s first homemade satellite will put an extra eye on dwindling Amazon forests

Subscribe to Nature Briefing, an unmissable daily round-up of science news, opinion and analysis free in your inbox every weekday.

Video: How to build a Quantum Internet

Never miss an episode: Subscribe to the Nature Podcast on Apple Podcasts , Google Podcasts , Spotify or your favourite podcast app. Head here for the Nature Podcast RSS feed.

doi: https://doi.org/10.1038/d41586-021-00570-6

Host: Benjamin Thompson

Welcome back to the Nature Podcast . This week, how the pandemic could widen existing disparities in science.

Host: Nick Petrić Howe

And a symbiotic relationship to rival the mitochondrion. I’m Nick Petrić Howe.

And I’m Benjamin Thompson.

Interviewer: Benjamin Thompson

The COVID-19 pandemic has of course changed the world in many ways, including the world of science. In some cases, this has given a kind of turbo boost to research. Think of the speed with which the current vaccines were developed for instance. But the pandemic has also fundamentally disrupted the scientific enterprise in other ways, and this disruption has taken an unequal toll on researchers, worsening existing disparities and leading to some members of the community bearing much more of a burden than others. To find out more, I called up our colleague Holly Else who told me how the world of scientific publishing was altered in 2020.

Interviewee: Holly Else

So, what we saw last year at least was a massive boom in the number of papers that researchers were submitting to journals for publication, and there’s several sort of reasons why that might be. It might be because they are at home and less likely to be doing wet lab-type research, so perhaps they’ve got time to write up papers that have been sitting needing work for many months. So, yeah, I mean, last year there was probably 100,000 papers just on COVID published.

Yeah, you would imagine that you could see that sort of health and medicine papers would increase, but it wasn’t, it was sort of across the board we saw in 2020.

Yeah, across the board, all subjects. And also, publishing changed quite a lot during the pandemic. So, in order to accommodate the fast research that was going on into vaccines and the virus itself, peer-review structures were tweaked and maybe there was a fast-track system set up which helped papers get published quicker. Preprints – so those are the papers that are uploaded before peer review – they also saw a huge rise in submissions or uploads too.

On one hand, you can say well, that’s really, really good. It gives us a lot more things to talk about. But, Holly, you and I have talked a lot on the podcast in the past about, well, the significant and persistent disparities that exist in science and in academia – I guess, what, just 30% of researchers are female, for example – and sadly, we’re seeing these disparities really coming through within this sort of submission boom.

Yeah, we are. So, across all career stages and genders and all subjects, submissions for research papers are increasing, but this massive increase has not been equal between men and women. So, the growth of submissions by men has increased a lot more than the growth in submissions by women, and we particularly see this at the senior level, so those who are professors or more. So, the rate of growth between typically a professor in the life sciences might be 10 percentage points higher for men than for women. We see a similar effect in physical sciences but it’s about a 5 percentage point increase for men versus women.

Well, what do we know about the potential reasons behind this disparity then?

Well, we don’t know too much to be honest. Obviously, if you’re at home trying to do your job while home schooling or looking after people, you probably have some idea what the issues might be, but in terms of systematic evidence we don’t have much yet. And one of the main arguments put forward is that in a lot of societies, women are still the main caregiver in a family and we sort of see that borne out in the data in that it’s the more senior women who are seeing the biggest effects, and they are more likely to have caring responsibilities, perhaps children to home school, than those at the start of their careers. There was one sort of early survey that looked at how much time academics had to do their research in the pandemic, and this particular survey found that in terms of how many hours scientists are dedicating to the research part of their job, they found that women with dependent children, so younger children, had a significantly reduced time available to do research as compared to with men.

And of course, this could have some quite serious knock-ons, Holly. I mean, I know in science the phrase ‘publish or perish’ is used a lot, which means you need to kind of get your papers and your data out there to be able to apply for tenure or promotions or what have you. So, this could have some quite serious, long-term consequences.

Yeah, this could potentially be a lost year for some researchers, and given the competitive nature of science, like you said, Ben, with the promotion criteria needing publications and outputs and evidence of all these things you’ve done, losing a year on your way to becoming tenure track, for example, could have quite a big implication longer term. So, now, we’re sort of settled into this new working environment, those are the conversations that are now starting to be had amongst funders and those who evaluate research. How can they take into consideration what’s happened in this year and how they need to be mindful of the fact that it’s not everybody who’s been affected in the same way.

Well, of course, yeah. I mean, here in the UK, we’ve got the REF, the Research Excellence Framework, going on, which is this kind of national academic evaluation exercise. That’s in full swing now and is ongoing. How is this going to account for the issues that you’ve raised?

Yeah, so academics in the UK were preparing for the REF, which is this massive assessment exercise that doesn’t happen very often and when it does it’s very stressful for universities, and that’s when the pandemic hit. So, what’s happened is that the people who administer the REF have extended the deadlines slightly, but they’ve also made some changes in terms of what you can submit. So, for example, if you were writing a book and you were expecting it to be published and actually you haven’t managed to do that because of the pandemic, you can now submit the digital copy or what work you have done. Another thing that’s been affected is where researchers have to show that their work has had an impact in the real world, and lots of researchers might be holding events or conferences or going out into the community and that work forms part of their impact submission, and obviously we haven’t had public gatherings so, in response, the evaluators have increased the window of opportunity that you could show the impact of you work.

Well, if that’s maybe a positive step then that’s going on in the UK to maybe take into account all that’s gone on over the past year or so, is this part of a broader trend we’re seeing in other countries? And what about independent research funders as well, how are they looking at what’s been going on and how it might be affecting people?

Well, another country that has a big research assessment activity similar to the UK is in Italy, and their timescale is slightly different from ours. So, they have a few more years yet really to figure out exactly how they’re going to take into account what has gone on, but those conversations are very much starting now. And when you talk about independent research funders who are not evaluating, per se, specific research, they might get a grant application, see somebody’s publication history, a lot of that is around working out whether the person who has applied for this grant is the right person to do the work, and the people that we’ve been speaking to feel maybe that they might be slightly less affected because it’s more of a perspective thing. They’re looking for the future whereas the research evaluators are looking back to see what you’ve actually done and how well you did it. But that’s not to say that research funding for grants won’t be affected by this because obviously people’s publication histories and promotion statistics, all that kind of stuff is taken into account when funders are awarding grants.

Well, finally, Holly, we don’t know if there’s going to be one year, two years, ten years before we kind of really get a sense of the knock-on effects of what’s happened in this particularly strange sort of period of time we’ve had here. But in an ideal world, I mean, where could science go? How do you think the scientific enterprise could improve and could learn from what’s happened?

For me, one of the biggest things science could learn is actually to look back on itself and collect the data, find out what is actually happening. I mean, what is going on? Why is it happening and how can we prevent it from happening in the future? And once we know that, the relevant people can sit around the table to thrash out some way to make sure that allowances are made so that things like this don’t continue to be a problem.

That was Nature ’s Holly Else. This week, Nature has an editorial on how research evaluation efforts need to take into account some of the things we’ve talked about today. You can find a link to that in the show notes.

Coming up, we’ll hear how a microscopic protist swallowing a bacterium led to a whole new way of thriving without oxygen. Before that though, Dan Fox is here with this week’s Research Highlights.

If having a parasitic worm take up residence in your gut didn’t sound bad enough, scientists have discovered that they may also make viral infections more deadly. Researchers studying mice infected with an intestinal roundworm and West Nile virus found that rodents were more likely to die when infected with both pathogens than when infected by just one. The team found that sensory cells called tuft cells lining the rodents’ intestines activate a cycle to attempt to clear the parasites. But this process could ultimately damage the intestines’ protective inner layer and impair the ability of immune cells to destroy virus-infected cells. The virus can then more readily infect the central nervous system and other tissues. These findings suggest that roundworms can make their hosts more susceptible to infection by viruses that target the intestines. Read that research in full in Cell .

Satellite observations have revealed an unprecedented space hurricane in Earth’s upper atmosphere. Previously, the existence of space hurricanes – hurricane-like circulation patterns in planets’ upper atmospheres – has been uncertain. Now, researchers have used satellite data to identify a space hurricane over Earth’s northern magnetic pole, and this hurricane wasn’t a whirling pattern of air but of plasma – ionised gas. Like a regular hurricane, the space hurricane featured a quiet centre, multiple spiral arms and widespread circulation. It also features precipitation, not of rain but of energetic electrons. And like a regular hurricane, space hurricanes could be disruptive as they drag energy from solar wind into Earth’s atmosphere, potentially causing disturbances in radio communications and satellite navigation systems. See if that research blows you away at Nature Communications .

Interviewer: Nick Petrić Howe

Symbiosis – two different organisms coexisting, in many cases to the mutual benefit of both. These relationships are a crucial element of biology. Part of the reason that complex organisms exist at all is down to a symbiosis. Some time in the distant past, an ancient cell swallowed up a smaller cell and the symbiosis between mitochondria and eukaryotes was born. Mitochondria are organelles that allow eukaryotic cells to create energy in the form of ATP using oxygen. But some eukaryotes live in areas where oxygen isn’t present. They survive primarily using fermentation, which produces energy but not as efficiently as mitochondria. But now, researchers have discovered an organism with a brand new way of thriving without oxygen. A paper in this week’s Nature describes a single-celled organism discovered in a lake in Switzerland that rather than using oxygen or fermentation can produce energy from nitrogen oxides, and it does it by creating a symbiosis with a bacterium, much like the ancestor of eukaryotic cells did with the ancestral mitochondrion. I called up one of the researchers, Jana Milucka, to find out more about their discovery.

Interviewee: Jana Milucka

Well, what we found was we believe we understand the mechanisms by which anaerobic eukaryotes can also respire under an aerobic conditions, so they live without oxygen but they have reverted to respiring a different electron acceptor under these conditions, and the mechanisms by which they accomplish this respiration is that they have entered a symbiosis with a bacterium which performs this process for them. And this sort of respiratory symbiosis or energy symbiosis, if you will, is really unusual and as far as we know, a similar type is only known from the original symbiosis between the mitochondria predecessor and the original archaea.

Tell me about this symbiosis. What cells are involved in this and is it sort of an endosymbiosis where one is inside the other or is it a different sort of relationship?

Yeah, so the host is an anaerobic unicell or ciliate and the endosymbiont is an obligately endosymbiotically living anaerobic bacterium that has the capacity to perform denitrification, so anaerobic respiration.

So, it’s a way for these organisms to use, as you said, nitrate as an electron acceptor, so use them to create energy. Were you surprised when you saw this?

Yes, very much so. I must say, this is not something that really we in anyway anticipated to find. What we first found was the genome of the endosymbiont and already from the properties of the genome we could tell that we were looking at something that is not a free-living organism, but it is something that has been living together with a host for a long, long time, and we could tell that from the fact that the genome was very small and it had specific features that are very typical among obligate endosymbionts. However, the obligate endosymbionts known to date that have genomes with these sorts of properties, they occur in insects, for example, and they primarily serve the function that they provide that host with nutrition, for example with essential vitamins or other nutrients, or maybe they provide defence mechanisms for the host. But when we looked at the genes that were encoded in the endosymbiont genome, we saw that the metabolic capacity to provide to synthesise any sort of vitamins were nearly absent from the genome, so clearly the function of these organisms was not to synthesise something. On the other hand, the capacity to encode enzymes for the respiratory pathway for the export of ATP and any genes related to respiration and energy metabolism were abundantly present in the genome and were also highly transcribed, pointing to the role that this organism plays a role in respiration and in energy metabolism.

And so you said you found this basically through doing some sort of sequencing. Were you able to test experimentally what was going on here? Were you able to show what exactly the processes these things are undergoing in the lab?

We thought that was absolutely crucial because the predictions from the genome were so unbelievable that we really wanted to make sure that this is indeed what is happening in situ. But we don’t have cultures of these organisms, so all the experiments that we did we had to perform with ciliates that were hand-picked into the experiment vials. Indeed, we did test for the process of denitrification, so we picked the individual ciliates and we measured whether they were capable of respiring nitrate, and we do see that this is what is going on.

Does this endosymbiont that you’ve found, this relationship, does this give us any clues as to how this might have happened with the mitochondrion?

So, the mitochondria and this symbiont, they have a completely different independent evolutionary origin. The predecessor of our symbiont and the predecessor of mitochondria do not share common origin. But it appears that the path they took down the line, the evolutionary path, seems to be very similar. So obviously, mitochondria much further than our symbiont because they had much longer to evolve and they have also lost more genes in that process, but when we look at where our symbiont is now, it’s intriguing that it has maintained such a strikingly similar set of functions and genes to those that mitochondria have retained. So, yes, I think it tells us something about the evolution of these kinds of respiratory symbiosis and to date, we thought that this is an event that happened once and it’s something that cannot be repeated, but I think based on this discovery we should rethink whether this sort of symbiosis and these sorts of evolutionary trajectories of symbiosis might happen more often.

That’s a really interesting proposal. So, what are your sort of next steps with this? Where is this research going next?

The biggest question I think is really how and where this symbiosis evolved, and for that we will have to look in a different setting than the lake that we studied because the lake in Switzerland we looked at is a post-glacial lake and this lake only originated roughly 10,000 years ago. But we know that the symbiosis that we are looking at is hundreds of millions of years old, so the question is where did it start? Is there maybe a freshwater system that is old enough to have been the cradle for this symbiosis or did this symbiosis evolve from the marine system and adapted to freshwater afterwards? I think these are all exciting questions.

That was Jana Milucka from the Max Planck Institute for Marine Microbiology in Germany. To find out more about this endosymbiosis then be sure to check out the show notes where there’ll be a link to Jana’s paper and an accompanying News and Views article.

Finally on the show, it’s time for the weekly Briefing chat where we discuss a couple of articles that have been highlighted in the Nature Briefing . Nick, what’s your Briefing highlight this time?

Well, I’ve been reading an article in The Guardian all about the Atlantic Meridional Overturning Circulation – a system that underpins the Gulf Stream in the Atlantic – and it seems like it’s at the weakest it’s been for a millennium.

Well, Nick, I’ve heard about the Gulf Stream certainly – that’s what sometimes brings half-decent summer weather to the southwest of the UK – but the other thing you said there, I must confess I am less familiar with it.

Yeah, and I’d emphasise the ‘half-decent’ weather to the south of the UK. But yes, the Atlantic Meridional Overturning Circulation – let’s just call it AMOC for short – is basically a system of currents that underpins the Gulf Stream, and it’s essentially what brings warm water from the Gulf of Mexico up all the way to Iceland where it gets cooler and saltier and continues to pull more warm water from the Gulf of Mexico, and it’s responsible for kind of making things sort of mild and a bit wet here in the UK and having sort of a warming effect overall on western Europe and making things a bit more mild. And that’s the thing that appears to be affected according to this article in The Guardian , and researchers have basically been looking at ice cores and other data that can give us information about the weather from the past millennium and have said that it looks like it’s at its weakest point that it’s been for a thousand years, and climate change is the probable cause.

I mean, it seems that on our Briefing chats it often is a probable cause, Nick, and I’m guessing that this is probably not good news then.

No, so, as I said, this AMOC brings nice weather, essentially mild weather, to the UK and Europe, so its disruption isn’t particularly great, and potentially this could lead to more uncertain weather in Europe. It could lead to more hurricanes in the UK, it could lead to more heatwaves, and even on the east coast of the US it could lead to higher sea levels, so not great all around. So, the AMOC has slowed by around 15% and by the end of this century it could go as much as 45% slowing down, and if it gets to that point then researchers are worried that we’ll reach some sort of tipping point where we can see some of these more disastrous effects.

I mean, that really doesn’t sound good then. I mean, is there anything the researchers think can be done to try and obviously avoid this scenario?

I mean, it’s a similar story with a lot of climate change work, and that is we need to produce less greenhouse gas emissions and we need to work more quickly towards a world where we are either carbon neutral or actively reducing the carbon in the atmosphere. So, we know what to do, like we’ve known what to do for a long time now, it’s just doing it and this is just another bit of information that will show one of the potential impacts for not doing enough when it comes to climate change.

Well, Nick, I’ve got a story that’s certainly kind of related to climate change, I guess, and it’s to do with a new satellite that was successfully launched a few days ago, and it’s a story that I read about in Science .

Oh, okay, and will this satellite give us more data to understand the impacts of climate change or past climate change?

Well, it certainly will give more data, and actually it’s kind of an achievement for Brazil because it’s the first time they have sort of designed and made a satellite in its entirety, and it’s now up in orbit, going around. And what it’s doing is it’s having a good look at the Amazon rainforest, so it’s called Amazonia-1, and it’s going to be part of a three-satellite series, and it’s really getting sort of eyes on what’s going on there in terms of things like deforestation.

Yeah, and the Amazon is something we hear a lot about and in terms of how it’s being depleted and fires and all sorts. It’s not really that much good news. But what will this new satellite, this Amazonia-1, tell us about what we don’t know about the Amazon rainforest?

Well, this satellite, which was developed by Brazil’s National Institute for Space Research, it’s got these kind of cameras on it and it’s got a bunch of tech that will let it see any areas of deforestation I think larger than 4 football pitches or 4 soccer fields, I guess, that have been deforested. Now, the imaging that it provides isn’t actually better than satellites that are already sort of going around and looking at the Amazon rainforest, but what this satellite does, it means that the time between images being taken is much shorter. It’s only a couple of days before when a satellite goes over to have a look, I think, at this point, and so it can almost give essentially more real-time data that can be used by law enforcement agencies to stop logging where it shouldn’t be done, for example, and also give data on things like the forest fires that you mentioned to researchers.

Well, we’ve been a bit dour on this week on this week’s Briefing chat, but this actually sounds like a bit of good news.

Well, Nick, good and bad news, I’m afraid, with this story. Scientists are obviously eager to get their hands on this data, but there are concerns that the data on deforestation may be of limited use, and that’s in part because, for example, Brazil’s president back in 2019, he said that the sort of data that had been collected on deforestation was false, for example, and there are also concerns about how science will be funded in Brazil and in terms of how the mission can continue. And also, researchers are saying, ‘Well this is great, we’ve got this satellite, but we’re going to need a lot more to be able to see over a much wider area to really get a handle on what’s going on in the rainforest in as close to real time as we can.’

Well, more data can only be a good thing as long as it’s accessible and used well. So, thanks for telling me about that, Ben. And listeners, if you’re interested in more stories like this but instead as an email then make sure you check out the Nature Briefing . We’ll put a link in the show notes where you can sign up.

Before we go, just time to tell you that we’ve got a new video on our YouTube channel. It’s about how scientists are trying to build a quantum internet and to do so, they need to develop these things called quantum repeaters, and you can find out what they are in the video, and we’ll put a link to that in the show notes.

That’s all for this week. As always, if you want to reach out then we’re on Twitter. We’re @NaturePodcast. Or if you prefer email then we’re [email protected]. I’m Nick Petrić Howe.

And I’m Benjamin Thompson. See you next time.

Related Articles

• Research management
• Cell biology

How we landed job interviews for professorships straight out of our PhD programmes

Career Column 08 APR 24

How two PhD students overcame the odds to snag tenure-track jobs

Adopt universal standards for study adaptation to boost health, education and social-science research

Correspondence 02 APR 24

Is ChatGPT corrupting peer review? Telltale words hint at AI use

News 10 APR 24

Rwanda 30 years on: understanding the horror of genocide

Editorial 09 APR 24

How I harnessed media engagement to supercharge my research career

Career Column 09 APR 24

ROS-dependent S-palmitoylation activates cleaved and intact gasdermin D

Article 10 APR 24

Sex differences orchestrated by androgens at single-cell resolution

Metabolic rewiring promotes anti-inflammatory effects of glucocorticoids

Junior Group Leader Position at IMBA - Institute of Molecular Biotechnology

The Institute of Molecular Biotechnology (IMBA) is one of Europe’s leading institutes for basic research in the life sciences. IMBA is located on t...

Austria (AT)

IMBA - Institute of Molecular Biotechnology

Open Rank Faculty, Center for Public Health Genomics

Center for Public Health Genomics & UVA Comprehensive Cancer Center seek 2 tenure-track faculty members in Cancer Precision Medicine/Precision Health.

Charlottesville, Virginia

Center for Public Health Genomics at the University of Virginia

Husbandry Technician I

Memphis, Tennessee

St. Jude Children's Research Hospital (St. Jude)

Lead Researcher – Department of Bone Marrow Transplantation & Cellular Therapy

Researcher in the center for in vivo imaging and therapy.

Sign up for the Nature Briefing newsletter — what matters in science, free to your inbox daily.

Quick links

• Explore articles by subject
• Guide to authors
• Editorial policies

Making educational experiences better for everyone.

Immersive learning for 25 languages

Marketplace for millions of educator-created resources

Fast, easy, reliable language certification

Fun educational games for kids

Comprehensive K-12 personalized learning

Trusted tutors for 300+ subjects

35,000+ worksheets, games, and lesson plans

Adaptive learning for English vocabulary

• SUGGESTED TOPICS
• The Magazine
• Newsletters
• Managing Yourself
• Managing Teams
• Work-life Balance
• The Big Idea
• Data & Visuals
• Reading Lists
• Case Selections
• HBR Learning
• Topic Feeds
• Account Settings
• Email Preferences

Transformations That Work

• Michael Mankins
• Patrick Litre

More than a third of large organizations have some type of transformation program underway at any given time, and many launch one major change initiative after another. Though they kick off with a lot of fanfare, most of these efforts fail to deliver. Only 12% produce lasting results, and that figure hasn’t budged in the past two decades, despite everything we’ve learned over the years about how to lead change.

Clearly, businesses need a new model for transformation. In this article the authors present one based on research with dozens of leading companies that have defied the odds, such as Ford, Dell, Amgen, T-Mobile, Adobe, and Virgin Australia. The successful programs, the authors found, employed six critical practices: treating transformation as a continuous process; building it into the company’s operating rhythm; explicitly managing organizational energy; using aspirations, not benchmarks, to set goals; driving change from the middle of the organization out; and tapping significant external capital to fund the effort from the start.

Lessons from companies that are defying the odds

Idea in Brief

The problem.

Although companies frequently engage in transformation initiatives, few are actually transformative. Research indicates that only 12% of major change programs produce lasting results.

Why It Happens

Leaders are increasingly content with incremental improvements. As a result, they experience fewer outright failures but equally fewer real transformations.

The Solution

To deliver, change programs must treat transformation as a continuous process, build it into the company’s operating rhythm, explicitly manage organizational energy, state aspirations rather than set targets, drive change from the middle out, and be funded by serious capital investments.

Nearly every major corporation has embarked on some sort of transformation in recent years. By our estimates, at any given time more than a third of large organizations have a transformation program underway. When asked, roughly 50% of CEOs we’ve interviewed report that their company has undertaken two or more major change efforts within the past five years, with nearly 20% reporting three or more.

• Michael Mankins is a leader in Bain’s Organization and Strategy practices and is a partner based in Austin, Texas. He is a coauthor of Time, Talent, Energy: Overcome Organizational Drag and Unleash Your Team’s Productive Power (Harvard Business Review Press, 2017).
• PL Patrick Litre leads Bain’s Global Transformation and Change practice and is a partner based in Atlanta.

Partner Center

An official website of the United States government

The .gov means it’s official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

• Publications
• Account settings

Preview improvements coming to the PMC website in October 2024. Learn More or Try it out now .

• Advanced Search
• Journal List
• Afr Health Sci
• v.19(3); 2019 Sep

Drug expiry debate: the myth and the reality

Introduction.

There is so much wastage of drugs as they are not used in time. Medications are expensive, and in the Asian and African continents, where many have the problem of affordability the debate is to see if the medication could be used even after the expiry date without losing the efficacy. Most of drug expiration dates information is from the study conducted by the Food and Drug Administration at the request of the military. With a large and expensive stockpile of drugs, the military faced tossing out and replacing its drugs every few years. What they found from the study is 90% of more than 100 drugs, both prescription and over-the-counter, were perfectly good to use even 15 years after the expiration date.

Hence, the expiration date doesn't really indicate a point at which the medication is no longer effective or has become unsafe to use.

What does an expiration date mean?

The expiration date is the final day that the manufacturer guarantees the full potency and safety of a medication. Drug expiration dates exist on most medication labels, including prescription, over-the-counter (OTC) and dietary (herbal) supplements.

Proper storage of medications may help to extend their potency. The bathroom and medicine cabinet are not ideal places to store medications due to heat and humidity. Similarly, medications should not be left in a hot car. Medications remain most stable in dry, cool spaces away from light. Keep the prescription bottle caps tightly closed and always keep medications out of reach of children and pets.

Expiry date

The 2015 commentary in Mayo Clinic Proceedings, “Extending Shelf Life Just Makes Sense,” suggested that drug makers could be required to set a preliminary expiration date and then update it after long-term testing. An independent organization could also do testing similar to that done by the FDA extension program or data from the extension program could be applied to properly stored medications 1

The United States' Center for Drug Evaluation and Research officially recommends that drugs past their expiration date be disposed. It has been argued that this practice is wasteful, since consumers and medical facilities are encouraged to purchase fresh medication to replace their expired products, also resulting in additional profits for pharmaceutical firms.

According to the Food and Drug Administration (FDA) study gets to the heart of medicine expiration and safety. Updated: August 13, 2017, it turns out that the expiration date on a drug does stand for something, but probably not what you think it does. Since a law was passed in 1979, drug manufacturers are required to stamp an expiration date on their products. This is the date at which the manufacturer can still guarantee the full potency and safety of the drug.

Medical authorities state that expired medicine is safe to take, even those that expired years ago. It's true the effectiveness of a drug may decrease over time, but much of the original potency still remains even a decade after the expiration date. Excluding nitroglycerin, insulin, and liquid antibiotics, most medications are as long-lasting as the ones tested by the military. Placing a medication in a cool place, such as a refrigerator, will help a drug remain potent for many years.

Solid dosage forms, such as tablets and capsules, appear to be most stable past their expiration date. Drugs that exist in solution or as a reconstituted suspension, and that require refrigeration (such as amoxicillin suspension), may not have the required potency if used when outdated. Loss of potency can be a major health concern, especially when treating an infection with an antibiotic. In addition, antibiotic resistance may occur with sub-potent medications. Drugs that exist in solution, especially injectable drugs, should be discarded if the product forms a precipitant or looks cloudy or discolored.

A study by Khanchandani on efficacy, safety concern and disposal practices followed for expired drug preparations among medical personnel, found that 89.39% had knowledge that expiry date depends on both manufacturing and storing condition. Ninety one percent subject responded correctly that the drug should be best stored in cool dry and dark places. During the study they found out that majority consider expired drug use is not safe but 89.39% subject were aware about not using insulin, liquid antibiotic, nitroglycerin after expiry. The testing conducted by the US FDA ultimately covered more than 100 drugs prescription and OTC drugs. The result showed that about 90% were safe and effective as long as 15 years past their original expiration date. Loel Dawis, expiration date chief said that with a handful of exception notably nitroglycerin, Insulin and Liquid antibiotic most drugs are probably as durable as the agency tested. 4 .

A study done by Simons on outdated EpiPen and EpiPen Jr auto injectors: past their prime, noted that, drugs differ in terms of their forms, dosage, and stability. Usually, drugs in liquid forms (e.g. solutions and suspensions) are not as stable as those in the solid forms (e.g. tablets and capsules). It has been reported that bioavailability of EpiPen® (epinephrine auto-injectors) were reduced when administered between 1 to 90 months after the labelled expiration date compared with those that were not yet expired 5

Potency and efficacy

Medication's potency gradually decreases starting from the moment of its manufacture. This process is not in any way spontaneous after the expiry date.

Expired drugs have not necessarily lost their potency and efficacy. The expiration date is only an assurance that the labeled potency will last at least until that date. Ongoing research shows that stored under optimal conditions, many drugs retain 90% of their potency for at least five years after the labeled expiration date, and sometimes longer. Even 10 years after the expiration date many pharmaceuticals retain a significant amount of their original potency. 2

Solid dosage forms, such as tablets and capsules, are most stable past their expiration date. Drugs that exist in solution or as a reconstituted suspension may not have the required potency if used when outdated.

The best evidence of acceptable potency of the medications beyond their expiration date is provided by the Shelf Life Extension Program (SLEP) undertaken by the FDA for the Department of Defense. The aim of the SLEP program was to reduce medication costs for the military. SLEP has found that 88% of 122 different drugs stored under ideal conditions should have their expiration dates extended more than 1 year, with an average extension of 66 months, and a maximum extension of 278 months. 3 Certain medications have a narrow therapeutic index and little decreases in the pharmacological activity can result in serious consequences for patients. Monoclonal antibodies should be included in this group. These drugs should not be used beyond the expiry date.

The 2015 commentary in Mayo Clinic Proceedings, “Extending Shelf Life Just Makes Sense,” suggested that drug makers could be required to set a preliminary expiration date and then update it after long-term testing. Even though the literature denotes western circumstances, we can take a leaf or two to modify to work around it, so that it will be beneficial to the African continent, especially more so in the peripheral outreach health centers in Africa where availability and storage of medicines are a challenge.

What Reagents Can You Use Past Their Chemical Expiry Date?

Listen to one of our scientific editorial team members read this article. Click here to access more audio articles or subscribe.

Many chemicals in the lab, including buffer salts, metal salts, sugars, and SDS, are okay to use past their expiry date. Others, such as ammonium persulfate and antibiotic solutions, should not be used past their chemical expiry date. And reducing agents and hydrogen peroxide can be qualitatively checked to see if they still work. So consider the chemistry and biochemistry of your old reagents before throwing them out.

Have you ever picked a chemical that’s years out of date up off the shelf and wondered: “can I still use this?”

Chemicals are expensive. Most labs contain thousands, and few people know which are in date and which aren’t. Replacing them whenever they go out of date is a massive waste of grant money.  

Plus, we use some chemicals so rarely, and it’s also rare that people sort out their benches before leaving. The result is a pile-up of old chemicals and solutions—some potentially useful and worth keeping.  

So what reagents are usable past their chemical expiry date? How can you check if they are still okay? And which ones should you throw out?

Read on to find out.

In the lab, you’ll encounter buffer solutions (e.g., 1M tris stocks) and buffer salts (e.g., sodium citrate).  

Buffer salts should be fine to use past their expiry date since they are usually stable unreactive solids. So long as they have been stored in their original container, you’re good to go.

Buffer solutions are different, and you should probably throw them out. Why so? Because they are usually pH adjusted to somewhere around physiological pH.  

Consider also that buffer molecules, their breakdown products, and molecules dissolved from the air provide carbon and nitrogen sources, and you have ideal growing conditions for unwanted bacteria and fungus.  

They can also change the pH of the solution away from the one written on the bottle.

Swirl your buffer solutions occasionally and look for floaty bits to check if anything is growing in them.

In a pinch, you could filter sterilize your buffer solutions to remove anything that has grown before reusing. Just check its pH is still correct.

2. General Salts

Most of the chemicals on your shelves will be salts of some sort, like magnesium chloride, potassium phosphate, and calcium carbonate.

Like buffer crystals, many of the salts in your lab are unreactive solids and are okay to use beyond their expiry date.

Be Cautious Using Old Hygroscopic Salts

Just beware that some salts are hygroscopic and will attract atmospheric moisture to themselves. Sometimes so much so they get physically wet.  

Examples of hygroscopic salts include zinc chloride and calcium chloride.

Plus, simple salts are relatively cheap, meaning we buy them in huge tubs that sit around for years.

You’ll find out which old salts are hygroscopic when you open them and are soaking wet.

Throw these out because you can’t weigh them out accurately, meaning there will be an error in the molarity of your solutions.

In a pinch, you could try drying them out by freeze-drying or warming them up gently in low-humidity conditions.

3. Reducing Agents

Let’s split these into two.

BME and DTT

Beta-mercaptoethanol (BME) is a liquid at room temperature and dithiothreitol (DTT) is a white solid at room temperature.

Avoid reusing solutions containing them, and add them immediately before your experiments.

Depending on the pH and temperature of the solution, their half-life is between a few hours and several days.  

Pure BME and DTT are stable for 1-3 years if stored properly. You’re likely to finish the bottle before it goes out of date.

Don’t use them beyond their shelf life because, if they have lost their potency, it will give you a real headache when you troubleshoot what’s gone wrong.

Tris(2-carboxyethyl)phosphine (TCEP) is a solid at room temperature and more stable in aqueous solutions than BME and DTT.

The powder is stable and should be okay to use beyond its use-by date.  

Solutions of it are more stable than BME and DTT solutions but don’t last forever. Discard them after a month or freeze them down at -20°C.

How to Qualitatively Check if Reducing Agents Still Work

Find a sample of a protein in your lab containing two or more peptide chains held together by disulfide bonds and run a gel of it.  

Antibodies work great but check to see if you have something cheaper to hand.

• Run one gel with a sample containing 1-10 mM reducing agent.  
• Run another gel with no reducing agent in the sample.

If the reducing agent is okay, it will break the disulfide bonds, resulting in multiple bands on your gel corresponding to the individual protein chains.  

The unreduced sample will produce a single heavier band corresponding to the disulfide-linked chains.

And finally, because they lose potency quickly, reducing agents are one of the first things to check when things unexpectedly go wrong!

4. Phenylmethylsulfonyl Fluoride

Phenylmethylsulfonyl fluoride (PMSF) is a common serine protease inhibitor.

The powder is okay to use past its use-by date unless you notice your sample starts getting proteolyzed. If this happens, buy some new PMSF.  

Aqueous solutions of PMSF have a half-life of about 30 minutes, so don’t reuse them.

And stock solutions of PMSF are usually prepared at 100 mM in isopropanol. These should be stable for several months at 4°C or several years at -20°C.

5. Heavy Metal Salts and Solutions

Heavy metal salts include chemicals like nickel sulfate, cobalt chloride, silver nitrate, and gold chloride.  

These stable compounds last almost indefinitely and can be used well beyond their shelf life.

Spillage and contamination are more likely to ruin them before they become chemically unsuitable for your experiments.  

Ditto for solutions. Heavy metals are antiseptic. Nothing should grow in gold, silver, cobalt, and nickel solutions. So you can keep them for years on your shelf!

6. Antibiotics

Generally, use the powders regardless of how old they are, but don’t use old solutions.  

A few points to note.  

Generally speaking, the crystals are unreactive solids. That’s why it doesn’t matter if they are old.  

However, once dissolved in solution, some will doubtless be more stable than others.  

Now, there are a lot of antibiotics that we use in research , and I don’t know the stability and chemistry of most of them.

Plus, they will have different stabilities depending on what they are dissolved in and how they have been stored.

Finally, antibiotics are usually a critical component of our experiments.

That’s why it’s best to err on the side of caution with solutions.

7. Ammonium Persulfate

Desiccated ammonium persulfate is stable and will last for years. So you can use the really old stuff that’s in your lab.  

However, ammonium persulfate hydrolyzes rapidly in solution, producing ammonium hydrogen sulfate and hydrogen peroxide.

Never use old ammonium persulfate solutions!

8. Hydrogen Peroxide

Hydrogen peroxide is okay to use past its chemical expiry date if stored in the dark.  

The liquid, supplied as a 30 % w/v solution, is stable but decomposes when exposed to light. So factor this into your decision.  

For this reason, and because reagent bottles are transparent, old working solutions that contain dilute hydrogen peroxide should be discarded.

How to Qualitatively Check Whether Hydrogen Peroxide Is Still Usable

Hydrogen peroxide oxidizes potassium iodide, forming iodine, potassium hydroxide, and hydrogen gas.  

Aqueous solutions of potassium iodide are clear, but iodine solutions are brown.  

Thus, if you prepare 100 mM potassium iodide, add a few drops of hydrogen peroxide into it, and it slowly turns brown—the hydrogen peroxide still works.  

9. Sodium Dodecyl Sulfate

Sodium dodecyl sulfate (SDS) powders and solutions should be fine to use beyond their use-by date.  

Sometimes, sodium dodecyl sulfate may precipitate out of the solution. Especially when it gets cold in the lab. If this happens, heat the solution gently on a hot plate while stirring, and cover the bottle in foil to keep the heat in.

Old powders should be fine to use but check solutions for growth.  

Like most chemicals on this list (and in general), the crystals are stable, unreactive solids.  

Sugar solutions are similarly stable, but the molecules are high-energy food sources for microbes. Add the fact that sugar solutions will be at or near neutral pH, and things start growing in them really quickly.  

Be sure to check old solutions for contamination before using them.  

Think About the Chemistry

Before discarding a reagent out of superstition, think about its chemistry. For example:

• Is it composed of strong ionic bonds?
• Can it oxidize?
• Is it photosensitive?
• Does it degrade in water?
• Might its composition have changed?

Think About the Biochemistry

Before using old chemicals and solutions, consider biochemistry.  

• What properties make it useful?
• Is it a nitrogen source?
• Is it a carbon source?
• Is it antiseptic?
• Will it retain selectivity?

Can You Use Out-of-Date Chemicals? Answers At a Glance

For easy reference, here’s all that summarized in a convenient table.

Table 1 . Summary of whether you can use certain reagents past their chemical expiry date.

Using Reagents Beyond Their Chemical Expiry Date Summarized

There’s your list of common chemicals, whether or not they are okay to use past their chemical expiry date, and how to check if they are still good!  

Plus, a bit of chemistry to help you understand why some old reagents still work. Hopefully, you can apply this wisdom to chemicals in your lab to help you decide on a case-by-case basis.

Using old reagents can save you time waiting for orders and money otherwise spent on them.

Note that these aren’t hard and fast rules. I’ve based what I’ve written on the cumulation of my lab experience. You can take this advice as generally or literally as you wish.  

For sensitive and critical experiments, try to use the freshest reagents to avoid a heavy heart if it goes wrong.  

Any other helpful tips and lab hacks you want to share? Have I missed a good chemical out? Drop them in the comments section below!

Leave a Comment Cancel Reply

You must be logged in to post a comment.

This site uses Akismet to reduce spam. Learn how your comment data is processed .

Forgot your password?

Lost your password? Please enter your email address. You will receive mail with link to set new password.

Back to login

• school Campus Bookshelves
• menu_book Bookshelves
• perm_media Learning Objects
• login Login
• how_to_reg Request Instructor Account
• hub Instructor Commons
• Download Page (PDF)
• Download Full Book (PDF)
• Periodic Table
• Physics Constants
• Scientific Calculator
• Reference & Cite
• Tools expand_more
• Readability

selected template will load here

This action is not available.

13.1: The Solution Process

• Last updated
• Save as PDF
• Page ID 21784

Learning Objectives

• To understand how enthalpy and entropy changes affect solution formation.
• To use the magnitude of the changes in both enthalpy and entropy to predict whether a given solute–solvent combination will spontaneously form a solution.

In all solutions, whether gaseous, liquid, or solid, the substance present in the greatest amount is the solvent, and the substance or substances present in lesser amounts are the solute(s). The solute does not have to be in the same physical state as the solvent, but the physical state of the solvent usually determines the state of the solution. As long as the solute and solvent combine to give a homogeneous solution, the solute is said to be soluble in the solvent. Table \(\PageIndex{1}\) lists some common examples of gaseous, liquid, and solid solutions and identifies the physical states of the solute and solvent in each.

Forming a Solution

The formation of a solution from a solute and a solvent is a physical process, not a chemical one. That is, both solute and solvent can be recovered in chemically unchanged forms using appropriate separation methods. For example, solid zinc nitrate dissolves in water to form an aqueous solution of zinc nitrate:

\[\ce{Zn(NO3)2(s) + H2O(l) \rightarrow Zn^{2+}(aq) + 2NO^{-}3(aq)} \label{13.1.1} \]

Because \(Zn(NO_3)_2\) can be recovered easily by evaporating the water, this is a physical process. In contrast, metallic zinc appears to dissolve in aqueous hydrochloric acid. In fact, the two substances undergo a chemical reaction to form an aqueous solution of zinc chloride with evolution of hydrogen gas:

\[\ce{ Zn(s) + 2H^{+}(aq) + 2Cl^{-}(aq) \rightarrow Zn^{2+}(aq) + 2Cl^{-}(aq) + H2(g)} \label{13.1.2} \]

When the solution evaporates, we do not recover metallic zinc, so we cannot say that metallic zinc is soluble in aqueous hydrochloric acid because it is chemically transformed when it dissolves. The dissolution of a solute in a solvent to form a solution does not involve a chemical transformation (that it is a physical change ).

Dissolution of a solute in a solvent to form a solution does not involve a chemical transformation.

Substances that form a single homogeneous phase in all proportions are said to be completely miscible in one another. Ethanol and water are miscible, just as mixtures of gases are miscible. If two substances are essentially insoluble in each other, such as oil and water, they are immiscible . Examples of gaseous solutions that we have already discussed include Earth’s atmosphere.

The Role of Enthalpy in Solution Formation

Energy is required to overcome the intermolecular interactions in a solute, which can be supplied only by the new interactions that occur in the solution, when each solute particle is surrounded by particles of the solvent in a process called solvation (or hydration when the solvent is water). Thus all of the solute–solute interactions and many of the solvent–solvent interactions must be disrupted for a solution to form. In this section, we describe the role of enthalpy in this process.

Because enthalpy is a state function , we can use a thermochemical cycle to analyze the energetics of solution formation. The process occurs in three discrete steps, indicated by \(ΔH_1\), \(ΔH_2\), and \(ΔH_3\) in Figure \(\PageIndex{2}\). The overall enthalpy change in the formation of the solution (\( \Delta H_{soln}\)) is the sum of the enthalpy changes in the three steps:

\[ \Delta H_{soln} = \Delta H_1 + \Delta H_2 + \Delta H_3 \label{13.1.3} \]

When a solvent is added to a solution, steps 1 and 2 are both endothermic because energy is required to overcome the intermolecular interactions in the solvent (\(\Delta H_1\)) and the solute (\(\Delta H_2\)). Because \(ΔH\) is positive for both steps 1 and 2, the solute–solvent interactions (\(\Delta H_3\)) must be stronger than the solute–solute and solvent–solvent interactions they replace in order for the dissolution process to be exothermic (\(\Delta H_{soln} < 0\)). When the solute is an ionic solid, \(ΔH_2\) corresponds to the lattice energy that must be overcome to form a solution. The higher the charge of the ions in an ionic solid, the higher the lattice energy. Consequently, solids that have very high lattice energies, such as \(MgO\) (−3791 kJ/mol), are generally insoluble in all solvents.

A positive value for \(ΔH_{soln}\) does not mean that a solution will not form. Whether a given process, including formation of a solution, occurs spontaneously depends on whether the total energy of the system is lowered as a result. Enthalpy is only one of the contributing factors. A high \(ΔH_{soln}\) is usually an indication that the substance is not very soluble. Instant cold packs used to treat athletic injuries, for example, take advantage of the large positive \(ΔH_{soln}\) of ammonium nitrate during dissolution (+25.7 kJ/mol), which produces temperatures less than 0°C (Figure \(\PageIndex{3}\)).

Entropy and Solution Formation

The enthalpy change that accompanies a process is important because processes that release substantial amounts of energy tend to occur spontaneously. A second property of any system, its entropy, is also important in helping us determine whether a given process occurs spontaneously. We will discuss entropy in more detail elsewhere, but for now we can state that entropy (\(S\)) is a thermodynamic property of all substances that is proportional to their degree of disorder. A perfect crystal at 0 K, whose atoms are regularly arranged in a perfect lattice and are motionless, has an entropy of zero. In contrast, gases have large positive entropies because their molecules are highly disordered and in constant motion at high speeds.

The formation of a solution disperses molecules, atoms, or ions of one kind throughout a second substance, which generally increases the disorder and results in an increase in the entropy of the system. Thus entropic factors almost always favor formation of a solution. In contrast, a change in enthalpy may or may not favor solution formation. The London dispersion forces that hold cyclohexane and n-hexane together in pure liquids, for example, are similar in nature and strength. Consequently, \(ΔH_{soln}\) should be approximately zero, as is observed experimentally. Mixing equal amounts of the two liquids, however, produces a solution in which the n-hexane and cyclohexane molecules are uniformly distributed over approximately twice the initial volume. In this case, the driving force for solution formation is not a negative \(ΔH_{soln}\) but rather the increase in entropy due to the increased disorder in the mixture. All spontaneous processes with \(ΔH \ge 0\) are characterized by an increase in entropy. In other cases, such as mixing oil with water, salt with gasoline, or sugar with hexane, the enthalpy of solution is large and positive, and the increase in entropy resulting from solution formation is not enough to overcome it. Thus in these cases a solution does not form.

All spontaneous processes with ΔH ≥ 0 are characterized by an increase in entropy.

Table \(\PageIndex{2}\) summarizes how enthalpic factors affect solution formation for four general cases. The column on the far right uses the relative magnitudes of the enthalpic contributions to predict whether a solution will form from each of the four. Keep in mind that in each case entropy favors solution formation. In two of the cases the enthalpy of solution is expected to be relatively small and can be either positive or negative. Thus the entropic contribution dominates, and we expect a solution to form readily. In the other two cases the enthalpy of solution is expected to be large and positive. The entropic contribution, though favorable, is usually too small to overcome the unfavorable enthalpy term. Hence we expect that a solution will not form readily.

In contrast to liquid solutions, the intermolecular interactions in gases are weak (they are considered to be nonexistent in ideal gases). Hence mixing gases is usually a thermally neutral process (\(ΔH_{soln} \approx 0\)), and the entropic factor due to the increase in disorder is dominant (Figure \(\PageIndex{4}\)). Consequently, all gases dissolve readily in one another in all proportions to form solutions.

Example \(\PageIndex{1}\)

Considering \(\ce{LiCl}\), benzoic acid (\(\ce{C6H5CO2H}\)), and naphthalene, which will be most soluble and which will be least soluble in water?

Given : three compounds

Asked for: relative solubilities in water

Strategy : Assess the relative magnitude of the enthalpy change for each step in the process shown in Figure \(\PageIndex{2}\). Then use Table \(\PageIndex{2}\) to predict the solubility of each compound in water and arrange them in order of decreasing solubility.

The first substance, \(\ce{LiCl}\), is an ionic compound, so a great deal of energy is required to separate its anions and cations and overcome the lattice energy (ΔH 2 is far greater than zero in Equation \(\ref{13.1.1}\)). Because water is a polar substance, the interactions between both Li + and Cl − ions and water should be favorable and strong. Thus we expect \(ΔH_3\) to be far less than zero, making LiCl soluble in water. In contrast, naphthalene is a nonpolar compound, with only London dispersion forces holding the molecules together in the solid state. We therefore expect \(ΔH_2\) to be small and positive. We also expect the interaction between polar water molecules and nonpolar naphthalene molecules to be weak \(ΔH_3 \approx 0\). Hence we do not expect naphthalene to be very soluble in water, if at all. Benzoic acid has a polar carboxylic acid group and a nonpolar aromatic ring. We therefore expect that the energy required to separate solute molecules (ΔH 2 ) will be greater than for naphthalene and less than for LiCl. The strength of the interaction of benzoic acid with water should also be intermediate between those of LiCl and naphthalene. Hence benzoic acid is expected to be more soluble in water than naphthalene but less soluble than \(\ce{LiCl}\). We thus predict \(\ce{LiCl}\) to be the most soluble in water and naphthalene to be the least soluble.

Exercise \(\PageIndex{1}\)

Considering ammonium chloride, cyclohexane, and ethylene glycol (\(HOCH_2CH_2OH\)), which will be most soluble and which will be least soluble in benzene?

The most soluble is cyclohexane; the least soluble is ammonium chloride.

Solutions are homogeneous mixtures of two or more substances whose components are uniformly distributed on a microscopic scale. The component present in the greatest amount is the solvent, and the components present in lesser amounts are the solute(s). The formation of a solution from a solute and a solvent is a physical process, not a chemical one. Substances that are miscible, such as gases, form a single phase in all proportions when mixed. Substances that form separate phases are immiscible. Solvation is the process in which solute particles are surrounded by solvent molecules. When the solvent is water, the process is called hydration. The overall enthalpy change that accompanies the formation of a solution, \(ΔH_{soln}\), is the sum of the enthalpy change for breaking the intermolecular interactions in both the solvent and the solute and the enthalpy change for the formation of new solute–solvent interactions. Exothermic (\(ΔH_{soln} < 0\)) processes favor solution formation. In addition, the change in entropy, the degree of disorder of the system, must be considered when predicting whether a solution will form. An increase in entropy (a decrease in order) favors dissolution.