The overarching project of science is building reliable knowledge about the world, but the way this knowledge-building happens in our world is in the context of competition. For example, scientists compete with each other to be the first to make a new discovery, and they compete with each other for finite pools of grant money with which to conduct more research and make further discoveries.
I’ve heard the competitive pressures on scientists described as a useful way to motivate scientists to be clever and efficient (and not to knock off early lest some more dedicated lab get to your discovery first). But there are situations where it’s less obvious that fierce competition for scarce resources leads to choices that really align with the goal of building reliable knowledge about the world.
This week, on NPR’s Morning Edition, Richard Harris reported a pair of stories on how researchers who work with cells in culture grapple with the problem of their intended cell line being contaminated and overtaken by a different cell line. Harris tells us:
One of the worst cases involves a breast cancer cell line called MDA-435 (or MDA-MB-435). After the cell line was identified in 1976, breast cancer scientists eagerly adopted it.
When injected in animals, the cells spread the way breast cancer metastasizes in women, “and that’s not a very common feature of most breast cancer cell lines,” says Stephen Ethier, a cancer geneticist at the Medical University of South Carolina. “So as a result of that, people began asking for those cells, and so there are many laboratories all over the world, who have published hundreds of papers using the MDA-435 cell line as a model for breast cancer metastasis.”
In fact, scientists published more than a thousand papers with this cell line over the years. About 15 years ago, scientists using newly developed DNA tests took a close look at these cells. And they were shocked to discover that they weren’t from a breast cancer cell at all. The breast cancer cell line had been crowded out by skin cancer cells.
“We now know with certainty that the MDA-435 cell line is identical to a melanoma cell line,” Ethier says.
And it turns out that contamination traces back for decades. Several scientists published papers about this to alert the field, “but nevertheless, there are people out there who haven’t gotten the memo, apparently,” he says.
Decades worth of work and more than a thousand published research papers were supposed to add up to a lot of knowledge about a particular kind of breast cancer cell, except it wasn’t knowledge about breast cancer cells at all because the cells in the cell line had been misidentified. Probably scientists know something from that work, but it isn’t the knowledge they thought they had before the contamination was detected.
On the basis of the discovery that this much knowledge-building had been compromised by being based on misidentified cells, you might imagine researchers would prioritize precise identification of the cells they use. But, as Harris found, this obvious bit of quality control meets resistance. For one thing, researchers seem unwilling to pay the extra financial costs it would take:
This may all come down to money. Scientists can avoid most of these problems by purchasing cells from a company that routinely tests them. But most scientists would rather walk down the hall and borrow cells from another lab.
“Academics share their cell lines like candy because they don’t want to go back and spend another $300,” said Richard Neve from Genentech. “It is economics. And they don’t want to spend another $100 to [verify] that’s still the same cell line.”
Note here that scientists could still economize by sharing cell lines with their colleagues instead of purchasing them but paying for the tests to nail down the identity of the shared cells. However, many do not.
(Consider, though, how awkward it might be to test cells you’ve gotten from a colleague only to discover that they are not the kind of cells your colleague thought they were. How do you break the news to your colleague that their work — including published papers in scientific journals — is likely to be mistaken and misleading? How likely would this make other colleagues to share their cell lines with you, knowing that you might bring them similarly bad news as a result of their generosity?)
Journals like Nature have tried to encourage scientists to test their cell lines by adding it to an authors’ checklist for researchers submitting papers. Most authors do not check the box indicating they have tested their cells.
One result here is that the knowledge that comes from these studies and gets reported in scientific journals may not be as solid as it seems:
When scientists at [Genentech] find an intriguing result from an academic lab, the first thing they do is try to replicate the result.
Neve said often they can’t, and misidentified cells are a common reason.
This is a problem that is not just of concern to scientists. The rest of us depend on scientists to build reliable knowledge about the world in part because it might matter for what kinds of treatments are developed for diseases that affect us. Moreover, much of this research is paid for with public money — which means the public has an interest in whether the funding is doing what it is supposed to be doing.
However, Harris notes that funding agencies seem unwilling to act decisively to address the issue of research based on misidentified cell lines:
“We are fully convinced that this is a significant enough problem that we have to take steps to address it,” Jon Lorsch, director of the NIH’s National Institute of General Medical Sciences, said during the panel discussion.
One obvious step would be to require scientists who get federal funding to test their cells. Howard Soule, chief science officer at the Prostate Cancer Foundation, said that’s what his charity requires of the scientists it funds.
There’s a commercial lab that will run this test for about $140, so “this is not going to break the bank,” Soule said.
But Lorsch at the NIH argued that it’s not so simple on the scale at which his institute hands out funding. “We really can’t go and police 10,000 grants,” Lorsch said.
“Sure you can,” Soule shot back. “How can you not?”
Lorsch said if they do police this issue, “there are dozens and dozens of other issues” that the NIH should logically police as well. “It becomes a Hydra,” Lorsch said. “You know, you chop off one head and others grow.”
Biomedical research gets more expensive all the time, and the NIH is reluctant to pile on a whole bunch of new rules. It’s a balancing act.
“If we become too draconian we’re going to end up squashing creativity and slowing down research, which is not good for the taxpayers because they aren’t going to get as much for their money,” Lorsch said.
To my eye, Lorsch’s argument against requiring researchers to test their cells focuses on the competitive aspect of scientific research to the exclusion of the knowledge-building aspect.
What does it matter if the taxpayers get more research generated and published if a significant amount of that research output is irreproducible because of misidentified cells? In the absence of tests to properly identify the cells being used, there’s no clear way to tell just by looking at the journal articles which ones are reliable and which ones are not. Post-publication quality control requires researchers to repeat experiments and compare their results to those published, something that will cost significantly more than if the initial researchers tested their cells in the first place.
However, research funding is generally awarded to build new knowledge, not to test existing knowledge claims. Scientists get credit for making new discoveries, not for determining that other scientists’ discoveries can be reproduced.
NIH could make it a condition of funding that researchers working with cell lines get those cell lines tested, and arguably this would be the most cost-efficient way to ensure results that are reliable rather than based on misidentification. I find Lorsch’s claim that there are dozens of other kinds of quality control of this sort NIH could demand, so they cannot demand this, unpersuasive. Even if there are many things to fix, it doesn’t mean you must fix them all at once. Incremental improvements in quality control are surely better than none at all.
His further suggestion that engaging in NIH-mandated quality control will quash scientific creativity strikes me as silly. Scientists are at their most creative when they are working within constraints to solve problems. Indeed, were NIH to require that researchers test their cells, there is no reason to think this additional constraint could not be easily incorporated into researchers’ current competition for NIH funding.
The big question, really, is whether NIH is prioritizing funding a higher volume of research, or higher quality research. Presumably, the public is better served by a smaller number of published studies that make reliable claims about the actual cells researchers are working with than by a large number of published studies making hard-to-verify claims about misidentified cells.
If scientific competition is inescapable, at least let’s make sure that the incentives encourage the careful steps required to build reliable knowledge. If those careful steps are widely seen as an impediment in succeeding in the competition, we derail the goal that the competitive pressures were supposed to enhance.
