Those of you who have ever brought a piece of scientific research to completion — a process which almost always includes publishing your results — have probably run into lists like this one that spell out the meanings of phrases commonly used in scientific papers. Included are such classics as:
“It has long been known”
I don’t know the original reference.
and
“Typical results are shown”
Either means the only results are shown or the best results are shown.
and
“A trend is evident”
Okay, a trend does seem apparent to me, but no statistical analysis in the world will support it.
I’m hoping by now that you’ll have caught that these lists are intended to be humorous. Scientists are engaged in an endeavor where they’re trying to figure out what the data show about the world, not just what they want to see in their experimental results. Ideally, scientists are making sure their data and conclusions can stand up to the toughest objections they can imagine being raised before they even send their manuscripts off to the journal. And, to the extent that science is a knowledge-building project where scientists need to be able to depend on the results communicated by other scientists, they know they should be striving for scrupulous honesty and utter clarity of language. Irony is not a literary device that ought to be getting a lot of use in scientific communication.
And yet, part of what drives the “humor” in the “translation guide” is that there are scientists who do engage in … what to call it? Putting the most favorable spin on their results? Stretching the meanings of the words as far as they can go without engaging in outright lies? It’s not the kind of thing in which scientists are typically proud to engage, but when an experiment is being particularly cranky in year 7 of a graduate program, one can imagine that it might be a better option than saying, “I’ve got nothing.” And certainly, one suspects that other scientists are engaging in scientific puffery.
The other day I heard of another example in this vein. Professor Smith finds an interesting and important result, writes it up, and sends it to the journal, which accepts it for publication. A few months later, Jones submits a report of a very similar (if not exactly identical) result. The kicker: Jones includes a paragraph near the end of his manuscript saying “Our results were recently confirmed by Smith.”
OK, there’s a sense in which Smith’s results and Jones’s results do confirm each other; when different scientists get the same experimental results, it suggests they’ve gotten hold of some real feature of the world. But the wording here is certainly meant to suggest that Jones got to the result first and Smith got there later and found that Jones was indeed correct in what he reported– while the priority as established by when the two manuscripts were received by the journal is just the opposite.
Jones seems to be trying to put one over on the other scientists reading his article. Smith, especially, views this as less than honest.
Why do people use words whose meaning is some distance from the truth? I imagine it’s because they anticipate that there’s a benefit that will come to them from cultivating a certain predictable misunderstanding, or a cost that will come to them if they state the truth plainly. The way the score is kept in the world of science, Jones doesn’t get any points at all for his finding if Smith gets there first — not because it’s not an important finding, nor even because Jones didn’t get there on the basis of his own effort and ingenuity (since he did), but simply because the scientist who gets there first gets to plant his flag and claim the glory. Keeping score this way makes it seem irrational not to try to squeeze some extra points out of the system by playing on ambiguities in the word “confirmed”.
This strikes me as one more reason it might be useful to reexamine the reward structure to see whether it’s actually motivating scientists to produce more and better knowledge.
Someone once published a paper a couple years after one of mine. I had proposed an explanation for the phenomenon we were studying. Their paper recited the same explanation without crediting our paper, and went on to say that it was a good explanation because it explained our results, as if we hadn’t. It was so blatant it was funny.
Great post!
Here’s a wrinkle in the Jones/Smith analogy. I think it would muddy the relationship between the competing scientists, but offers more support for reward structure reform.
What if Smith has a tendency to rush sloppy results out the door, or in a more serious situation, benefited from reading a grant proposal or paper submission by Jones or saw a poster by Jones’ graduate student? It would be unethical for Smith to incorporate Jonesian ideas into his own paper, but the knowledge that something is breaking from the Jones camp could rush a more preliminary finding in order to stake a claim.
Now what if Jones has a reputation for assembling a complete story – including truly ‘representative’ images? And we’re not talking about overly cautious – just careful science. Is getting scooped the price you pay for doing good science?
If Jones’ data is more complete, if it gets to the issue from more angles, it will be clear their colleagues (if not Smith) that Jones probably did make this observation first. In the long run, Smith’s claim may wash out because Jones has a more established platform to ask follow-up questions.
One rarely pursued option is co-publication. If everyone is honest with each other, and has the truth of their finding as a priority, it seems like Jones and Smith (and journal editors?) would want to publish their work at the same time. The only instances I’ve encountered of this (but I haven’t looked) is in the vignettes describing shared Nobel prizes. But once you’re of the stature to win a Nobel, who needs publication records?
A lot of interesting dilemmas. If both Jones and Smiths results came from independent multiyear efforts, one would hope that Smith would still get considerable credit wrt future funding etc. And his result should be considered as valuable confirmation…
Then I can easily imagine the situation where a researcher is going to suffer a serious career setback if his project doesn’t produce an interesting result….
c’mon now thomas. this stuff is rampant. just one favorite example. take a look at nature genetics 24(4) for three papers on CRF2 receptor KO mouse. look very closely at the “submitted” dates. check the acknowledgments. do the math.
and getting back to the point, WHY is it important to copublish? this just perpetuates the stupidity that “first” is better where first is measured in days. this kind of precision is just not necessary for a reasonable person to judge “independent” work from clearly derivative or out and out copying work. Dr. F-R, this is also applicable to your example, mere publication date isn’t everything. What about submission date? What about evidence of prior interest and work on the topic? Your Jones might be well justified in trying to correct the record.
with respect to thomas’ Jones/Smith furtherance, well unfortunately the quick/sloppy who gets into Science or Nature wins. Now it is true that eventually, over time, Jones has a better rep within his field. Unfortunately in so many ways scientists are judged by people who are not in the immediate field, refuse to entertain subjective “quality” judgments and increasingly resort to supposed objective criteria like “number of Science papers”. it isn’t just a risk of getting “scooped” it is a risk of not making it in those areas of science where it is semi-expected that Science and Nature will turn up on your CV.
Now it is true that eventually, over time, Jones has a better rep within his field
<type=”postdoc” style=”disgruntled”>Only if he survives long enough. If Smith eats his lunch enough times, he will not retain funding long enough to establish any reputation: he’ll hit tenure review and so long, chump, you should have cheated like everyone else. </style>
While people may have become scientists because they wanted to learn more about the natural world, that is only a side effect of the enterprise they are actually engaged in, and the enterprise on which they are judged and for which they are rewarded.
What they are really engaged in is a race for prestige. That’s what they are rewarded on– Nobel prizes don’t go to the most careful laboratory workers, to the people who made the most rigorous calculations, to the people who most humbly represented their results. They go to the people who have the sexiest results. Universities don’t hire people who are good at science, they hire people who have highly cited papers. Recognition never comes from carefully analyzed null results, but can come from quick-dash exciting results.
Bill Hooker touched on the negative consequences of an overriding emphasis on competition in science in column in 3 Quarks Daily.
To be honest, some of the things you’re talking about are sometimes minor enough transgressions that I wouldn’t worry about them. But you are right that there is a pattern that we have to oversell our results in an attempt to make ourselves each look more important than perhaps we really are.
It’s a pity. If we were really engaged in the enterprise we all want to be engaged in, there would be no penalty for humble honesty, there would be no penalty for coming up with a null result. Unfortunately, we each have our own livelihoods to look out for, which given the way the system is put together, often pushes us in the wrong direction.
-Rob
“The other day I heard of another example in this vein.”
Don’t be coy! Let’s have some citations!
Don’t be coy! Let’s have some citations!
Here I will be coy. (“Smith” and I were guests at the same wedding, and I’m not prepared to set up conditions where people won’t chat with me at weddings.)
And I appreciate the other potentially relevant details y’all are putting on the table such as date of first submission, the quality of the results, and so forth. It seems to me that the reason we have to attend to these details in the first place turns largely on the set-up of a scientist vs. scientist competition instead of scientists against the world (i.e., working together to work out the truth).
With that … I’ve given myself a Helen Reddy earworm. Ack!
If it is a problem that ‘first’ is measured in days, I still do not see the harm in co-publishing. If it doesn’t do anything to fix a broken rewards system, at least it might throw some sand in its eyes. In terms of total numbers of papers, I think my use of ‘rarely’ is justified, but I’ll concede that in situations where competitors make similar findings, co-publications may actually not be rare. But this presumes the competitors are moderately friendly and actually know about the other’s progress.
Has there ever been any game theory applied to this and other publishing situations in science?
set-up of a scientist vs. scientist competition instead of scientists against the world (i.e., working together to work out the truth).
Yes!
Damn bloody evolution, it made us into such competitive creatures. A “competition is good” ethic keeps creeping in to all sorts of regimes where perhaps it does more harm than good; certainly worshipping it as an unabashed good allows it to do its harm without enough questioning.
Mind you, I can see some dangers in a total lack of competition– a bandwagon effect where everybody smiles and agrees on everything, and we get taken down a garden path and don’t learn the right things. But, as it is right now, we have to look out for our own record before we look out for learning science.
-Rob
I think David Hull’s “Science as a Process” speaks directly to the issues being raised. Just how cooperative and competitive impulses should be directed to promote sound, ethical science poses a difficult problem.
Scientists are not different from professional athletes. The football player who scores the touchdown raises his index fingure, performs the victory dance at the endzone or invents any number of antics to promote himself and his achievement. The team, the choaches, the teachers, they are all secondary to the momentary glory that is being caught on film. And the victroy and the glory, even if achieved through some cheating, are usually not diminished. That’s why quarterbacks and runningbacks are the highest paid players on the team, they promote themselves and they are in positions most visible on the team, not necessarily because they are the most improtant positions or the best players on the team.
The “competition” from the perspective of a computer science professor who left the academia:
Why I am Not a Professor OR The Decline and Fall of the British University
“… You can routinely find lecturers with more than a hundred published papers and you marvel at these paradigms of human creativity. These are people, you think, who are fit to challenge Mozart who wrote a hundred pieces or more of music. And then you get puzzled that, in this modern world, there should be so many Mozarts – almost one for every department. ” — Mark Tarver
Competition seems to be getting a bad rap here. As I see it, the structure of science today is a brilliant mechanism for leveraging a basically negative tendency toward status-seeking and invidious comparison toward a prosocial goal of accelerated research productivity.
Sure, if everyone was equally motivated and less competitive, science might proceed more efficiently. But that’s not an option: if people were less competitive, they wouldn’t be equally motived to devote the sort of time and energy and passion to science that most successful scientists do.
-Crow
But the wording here is certainly meant to suggest that Jones got to the result first and Smith got there later and found that Jones was indeed correct in what he reported…
I might feel differently if I saw the real example you have in mind, with that statement in context. But as you’ve presented it, I don’t see the problem.
It’s an accurate presentation of the way the research was performed. Jones (really, Jones’ grad student) came up with the line of analysis, did the work, interpreted the results and wrote the paper. At some point, the Smith results appeared. The text you have is a perfectly appropriate statement of that process, and details of priority are not.
Very Important Professor gives a superb presentation on how to give scientific talks. One of his points is: if you and someone else published the same thing around the same time, just say that. Talking about how your paper came out a month before or after the other person’s just makes you sound petty.
Now, if Jones read Smith’s paper, and then did all the work and banged out a paper, that’s completely different…
For more on ethical ambiguity in scientific publication, see the past post of one of your fellow Sciencebloggers:
The armadillodile diaries, a story of science ethics
the set-up of a scientist vs. scientist competition instead of scientists against the world (i.e., working together to work out the truth)
Part of what creates this competition is the idea that “negative” results (no effect found) or duplicative results aren’t worth publishing. I would argue that there’s tremendous value in both, particularly in fields where experimentation is costly or lengthy. Never let a result go to waste, even if it’s not sexy!
But because negative and confirmatory results aren’t exciting to journals (or authors; there’s some natural disappointment at work), the unluckier author is motivated to find a spin that makes her work look more “original” (and more timely).
A philosopher of science in the great German ‘LaborJournal’ put it this way: “science acquires more and more attributes of professional sports. Doping of athletes is the consequence. Is this really what we want?”
Rob Knop, I like your comments.
A different angle: re the “confirmation” issue Janet raised: I’ve seen manuscripts written by researchers whose first language is not English who clearly do not understand the temporal meaning in ‘confirmed”, so some people get the benefit of the doubt.
It’s a misconception to think that scientists are usually responsible for co-publication. Editors are, and they (at least some of them) really don’t care much about small differences in priority. A paper of mine was held up for months. Turned out the editor just wanted to package it with a bunch of other interesting papers in the same field that were not competing. I was happy: our paper was probably read by many more colleagues than it would have been.