Tag Archives: tribe of science

Scientific authorship: guests, courtesy, contributions, and harms.

Posted on by

DrugMonkey asks, where’s the harm in adding a “courtesy author” (also known as a “guest author”) to the author line of a scientific paper?

I think this question has interesting ethical dimensions, but before we get into those, we need to say a little bit about what’s going on with authorship of scientific papers.

I suppose there are possible worlds in which who is responsible for what in a scientific paper might not matter. In the world we live in now, however, it’s useful to know who designed the experimental apparatus and got the reaction to work (so you can email that person your questions when you want to set up a similar system), who did the data analysis (so you can share your concerns about the methodology), who made the figures (so you can raise concerns about digital fudging of the images), etc. Part of the reason people put their names on scientific papers is so we know who stands behind the research — who is willing to stake their reputation on it.

The other reason people put their names on scientific papers is to claim credit for their hard work and their insights, their contribution to the larger project of scientific knowledge-building. If you made a contribution, the scientific community ought to know about it so they can give you props (and funding, and tenure, and the occasional Nobel Prize).

But, we aren’t in a possition to make accurate assignments of credit or responsibility if we have no good information about what an author’s actual involvement in the project may have been. We don’t know who’s really in a position to vouch for the data, or who really did heavy intellectual lifting in bringing the project to fruition. We may understand, literally, the claim, “Joe Schmoe is second author of this paper,” but we don’t know what that means, exactly.

I should note that there is not one universally recognized authorship standard for all of the Tribe of Science. Rather, different scientific disciplines (and subdisciplines) have different practices as far as what kind of contribution is recognized as worthy of inclusion as an author on a paper, and as far as what the order in which the authors are listed is supposed to communicate about the magnitude of each contribution. In some fields, authors are always listed alphabetically, no matter what they contributed. In others, being first in the list means you made the biggest contribution, followed by the second author (who made the second-biggest contribution), and so forth. It is usually the case that the principal investigator (PI) is identified as the “corresponding author” (i.e., the person to whom questions about the work should be directed), and often (but not always) the PI takes the last slot in the author line. Sometimes this is an acknowledgement that while the PI is the brains of the lab’s scientific empire, particular underlings made more immediately important intellectual contributions to the particular piece of research the paper is communicating. But authorship practices can be surprisingly local. Not only do different fields do it differently, but different research groups in the same field — at the same university — do it differently. What this means is it’s not obvious at all, from the fact that your name appears as one of the authors of a paper, what your contribution to the project was.

There have been attempts to nail down explicit standards for what kinds of contributions should count for authorship, with the ICMJE definition of authorship being one widely cited effort in this direction. Not everyone in the Tribe of Science, or even in the subset of the tribe that publishes in biomedical journals, thinks this definition draws the lines in the right places, but the fact that journal editors grapple with formulating such standards suggests at least the perception that scientists need a clear way to figure out who is responsible for the scientific work in the literature. We can have a discussion about how to make that clearer, but we have to acknowledge that at the present moment, just noting that someone is an author without some definition of what that entails doesn’t do the job.

Here’s where the issue of “guest authorship” comes up. A “guest author” is someone whose name appears in a scientific paper’s author line even though she has not made a contribution that is enough (under whatever set of standards one recognizes for proper authorship) to qualify her as an author of the paper.

A guest is someone who is visiting. She doesn’t really live here, but stays because of the courtesy and forebearance of the host. She eats your food, sleeps under your roof, uses your hot water, watches your TV — in short, she avails herself of the amenities the host provides. She doesn’t pay the rent or the water bill, though; that would transform her from a guest to a tenant.

To my way of thinking, a guest author is someone who is “just visiting” the project being written up. Rather than doing the heavy lifting in that project, she is availing herself of the amenities offered by association (in print) with that project, and doing so because of the courtesy and forebearance of the “host” author.

The people who are actually a part of the project will generally be able to recognize the guest author as a “guest” (as opposed to an actual participant). The people receiving the manuscript will not. In other words, the main amenity the guest author partakes in is credit for the labors of the actual participants. Even if all the participants agreed to this (and didn’t feel the least bit put out at the free-rider whose “authorship” might be diluting his or her own share of credit), this makes it impossible for those outside the group to determine what the guest author’s actual contribution was (or, in this case, was not). Indeed, if people outside the arrangement could tell that the guest author was a free-rider, there wouldn’t be any point in guest authorship.

Science strives to be a fact-based enterprise. Truthful communication is essential, and the ability to connect bits of knowledge to the people who contributed is part of how the community does quality control on that knowledge base. Ambiguity about who made the knowledge may lead to ambiguity about what we know. Also, developing too casual a relationship with the truth seems like a dangerous habit for a scientist to get into.

Coming back to DrugMonkey’s question about whether courtesy authorship is a problem, it looks to me like maybe we can draw a line between two kinds of “guests,” one that contributes nothing at all to the actual design, execution, evaluation, or communication of the research, and one who contributes something here, just less than what the conventions require for proper authorship. If these characters were listed as authors on a paper, I’d be inclined to call the first one a “guest author” and the second a “courtesy author” in an attempt to keep them straight; the cases with which DrugMonkey seems most concerned are the “courtesy authors” in my taxonomy. In actual usage, however, the two labels seem to be more or less interchangeable. Naturally, this makes it harder to distinguish who actually did what — but it strikes me that this is just the kind of ambiguity people are counting on when they include a “guest author” or “courtesy author” in the first place.

What’s the harm?

Consider a case where the PI of a research group insists on giving authorship of a paper to a postdoc who hasn’t gotten his experimental system to work at all and is almost out of funding. The PI gives the justification that “He needs some first-author papers or his time here will have been a total waste.” As it happens, giving this postdoc authorship bumps the graduate student who did all the experimental work (and the conceptual work, and data analysis, and drafting of the manuscript) out of first author slot — maybe even off the paper entirely.

There is real harm here, to multiple parties. In this case, someone got robbed of appropriate credit, and the person identified as most responsible for the published work will be a not-very-useful person to contact with deeper questions about the work (since he didn’t do any of it or at best participated on the periphery of the project).

Consider another kind of case, where authorship is given to a well-known scientist with a lot of credibility in his field, but who didn’t make a significant intellectual contribution to work (at least, not one that rises to the level of meriting authorship under the recognized standards). This is the kind of courtesy authorship that was extended to Gerald Schatten in a 2005 paper in Science another of whose authors was Hwang Woo Suk. This paper had 25 authors listed, with Schatten identified as the senior author. Ultimately, the paper was revealed to be fraudulent, at which point Schatten claimed mostly to have participated in writing the paper in good English — a contribution recognized as less than what one would expect from an author (especially the senior author).

Here, including Schatten as an author seemed calculated to give the appearance (to the journal editors while considering the manuscript, and to the larger scientific community consuming the published work)that the work was more important and/or credible, because of the big name associated with it. But this would only work because listing that big name in the author line amounts to claiming the big name was actually involved in the work. When the paper fell apart, Schatten swiftly disavowed responsibility — but such a disavowal was only necessary because of what was communicated by the author line, and I think it’s naïve to imagine that this “ambiguity” or “miscommunication” was accidental.

In cases like this, I think it’s fair to say courtesy authorship does harm, undermining the baseline of trust in the scientific community. It’s hard to engage in efficient knowledge-building with people you think are trying to put one over on you.

The cases where DrugMonkey suggests courtesy authorship might be innocuous strike me as interestingly different. They are cases where someone has actually made a real contribution of some sort to the work, but where that contribution may be judged (under whatever you take to be the accepted standards of your scientific discipline) as not quite rising to the level of authorship. Here, courtesy authorship could be viewed as inflating the value of the actual contribution (by listing the person who made it in the author line, rather than the acknowledgements), or alternatively as challenging where the accepted standards of your discipline draw the line between a contribution that qualifies you as an author and one that does not. For example, DrugMonkey writes:

First, the exclusion of those who “merely” collect data is stupid to me. I’m not going to go into the chapter and verse but in my lab, anyway, there is a LOT of ongoing trouble shooting and refining of the methods in any study. It is very rare that I would have a paper’s worth of data generated by my techs or trainees and that they would have zero intellectual contribution. Given this, the asymmetry in the BMJ position is unfair. In essence it permits a lab head to be an author using data which s/he did not collect and maybe could not collect but excludes the technician who didn’t happen to contribute to the drafting of the manuscript. That doesn’t make sense to me. The paper wouldn’t have happened without both of the contributions.

I agree with DrugMonkey that there’s often a serious intellectual contribution involved in conducting the experiments, not just in designing them (and that without the data, all we have are interesting hunches, not actual scientific knowledge, to report). Existing authorship standards like those from ICMJE or BMJ can unfairly exclude those who do the experimental labor from authorship by failing to recognize this as an intellectual contribution. Pushing to have these real contributions recognized with appropriate career credit is important. As well, being explicit about who made these contributions to the research being reported in the paper makes it much easier for other scientists following up on the published work (e.g., comparing it to their own results in related experiments, or trying to use some of the techniques described in the paper to set up new experiments) to actually get in touch with the people most likely to be able to answer their questions.

Changing how might weight experimental prowess is given in the career scorekeeping may be an uphill battle, especially when the folks distributing the rewards for the top scores are administrators (focused on the money the people they’re scoring can bring to an institution) and PIs (who frequently have more working hours devoted to conception and design of project for their underlings rather than to the intellectual labor of making those projects work, and to writing the proposals that bring in the grant money and the manuscripts that report the happy conclusion of the projects funded by such grants). That doesn’t mean it’s not a fight worth having.

But, I worry that using courtesy authorship as a way around this unfair setting of the authorship bar actually amounts to avoiding the fight rather than addressing these issues and changing accepted practices.

DrugMonkey also writes:

Assuming that we are not talking about pushing someone else meaningfully* out of deserved credit, where lies the harm even if it is a total gift?

Who is hurt? How are they damaged?
__
*by pushing them off the paper entirely or out of first-author or last-author position. Adding a 7th in the middle of the authorship list doesn’t affect jack squat folks.

Here, I wonder: if dropping in a courtesy author as the seventh author of a paper can’t hurt, how either can we expect it to help the person to whom this “courtesy” is extended?

Is it the case that no one actually expects that the seventh author made anything like a significant contribution, so no one is being misled in judging the guest in the number seven slot as having made a comparable contribution to the scientist who earned her seventh-author position in another paper? If listing your seventh-author paper on your CV is automatically viewed as not contributing any points in your career scorekeeping, why even list it? And why doesn’t it count for anything? Is it because the seventh author never makes a contribution worth career points … or is it because, for all we know, the seventh author may be a courtesy author, there for other reasons entirely?

If a seventh-author paper is actually meaningless for career credit, wouldn’t it be more help to the person to whom you might extend such a “courtesy” if you actually engaged her in the project in such a way that she could make an intellectual contribution recognized as worthy of career credit?

In other words, maybe the real problem with such courtesy authorship is that it gives the appearance of help without actually being helpful.

Methodology versus beliefs: What did Marcus Ross do wrong?

Posted on by

We’ve been discussing whether good science has more to do with the methodology you use or with what you believe, and considering the particular case of Ph.D. geoscientist and young earth creationist Marcus Ross (here and here). At least some of the responses to these two posts seem to offer the view that: (1) of course what makes for a reliable piece of scientific knowledge is the methodology used to produce it (and especially to check it for error), but (2) the very fact that Marcus Ross is committed to young earth creationism, which means among other things that he is committed to the belief that the earth is not more than 10,000 years old, is a fatal blow to his scientific credibility as a geoscientist.

Either this boils down to claiming that having young earth creationist beliefs makes it impossible to use scientific methodology and generate a reliable piece of knowledge (even though Ross seems to have done just that in writing his dissertation), or perhaps to claiming instead that a person who holds young earth creationist beliefs and also uses standard scientific methodology to generate bits of scientific knowledge must have some ulterior motive for generating them. In this latter case, I take it the worry is not with the respectability of the product (i.e., the scientific knowledge claims), nor of the process (i.e., the standard sorts of evidence or inferential machinery being used to support the scientific knowledge claims), but rather of the producer (i.e., the person going through all the scientific motions yet still believing in young earth creationism).

I think it’s worth examining the general unease and trying to be more precise about what people think Marcus Ross might be doing wrong here. However, let the record reflect that I have not been surveilling Marcus Ross — not sitting in on the classes he teaches, not tracking down and reading his scientific publications, not following him to geological meetings or church or the supermarket. What this means is that we’re going to be examining hypotheticals here, rather than scads of empirical facts about what Marcus Ross actually does.

Possibility 1: Ross is using his geoscience Ph.D. to gain unwarranted increase in credibility for young earth creationist beliefs.

Ross teaches geology at Liberty University. Part of this teaching seems to involve setting out the kinds of theories, evidence, and inferential machinery (including accepted dating methods and the evidential support for them) that you’d expect students to learn in a geology class in a secular university. Part of it also seems to involve laying out the details of young earth creationism (which is not accepted as scientific by the scientists who make up the field of geoscience), the claims it supports, and on what evidential basis. Obviously, the claims of young earth creationism are bolstered by quite different evidence and a quite distinct (religious) inferential structure.

One approach to this pedagogy would be to bring out the important differences, both in the conclusions of geology and of young earth creationism and in the recognized rules for drawing, testing, and supporting conclusions between the two. Indeed, Ross’s comments make it sound like this is the approach he takes:

In my classes here at Liberty University I introduce my students to the reasons why geologists think the Earth is ancient, or why various organisms are viewed as strong evidence for evolution.  I do this so that they understand that these arguments are well thought-out, and to teach them to respect the ideas of those with whom they disagree.

If Ross is actually making it clear how scientific inference differs from faith-based claims, then is should be clear to any of his students who are paying attention that the science Ross studied in graduate school does not support his young earth creationism. Rather, the science supports the scientific inference. His faith supports young earth creationism. The two are different.

If, on the other hand, Ross were to mischaracterize the theories, evidence, and inferential machinery of geoscience in his classes, that would be bad. It would amount to lying about the nature of geoscience (and perhaps also of science more broadly).

In the same way, if Ross were to claim that the body of geological knowledge, or the methods of geoscience, or the empirical evidence recognized by geoscientists lent scientific support to the claims of young earth creationism, that would also be lying.

Ross (and his students) might still accept young earth creationism, but they would be doing so on religious rather than scientific grounds — something that a careful study of geoscience and its methods should make clear. If anything, such a study should underline that the rules for the scientific credibility of a claim are orthogonal to the rules for the religious credibility of a claim.

Possibility 2: Ross doesn’t intend to use his geoscience Ph.D. to gain unwarranted increase in credibility for young earth creationist beliefs, but it has that effect on his audience anyway.

You might worry that Marcus Ross’s status as a Ph.D. geoscientist lends extra credibility to all the beliefs he voices — at least when those beliefs are judged by an audience of undergraduates who are enamored by Ph.D.s. That’s a hard degree to get, after all, and you have to be really smart to get one, right? And, smart people (especially those certified to be Ph.D.-smart by Ph.D. granting institutions) have more credible beliefs than everyone else, right?

If Ross’s students are making this sort of judgment about his credibility — and they might well be — it’s a silly judgment to make. It would be akin to assuming that my Ph.D. in chemistry would make me a more credible commentator on the theories of Descartes or Husserl. Let me assure you, it does not! (That’s why I spent six additional years of my life in graduate school developing the expertise relevant for work in philosophy.)

Indeed, the kind of extra credibility young earth creationism might gain in the minds of undergraduates by this route speaks more to a lack of critical thinking on the part of the undergraduates than it does to any dishonesty on Ross’s part. It also makes me yearn for the days of robust teen rebellion and reflexive mistrust of anyone over 30.

We should be fair, though, and recognize that it’s not just college students who can be dazzled by an advanced degree. Plenty of grown-ups in the larger society have the same reaction. Uncritically accepting the authority of the Ph.D. to speak on matters beyond the tether of his expertise is asking to be sold snake oil.

In light of the increased authority non-scientists seem to grant those with scientific training even outside the areas of their scientific expertise, it might be reasonable to ask scientists to be explicit about when they are speaking as scientists and when they are speaking as people with no special authority (or, perhaps, with authority that has some source other than scientific training). But, if we think Marcus Ross has an obligation to note that his scientific training does not support his views in the realm of young earth creationism, we probably ought to hold other scientists to the same obligation when they speak of matters beyond their scientific expertise. Fair is fair.

Possibility 3: Ross is using his engagement with the community of geoscientists to make it appear to outsiders as though his young earth creationist views are scientifically respectable, even though he knows they aren’t.

This is a possibility raised by Donald Prothero’s account of “stealth creationism” at meetings of the Geological Society of America (GSA). Prothero writes:

Most of the time when I attend the meetings, there are plenty of controversial topics and great debates going on within the geological community, so the profession does not suppress unorthodox opinions or play political games. This is the way it should be in any genuine scientific discipline. I’ve seen amazingly confrontational knock-down-drag-out sessions about particularly hotly debated ideas, but always conducted in a spirit of honest scientific exchange and always hewing to rules of science and naturalism. To get on the meeting program, scientists must propose to organize sessions around particular themes, along with field trips to geologically interesting sites within driving distance of the convention city, and the GSA host committee reads and approves these proposals. But every once in a while, I see a poster title and abstract with something suspicious about it. When I check the authors, they turn out to be Young-Earth Creationists (YEC) who claim the earth is only 6000 years old and all of geology can be explained by Noah’s flood. When I visit the poster session, it’s usually mobbed by real geologists giving the YECs a real grilling, even though the poster is ostensibly about some reasonable geologic topic, like polystrate trees in Yellowstone, and there is no overt mention of Noah’s flood in the poster. But the 2010 meeting last year in Denver took the cake: there was a whole field trip run by YECs who did not identify their agenda, and pretended that they were doing conventional geology—until you read between the lines.

Marcus Ross was one of the leaders of the field trip in question, as was Steve Austin of the Institute for Creation Research. Prothero quotes his colleague Steve Newton’s account of this GSA meeting field trip:

Through the entire trip, the leaders never identified themselves as YECs or openly advocated Noah’s flood or a 6000-year-old earth. Instead, the entire trip was filled with stops at outcrops where the leaders emphasized the possible evidence for sudden deposition of the strata at Garden of the Gods near Colorado Springs, without stating explicitly that they believed this sudden deposition was Noah’s flood in action. (There are LOTS of instances of local rapid and sudden deposition of strata in real geology, but they are local and clearly cannot be linked to any global flood). As Newton described it:

Furthermore, the field trip leaders were careful not to make overt creationist references. If the 50 or so field trip participants did not know the subtext and weren’t familiar with the field trip leaders, it’s quite possible that they never realized that the leaders endorsed geologic interpretations completely at odds with the scientific community. Even the GSA Sedimentary Geology Division had initially signed on as a sponsor of the trip (though they backed out once they learned the views of the trip leaders).

But the leaders’ Young-Earth Creationist views were apparent in rhetorical subtleties. For example, when Austin referred to Cambrian outcrops, he described them as rocks that are “called Cambrian.” It’s an odd phrasing, allowing use of the proper geologic term while subtly denying its implications. In one instance, when Austin was asked by a trip attendee about the age of a rock unit, he responded somewhat cryptically, “Wherever you want to go there.” Such phrasing was telling, if you knew what to listen for.

Subtext about the age of formations was a big part of the Young-Earth Creationist rhetoric on the trip. As we moved on to each field trip stop, a narrative began to emerge: the creationist concept of Noah’s Flood as explanation for the outcrops. Although no one uttered the words “Noachian Flood,” the guides’ descriptions of the geology were revealing and rather coy. For example, at the first stop—a trail off Highway 24 near Manitou Spring—Austin stated that the configuration of the units was “the same over North America,” and had been formed by a massive marine transgression. “Whatever submerged the continent,” Austin went on, it must have been huge in scale.

Here, a charitable reading of the field trip might be that the believers in geology were taking in the sights and interpreting the evidence with the (scientific) inferential machinery of geology, while the young earth creationists were taking in the very same sights and interpreting the evidence with the (religious) inferential machinery of young earth creationism. But, Prothero argues that there’s more than this going on:

Sadly, the real problem here is that YEC “geologists” come back from this meeting falsely bragging that their “research” was enthusiastically received, and that they “converted” a lot of people to their unscientific views. As Newton pointed out, they will crow in their publicity that they are attending regular professional meetings and presenting their research successfully. For those who don’t know any better, it sounds to the YEC audience like they are conventional geologists doing real research and that they deserve to be taken seriously as geologists—even though every aspect of their geology is patently false (see Chapter 3 in my 2007 Evolution book). And so, once more the dishonesty of the YEC takes advantage of the openness and freedom of the scientific community to exploit it to their own ends, and abuse the privilege of open communication to push anti-scientific nonsense on the general population that doesn’t know the difference.

Prothero notes (as does Marcus Ross in his comments on this blog) that the research by young earth creationists that is well received by the geological community is completely conventional, using only the inferential machinery of geoscience and making no use of the assumptions of young earth creationism. But presenting work (or leading a field trip) with a young earth creationist subtext (i.e., possibly these observations can be interpreted as evidence of a really big flood of some kind …) to an audience of geologists, and then spinning a lack of loud objections to a conclusion you didn’t explicitly present as if it were endorsement of that conclusion by the geologists is a dishonest move.

Honest engagement with a scientific community means putting your evidential and methodological cards on the table. It means, if you want to know whether other scientists would endorse (or even accept as not-totally-implausible) a particular conclusion, you put that particular conclusion out there for their examination. All you can reasonably conclude from the fact that other scientists didn’t shoot down a conclusion that you never openly stated is that those other scientists did not read your mind.

Possibility 4: It’s wrong for Ross to maintain his young earth creationist beliefs after the thorough exposure to scientific theories, evidence, and methodology that he received in his graduate training in geosciences.

Learning to be a scientist means, among other things, learning scientific patterns of thought, scientific standards for evaluating knowledge claims, and scientific methods for generating and testing new knowledge claims. Such immersion in the tribe of science and in the activity of scientific research, some might argue, should have driven the young earth creationist beliefs right out of Marcus Ross’s head.

Maybe we could reasonably expect this outcome if his young earth creationist beliefs depended on the same kind of evidence and inferential machinery as do scientific claims. However, they do not. Young earth creationist claims are not scientific claims, but faith-based claims. Young earth creationism sets itself apart from the inferential structure of science — if its adherents are persuaded that a claim is credible on the basis of faith (e.g., in a particular reading of scriptures), then no arrangement of empirical evidence could be sufficient to reliably undermine that adherence.

To be sure, this means that a scientist like Marcus Ross who is also a young earth creationist has non-scientific beliefs in his head. But, if we’re going to assert that scientific training ought, when done right, to purge the trainee of all non-scientific beliefs, then there is precious little evidence that scientific training is being done right anywhere.

There are quite a lot of scientists with non-scientific beliefs that persist. They have beliefs about who would be the best candidate to vote for in a presidential election, about what movie will be most entertaining, about what entree at the restaurant will be most delicious and nutritious. They have beliefs about whether the people they care for also care for them, and about whether their years of toil on particular research questions will make the world a better place (or, more modestly, whether they will have been personally fulfilling). Many of these beliefs are hunches, no better supported by the available empirical evidence than are the beliefs routinely formed by non-scientists.

This is not to say that the evidence necessarily argues against holding these beliefs. Rather, the available evidence may be so sparse as to be inadequate to support or undermine the belief. Still, scientific training does not prevent the person so trained from forming beliefs in these instances — and this may be useful, especially since there are situations where sitting on the fence waiting for decisive evidence is not the best call. (Surely we have more complete evidence about what kind of president Richard M. Nixon would make now than was available in November 1968, but it’s too late for us to use that evidence to vote in the 1968 presidential election.)

If harboring non-scientfic beliefs is a crime, we’d be hard pressed to find a single member of the tribe of science who is not at least a little guilty.

Maybe it’s more reasonable to hold scientists accountable to recognize which of their beliefs are well supported by empirical evidence and which are not. A bit of reflection is probably sufficient to help scientists sort out the scientific beliefs from the non-scientific beliefs. And, to the extent that Marcus Ross wants to be a practicing member of the tribe of science (or even an intellectually honest outsider with enough scientific training that he ought to be able to tell the difference), it’s just as reasonable to hold him accountable for recognizing which sort of beliefs constitute his young earth creationism.

Being able to tell the difference between scientific and non-scientific beliefs is not only a more attainable goal for human scientists than having only scientific beliefs, but it is a much easier standard for the tribe of science to police, since it involves examining what kinds of claims a person asserts as backed by the science — something other scientists can check by examining evidence and arguments — rather than examining what’s in a person’s head.

These possibilities strike me as the most likely candidates for what’s bugging science-minded people about Marcus Ross. If I’ve missed what’s bugging you about him, please make your case in the comments.

Methodology versus beliefs: a comment from Marcus Ross.

Posted on by

Last week, we considered whether good science has more to do with what you do or with what you believe, exploring this issue using the case of Marcus Ross, a Ph.D. geoscientist and young earth creationist. Dr. Ross sent me a response to this post via email. With his permission, I’m sharing that email here:

* * * * *
Hello Janet,
 
Thank you for your thoughtful piece yesterday in Scientific American.  It has been quite a while since the New York Times piece in 2007, so I was surprised to it revisited.  And I found your analysis of the events of my Ph.D. work far more considerate than many of the earlier reactions.  It’s nice not to be referred to as a trained parrot, a textbook case of cognitive dissonance, or a variety of unprintable words!
 
This paragraph from your piece sums things up quite nicely:

“It looks like Ross saw his dissertation as an exercise in presenting the inferences one could draw from the available data using the recognized methods of geoscience. In other words, here’s what we would conclude if all the assumptions about the age of the earth, deposition of fossils, isotope dating methods, etc., were true…”

 
This is a good sketch of what I did, not only for the Ph.D., but for all of my geological education (which was conducted entirely at non-creationist, state schools; and like at URI, at each location I made it known to my advisors that I was a young-Earth creationist).  I always felt that, since I was attempting to earn a degree from an institution which adhered to an ancient Earth and evolutionary explanations of life’s diversity, that I must show myself proficient in these areas. 
 
One clarification which stems from Cornelia Dean’s original article: I never referred to a “paleontological paradigm”.  That term is one she invented from her interview of me, but one I never introduced.  Indeed, the term actually makes very little sense (does anyone speak of a microbiology paradigm?).  In speaking with my students, I refer to the old-Earth and evolutionary paradigms, and I make sure to distinguish the two as well.
 
One issue that you bring up is whether I’ve essentially given up on interaction with the geological community, especially given my position at Liberty University.  Let me assure you that such is not the case.  In both print and in annual meetings, I do what I can to contribute to, and interact with, current geological discussions.  My publication record is not extensive, but it includes papers in a handful of conventional geological journals, including recent geological papers in 2009 and 2010 and co-leading a field trip at the annual meeting of the Geological Society of America (our largest professional association) last year with four other creation geologists.  Even Steven Newton of the NCSE has written, more or less, charitably of my, and my creationist colleagues’, continuing interactions at society meetings over the past few years.
 
Nevertheless, despite my best attempts, and because of some of my old-Earth and evolutionary colleagues’ attitudes towards me, the road of interaction has been bumpy.  I have had chapters of my (decidedly conventional) dissertation rejected from journals and special publications for no other reason than the fact that I am a creationist, sometimes in very explicit terms.  Presentations at society meetings are viewed with deep suspicion that I will make creationist arguments (or even preach!) once given the lectern.  I have, on two occasions, been “outed” as a creationist following my own presentation by scientists who wished to score points with their students and peers, and do damage to my reputation.  But having been open about being a creationist my whole career usually blunts such shoddy attempts at a “gotcha” moment.  The job description for my employment was gleefully mocked at a society presentation while I was in attendance.  And this is from the more legitimate forms of scientific dialogue.  Googling my name gets really ugly, really fast.
 
But such is no major deterrent to me, though it does impede my attempts to publish in conventional literature, for example.  I value the contributions of my colleagues, and have enjoyed many constructive interactions, despite the occasional run-in with less pleasant sorts.  In my classes here at Liberty University I introduce my students to the reasons why geologist think the Earth is ancient, or why various organisms are viewed as strong evidence for evolution.  I do this so that they understand that these arguments are well thought-out, and to teach them to respect the ideas of those with whom they disagree.  And I was grateful for your blog post because, unlike many others, you respect my position enough to treat it with courtesy.  Thank you.
 
Blessings,
Marcus
 
Marcus R. Ross, Ph.D.  

Associate Professor of Geology

Dept. of Biology and Chemistry
Liberty University

Is being a good scientist a matter of what you do or of what you feel in your heart?

Posted on by

If the question posed in the title of the post seems to you to have an obvious answer, sit tight while I offer a situation in which it might be less obvious.

We recently discussed philosopher Karl Popper’s efforts to find the line of demarcation between science and pseudo-science. In that discussion, one of the things you may have noticed is that Popper’s story is as much about a distinctive scientific attitude as it is about the details of scientific methodology. I wrote:

Popper has this picture of the scientific attitude that involves taking risks: making bold claims, then gathering all the evidence you can think of that might knock them down. If they stand up to your attempts to falsify them, the claims are still in play. But, you keep that hard-headed attitude and keep you eyes open for further evidence that could falsify the claims. If you decide not to watch for such evidence — deciding, in effect, that because the claim hasn’t been falsified in however many attempts you’ve made to falsify it, it must be true — you’ve crossed the line to pseudo-science.

And, my sense from scientists is that Popper’s description of their characteristic attitude is what they like best about his account. Hardly any scientist goes into the lab Monday morning with the firm intention of trying (yet again) to falsify the central hypotheses which she and the other scientists in her field have been using successfully (to predict and to explain and to create new phenomena) for years. Hardly any scientist will toss out hypotheses on the basis of a single experimental result that does not match the predictions of the hypotheses. But scientists agree that when they’re following the better angels of their scientific nature, their eyes are open to evidence that might conflict with even their most trusted hypotheses, and they are ready to kiss those hypotheses goodbye if the facts in the world line up against them.

An attitude is something that’s in your heart.

Certainly, an attitude may exert a strong influence on what you do, but if having the right attitude is something that matters to us over and above doing the right thing, we can ask why that is. My best hunch is that an attitude may act as a robust driver of behavior — in other words, having the right attitude may be a reliable mechanism that gets you to do the right thing, at least more than you might in the absence of that attitude.

So, what should we say about a scientist who appears to practice the methodology as he should, but who reveals himself as having something else in his heart?

This question came up back in 2007, when the New York Times reported on the curious case of Marcus R. Ross. Ross had written and defended an “impeccable” dissertation on the abundance and spread of marine reptiles called mosasaurs which (as his dissertation noted) vanished about 65 million years ago, earning a Ph.D. in geosciences from the University of Rhode Island. Then, he accepted a faculty position at Liberty University, where he is an Assistant Director of the Center for Creation Studies.

Ross is a young earth creationist, and as such believes that the earth is no older than 10,000 years. He was a young earth creationist when he wrote the impeccable dissertation in which he noted the disappearance of mosasaurs about 65 millions years ago. Indeed, he was a young earth creationist when he applied to the geosciences Ph.D. program at the University of Rhode Island, and did not conceal this information from the admissions committee.

Some details from the New York Times article:

For him, Dr. Ross said, the methods and theories of paleontology are one “paradigm” for studying the past, and Scripture is another. In the paleontological paradigm, he said, the dates in his dissertation are entirely appropriate. The fact that as a young earth creationist he has a different view just means, he said, “that I am separating the different paradigms.”

He likened his situation to that of a socialist studying economics in a department with a supply-side bent. “People hold all sorts of opinions different from the department in which they graduate,” he said. “What’s that to anybody else?” …

In theory, scientists look to nature for answers to questions about nature, and test those answers with experiment and observation. For Biblical literalists, Scripture is the final authority. As a creationist raised in an evangelical household and a paleontologist who said he was “just captivated” as a child by dinosaurs and fossils, Dr. Ross embodies conflicts between these two approaches. The conflicts arise often these days, particularly as people debate the teaching of evolution. …

In a telephone interview, Dr. Ross said his goal in studying at secular institutions “was to acquire the training that would make me a good paleontologist, regardless of which paradigm I was using.” …

He would not say whether he shared the view of some young earth creationists that flaws in paleontological dating techniques erroneously suggest that the fossils are far older than they really are.

Asked whether it was intellectually honest to write a dissertation so at odds with his religious views, he said: “I was working within a particular paradigm of earth history. I accepted that philosophy of science for the purpose of working with the people” at Rhode Island.

And though his dissertation repeatedly described events as occurring tens of millions of years ago, Dr. Ross added, “I did not imply or deny any endorsement of the dates.”

Ross pursued an education that gave him detailed knowledge of the theories the geoscience community uses, the questions geoscientists take to be interesting ones to pursue, the methods they use to make observations, to analyze data, and to draw inferences. He showed sufficient mastery of the “paleontological paradigm” that he was able to use it to build an additional piece of knowledge (the work contained in his dissertation) that was judged a contribution to his scientific community.

But, if he believed in his heart that the earth was thousands, not millions, of years old as he built this piece of knowledge, was he really a part of that scientific community? Was he essentially lying in his interactions with its members?

It looks like Ross saw his dissertation as an exercise in presenting the inferences one could draw from the available data using the recognized methods of geoscience. In other words, here’s what we would conclude if all the assumptions about the age of the earth, deposition of fossils, isotope dating methods, etc., were true. His caginess about the dates in the interview quoted above, and his professed belief in young earth creationism, suggest that Ross thinks at least some of these scientific assumptions are false.

However, assuming his rejection of the scientific assumptions flows primarily from his commitments as a young earth creationist, the rejection of the claims other geoscientists agree on is based in religious reasons, not scientific reasons. If there were scientific reasons to doubt these assumptions, it seems like examining those could only lead to a stronger body of knowledge in geosciences, and that Ross could have contributed to the field by making such an examination the focus of his doctoral research.

Is it an obligation for a scientist who has concerns about the goodness of an assumption on which people in his field rest their inferences to voice that concern? Is it an obligation for that scientist to gather data to test that hypothesis, or to work out an alternative hypothesis that is better supported by the data? Or is it OK to keep your doubts to your self and just use the inferential machinery everyone else is using?

Maybe people will answer this differently if the scientist in question is planning an ongoing engagement with the other members of this field, or if he is just passing through on the way to somewhere else. More on this in just a moment.

Here’s a shorter version of my question about the scientist’s obligations here: Does intellectual honesty in scientific knowledge-building just cover the way you use the inferential structure and the inputs (i.e., data) from which you draw your inferences? Or does it require disclosure of which assumptions you really accept (not just for the sake of argument, but in your heart of hearts) when drawing your inferences and which you are inclined to think are mistaken?

Does intellectual honesty require that you disclose as well the fact that you don’t actually accept the inferential structure of science as a good way to build knowledge?

Because ultimately, a commitment to young earth creationism seems to be a commitment that the data cannot properly be used to infer any claims that are at odds with scripture.

And here’s where scientists who might be willing to accept Ross’s dissertation as a legitimate chunk of scientific knowledge may have serious concerns with Ross as a credible member of the scientific community. The dissertation may stand (or fall) as a scientific argument that presents a particular array of data, describes accepted inferential strategies (perhaps even defending some such strategies that are themselves new contributions), and uses these strategies to draw conclusions form the data. Even if the person who assembled this argument was wracked with doubts about all the central premises of the argument, the argument itself could still function perfectly well in the ongoing scientific discourse, and other scientists in the community could judge that argument on its strengths and weaknesses — not on what might be in the heart of the person who constructed the argument.

But, if, ultimately, Marcus Ross rejects the “paleontological paradigm” — and the possibility that the data could properly support inferences at odds with scripture — can he function as a member of a community that makes, and evaluates, inferences using this paradigm?

Maybe he could, but his career trajectory makes it look like he has chosen not to be a member of the larger community of geoscientists. Instead, he has positioned himself as a member of a community of “creation scientists”. Whether Ross’s ongoing work on extinct marine reptiles is of any scientific interest to the scientific field that trained him will probably depend on the methodology and inferential structure on display in his manuscripts.

Because methodology and inferential structure are much easier to evaluate in the peer review process than what is in the author’s heart.

* * * * *

If you enjoyed this post, consider contributing a few bucks to a project in my Giving Page in the Science Bloggers for Students 2011 challenge. Supporting science education in public school classrooms will help young people get a better handle on what kind of attitude and methodology makes science science — and on all the cool things science can show us about our world.

What a scientist knows about science (or, the limits of expertise).

Posted on by

In a world where scientific knowledge might be useful in guiding decisions we make individually and collectively, one reason non-scientists might want to listen to scientists is that scientists are presumed to have the expertise to sort reliable knowledge claims from snake oil. If you’re not in the position to make your own scientific knowledge, your best bet might be to have a scientific knowledge builder tell you what counts as good science.

But, can members of the public depend on any scientist off the street (or out of the lab) to vet all the putative scientific claims for credibility?

Here, we have to grapple with the relationship between Science and particular scientific disciplines — and especially with the question of whether there is enough of a common core between different areas of science that scientists trained in one area can be trusted to recognize the strengths and weaknesses of work in another scientific area. How important is all that specialization research scientists do? Can we trust that, to some extent, all science follows the same rules, thus equipping any scientist to weigh in intelligently about any given piece of it?

It’s hard to give you a general answer to that question. Instead, as a starting point for discussion, let me lay out the competence I personally am comfortable claiming, in my capacity as a trained scientist.

As someone trained in a science, I am qualified:

  1. to say an awful lot about the research projects I have completed (although perhaps a bit less about them when they were still underway).
  2. to say something about the more or less settled knowledge, and about the live debates, in my research area (assuming, of course, that I have kept up with the literature and professional meetings where discussions of research in this area take place).
  3. to say something about the more or less settled (as opposed to “frontier”) knowledge for my field more generally (again, assuming I have kept up with the literature and the meetings).
  4. perhaps, to weigh in on frontier knowledge in research areas other than my own, if I have been very diligent about keeping up with the literature and the meetings and about communicating with colleagues working in these areas.
  5. to evaluate scientific arguments in areas of science other than my own for logical structure and persuasiveness (though I must be careful to acknowledge that there may be premises of these arguments — pieces of theory or factual claims from observations or experiments that I’m not familiar with — that I’m not qualified to evaluate).
  6. to recognize, and be wary of, logical fallacies and other less obvious pseudo-scientific moves (e.g., I should call shenanigans on claims that weaknesses in theory T1 count as support for alternative theory T2).
  7. to recognize that experts in fields of science other than my own generally know what the heck they’re talking about.
  8. to trust scientists in fields other than my own to rein in scientists in those fields who don’t know what they are talking about.
  9. to face up to the reality that, as much as I may know about the little piece of the universe I’ve been studying, I don’t know everything (which is part of why it takes a really big community to do science).

This list of my qualifications is an expression of my comfort level more than anything else. It’s not elitist — good training and hard work can make a scientist out of almost anyone. But, it recognizes that with as much as there is to know, you can’t be an expert on everything. Knowing how far the tether of your expertise extends is part of being a responsible scientist.

So, what kind of help can a scientist give the public in evaluating what is presented as scientific knowledge? What kind of trouble can a scientist encounter in trying to sort out the good from the bad science for the public? Does the help scientists offer here always help?

Trust me, I’m a scientist.

Posted on by

In an earlier post, I described an ideal of the tribe of science that the focus of scientific discourse should be squarely on the content — the hypotheses scientists are working with, the empirical data they have amassed, the experimental strategies they have developed for getting more information about our world — rather than on the particular details of the people involved in this discourse. This ideal is what sociologist of science Robert K. Merton* described as the “norm of universalism”.

Ideals, being ideals, can be hard to live up to. Anonymous peer review of scientific journal articles notwithstanding, there are conversations in the tribe of science where it seems to matter a lot who is talking, not just what she’s saying about the science. Some scientists were trained by pioneers in their fields, or hired to work in prestigious and well-funded university departments. Some have published surprising results that have set in motion major changes in the scientific understanding of a particular phenomenon, or have won Nobel Prizes.

The rest can feel like anonymous members in a sea of scientists, doing the day to day labor of advancing our knowledge without benefit of any star power within the community. Indeed, probably lots of scientists prefer the task of making the knowledge, having no special need to have their names widely known within their fields and piled with accolades.

But there’s a peculiar consequence of the idea that scientists are all in the knowledge-buiding trenches together, focused on the common task rather than on self-agrandizement. When scientists are happily ensconced in the tribe of science, very few of them take themselves to be stars. But when the larger society, made up mostly of non-scientists, encounters a scientist — any scientist — that larger society might take him to be a star.

Merton touched on this issue when he described another norm of the tribe of science, disinterestedness. One way to think about the norm of disinterestedness is that scientists aren’t doing science primarily to get the big bucks, or fame, or attractive dates. Merton’s description of this community value is a bit more subtle. He notes that disinterestedness is different from altruism, and that scientists needn’t be saints.

The best way to understand disinterestedness might be to think of how a scientist working within her tribe is different from an expert out in the world dealing with laypeople. The expert, knowing more than the layperson, could exploit the layperson’s ignorance or his tendency to trust the judgment of the expert. The expert, in other words, could put one over on the layperson for her own benefit. This is how snake oil gets sold.

The scientist working within the tribe of science can expect no such advantage. Thus, trying to put one over on other scientists is a strategy that shouldn’t get you far. By necessity, the knowledge claims you advance are going to be useful primarily in terms of what they add to the shared body of scientific knowledge, if only because your being accountable to the other scientists in the tribe means that there is no value added to the claims from using them to play your scientific peers for chumps.

Merton described situations in which the bona fides of the tribe of science were used in the service of non-scientific ends:

Science realizes its claims. However, its authority can be and is appropriated for interested purposes, precisely because the laity is often in no position to distinguish spurious from genuine claims to such authority. The presumably scientific pronouncements of totalitarian spokesmen on race or economy or history are for the uninstructed laity of the same order as newspaper reports of an expanding universe or wave mechanics. In both instances, they cannot be checked by the man-in-the-street and in both instances, they may run counter to common sense. If anything, the myths will seem more plausible and are certainly more comprehensible to the general public than accredited scientific theories, since they are closer to common-sense experience and to cultural bias. Partly as a result of scientific achievements, therefore, the population at large becomes susceptible to new mysticisms expressed in apparently scientific terms. The borrowed prestige of science bestows prestige on the unscientific doctrine. (p. 277))

(Bold emphasis added)

The success of science — the concentrated expertise of the tribe — means that those outside of it may take “scientific” claims at face value. Unable to make an independent evaluation of their credibility, lay people can easily fall prey to a wolf in scientist’s clothing, to a huckster assumed to be committed first and foremost to the facts (as scientists try to be) who is actually distorting them to look after his own ends.

This presents a serious challenge for non-scientists — and for scientists, too.

If the non-scientist can’t determine whether a purportedly scientific claim is a good one — whether, for example, it is supported by the empirical evidence — the non-scientist has to choose between accepting that claim on the authority of someone who claims to be a scientist (which in itself raises another evaluative problem for the non-scientist — what kind of credentials do you need to see from the guy wearing the lab coat to believe that he’s a proper scientist?), or setting aside all putative scientific claims and remaining agnostic about them. You trust that the “Science” label on a claim tells you something about its quality, or you recognize that it conveys even less useful information to you than a label that says, “Now with Jojoba!”

If late-night infomercials and commercial websites are any indication, there are not strong labeling laws covering what can be labeled as “Science”, at least in a sales pitch aimed at the public at large.** This leaves open the possibility that the claims made by the guy in the white lab coat that he’s saying are backed by Science would not be recognized by other scientists as backed by science.

The problem this presents for scientists is two-fold.

On the one hand, scientists are trying to get along in a larger society where some of what they discover in their day jobs (building knowledge) could end up being relevant to how that larger society makes decisions. If we want our governments to set sensible policy as far as tackling disease outbreaks, or building infrastructure that won’t crumble in floods, or ensuring that natural resources are utilized sustainably, it would be good for that policy to be informed by the best relevant knowledge we have on the subject. Policy makers, in other words, want to be able to rely on science — something that scientists want, too (since usually they are working as hard as they are to build the knowledge so that the knowledge can be put to good use). But that can be hard to do if some members of the tribe of science go rogue, trading on their scientific credibility to sell something as science that is not.

Even if policy makers have some reasonable way to tell the people slapping the Science label on claims that aren’t scientific, there will be problems in a democratic society where the public at large can’t reliably tell scientists from purveyors of snake-oil.

In such situations, the public at large may worry that anyone with scientific credentials could be playing them for suckers. Scientists who they don’t already know by reputation may be presumed to be looking out for their own interests rather than to be advancing scientific knowledge.

A public distrustful of scientists’ good intentions or trustworthiness in interactions with non-scientists will convey that distrust to the people making policy for them.

This means that scientists have a strong interest in identifying the members of the tribe of science who go rogue and try to abuse the public’s trust. People presenting themselves as scientists while selling unscientific claims are diluting the brand of Science. They undermine the reputation science has for building reliable knowledge. They undercut the claim other scientists make that, in their capacity as scientists, they hold themselves accountable to the way the world really is — to the facts, no matter how inconvenient they may be.

Indeed, if the tribe of science can’t make the case that it is serious about the task of building reliable knowledge about the world and using that knowledge to achieve good things for the public, the larger public may decide that putting up public monies to support scientific research is a bad idea. This, in turn, could lead to a world where most of the scientific knowledge is built with private money, by private industry — in which case, we might have to get most of our scientific knowledge from companies that actually are trying to sell us something.

_____
*Robert K. Merton, “The Normative Structure of Science,” in The Sociology of Science: Theoretical and Empirical Investigations. University of Chicago Press (1979), 267-278.

**There are, however, rules that require the sellers of certain kinds of products to state clearly when they are making claims that have not been evaluated by the Food and Drug administration.

Scientific credibility: is it who you are, or how you do it?

Posted on by

Part of the appeal of science is that it’s a methodical quest for a reliable picture of how our world works. Creativity and insight is crucial at various junctures in this quest, but careful work and clear reasoning does much of the heavy lifting. Among other things, this means that the grade-schooler’s ambition to be a scientist someday is significantly more attainable than the ambition to be a Grammy-winning recording artist, a pro-athlete, an astronaut, or the President of the United States.

Scientific methodology, rather than being a closely guarded trade secret, is a freely available resource.

Because of this, there is a sense that it doesn’t matter too much who is using that scientific methodology. Rather, what matters is what scientists discover by way of the methodology.
Continue reading→

What about Dalibor Sames? The Bengü Sezen fraud and the responsibilities of the PI in the training of new scientists.

Posted on by

Unless you are a chemist or a habitual follower of scientific misconduct stories, it’s possible that you missed the saga of Bengü Sezen.

From 2000 to 2005, Sezen was a graduate student in chemistry at Columbia University, working in the laboratory of then-Assistant Professor Dalibor Sames. She appeared to be a talented scientist in training, and during her graduate studies was lead author on three papers published in the Journal of the American Chemical Society. Columbia University conferred upon her a Ph.D. in chemistry (with distinction).

But, as it turns out, her published results were not reproducible, an issue raised by chemists at Columbia and elsewhere as early as 2002. Further, the results were irreproducible for very good reason: as reported by Chemical & Engineering News, investigations by Columbia University and by the U.S. Department of Health & Human Services (which is home to the Office of Research Integrity) revealed

a massive and sustained effort by Sezen over the course of more than a decade to dope experiments, manipulate and falsify NMR and elemental analysis research data, and create fictitious people and organizations to vouch for the reproducibility of her results.

In the wake of the investigations, Sames has retracted the papers coauthored with Sezen (Sezen refused to retract them on the grounds that she stood by the work), and Columbia has revoked the Ph.D. it granted Sezen.

The evidence from the investigations supports the hypothesis that Bengü Sezen was a liar masquerading as a chemist, that she claimed to have done experiments that she hadn’t, to have obtained NMR spectra that she created (in part) with correction fluid, to have built molecules that she didn’t build. She committed fraud that introduced not just mistakes but lies into the scientific literature.

But she didn’t — she couldn’t — do this alone. She didn’t commit her fraud as a principal investigator (PI). Rather she did it as a scientific trainee, a graduate student working under the supervision of Dalibor Sames (who is currently an Associate Professor at Columbia). It’s worth examining what responsibility Sames bears for what happened here.
Continue reading→

Objectivity requires teamwork, but teamwork is hard.

Posted on by

In my last post, I set out to explain why the scientific quest to build something approaching objective knowledge requires help from other people. However, teamwork can be a challenge in the best of circumstances. And, certain aspects of scientific practices — especially in terms of how rewards are distributed — can make scientific teamwork even harder.

In this post, I’ll run down just some of the obstacles to scientists playing together effectively to build reliable knowledge about the world.

First, recall that a crucial thing individual scientists hope to get from their teammates in knowledge-building is help in identifying when they are wrong. The sociologist of science Robert Merton noted that a rule of the knowledge-building game, at least as far as Team Science is concerned, is organized skepticism, which I once described like this:

Everyone in the tribe of science can advance knowledge claims, but every such claims that is advanced is scrutinized, tested, tortured to see if it really holds up. The claims that do survive the skeptical scrutiny of the tribe get to take their place in the shard body of scientific knowledge.

In principle, each scientist tries to have their organized skepticism turned up to a healthy level when looking at her own results, as well as the results of others. In practice, there are issues that get in the way of both self-scrutiny and scrutiny of the results of others.

It’s hard to make a scientific career in replicating the results of others.

The first thing to recognize is that as serious as scientists are about the ideal of reproducible results, reproducibility is hard. It takes a while to gain technical mastery of all the moving parts in your experimental system and to figure out which of those wiggly bits make a difference in the results you see.

In itself, this needn’t be an obstacle to scientists working well together. The problem is that scientific rewards are usually reserved for those who generate novel findings — figuring out something that wasn’t known before — rather than for those who replicate results someone else has already put forward. What matters, for the career score-keeping (which drives who gets hired, who gets grant money, who gets promoted, who wins prizes) is whether you are first across the finish line to discover X. Being second (or third, or tenth) across that finish line is a nice reassurance that the first one across had a solid finding, but it doesn’t count in the same way.

Setting up the rewards so that the only winner is the first across the finish line may also provide a disincentive to doing enough experiments yourself to be sure that your results are really robust — the other guy may be sure enough to submit his manuscript on the basis of fewer runs, or might have gotten a head-start on it.

Now surely there are some exceptions, places perhaps where X was such a startlingly unexpected finding that the scientific community won’t really believe it until multiple researchers come forward to report that they have found it. But this is the exception rather than the rule, which means that if the second scientist to have found X cannot add some additional ingredient to our understanding of it that wasn’t part of the first report of X, that second researcher is out of luck.

Scientists are generally pretty smart. Among other things, this means most of them will come up with some strategy for spending their time that takes account of what activities will be rewarded. To the extent that working to replicate someone else’s results looks like a high-investment, low-yield activity, scientists may judge it prudent to spend their time doing something else.

It’s worth noting that scientists will frequently try to reproduce the results of others when those results are the starting point for a brand new piece of research of their own. These efforts can be time consuming and frustrating (see: “reproducibility is hard”). And, in the event that you discover that the other scientist’s results seem not to hold up, communicating to this other scientist is not always viewed as a friendly gesture.

Questions about results can feel like personal attacks.

Scientists work hard to get their studies to work and to draw their best conclusions about what their observations mean — and, as we’ve just noted, they do this while racing against the clock in hopes that some other researcher doesn’t make the discovery (and secure the credit for it) first. Since scientists are human, they can get attached to those results they worked so hard to get.

It shouldn’t be a surprise, then, that they can get touchy when someone else pops into the fray to tell them that there’s a problem with those results.

If the results are wrong, scientists face the possibility that they have wasted a bunch of time and money, blood, sweat, and tears. As well, they may have to issue a correction or even a retraction of their published results, which means that the publication that they’re correcting or retracting will no longer do the same work to advance their career.

In such a situation, getting defensive is understandable. However, getting defensive doesn’t do much to advance the knowledge-building project that science is supposed to be.

None of this is to say that an objection raised to one’s results should be automatically accepted as true. Organized skepticism applies to the critiques as well as to the original results.

That said, though, it strikes me that the best way to the knowledge-building, error-detecting teamwork that the tribe of science could use here might be establishing environments in scientific communities (from research groups to departments to disciplines) where researchers don’t take scrutiny of their results, data, methods, etc., personally — and where the scrutiny is applied to each member’s results, data, methods, etc. (since anyone can make mistakes).

When the players understand the game as aimed at building a reliable body of knowledge about the world that they can share, maybe they can be more welcoming of others pointing out their errors. When the game is understood as each scientist against all the others, pulling back to look critically at problems with one’s own work (especially when they are pointed out by a competitor) doesn’t look like such a great strategy.

(Unwillingness to take critiques of promising results seriously seems to have been a major feature of the Bengü Sezen/Dalibor Sames fraud scandal, and may also have played a role in the downfall of Harvard psychologist Marc Hauser.)

Sharing too much common ground makes it harder to be objective, too.

Part of the reason that scientists try to be alert to ways they could be deceived, even by themselves, is that the opportunities for deception are plentiful. One of the issues you have to face is that your expectations can influence what you see.

We don’t even have to dig into Thomas S. Kuhn’sThe Structure of Scientific Revolutions, embrace his whole story about paradigms, or even peruse the perception experiments he describes to accept that this is a possibility. The potential effect of expectations on observations is one reason that placebo-controlled trials are “double-blind” whenever possible, so neither experimental subject nor researcher is swayed by what they think ought to be occurring. Expectations also play a role in what kind of scientific findings are accepted easily into the shared body of knowledge (because they seem to fit so naturally with what we already know) and which ones are resisted (because they don’t fit so well, and might even require us to identify some of the things we thought we “knew” as wrong). And, expectations can influence scientific knowledge by shaping what kinds of questions researchers ask, what kinds of methods they decide to use to tackle those questions, and what kinds of outcomes they view as within the realm of possible outcomes. (If you’re doing an experiment and you see an outcome outside the range of expectations, often the first thing you check is whether the equipment is malfunctioning.)

Working with other people helps scientists build better knowledge by giving them some information about which observed outcomes are driven by the features of the phenomenon they’re trying to understand and which are driven by subjective features (like expectations). But the other people are must helpful here if their expectations and background assumptions are not identical to our own!

In the case that a scientific community shares all the same background assumptions, expectations, even unconscious biases, these things — and the ways that they can influence how experiments are designed and what findings they produce — may become almost invisible to the scientists in the community.

What this means is that it may be a healthy thing for a community of knowledge-builders to be diverse. How diverse? Ideally, you’d want the community to achieve enough diversity that it’s hard for each individual’s background assumptions to stay in the background, because you’re always in spitting distance of another individual with different background assumptions. Recognizing them as assumptions rather than necessarily true facts about the world can keep potential errors or oversights due to these assumptions on a shorter leash.

Each of these obstacles is linked to an over-arching challenge for scientific teamwork:

Teamwork often isn’t recognized or rewarded in scientific career score-keeping.

Scientists work with each other a lot to divide up aspects of complex research projects, but when it comes time to tally up the score what sometimes seems to matter most is who ends up being first author and who’s lost in the et al. Scientists have detailed discussions about published research in their field, enacting something like post-publication peer review, but if you determine that the other guy’s argument is persuasive or that his published findings actually hold up, it can be hard to capture the contribution you’ve made to Team Science’s joint knowledge-building project on your Curriculum Vitae. Scientists even have detailed pre-publication engagement about results (sometimes as a journal submission is being peer reviewed, sometimes less formally), but helping someone else uncover her mistakes or biases may put her in position to cross the discovery finish line before you do — and again, one doesn’t get much tangible career reward for providing this assist.

Teamwork is essential to making the knowledge scientists produce more objective. Yet the big career rewards in science seem clearly tied to individual achievement. Maybe the assumption is that some combination of competitive impulse (i.e., wanting to knock down other scientists’ putative knowledge claims) and community goodwill is enough to get scientists working together to find the errors and weed them out.

But maybe, if we think objectivity is a quality towards which scientific knowledge should be striving, it would make sense to put some more concrete rewards in place to incentivize the scientific teamwork on which objectivity depends.