Category Archives: Misconduct

What did Jonah Lehrer teach us about science?

Posted on by

Los Angeles Times book critic David L. Ulin wishes people would lay off of Jonah Lehrer. It’s bad enough that people made a fuss last July about falsified quotes and plagiarism that caused Lehrer’s publisher to recall his book Imagine and cost him a plum job at The New Yorker. Now people are crying foul that the Knight Foundation paid Lehrer $20,000 to deliver a mea culpa that Lehrer’s critics have not judged especially persuasive on the “lesson learned” front. Ulin thinks people ought to cut Lehrer some slack:

What did we expect from Lehrer? And why did we expect anything at all? Like every one of us, he is a conflicted human, his own worst enemy, but you’d hardly know that from the pile-on provoked by his talk.

Did Jonah Lehrer betray us? I don’t think so.

Ulin apparently feels qualified to speak on behalf of all of us. In light of some of the eloquent posts from people who feel personally betrayed by Lehrer, I’d recommend that Ulin stick to “I-statements” when assessing the emotional fallout from Lehrer’s journalistic misdeeds and more recent public relations blunder.

And, to be fair, earlier in Ulin’s piece, he does speak for himself about Lehrer’s books:

That’s sad, tragic even, for Lehrer was a talented journalist, a science writer with real insights into creativity and how the brain works. I learned things from his books “How We Decide” and “Imagine” (the latter of which has been withdrawn from publication), and Lehrer’s indiscretions haven’t taken that away.

(Bold emphasis added.)

Probably Ulin wouldn’t go to the mat to assert that what he learned from Imagine was what Bob Dylan actually said (since a fabricated Dylan quote was one of the smoking guns that revealed Lehrer’s casual attitudes toward journalistic standards). Probably he’d say he learned something about the science Lehrer was describing in such engaging language.

Except, people who have been reading Lehrer’s books carefully have noted that the scientific story he conveyed so engagingly was not always conveyed so accurately:

Jonah Lehrer was never a very good science writer. He seemed not to fully understand the science he was trying to explain; his explanations were inaccurate, overblown, and often just plain wrong, usually in the direction of giving his readers counterintuitive thrills and challenging their settled beliefs. You can read my review and the various parts of my exchange with him that are linked above for detailed explanations of why I make this claim. Others have made similar points too, for example Isaac Chotiner at the New Republic and Tim Requarth and Meehan Crist at The Millions. But the tenor of many critics last year was “he committed unforgivable journalistic sins and should be punished for them, but he still got the science right.” There was a clear sense that one had nothing to do with the other.

In my opinion, the fabrications and the scientific misunderstanding are actually closely related. The fabrications tended to follow a pattern of perfecting the stories and anecdotes that Lehrer — like almost all successful science writers nowadays — used to illustrate his arguments. Had he used only words Bob Dylan actually said, and only the true facts about Dylan’s 1960s songwriting travails, the story wouldn’t have been as smooth. It’s human nature to be more convinced by concrete stories than by abstract statistics and ideas, so the convincingness of Lehrer’s science writing came from the brilliance of his stories, characters, and quotes. Those are the elements that people process fluently and remember long after the details of experiments and analyses fade.

(Bold emphasis added.)

If this is the case — that Lehrer was an entertaining communicator but not a reliably accurate communicator of the current state of our best scientific knowledge — did Ulin actually learn what he thought he learned from Lehrer’s books? Or, was he misled by glib storytelling into thinking he understood what science might tell us about creativity, imagination, the workings of our own brains?

Maybe Ulin doesn’t expect a book marketed as non-fiction popular science to live up to this standard, but a lot of us do. And, while lowering one’s standards is one way to avoid feeling betrayed, it’s not something I would have expected a professional book critic to advise readers to do.

Marc Hauser makes an excuse for cheating. What he could have done instead.

Posted on by

DrugMonkey notes that Marc Hauser has offered an explanation for faking data (as reported on the Chronicle of Higher Education Percolator blog). His explanation amounts to:

  • being busy with teaching and directing the Mind, Brain & Behavior Program at Harvard
  • being busy serving on lots of fancy editorial boards
  • being busy writing stuff explaining science to an audience of non-scientists
  • being busy working with lots of scientific collaborators
  • being busy running a large research lab with lots of students

DrugMonkey responds that busy is part of the job description, especially if you’re rolling in the prestige of a faculty post at Harvard, and of being a recognized leader in your field. I would add that “I was really busy and I made a bad decision (but just this one time)” is an excuse we professors frequently hear from students we catch cheating. It’s also one that doesn’t work — we expect our students to do honest work and figure out their time management issues. And, we’re expected to work out our own time management issues — even if it means saying “No” to invitations that are sometimes tempting.

By the way, Marc Hauser didn’t actually admit that he faked data, or committed research misconduct of any kind, so much as he “accepts the findings” of the Office of Research Integrity. Moreover, his comments seem to be leaning on that last bullet point (the rigors of supervising a big lab) to deflect what responsibility he does take. From the CHE Percolator:

He also implies that some of the blame may actually belong to others in his lab. Writes Hauser: “I let important details get away from my control, and as head of the lab, I take responsibility for all errors made within the lab, whether or not I was directly involved.”

But that take—the idea that the problems were caused mainly by Hauser’s inattention—doesn’t square with the story told by those in his laboratory. A former research assistant, who was among those who blew the whistle on Hauser, writes in an e-mail that while the report “does a pretty good job of summing up what is known,” it nevertheless “leaves off how hard his co-authors, who were his at-will employees and graduate students, had to fight to get him to agree not to publish the tainted data.”

The former research assistant points out that the report takes into account only the research that was flagged by whistle-blowers. “He betrayed the trust of everyone that worked with him, and especially those of us who were under him and who should have been able to trust him,” the research assistant writes.

So, Hauser is kind of claiming that there were too many students, postdocs, and technicians to supervise properly, and some of them got away from him and falsified methodology and coding and fabricated data. The underlings challenge this account.

In the comments at DrugMonkey’s, hypotheses are being floated as to what might have spurred Hauser’s bad actions. (A perception that he needed to come up with sexy findings to stay a star in his field is one of the frontrunners.) I’m more inclined to come up with a list of options Hauser might have fruitfully pursued instead of faking or allowing fakery to happen on his watch:

  1. He could have agreed not to send out manuscripts with questionable data when his underlings asked him.
  2. He could have asked to be released from some of his teaching and/or administrative duties at Harvard so he could spend the needed time on his research and on properly mentoring the members of his lab.
  3. He could have taken on fewer students in order to better supervise and mentor the students in his charge.
  4. He could have sought the advice of a colleague or a collaborator on ways he might deal with his workload (or with the temptations that workload might be awakening in him).
  5. He could have communicated to his department, his professional societies, and the funding agencies his considered view that the demands on researchers, and operative definitions of productivity, make it unreasonable hard to do the careful research needed to come up with reliable answers to scientific questions.

And those are just off the top of my head.

I’m guessing that the pressure Marc Hauser felt to get results was real enough. What I’m not buying is the same thing that I don’t buy when I get this excuse from student plagiarists: that there was no other choice. Absent a gun to Hauser’s head, there surely were other things he could have done.

Feel free to add to the list of other options someone facing Hauser-like temptations could productively pursue instead of cheating.

Harvard Psych department may have a job opening.

Posted on by

… because Marc Hauser has resigned his faculty position, effective August 1.

You may recall, from our earlier discussions of Hauser (here, here, here, and here), that some of his papers were retracted because they drew conclusions that weren’t supported by the data … and then it emerged that maybe the data didn’t support the conclusions on account of scientific misconduct (rather than honest mistakes). Harvard mounted an inquiry. Hauser took a leave of absence from his position while the inquiry was ongoing. Harvard found Hauser “solely responsible, after a thorough investigation by a faculty member investigating committee, for eight instances of scientific misconduct under FAS standards.” In February, Hauser’s colleagues in the Psychology Department voted against allowing him to return to the classroom in the Fall. Meanwhile, since Hauser’s research was supported by grants from federal funding agencies, the Office of Research Integrity is thought to be in the midst of its own investigation of Hauser’s scientific conduct.

So perhaps Hauser’s resignation was to be expected (although it’s not too hard to come up with examples of faculty who were at least very close to scientific fraudsters — close enough to be enabling the fraud — who are still happily ensconced in their Ivy League institutions).

From Carolyn Y. Johnson at the Boston Globe:

“While on leave over the past year, I have begun doing some extremely interesting and rewarding work focusing on the educational needs of at-risk teenagers. I have also been offered some exciting opportunities in the private sector,” Hauser wrote in a resignation letter to the dean, dated July 7. “While I may return to teaching and research in the years to come, I look forward to focusing my energies in the coming year on these new and interesting challenges.”

Hauser did not respond to e-mail or voicemail messages today.

His resignation brings some resolution to the turmoil on campus, but it still leaves the scientific community trying to sort out what findings, within his large body of work, they should trust. Three published papers led by Hauser were thrown into question by the investigation — one was retracted and two were corrected. Problems were also found in five additional studies that were either not published or corrected prior to publication.

“What it does do is it provides some sort of closure for people at Harvard. … They were in a state of limbo,” said Gerry Altmann, editor of the journal Cognition, who, based on information provided to him by Harvard last year, said the only plausible conclusion he could draw was that some of the data had been fabricated in a study published in his journal in 2002 and retracted last year. “There’s just been this cloud hanging over the department. … It has no real impact on the field more broadly.”

Maybe it’s just me, but there seems to be a mixed message in those last two paragraphs. Either this is the story of one bad apple who indulged in fabrication and brought shame to his university, or this is the story of a trusted member of the scientific community who contributed many, many articles to the literature in his field and now turns out not to be so trustworthy. If it’s the latter, then we’re talking about potential impacts that are much bigger than Harvard’s reputation. We’re talking about a body of scientific literature that suddenly looks less solid — a body of scientific literature that other researchers had trusted, used as the basis for new studies of their own, perhaps even taken as the knowledge base with which other new findings would need to be reconciled to be credible.

And, it’s not like there’s no one suggesting that Marc Hauser is a good guy who has made important (and presumably trustworthy) contributions to science. For example:

“I’m deeply saddened by the whole events of the last year,” Steven Pinker, a psychology professor at Harvard, said today. “Marc is a scientist of enormous creativity, energy, and talent.”

Meanwhile, if the data from the Harvard investigation best supports the conclusion that Hauser’s recent work was marred by scientific misconduct characterized by “problems involving data acquisition, data analysis, data retention, and the reporting of research methodologies and results,” this seems to count against Hauser’s credibility (and his judgment). And, although we might make the case that teaching involves a different set of competencies than research, his colleagues may have decided that the his to his credibility as a knowledge-builder would also do damage to his credibility as a teacher. The Boston Globe article notes:

Another researcher in the field, Michael Tomasello, co-director of the Max Planck Institute for Evolutionary Anthropology, said today that Hauser’s departure was not unexpected. “Once they didn’t let him teach –- and there are severe restrictions in his ability to do research — you come to office and what do you do all day?” he said. “People in the field, we’re just wondering — this doesn’t change anything. We’re still where we were before about the [other] studies.”

What could Hauser do at work all day if not teach and conduct research? Some might suggest a full slate of committee work.

Others would view that as cruel and unusual punishment, even for the perpetrator of scientific misconduct.

A question for the trainees: How involved do you want the boss to get with your results?

Posted on by

This question follows on the heels of my recent discussion of the Bengü Sezen misconduct investigations, plus a conversation via Twitter that I recapped in the last post.

The background issue is that people — even scientists, who are supposed always to be following the evidence wherever it might lead — can run into trouble really scrutinizing the results of someone they trust (however that trust came about). Indeed, in the Sezen case, her graduate advisor at Columbia University, Dalibor Sames, seemed to trust Sezen and her scientific prowess so much that he discounted the results of other graduate students in his lab who could not replicate Sezen’s results (which turned out to have been faked).

Really, it’s the two faces of the PI’s trust here: trusting one trainee so much that her results couldn’t be wrong, and using that trust to ignore the empirical evidence presented by other trainees (who apparently didn’t get the same level of presumptive trust). As it played out, at least three of those other trainees whose evidence Sames chose not to trust left the graduate program before earning their degrees.

The situation suggests to me that PIs would be prudent to establish environments in their research groups where researchers don’t take scrutiny of their results, data, methods, etc., personally — and where the scrutiny is applied to each member’s results, data, methods, etc. (since anyone can make mistakes). But how do things play out when they rubber hits the road?

So, here’s the question I’d like to ask the scientific trainees. (PIs: I’ve posed the complementary question to you in the post that went up right before this one!)

In his or her capacity as PI, your advisor’s scientific credibility (and likely his or her name) is tied to all the results that come out of the research group — whether they are experimental measurements, analyses of measurements, modeling results, or whatever else it is that scientists of your stripe regard as results. Moreover, in his or her capacity as a trainer of new scientists, the boss has something like a responsibility to make sure you know how to generate reliable results — and that you know how to tell them from results that aren’t reliable. What does your PI do to ensure that the results you generate are reliable? Do you feel like it’s enough (both in terms of quality control and in terms of training you well)? Do you feel like it’s too much?

Commenting note: You may feel more comfortable commenting with a pseudonym for this particular discussion, and that’s completely fine with me. However, please pick a unique ‘nym and keep it for the duration of this discussion, so we’re not in the position of trying to sort out which “Anonymous” is which. Also, if you’re a regular commenter who wants to go pseudonymous for this discussion, you’ll probably want to enter something other than your regular email address in the commenting form — otherwise, your Gravatar may give your other identity away!

A question for the PIs: How involved do you get in your trainees’ results?

Posted on by

In the wake of this post that touched on recently released documents detailing investigations into Bengü Sezen’s scientific misconduct, and that noted that a C & E News article described Sezen as a “master of deception”, I had an interesting chat on the Twitters:

@UnstableIsotope (website) tweeted:

@geernst @docfreeride I scoff at the idea that Sezen was a master at deception. She lied a lot but plenty of opportunities to get caught.

@geernst (website) tweeted back:

@UnstableIsotope Maybe evasion is a more accurate word.

@UnstableIsotope:

@geernst I’d agree she was a master of evasion. But she was caught be other group members but sounds like advisor didn’t want to believe it.

@docfreeride (that’s me!):

@UnstableIsotope @geernst Possible that she was master of deception only in environment where people didn’t guard against being deceived?

@UnstableIsotope:

@docfreeride @geernst I agree ppl didn’t expect deception, my read suggests she was caught by group members but protected by advisor.

@UnstableIsotope:

@docfreeride @geernst The advisor certainly didn’t expect deception and didn’t encourage but didn’t want to believe evidence

@docfreeride:

@UnstableIsotope @geernst Not wanting to believe the evidence strikes me as a bad fit with “being a scientist”.

@UnstableIsotope:

@docfreeride @geernst Yes, but it is human. Not wanting to believe your amazing results are not amazing seems like a normal response to me.

@geernst:

@docfreeride @UnstableIsotope I agree. Difficult to separate scientific objectivity from personal feelings in those circumstances.

@docfreeride:

@geernst @UnstableIsotope But isn’t this exactly the argument for not taking scrutiny of your results, data, methods personally?

@UnstableIsotope:

@docfreeride @geernst Definitely YES. I look forward to people repeating my experiments. I’m nervous if I have the only result.

@geernst:

@docfreeride @UnstableIsotope Couldn’t agree more.

This conversation prompted a question I’d like to ask the PIs. (Trainees: I’m going to pose the complementary question to you in the very next post!)

In your capacity as PI, your scientific credibility (and likely your name) is tied to all the results that come out of your research group — whether they are experimental measurements, analyses of measurements, modeling results, or whatever else it is that scientists of your stripe regard as results. What do you do to ensure that the results generated by your trainees are reliable?

Now, it may be the case that what you see as the appropriate level of involvement/quality control/”let me get up in your grill while you repeat that measurement for me” would still not have been enough to deter — or to detect — a brazen liar. If you want to talk about that in the comments, feel free.

Commenting note: You may feel more comfortable commenting with a pseudonym for this particular discussion, and that’s completely fine with me. However, please pick a unique ‘nym and keep it for the duration of this discussion, so we’re not in the position of trying to sort out which “Anonymous” is which. Also, if you’re a regular commenter who wants to go pseudonymous for this discussion, you’ll probably want to enter something other than your regular email address in the commenting form — otherwise, your Gravatar may give your other identity away!

What are honest scientists to do about a master of deception?

Posted on by

A new story posted at Chemical & Engineering News updates us on the fraud case of Bengü Sezen (who we discussed here, here, and here at much earlier stages of the saga).

William G. Schultz notes that documents released (PDF) by the Department of Health and Human Services (which houses the Office of Research Integrity) detail some really brazen misconduct on Sezen’s part in her doctoral dissertation at Columbia University and in at least three published papers.

From the article:

The documents—an investigative report from Columbia and HHS’s subsequent oversight findings—show a massive and sustained effort by Sezen over the course of more than a decade to dope experiments, manipulate and falsify NMR and elemental analysis research data, and create fictitious people and organizations to vouch for the reproducibility of her results. …

A notice in the Nov. 29, 2010, Federal Register states that Sezen falsified, fabricated, and plagiarized research data in three papers and in her doctoral thesis. Some six papers that Sezen had coauthored with Columbia chemistry professor Dalibor Sames have been withdrawn by Sames because Sezen’s results could not be replicated. …

By the time Sezen received a Ph.D. degree in chemistry in 2005, under the supervision of Sames, her fraudulent activity had reached a crescendo, according to the reports. Specifically, the reports detail how Sezen logged into NMR spectrometry equipment under the name of at least one former Sames group member, then merged NMR data and used correction fluid to create fake spectra showing her desired reaction products.

Apparently, her results were not reproducible because those trying to reproduce them lacked her “hand skills” with Liquid Paper.

Needless to say, this kind of behavior is tremendously detrimental to scientific communities trying to build a body of reliable knowledge about the world. Scientists are at risk of relying on published papers that are based in wishes (and lies) rather than actual empirical evidence, which can lead them down scientific blind alleys and waste their time and money. Journal editors devoted resources to moving her (made-up) papers through peer review, and then had to devote more resources to dealing with their retractions. Columbia University and the U.S. government got to spend a bunch of money investigating Sezen’s wrongdoing — the latter expenditures unlikely to endear scientific communities to an already skeptical public. Even within the research lab where Sezen, as a grad student, was concocting her fraudulent results, her labmates apparently wasted a lot of time trying to reproduce her results, questioning their own abilities when they couldn’t.

And to my eye, one of the big problems in this case is that Sezen seems to have been the kind of person who projected confidence while lying her pants off:

The documents paint a picture of Sezen as a master of deception, a woman very much at ease with manipulating colleagues and supervisors alike to hide her fraudulent activity; a practiced liar who would defend the integrity of her research results in the face of all evidence to the contrary. Columbia has moved to revoke her Ph.D.

Worse, the reports document the toll on other young scientists who worked with Sezen: “Members of the [redacted] expended considerable time attempting to reproduce Respondent’s results. The Committee found that the wasted time and effort, and the onus of not being able to reproduce the work, had a severe negative impact on the graduate careers of three (3) of those students, two of whom [redacted] were asked to leave the [redacted] and one of whom decided to leave after her second year.”

In this matter, the reports echo sources from inside the Sames lab who spoke with C&EN under conditions of anonymity when the case first became public in 2006. These sources described Sezen as Sames’ “golden child,” a brilliant student favored by a mentor who believed that her intellect and laboratory acumen provoked the envy of others in his research group. They said it was hard to avoid the conclusion that Sames retaliated when other members of his group questioned the validity of Sezen’s work.

What I find striking here is that Sezen’s vigorous defense of her’s own personal integrity was sufficient, at least for awhile, to convince her mentor that those questioning the results were in the wrong — not just incompetent to reproduce the work, but jealous and looking to cause trouble. And, it’s deeply disappointing that this judgment may have been connected to the departure of those fellow graduate students who raised questions from their graduate program.

How could this have been avoided?

Maybe a useful strategy would have been to treat questions about the scientific work (including its reproducibility) first and foremost as questions about the scientific work.

Getting results that others cannot reproduce is not prima facie evidence that you’re a cheater-pants. It may just mean that there was something weird going on with the equipment, or the reagents, or some other component of the experimental system when you did the experiment that yielded the exciting but hard to replicate results. Or, it may mean that the folks trying to replicate the results haven’t quite mastered the technique (which, in the case that they are your colleagues in the lab, could be addressed by working with them on their technique). Or, it may mean that there’s some other important variable in the system that you haven’t identified as important and so have not worked out (or fully described) how to control.

In this case, of course, it’s looking like the main reason that Sezen’s results were not reproducible was that she made them up. But casting the failure to replicate presumptively as one scientist’s mad skillz and unimpeachable integrity against another’s didn’t help get to the bottom of the scientific facts. It made the argument personal rather than putting the scientists involved on the same team in figuring out what was really going on with the scientific systems being studied.

Of all of the Mertonian norms imputed to the Tribe of Science, organized skepticism is probably the one nearest and dearest to most scientists’ basic understanding of how they get the knowledge-building job done. Figuring out what’s going on with particular phenomena in the world can be hard, not least because lining up solid evidence to support your conclusions requires identifying evidence that others trying to repeat your work can reliably obtain themselves. This is more than just a matter of making sure your results are robust. Rather, you want others to be able to reproduce your work so that you know you haven’t fooled yourself.

Organized skepticism, in other words, should start at home.

There is a risk of being too skeptical of your own results, and there are chances to overlook something important as noise because it doesn’t fit with what you expect to observe. However, the scientist who refuses to entertain the possibility that her work could be wrong — indeed, who regards questions about the details of her work as a personal affront — should raise a red flag for the rest of her scientific community, no matter what her career stage or her track record of brilliance to date.

In a world where every scientist’s findings are recognized as being susceptible to error, the first response to questions about findings might be to go back to the phenomena together, helping each other to locate potential sources of error and to avoid them. In such a world, the master of deception trying to ride personal reputation (or good initial impressions) to avoid scrutiny of his or her work will have a much harder time getting traction.

Does the punishment fit the crime? Luk Van Parijs has his day in court.

Posted on by

Earlier this month, the other shoe finally dropped on the Luk Van Parijs case.

You may recall that Van Parijs, then an associate professor of biology at MIT, made headlines back in October of 2005 when MIT fired him after spending nearly a year investigating charges that he had falsified and fabricated data and finding those charges warranted. We discussed the case as it was unfolding (here and here), and discussed also the “final action” by the Office of Research Intergrity on the case (which included disbarment from federal funding through December 21, 2013).

But losing the MIT position and five year’s worth of eligibility for federal funding (counting from when Van Parijs entered the Voluntary Exclusion Agreement with the feds) is not the extent of the formal punishment to be exacted for his crimes — hence the aforementioned other shoe. As well, the government filed criminal charges against Van Parijs and sought jail time.

As reported in a news story posted 28 June 2011 at Nature (“Biologist spared jail for grant fraud” by Eugenie Samuel Reich, doi:10.1038/474552a):

In February 2011, US authorities filed criminal charges against Van Parijs in the US District Court in Boston, citing his use of fake data in a 2003 grant application to the National Institutes of Health, based in Bethesda, Maryland. Van Parijs entered a guilty plea, and the government asked Judge Denise Casper for a 6-month jail term because of the seriousness of the fraud, which involved a $2-million grant. “We want to discourage other researchers from engaging in similar behaviour,” prosecutor Gregory Noonan, an assistant US attorney, told Nature.

On 13 June, Casper opted instead for six months of home detention with electronic monitoring, plus 400 hours of community service and a payment to MIT of $61,117 — restitution for the already-spent grant money that MIT had to return to the National Institutes of Health. She cited assertions from the other scientists that Van Parijs was truly sorry. “I believe that the remorse that you’ve expressed to them, to the probation office, and certainly to the Court today, is heartfelt and deeply held, and I don’t think it’s in any way contrived for this Court,” she said.

Let me pause for a moment to let you, my readers, roll your eyes or howl or do whatever else you deem appropriate to express your exasperation that Van Parijs’s remorse counts for anything in his sentencing.

Verily, it is not hard to become truly sorry once you have been caught doing bad stuff. The challenge is not to do the bad stuff in the first place. And, the actual level of remorse in Van Parijs’s heart does precisely nothing to mitigate the loss (in time and money, to name just two) suffered by other researchers relying on Van Parijs to make honest representations in his journal articles and grant proposals.

Still, there’s probably a relevant difference (not just ethically, but also pragmatically) between the scientist caught deceiving the community who gets what such deception is a problem and manifests remorse and the scientist caught deceiving the community who doesn’t see what the big deal is (because surely everyone does this sort of thing, at least occasionally, to survive in the high-pressure environment). With the remorseful cheater, there might at least be some hope of rehabilitation.

Indeed, the article notes:

Luk Van Parijs was first confronted with evidence of data falsification by members of his laboratory in 2004, when he was an associate professor of biology at the Massachusetts Institute of Technology (MIT) in Cambridge. Within two days, he had confessed to several acts of fabrication and agreed to cooperate with MIT’s investigation.

A confession within two days of being confronted with the evidence is fairly swift. Other scientific cheaters in the headlines seem to dig their heels in and protest their innocence (or that the post-doc or grad student did it) for significantly longer than that.

Anyway, I think it’s reasonable for us to ask here what the punishment is intended to accomplish in a case like this. If the goal is something beyond satisfying our thirst for vengeance, then maybe we will find that the penalty imposed on Van Parijs is useful even if it doesn’t include jail time.

As it happens, one of the scientists who asked the judge in the case for clemency on his behalf suggests that jail time might be a penalty that actually discourages the participation of other members of the scientific community in rooting out fabrication and falsification. Of course, not everyone in the scientific community agrees:

[MIT biologist Richard] Hynes argued that scientific whistleblowers might be reluctant to come forwards if they thought their allegations might result in jail for the accused.

But that is not how the whistleblowers in this case see it. One former member of Van Parijs’ MIT lab, who spoke to Nature on condition of anonymity, says he doesn’t think the prospect of Van Parijs’ imprisonment would have deterred the group from coming forwards. Nor does he feel the punishment is adequate. “Luk’s actions resulted in many wasted years as people struggled to regain their career paths. How do you measure the cost to the trainees when their careers have been derailed and their reputations brought into question?” he asks. The court did not ask these affected trainees for their statements before passing sentence on Van Parijs.

This gets into a set of questions we’ve discussed before:

I’m inclined to think that the impulse to deal with science’s youthful offenders privately is a response to the fear that handing them over to federal authorities has a high likelihood of ending their scientific careers forever. There is a fear that a first offense will be punished with the career equivalent of the death penalty.

Permanent expulsion or a slap on the wrist is not much of a range of penalties. And, I suspect neither of these options really address the question of whether rehabilitation is possible and in the best interests of both the individual and the scientific community. …

If no errors in judgment are tolerated, people will do anything to conceal such errors. Mentors who are trying to be humane may become accomplices in the concealment. The conversations about how to make better judgments may not happen because people worry that their hypothetical situations will be scrutinized for clues about actual screw-ups.

Possibly we need to recognize that it’s an empirical question what constellation of penalties (including jail time) encourage or discourage whisteblowing — and to deploy some social scientists to get reliable empirical data that might usefully guide decisions about institutional structures of rewards and penalties that will best encourage the kinds of individual behaviors that lead to robust knowledge-building activities and effectively coordinated knowledge-building communities.

But, it’s worth noting that even though he won’t be doing jail time, Van Parijs doesn’t escape without punishment.

He will be serving the same amount of time under home detention (with electronic monitoring) as he would have served in jail if the judge had given the sentence the government was asking for. In other words, he is not “free” for those six months. (Indeed, assuming he serves this home detention in the home shared by his wife and their three young children, I reckon there is a great deal of work that he might be called on to do with respect to child care and household chores, work that he might escape in a six-month jail sentence.)

Let’s not forget that it costs money to incarcerate people. The public picks up the tab for those expenses. Home detention almost certainly costs the public less. And, Van Parijs is probably less in a position to reoffend during his home detention, even if he slipped out of his ankle monitor, than is the guy who robs convenience stores. What university is going to be looking at his data?

Speaking of the public’s money, recall that another piece of the sentence is restitution — paying back to MIT the $61,117 that MIT spent when it returned Van Parijs’s grant money to NIH. Since Van Parijs essentially robbed the public of the grant money (by securing it with lies and/or substitute lies for the honest scientific results the grant-supported research was supposed to be generating), it is appropriate that Van Parijs dip into his own pocket to pay this money back.

It’s a little more complicated, since he needs to pay MIT back. MIT seems to have recognized that paying the public back as soon as the problem was established was the right thing to do, or a good way to reassure federal funding agencies and the the public that universities like MIT take their obligations to the public very seriously, or both. A judgment that doesn’t make MIT eat that loss, in turn, should encourage other universities that find themselves in similar situations to step up right away and make things right with the funding agencies.

And, in recognition that the public may have been hurt by Van Parijs’s deception beyond the monetary cost of it, Van Parijs will be doing 400 hours of community service. In inclined to believe that given the current fiscal realities of federal, state, and local governments, there is some service the community needs — and doesn’t have adequate funds to pay for — that Van Parijs might provide in those 400 hours. Perhaps it will not be a service he finds intellectually stimulating to provide, but that’s part of what makes it punishment.

Undoubtedly, there are members of the scientific community or of the larger public that will feel that this punishment just isn’t enough — that Van Parijs committed crimes against scientific integrity that demand harsher penalties.

Pragmatically, though, I think we need to ask what it would cost to secure those penalties. We cannot ignore the costs to the universities and to the federal agencies to conduct their investigations (here Van Parijs confessed rather denying the charges and working to obstruct the fact-finding), or to prosecutors to go to trial (here again, Van Parijs pled guilty rather than mounting a vigorous defense). Maybe there was a time where there were ample resources to spend on full-blown investigations and trials of this sort, but that time ain’t now.

And, we might ask what jailing Van Parijs would accomplish beyond underlining that fabrication and falsification on the public’s dime is a very bad thing to do.

Would jail time make it harder for Van Parijs to find another position within the tribe of science than it will already be for him? (Asked another way, would being sentenced to home detention take any of the stink off the verdict of fraud against him?) I reckon the convicted fraudster scientist has a harder time finding a job than your average ex-con — and that scientists who feel his punishment is not enough can lobby the rest of their scientific community to keep a skeptical eye on Van Parijs (should he publish more papers, apply for jobs within the tribe of science, or what have you).

Punishment, redemption, and celebrity status: still more on the Hauser case.

Posted on by

Yesterday in the New York Times, Nicholas Wade wrote another article about the Marc Hauser scientific misconduct and its likely fallout. The article didn’t present much in the way of new facts, as far as I could tell, but I found this part interesting:

Some forms of scientific error, like poor record keeping or even mistaken results, are forgivable, but fabrication of data, if such a charge were to be proved against Dr. Hauser, is usually followed by expulsion from the scientific community.

“There is a difference between breaking the rules and breaking the most sacred of all rules,” said Jonathan Haidt, a moral psychologist at the University of Virginia. The failure to have performed a reported control experiment would be “a very serious and perhaps unforgivable offense,” Dr. Haidt said.

Dr. Hauser’s case is unusual, however, because of his substantial contributions to the fields of animal cognition and the basis of morality. Dr. [Gerry] Altmann [editor of the journal Cognition] held out the possibility of redemption. “If he were to give a full and frank account of the errors he made, then the process can start of repatriating him into the community in some form,” he said.

I’m curious what you all think about this.

Do you feel that some of the rules of scientific conduct are more sacred than others? That some flavors of scientific misconduct are more forgivable than others? That a scientist who has made “substantial contributions” in his or her field of study might be entitled to more forgiveness for scientific misconduct than your typical scientific plodder?

I think these questions touch on the broader question of whether the tribe of science (or the general public putting up the money to support scientific research) believes rehabilitation is possible for those caught in scientific misdeeds. (This is something we’ve discussed before in the context of why members of the tribe of science might be inclined to let “youthful offenders” slide by with a warning rather than exposing them to punishments that are viewed as draconian.)

But the Hauser case adds an element to this question. What should we make of the case where the superstar is caught cheating? How should we weigh the violation of trust against the positive contribution this researcher has made to the body of scientific knowledge? Can we continue to trust that his or her positive contribution to that body of knowledge was an actual contribution, or ought we to subject it to extra scrutiny on account of the cheating for which we have evidence? Are we forced to reexamine the extra credence we may have been granting the superstar’s research on account of that superstar status?

And, in a field of endeavor that strives for objectivity, are we really OK with the suggestion that members of the tribe of science who achieve a certain status should be held to different rules than those by which everyone else in the tribe is expected to play?

Harvard Dean sheds (a little) more light on Hauser misconduct case.

Posted on by

Today ScienceInsider gave an update on the Marc Hauser misconduct case, one that seems to support the accounts of other researchers in the Hauser lab. From ScienceInsider:

In an e-mail sent earlier today to Harvard University faculty members, Michael Smith, dean of the Faculty of Arts and Sciences (FAS), confirms that cognitive scientist Marc Hauser “was found solely responsible, after a thorough investigation by a faculty member investigating committee, for eight instances of scientific misconduct under FAS standards.”

ScienceInsider reprints the Dean’s email in its entirety. Here’s the characterization of the nature of Hauser’s misconduct from that email:

Continue reading→

What kind of problem is it when data do not support findings?

Posted on by

And, whose problem is it?

Yesterday, The Boston Globe published an article about Harvard University psychologist Marc Hauser, a researcher embarking on a leave from his appointment in the wake of a retraction and a finding of scientific misconduct in his lab. From the article:

In a letter Hauser wrote this year to some Harvard colleagues, he described the inquiry as painful. The letter, which was shown to the Globe, said that his lab has been under investigation for three years by a Harvard committee, and that evidence of misconduct was found. He alluded to unspecified mistakes and oversights that he had made, and said he will be on leave for the upcoming academic year. …

Much remains unclear, including why the investigation took so long, the specifics of the misconduct, and whether Hauser’s leave is a punishment for his actions.

The retraction, submitted by Hauser and two co-authors, is to be published in a future issue of Cognition, according to the editor. It says that, “An internal examination at Harvard University . . . found that the data do not support the reported findings. We therefore are retracting this article.’’

The paper tested cotton-top tamarin monkeys’ ability to learn generalized patterns, an ability that human infants had been found to have, and that may be critical for learning language. The paper found that the monkeys were able to learn patterns, suggesting that this was not the critical cognitive building block that explains humans’ ability to learn language. In doing such experiments, researchers videotape the animals to analyze each trial and provide a record of their raw data. …

The editor of Cognition, Gerry Altmann, said in an interview that he had not been told what specific errors had been made in the paper, which is unusual. “Generally when a manuscript is withdrawn, in my experience at any rate, we know a little more background than is actually published in the retraction,’’ he said. “The data not supporting the findings is ambiguous.’’

Gary Marcus, a psychology professor at New York University and one of the co-authors of the paper, said he drafted the introduction and conclusions of the paper, based on data that Hauser collected and analyzed.

“Professor Hauser alerted me that he was concerned about the nature of the data, and suggested that there were problems with the videotape record of the study,’’ Marcus wrote in an e-mail. “I never actually saw the raw data, just his summaries, so I can’t speak to the exact nature of what went wrong.’’
The investigation also raised questions about two other papers co-authored by Hauser. The journal Proceedings of the Royal Society B published a correction last month to a 2007 study. The correction, published after the British journal was notified of the Harvard investigation, said video records and field notes of one of the co-authors were incomplete. Hauser and a colleague redid the three main experiments and the new findings were the same as in the original paper. …

“This retraction creates a quandary for those of us in the field about whether other results are to be trusted as well, especially since there are other papers currently being reconsidered by other journals as well,’’ Michael Tomasello, co-director of the Max Planck Institute for Evolutionary Anthropology in Leipzig, Germany, said in an e-mail. “If scientists can’t trust published papers, the whole process breaks down.’’ …

In 1995, he [Hauser] was the lead author of a paper in the Proceedings of the National Academy of Sciences that looked at whether cotton-top tamarins are able to recognize themselves in a mirror. Self-recognition was something that set humans and other primates, such as chimpanzees and orangutans, apart from other animals, and no one had shown that monkeys had this ability.

Gordon G. Gallup Jr., a professor of psychology at State University of New York at Albany, questioned the results and requested videotapes that Hauser had made of the experiment.

“When I played the videotapes, there was not a thread of compelling evidence — scientific or otherwise — that any of the tamarins had learned to correctly decipher mirrored information about themselves,’’ Gallup said in an interview.

A quick rundown of what we get from this article:

  • Someone raised a concern about scientific misconduct that led to the Harvard inquiry, which in turn led to the discovery of “evidence of misconduct” in Hauser’s lab.
  • We don’t, however, have an identification of what kind of misconduct is suggested by the evidence (fabrication? falsification? plagiarism? other serious deviations from accepted practices?) or of who exactly committed it (Hauser or one of the other people in his lab).
  • At least one paper has been retracted because “the data do not support the reported findings”.
  • However, we don’t know the precise issue with the data here — e.g., whether the reported findings were bolstered by reported data that turned out to be fabricated or falsified (and are thus not being included anymore in “the data”).
  • Apparently, the editor of the journal that published the retracted paper doesn’t know the precise issue with the data, either, and found this unusual enough a situation with respect to the retraction of the paper to merit comment.
  • Other papers from the Hauser group may be under investigation for similar reasons at this point, and other researchers in the field seem to be nervous about those papers and their reliability in light of the ongoing inquiry and the retraction of the paper in Cognition.

There’s already been lots of good commentary on what might be going on with the Hauser case. (I say “might” because there are many facts still not in evidence to those of us not actually on the Harvard inquiry panel. As such, I think it’s necessary to refrain from drawing conclusions not supported by the facts that are in evidence.)

John Hawks situates the Hauser case in terms of the problem of subjective data.

Melody has a nice discussion of the political context of getting research submitted to journals, approved by peer reviewers, and anointed as knowledge.

David Dobbs wonders whether the effects of the Hauser case (and of the publicity it’s getting) will mean backing off from overly strong conclusions drawn from subjective data, or backing off too far from a “hot” scientific field that may still have a bead on some important phenomena in our world.

Drugmonkey critiques the Boston Globe reporting and reminds us that failure to replicate a finding is not evidence of scientific misconduct or fraud. That’s a hugely important point, and one that bears repeating. Repeatedly.

This is the kind of territory where we start to notice common misunderstandings about how science works. It’s usually not the case that we can cut nature at the joints along nicely dotted lines that indicate just where those cuts should be. Collecting reliable data and objectively interpreting that data is hard work. Sometimes as we go, we learn more about better conditions for collecting reliable data, or better procedures for interpreting the data without letting our cognitive biases do the driving. And sometimes, a data set we took to be reliable and representative of the phenomenon we’re trying to understand just isn’t.

That’s part of why scientific conclusions are always tentative. Scientists expect to update their current conclusions in the light of new results down the road — and in the light of our awareness that some of our old results just weren’t as solid or reproducible as we took them to be. It’s good to be sure they’re reproducible enough before you announce a finding to your scientific peers, but to be absolutely certain of total reproducibility, you have to solve the problem of induction, which isn’t terribly practical.

Honest scientific work can lead to incorrect conclusions, either because that honest work yielded wonk data from which to draw conclusions, or because good data can still be consistent with incorrect conclusions.

And, there’s a similar kind of disconnect we should watch out for. For the “corrected” 2007 paper in Proceedings of the Royal Society B, the Boston Globe article reports that videotapes and field notes (the sources of the data to support the reported conclusions) were “incomplete”. But, Hauser and a colleague redid the experiments and found data that supported the conclusions reported in this paper. One might think that as long as reported results are reproducible, they’re necessarily sufficiently ethical and scientifically sound and all that good stuff. That’s not how scientific knowledge-building works. The rules of the game are that you lay your data-cards on the table and base your findings on those data. Chancing upon an answer that turns out to be right but isn’t supported by the data you actually have doesn’t count, nor does having a really strong hunch that turns out to be right. In the scientific realm, empirical data is our basis for knowing what we know about the phenomena. Thus, doing the experiments over in the face of insufficient data is not “playing it safe” so much as “doing the job you were supposed to have done in the first place”.

Now, given the relative paucity of facts in this particular case, I find myself interested by a more general question: What are the ethical duties of a PI who discovers that he has published a paper whose findings are not, in fact, supported by the data?.

It seems reasonable that at least one of his or her duties involves correcting the scientific literature.

This could involve retracting the paper, in essence saying, “Actually, we can’t conclude this based on the data we have. Our bad!”

It could also involve correcting the paper, saying, “We couldn’t conclude this based on the data we have; instead, we should conclude this other thing,” or, “We couldn’t conclude this based on the data we originally reported, but we’ve gone and done more experiments (or have repeated the experiments we described), obtained this data, and are now confident that on the basis of these data, the conclusion in well-supported.”

If faulty data were reported, I would think that the retraction or correction should probably explain how the data were faulty — what’s wrong with them? If the problem had its source in an honest mistake, it might also be valuable to identify that honest mistake so other researchers could avoid it themselves. (Surely this would be a kindness; is it also a duty?)

Beyond correcting the scientific literature, does the PI in this situation have other relevant duties?

Would these involve ratcheting up the scrutiny of data within the lab group in advance of future papers submitted for publication? Taking the skepticism of other researchers in the field more seriously and working that much harder to build a compelling case for conclusions from the data? (Or, perhaps, working hard to identify the ways that the data might argue against the expected conclusion?) Making serious efforts to eliminate as much subjectivity from the data as possible?

Assuming the PI hasn’t fabricated or falsified the data (and that if someone in the lab group has, that person has been benched, at least for the foreseeable future), what kind of steps ought that PI to take to make things right — not just for the particular problematic paper(s), but for his or her whole research group moving forward and interacting with other researchers in the field? How can they earn back trust?