This question follows on the heels of my recent discussion of the Bengü Sezen misconduct investigations, plus a conversation via Twitter that I recapped in the last post.
The background issue is that people — even scientists, who are supposed always to be following the evidence wherever it might lead — can run into trouble really scrutinizing the results of someone they trust (however that trust came about). Indeed, in the Sezen case, her graduate advisor at Columbia University, Dalibor Sames, seemed to trust Sezen and her scientific prowess so much that he discounted the results of other graduate students in his lab who could not replicate Sezen’s results (which turned out to have been faked).
Really, it’s the two faces of the PI’s trust here: trusting one trainee so much that her results couldn’t be wrong, and using that trust to ignore the empirical evidence presented by other trainees (who apparently didn’t get the same level of presumptive trust). As it played out, at least three of those other trainees whose evidence Sames chose not to trust left the graduate program before earning their degrees.
The situation suggests to me that PIs would be prudent to establish environments in their research groups where researchers don’t take scrutiny of their results, data, methods, etc., personally — and where the scrutiny is applied to each member’s results, data, methods, etc. (since anyone can make mistakes). But how do things play out when they rubber hits the road?
So, here’s the question I’d like to ask the scientific trainees. (PIs: I’ve posed the complementary question to you in the post that went up right before this one!)
In his or her capacity as PI, your advisor’s scientific credibility (and likely his or her name) is tied to all the results that come out of the research group — whether they are experimental measurements, analyses of measurements, modeling results, or whatever else it is that scientists of your stripe regard as results. Moreover, in his or her capacity as a trainer of new scientists, the boss has something like a responsibility to make sure you know how to generate reliable results — and that you know how to tell them from results that aren’t reliable. What does your PI do to ensure that the results you generate are reliable? Do you feel like it’s enough (both in terms of quality control and in terms of training you well)? Do you feel like it’s too much?
Commenting note: You may feel more comfortable commenting with a pseudonym for this particular discussion, and that’s completely fine with me. However, please pick a unique ‘nym and keep it for the duration of this discussion, so we’re not in the position of trying to sort out which “Anonymous” is which. Also, if you’re a regular commenter who wants to go pseudonymous for this discussion, you’ll probably want to enter something other than your regular email address in the commenting form — otherwise, your Gravatar may give your other identity away!
Pingback: A question for the PIs: How involved do you get in your trainees’ results? | Adventures in Ethics and Science
I’m in a situation where I’m generally happy if my advisor has an idea of what my results are, let alone has done anything to ensure they’re reliable. I’m in a group that’s probably too large, and there’s not much overlap between projects, so there’s also not the check of having others in the group be able to replicate your results. For the computationalists in the group, he generally trusts our results. To some extent in computation, the post-docs are our “quality control” mechanism. He’s more likely to ask the experimentalists to go back and redo an experiment if he thinks it “looks funny”. Because we do a pretty obscure measurement technique, it’s rare for us to have experimental post-docs who have a strong background in the technique.
I certainly think he could do more, for the sake of quality control. However, prior experiences have pretty thoroughly trained me to always verify my analysis, and he typically hires post-docs who are strongly interested in mentorship roles. So while things could be better, we’re muddling through.
I always expected my boss to be tougher than the meanest reviewers might be. I also expected my peers, especially co-authors to do likewise. I encouraged my undergraduates to ask questions too, and learn early on how to ask the right questions. It’s hard because students especially can be over zealous tactless, and feelings get hurt. I have snapped at people who causally suggest I repeat a lengthy experiment when I should have calmly argued my case as to why this was unnecessary. It is the responsibility of the group leader to ensure that pride and stress does not inhibit open scrutiny. It is the PIs responsibility to ensure that a grumpy postdoc can address the criticisms of a new graduate student (and does not scare them into silence by snapping at their unreasonable demands!).
I typically go over the primary data with the boss and then show it off in lab meeting too. We try to throw bricks at it to see if we can knock it down and then the rest of the lab gets to do the same when we present at lab meeting.
This is a useful question. In my currently postdoc-less lab, there is no consistent review of primary data. We each meet with the PI when we have new or interesting data, but rarely show primary data unless it seems pertinent. I probably show processed data (in the form of values derived from fitted curves, bar graphs, statistical analysis) about 80% of the time. Usually I only show primary data if it seems anomalous or confusing to me, or I think it’s particularly remarkable. The PI only requests primary data every now and again for a figure.
In fact, I’d say the situation in my lab would be pretty easy to exploit for data falsification as long as you had a few good pieces of “representative” primary data to use for figures. Kind of scary, and definitely something to think about when setting up oversight of others in the future.