Honesty, diplomacy, independence, and solidarity in public discussions about science.

At DrugMonkey, PhysioProf explores the rules of engagement between grad students in journal club and seminar presentations (building off of interesting explorations of this question from A Lady Scientist, Dr. Jekyll & Mrs. Hyde, and Acmegirl — all of which you should click through to read in their entirety). I’m late to the party, but I wanted to share some thoughts on the balance here between the intellectual aspects and the human aspects of questioning within the tribe of science.

PhysioProf starts with A Lady Scientist’s sense that grad students have a duty to watch each other’s backs:

Students don’t tend to ask questions at these Journal Clubs. In fact, I think that the prevailing sentiment is that we’re supposed to go easy on the students, because if we were up there wouldn’t we want the same consideration? So, easy questions (eg. Can you define that negative control?) are ok, but the hard ones (eg. Those controls are very off. Can you still interpret the data?) are not. In other words, we, as grad students, should show some solidarity and help each other out in not looking stupid in front of the department. So, I feel like I’ve doubly hurt someone because not only did I ask a bothersome question, but it came from an unexpected source.

For those unfamiliar with the practice, a Journal Club usually involves a group of grad students and/or faculty in a sort of reading group focused on new or important articles in one’s field. Sometimes everyone in the Journal Club will read the article to be discussed and then the group will work together to understand and critique it. In other cases, one person takes primary responsibility for understanding the article and presenting its methodology and claims (along with critiques of these) to the larger group. Typically, Journal Clubs function as a way to divide the labor of keeping up with the literature in a meaningful way (i.e., because you’re not just running your eyeballs over the words and the graphs but meaningfully engaging with the scientific claims) and as a way to model (for the scientific trainees participating) good engagement with the literature and with other scientists in one’s field.
(There are other arenas for questions and answers involving both grad students and faculty, including group meetings within one’s research group and seminars presented to the entire department — either of which may focus on discussions of one’s own research.)
With that in mind, is it a good think for grad students in Journal Clubs to prioritize solidarity over asking incisive questions? PhysioProf thinks not:

The entire essence of science–what defines it as a profession–is that scientists ask all questions that present themselves, either of themselves or of others. …
When you fail to ask a question, or raise a criticism, based on some misguided sense of “loyalty” or “solidarity”, you are actively harming the scientist you think you are protecting. Because someone somewhere will eventually ask the question–a paper reviewer, a grant reviewer, a thesis committee member, a job search committee member, a job seminar audience member–and the sooner the issue gets raised, the sooner the scientist can address it.

PhysioProf is harking back to the norm of organized skepticism that is supposed to characterize scientific activity. For any claim someone advances, scientists are supposed to poke it and prod it and see if it holds up to scrutiny. To the extent that such skepticism is a habit of mind that needs to be cultivated, a Journal Club is a good opportunity for its cultivation. Usually, the article you are presenting to the group is an account of someone else’s research — which means that as the presenter, you don’t need to get defensive about the methodology, the data or data analysis, or the conclusions drawn from them. You just have to try to understand them, and coming up with the questions you would ask the authors to better understand the research really is added value on top of presenting what the authors say in the paper. That other participants in the Journal Club may come up with incisive questions of their own about the work simply reinforces why it’s important that science is done by communities — a group of people with different viewpoints and assumptions will be harder to convince than any single scientist.
So, from the point of view of building the skills one needs as a scientist, it seems like real solidarity with one’s fellow grad student might involve making an extra effort to ask hard questions of each other. Given that, in the context of a Journal Club, these are questions about someone else’s research, this reduces the expectation that the presenter can fully answer them off the top of her head. However, she can probably work out a strategy for coming to a sensible answer. If she can’t, she could call on the group for suggestions for a useful plan of attack. Being able to think on one’s feet, being able to work out strategies for solving problems different from the ones one has solved before, and being able to ask for help without falling into an utter panic are all useful skills for the working scientist. If graduate students are serious about making the transition from trainee to grown-up scientist, they have an interest in developing these skills. Solidarity with one’s fellow grad students means taking their interests seriously, too — including their interests in developing the skills they’ll need to succeed in science.
But commenter Becca argues that this approach overlooks the reality of interpersonal dynamics, especially between trainees and their advisors:

You are absolutely correct that questions are critical to scientific advancement. And you are absolutely correct that many in the scientific culture love the back-and-forths of a good analytical debate… as long as it does not consist of your trainee arguing with you, in front of other PIs.
Look, if possible, I love debate even more than I love science. But scientific presentations are *not* a conducive environment for real discussion and debate… there’s too much status-hunting and image-consciousness. It is great to pretend that it’s all about the ideas and making them better. But it is really every bit as much about one-ups-manship, ego, currying favor, and all manner of human political baggage.

Becca’s concern here seems to be that scientific presentations afford not just an opportunity to evaluate a piece of science (in terms of its methodology, the results presented, the conclusions drawn, etc.) but also an opportunity to evaluate other scientists.
Does the person presenting know her stuff? Is she thoroughly prepared? Can she think on her feet? Has she anticipated the killer questions about this piece of work? Is the person asking the killer question exposing a flaw with the research, or with the competence of the person presenting it? Is the questioner a faculty member cutting the grad student who is presenting down to size? Is the questioner a grad student who is taking shots at a PI’s presumed expertise? Is the questioner a grad student establishing his own awesomeness by making a fellow grad student look weak?
It’s true that, especially for grad students, it can seem as though every interaction with faculty (and even with other grad students) is one where folks are watching you and keeping score. Who is the smartest? Who is the weakest? Will this question help me rise in the rankings, or knock me down? The individual status hunt does feel like a distraction from the intellectual project of doing science (and of learning how to do science). After all, built into norms like organized skepticism and universalism is the notion that scientists are supposed to evaluate the content of each and every scientific claim rather than simply accepting them on the authority of the person putting them forward. Accomplished scientists with impressive track records can be wrong. Moreover, as scientists, their highest commitment is supposed to be getting their pictures of the world they’re studying right (or as close to right as possible) — and this requires finding out when they’ve gone wrong. Concealing this information so as not to hurt a scientist’s feelings instead hurts progress on the knowledge-building project.
But there do exist PIs who get angry when trainees point out flaws in their scientific reasoning. Sometimes the anger stems from the way the errors are pointed out (e.g., a “gotcha!” clearly intended to deflate the more senior scientist), but sometimes the PI would be angry regardless of how diplomatically the question was asked. Some PIs really are invested in seeing themselves as smarter than everyone else. (To be fair, some trainees are like this, too.) When presented with evidence that they are not infallible, they may try to punish the person presenting this evidence.
Needless to say, such behavior is problematic.
Grad students are well aware of power imbalances in their relations with PIs. Poking someone who is protective of his ego and who is well placed to crush you like a bug is not a good idea. Similarly, if you are aware that certain PIs like crushing grad students for sport the moment any vulnerability is exposed, you may decide it’s a bad idea to point out your fellow grad students’ mistakes in front of them and to provide them with fresh meat.
It’s a useful skill, after all, to identify scientists who seem not to be playing with the same rules of engagement as most of the rest of the community.
There’s a place here for faculty members to show some solidarity — not with their faculty peers with the too-tender egos, but rather with the trainees who are working to enter the community of grown-up scientists. After all, does it help a scientific field get closer to good accounts of their phenomena to have to tiptoe around a PI’s deep attachment to his hypotheses or results, rather than being able to discuss flaws in them openly? Arguably, it does not. Is it better for the continued health of your professional community to train people to focus on the scientific challenges or on interpersonal one-upsmanship? One would think it is the former.
While, in a Journal Club, grad students may most acutely aware of their interactions with other grad students, and of the interactions between grad students and faculty members, these public spaces for scientific communication also involve interactions between faculty members. Sometimes the exchange will include a faculty member stepping up in the conversation to assist (whether directly or obliquely) a grad student who is being attacked. Sometimes the exchange will even include faculty members taking each other to task for not engaging with the substance.
Certainly, there is room for grown-up scientists to remind each other that the organized skepticism of the endeavor should be directed toward the scientific claims themselves — that torturing grad students for fun, or even just to protect your ego, is not responsible behavior on the part of a scientist. Here, I think venues like Journal Clubs and departmental seminars can be importantly different from group meetings, since faculty member interaction are conducted in front of their peers, rather than simply in front of the grad students in their power.
To a certain extent, I see grad student mutual-assistance pacts as a strategy for opting out of gratuitous bloodletting. Still, such pacts become problematic if they keep grad students from figuring out how to engage in effective dissection of claims — especially their own — in a public setting. The surest way to avoid saying something stupid may be never to open your mouth, but this comes at the cost of being able to participate in the ongoing dialogue that is central to the scientific endeavor. Being completely safe isn’t an option for scientific grown-ups, so trainees need to work out reasonable ways to help each other in their brushes with danger.