Monday, January 15, 2018

Significance versus explanation

Next time I have to teach the difference between explanation and significance, this will be my go-to example: "Scientific aptitude better explains poor responses to teaching of evolution than psychological conflicts" by Mead et al. in Nature Ecology & Evolution.

The article has several figures that look like this:
The P-value looks amazing (9 x 10^-15), but the spread is equally large. I would not even want to guess what the R-square of that relationship is. The authors in the figure legend also specify that:
"The regression line is the best-fit line of y predicted by x. However, as assumptions of linear regression are not fully met it is provided for illustrative purposes alone to indicate the trend."
Good point, because without that line it would be impossible to tell what the relationship would be, which is always a dire sign. But those two sentences should be part of a master class in scientific writing: Necessary, to the point, anticipating a reader's needs and confusion, packaged in a short and succinct statement.

This study works perfectly well in a hypothetico-deductive framework with well-laid out hypotheses, logical predictions, and strong and convincing statistical tests. It also leaves the reader (or some of the readers, for instance me) with this "but is it really important?" question. I could not find any mention of this variability issue, and only lots of highly significant p-values.  What is actually the best explanation of these students' evolutionary understanding?

Monday, January 8, 2018

Let's talk about mental health

Today. Tomorrow. Every day. Because it requires attention year round. 

And the start of the semester is an appropriate time to bring up this study by Levecque and co-authors, covered in Science, that focuses on graduate students. It quantifies the problem, but also identified consistent predictors of mental health issues: 

  • work-life conflicts
  • high job demands
  • low job control 
The story I come up with based on these results is that low job controls leads to (a perception of) high job demands, to work-life conflicts, to mental health symptoms. And what could be related to low job control and/or high job demands? Maybe supervisor style, which can either diminish the prevalence of mental health issues when it is perceived as inspirational, or increase when it is perceived as laissez-faire. This again stresses the crucial importance of the supervisor-graduate student relationship, one of the recurrent themes in these news items. 

Another recurrent theme in these news items is the importance of reasonable job expectations, potentially non-academic ones. It is thus maybe not a coincidence that "Positive perception of career outside academia" has a strong positive effect on graduate mental health. If you know that all the hard work will eventually lead to a satisfying job that requires this education, and not necessarily a low probability faculty position after an endless series of postdocs, maybe that makes all the hard work worth it.

Friday, September 15, 2017

Career preparation for graduate students.

This is an important issue, and I have already spent 4 news items on this. A recent publication in PLOS ONE investigated some of the strategies and resources PhD students (and postdocs) use for non-academic jobs. This is very important because more than 80% of our graduate students end in these non-academic jobs (although these numbers also includes MSc students). I should not have been surprised, but there are sociological theories and active research programs around this issue, resulting in publications where I am only comfortable reading the abstract and discussion, and hoping that the reviewers did their job. The crux from the results section to me was this section:

Interestingly, our results also show that a trainee’s perception of high program support for career goals had direct and significant effects on their career development search efficacy, while perceived advisor support was not significant at all. It may be that perceived program support for career goals enables trainees to develop a broader support base within their graduate programs (as opposed to being driven by dissatisfaction with one’s advisor).
This presents a glass half full/empty perspective on career preparation: a grad student readiness does not depend on an advisor's experience that will vary wildly between faculty members, but more on the support provided by the department/college/university. And since universities are explicitly tasked to prepare their graduates for the job market, there is some incentive to address any shortcoming institutionally. The question, of course, still remains if Integrative Biology and the university as a whole provides sufficient support. So is your glass half full, or half empty? Let me know.

Thursday, August 24, 2017

Reviewers should (also) be paid by publishers

Stephen Heard touched on a controversial issue in his latest blog post: Can we stop saying reviewers are unpaid? This (contrarian?) point of view resulted in some push-back in the comments, and on other blogs (e.g. Let's keep saying it, and say it louder: REVIEWERS ARE UNPAID by Mick Watson). I agree with most of Stephen Heard's points, and also with his updated point "publishers (mostly) don't pay for reviews". Where I disagree with him (there has to be at least one disagreement, because why otherwise waste digital bandwidth on writing this?), is that "reviewers are (mostly) paid" is actually an important admission.

I struggled to find a useful analogy to explain my unease with his admission, and it suddenly dawned on me. Several years ago, I read a blog post by Alex Bond on Why volunteer field techs are a bad idea. I hope I summarize their (further developed with Auriel Fournier in a published opinion piece that received a lot of pushback) arguments correctly with these points:

  • it is an essential part of the research process, so it should be rewarded accordingly;
  • not paying reduces the field tech's value, and thus "the professionalism of science as a whole";
  • prevents underprivileged scientists from participating;
  • financial restrictions, tradition, and CV building for the field techs are not good enough justifications in light of these criticisms.
The analogy with peer review is immediately obvious in the first line: it is an essential part of the scientific process, so it should be rewarded. Stephen Heard's argument is that reviewers are being rewarded (paid). What the analogy with volunteer field techs exposes, though, is that it is important to look at who benefits from the work. In the field tech case, it is first the scientist who got funding for the study that benefits from the volunteer work, and then in a second step science as a whole. In the peer review case, what I think Stephen Heard overlooks is that publishers benefit first, and science as a whole only in a second step. 

And this benefit to publishers is important. A recent Science paper by Gretchen Vogel and Kai Kupferschmidt report on a quick back-of-the-envelop type of calculation:
"Collectively, the world's academic libraries pay some €7.6 billion in subscription fees for access to between 1.5 million and 2 million new papers annually, or between €3800 and €5000 per paper, according to an estimate by the Max Planck Society."
While the net benefit to the publishers will be lower, of course, nobody can argue that publishers do not benefit directly from the reviewers. And publishers do not pay reviewers, directly or indirectly. Yes, society as whole pays scientists partly for their many contributions, but eliminating the publishers from that argument is wrong. If we make the analogy with the volunteer field tech case again, that would be similar to imagining a PI flush with money saying that she will not pay her field tech because the "pay" provided by the experience will improve his CV and his changes for a scholarship or job. Or more succinctly: Volunteer field techs are (mostly) paid.

If that above scenario raises some problems, then arguing that reviewers are (mostly) paid, is not the "right" statement to ponder. If you agree that in the above scenario that the field tech should receive monetary compensation for his contribution (or be paid) from the PI, and that that is the important debate, then I think that "reviewers should (also) be paid by publishers" is the more important debate to have.

Tuesday, August 15, 2017

Science and passion

Since starting these news items, we have discussed themes related to mental health issues, their causes, and some solutions. Ardon Schnorr, a Biology PhD student at Carnegie Mellon University wrote a very powerful article around this theme that touches on a lot of issues surrounding this problem: It is a must read for any graduate student. And if you are inspired by the positive effects of science communication, but do not know where to start, you can always start with the University of Guelph Let's Talk Science outreach program.