[This is a copy of my blogpost, previously on thespectroscope.com.]
Everyone knows the destructive influence of the impact factor on academic research. This journal-citation metric is abused as a proxy for the quality of the published paper (see Mark Johnston's editorial and DORA). However, the impact factor maintains its grip and publishing in high-impact-factor (HIF) journals is commonly seen as a requirement for getting fellowships, faculty appointments, tenure, and funding. Academics may hate the torture of submitting their work to Science, Nature, Cell (SNC), but most believe that it is career suicide not to chase the glam publication. Most academics are wrong about this.
A recent research paper showed a very tight correlation between publishing in HIF journals and getting a faculty position. This report further cemented the illusion that one’s academic career is contingent on HIF publications. It is a classic case of mistaking correlation and causation. Surprisingly, the very people lecturing undergraduates and railing against journalists about confusing correlation with causation are entirely oblivious to this mistake in the case of the impact factor.
Michael Eisen has written a whole post on this in 2012, arguing that it is doing amazing science that lands people faculty appointments. That same ground-breaking work gives the glossy pubs, but it’s a mistake to think that the glossy pub is the cause of the hiring decision. He often mentions that four of his postdocs got faculty positions, not just without HIF papers, but before publishing their postdoctoral research altogether. Every time Michael mentions this, the response is, “Sure, it’s because they are coming from your lab. You started PLOS, you are HHMI, you are a superstar. Wouldn’t be possible out of any other lab.” So I e-mailed a few professors last night and posted the following question on Twitter:
Looking for folks who got faculty offers w/out their main first-author postdoc paper published. (Personally know 4-5, so must be many more.)
46 people are talking about this
Just overnight, I got a list of professors who got hired before publishing their work (see the bottom of this post). Of course, all of them subsequently published the work they presented at the job talks, and most often in the HIF journals. So if you look at the correlation between SNC and their appointments, it’s very high, but it happened after they got hired.
This list shows clearly that having a Nature paper is not a prerequisite for getting a faculty position. It also shows that having a Nature paper in no way guarantees that you’ll become a professor – for every person below who was hired without having published yet, there were probably 50-80 applicants with SNC pubs who did not get that job.
People are often shocked when I say it’s possible to get hired even before publishing. In response to my question last night, one student replied, “Do such unicorns exist?” They do, and it shouldn’t be that surprising. As James Fraser just told me, “It’s very obvious that publications are not the key criteria at good places - opening up a new field is!”
As soon as I publish this post, there will be a stream of comments that the people below are super-stars. They are, but considering the competition today, most people who get faculty positions are super-stars. Does chasing the HIF publication help your application? Possibly, but clearly not as much as most assume. And the chase itself is often time-consuming, draining, and destructive to the science itself. In fact, it is entirely possible that over-focusing on getting into Nature can actually hurt your prospects of becoming a professor. So my advice is to worry less about where you will publish the work and to focus more on the work itself.
--------------------------
Postdocs who got a faculty position before publishing their main postdoctoral work:
Angela DePace, Harvard
Sri Kosuri, UCLA
Andrew Hires, USC
Jeffrey Ross-Ibarra, UC Davis
Justin Fay, WashU
Mansi Srivastava, Harvard (Invited before acceptance, but paper accepted in Current Biology two days before the interview)
Aaron Baker, UT Austin
Jasper Rine, UC Berkeley
Mark Johnston, U. Colorado
Fred Winston, Harvard
David Schoppik, NYU Langone
Audrey Gasch, U. Wisconsin-Madison
Arjun Raj, U. Penn
Daniel MacArthur, MGH/Harvard
David Baltrus, U. Arizona
Katie Wilkinson, SJSU
Aaron Meyer, MIT
Sagar Khare, Rutgers
Rebecca Williams, U. of Hull
Nicola Allen, Salk (waiting for confirmation from Nicola)
Erica Bree Rosenblum, UC Berkeley (initial faculty position at U. Idaho)
James Fraser, UCSF
Dan Needleman, Harvard
Bret Payseur, U. Wisconsin-Madison
Sri Kosuri, UCLA
Andrew Hires, USC
Jeffrey Ross-Ibarra, UC Davis
Justin Fay, WashU
Mansi Srivastava, Harvard (Invited before acceptance, but paper accepted in Current Biology two days before the interview)
Aaron Baker, UT Austin
Jasper Rine, UC Berkeley
Mark Johnston, U. Colorado
Fred Winston, Harvard
David Schoppik, NYU Langone
Audrey Gasch, U. Wisconsin-Madison
Arjun Raj, U. Penn
Daniel MacArthur, MGH/Harvard
David Baltrus, U. Arizona
Katie Wilkinson, SJSU
Aaron Meyer, MIT
Sagar Khare, Rutgers
Rebecca Williams, U. of Hull
Nicola Allen, Salk (waiting for confirmation from Nicola)
Erica Bree Rosenblum, UC Berkeley (initial faculty position at U. Idaho)
James Fraser, UCSF
Dan Needleman, Harvard
Bret Payseur, U. Wisconsin-Madison
(The above is clearly a tiny subset, biased toward my circle of genetics/genomics folks. I will continue adding names below, so please share other names or your own and e-mail lenny at zappylab dot com.)
Amy Brock, UT Austin
Sean Eddy, HHMI Janelia
Jason H. Moore, Dartmouth
Albert Cardona, HHMI Janelia
Marta Zlatic, HHMI Janelia
Janna Fierst, U. Alabama
Daniel Pollard, Western Washington Univ.
Harpreet Singh, Drexel Univ
Ian Holmes, UC Berkeley
Aziz Aboobaker, Oxford
Matt MacManes, U. New Hampshire
Robert Bradley, U. Washington/Fred Hutch
Lior Pachter, UC Berkeley
Graham Coop UC Davis
Ran Blekhman, U. Minnesota
Melissa Wilson Sayres, Arizona State Univ.
Sean Eddy, HHMI Janelia
Jason H. Moore, Dartmouth
Albert Cardona, HHMI Janelia
Marta Zlatic, HHMI Janelia
Janna Fierst, U. Alabama
Daniel Pollard, Western Washington Univ.
Harpreet Singh, Drexel Univ
Ian Holmes, UC Berkeley
Aziz Aboobaker, Oxford
Matt MacManes, U. New Hampshire
Robert Bradley, U. Washington/Fred Hutch
Lior Pachter, UC Berkeley
Graham Coop UC Davis
Ran Blekhman, U. Minnesota
Melissa Wilson Sayres, Arizona State Univ.
(New names, added January 2018)
Cynthia Wolberger, Johns Hopkins
Scott Keeney, Memorial Sloan Kettering Cancer Center
Arash Komeili, UC Berkeley
Peter Bieling, Max Planck
Scott Keeney, Memorial Sloan Kettering Cancer Center
Arash Komeili, UC Berkeley
Peter Bieling, Max Planck
--------
[Comments below]
Tron Carter
4 years ago
Sorry not a well put togehter argument. These people still have had to pruse glam pubs to establish career/get funding/tenure. "Do good science" ? What does that even mean? Face it, you have to show the potential to bring in the pubs and funding if you wnat to get hired.
Reply
Hi Lenny,
I really enjoyed this post! Maybe I missed this in the post (and I haven't combed through all the comments yet), but how do these post-docs with unpublished (or published in a non-HIF journal) great science stack up against post-docs who also have done great science and have already published in HIF journals? Do people from the former group get hired at a rate similar to or greater than the latter? Thanks!
I really enjoyed this post! Maybe I missed this in the post (and I haven't combed through all the comments yet), but how do these post-docs with unpublished (or published in a non-HIF journal) great science stack up against post-docs who also have done great science and have already published in HIF journals? Do people from the former group get hired at a rate similar to or greater than the latter? Thanks!
Hi Lenny,
I'm not sure this shows what you want it to show. To me, it shows that doing your post-doc in an HHMI/Wellcome/Nobel lab vastly increases your likelihood of success EVEN IF you havn't published in NCS.
I put together a spreadsheet annotating the Post-Doc lab of each of the folks listed above. All told, I saw -
- Famous advisor (ie., did post-doc in HHMI/Wellcome/Nobel lab) : 12
- Non-famous advisor (ie., did post-doc in non HHHMI /Wellcome/Nobel lab): 21
- Weird inclusion (eg. previous generation or not a biologist) : 4
I was pretty generous in putting people in the non-famous category. Making some assumptions about the total number of post-docs etc., this gives you an odds ratio of p( job without NCS | famous lab) / p( job without NCS | not famous lab) = 6.3.
The uncharitable way of reading this is to say that, when you control for having similar publication records (ie., non-NCS) you are 6x more likely to get a job if you come from a "famous lab".
So, I think that this DOES NOT show that doing science is the only thing that matters. It shows that the fame of your advisor does matter - rather a lot. As someone who did my Ph.D. in an HHMI lab and turned down HHMI post-docs because I was sick of the politics, I can only say - "ugh".
-A
Spreadsheet here:
https://docs.google.com/spreadsheets/d/1yC0r3GCHhQ3DLzMqiM0x-Th7cJ38RT3kKdjAIIpID6c/pubhtml
I'm not sure this shows what you want it to show. To me, it shows that doing your post-doc in an HHMI/Wellcome/Nobel lab vastly increases your likelihood of success EVEN IF you havn't published in NCS.
I put together a spreadsheet annotating the Post-Doc lab of each of the folks listed above. All told, I saw -
- Famous advisor (ie., did post-doc in HHMI/Wellcome/Nobel lab) : 12
- Non-famous advisor (ie., did post-doc in non HHHMI /Wellcome/Nobel lab): 21
- Weird inclusion (eg. previous generation or not a biologist) : 4
I was pretty generous in putting people in the non-famous category. Making some assumptions about the total number of post-docs etc., this gives you an odds ratio of p( job without NCS | famous lab) / p( job without NCS | not famous lab) = 6.3.
The uncharitable way of reading this is to say that, when you control for having similar publication records (ie., non-NCS) you are 6x more likely to get a job if you come from a "famous lab".
So, I think that this DOES NOT show that doing science is the only thing that matters. It shows that the fame of your advisor does matter - rather a lot. As someone who did my Ph.D. in an HHMI lab and turned down HHMI post-docs because I was sick of the politics, I can only say - "ugh".
-A
Spreadsheet here:
https://docs.google.com/spreadsheets/d/1yC0r3GCHhQ3DLzMqiM0x-Th7cJ38RT3kKdjAIIpID6c/pubhtml
Great post Lenny!
Some thoughts:
http://rajlaboratory.blogspot.com/2014/11/why-are-papers-important-for-getting.html
Some thoughts:
http://rajlaboratory.blogspot.com/2014/11/why-are-papers-important-for-getting.html
nice post and good message to convey to post docs on the search for positions. IN my case, I did not puclish yet, but got already invited for interviews in the EU. and yes, I heard, the impact of pubications is stronger, but the good institutes look more on the person. and you can always hand in your manuscript to show the quality of your science.
Reply 1
Yeah, it's possible, but who is going to bet their career on something so unlikely? Wherever I've been, review panels, faculty council meeting, informal meetings, you name it, *without exception* if someone wants to describe another scientist, they describe where that person published. Any other concept is completely alien to all but a few:
http://blogarchive.brembs.net/comment-n911.html
You're list is sort of like saying: "I know people who got a job despite only working from 9-5." Of course it happens, but do you really want to bet your employment on something so unlikely? I wouldn't.
P.S.: Of course it goes without saying that a GlamMag pub doesn't come close to guarantee a job - necessary, but not sufficient. I have some top publications but still nearly didn't make it.
No idea why it says anonymous: Björn Brembs
http://blogarchive.brembs.net/comment-n911.html
You're list is sort of like saying: "I know people who got a job despite only working from 9-5." Of course it happens, but do you really want to bet your employment on something so unlikely? I wouldn't.
P.S.: Of course it goes without saying that a GlamMag pub doesn't come close to guarantee a job - necessary, but not sufficient. I have some top publications but still nearly didn't make it.
No idea why it says anonymous: Björn Brembs
Reply 1
Björn, you opposed the use of journal rank as a sole criterion. Maybe this silver lining on the horizon is the harbinger of the sunrise.
I agree with you that at present the journal rank is very influential in many, albeit not all, cases. We do not have the data to objectively quantify the distribution of the influence of indirect/surrogate measures. Still, the hope is that attitudes like yours during the site visit in Cologne will increase the fraction of cases in which scientists rely on their own critical evaluation of the scientific results, not their external and often distorted reflections. Keep fighting !
I agree with you that at present the journal rank is very influential in many, albeit not all, cases. We do not have the data to objectively quantify the distribution of the influence of indirect/surrogate measures. Still, the hope is that attitudes like yours during the site visit in Cologne will increase the fraction of cases in which scientists rely on their own critical evaluation of the scientific results, not their external and often distorted reflections. Keep fighting !
Reply 1
Thank you for this refreshing post Lenny! I remember having a related discussion 2 years ago. I think it is quite clear that the visibility of the journals in which the papers are published is a factor. The question is how important this factor is. My opinion is that the importance of this factor depends a lot on the specific situation and varies a lot from case to case, from 0 to 99 %. Recently, I received a very formal ranking/evaluation of my faculty application based on bibliometric measures. In other cases, I know of tenure track offers that were based almost exclusively on the support of the post-doctoral mentor (including one on your list). It is hard to generalize. I am very glad that you remind us of what should matter even if it is not always what matters for receiving an offer.
Reply 2
Thanks Lenny. I'm reminded of a couple a paragraphs from a book about MBAs I once partly read, where I think the author's words about the top business schools transfers to this discussion. ScienceNatureCell may accept good scientists, but they don't make them.
“Top business schools are notoriously hard to get into—the programs can afford to be picky because of their reputations. It’s circular. The reputation of a business school is built on the success of its graduates, so the top schools only admit those students intelligent and ambitious enough to make it through the rigorous selection process—the ones who are already likely to succeed, MBA or no MBA. Business schools don’t create successful people. They simply accept them, then take credit for their success.'
[Excerpt From: Kaufman, Josh. “The Personal MBA.” PENGUIN group, 2011-03-28]
- Dan @djjmorgan
- Dan @djjmorgan
