Monday, November 10, 2014

Yes, papers are important for getting faculty positions

Arjun Raj has written a good post on why publishing papers is important for getting a faculty job. It is in response to my argument that having a Science/Nature/Cell (SNC) publication is not nearly as important for getting a professor job as most people think (I published a list of professors who got job offers before publishing their main postdoctoral work).

I am in violent agreement with Arjun on much of what he wrote, and disagree just as strongly with other parts of it.

Agree
"I definitely feel like my job search might have been easier with a published paper, especially in biology/medical departments. And I have definitely heard of places, for example in other countries, in which applicants have been explicitly told that the job is theirs if and only if their postdoc paper is accepted."
I never meant to suggest that publishing does not help to get a faculty job. Reading your manuscripts allows search committees to assess how you communicate your science, how you write, and how you think - all important for trying to predict whether you will be able to do good science, get funding, teach, and get others interested in your research - factors crucial for the hiring decisions, as Arjun points out. A university would be a crazy risk-taker to hire someone who has never published at all.

Disagree

"If the search committee understands the work and the researcher and believes in them both, then why does the existence of an accepted high profile paper matter so much in and of itself? A big part of the answer is that visibility matters... Having a high profile paper when you start is undeniably a part of the answer to these questions. And it’s also a simple metric of success that is readily interpreted by people across disciplines."

While publishing your work is clearly important and helpful for assessing you as a researcher, using the journal/impact factor as a proxy is the opposite of good assessment. If instead of reading your papers, the committee invites you based on the name of the journal where you published, they are using a wrong and lazy metric. Your paper in Nature may be great or terrible, and it's impossible to say which one it is, without reading the paper itself.

Faculty searches are complicated with a million factors that play a role. Hope Jahren wrote a terrific comment on how faculty searches work, describing how unique each one is and that there is no formula or method that all universities or committees use. The list of faculty who got hired prior to publishing their postdoctoral discoveries makes it clear that having a paper in a fancy journal is not a prerequisite for getting a job.

Do publications help to get a job offer? Without a doubt. Does having your paper in a glam journal help? Yes, for some search committees it increases the chances you will be invited. But the correlation between hired faculty and glam pubs is driven by the quality of their work, as my list illustrates, rather than the name of the journal being the causal factor.

As I have commented already, "What I am arguing is that it's unclear if chasing the SNC paper helps or hurts your chances of becoming faculty, all other factors being equal. That chase - the rejections, rebuttals, resubmissions, followed by more rejections, and resubmitting to another glam journal - carries a huge cost. And if that costs you advances in your research, it's not inconceivable that this very chase will hurt your chances of getting a faculty job."

I'd love to see data on whether chasing the high-impact-factor publication helps or hurts postdocs.

4 comments:

  1. Hi Lenny,

    I agree that assessing *just* based on where a paper is published is silly, and no good committee hires just based on that–as I was saying, EVERYTHING matters. Indeed, as your post points out, many have gotten jobs without that high profile paper already in the bag. What I am saying is that in cases where they do want a high profile paper (as is the case at some departments/universities and countries), the reason why is not because they are intellectually shallow, but rather that glam publications make the researcher more visible, both internally and externally. This makes it easier to get students, grants, and other subsequent papers, be they in fancy journals or not. And I'm sorry, but a paper in Cell is more visible (maybe not better, but more visible) than a paper in most other journals–to argue otherwise is disingenuous. Even when they hire someone who hasn't yet published their fancy postdoc work at a fancy journal, they are taking a calculated risk that the person will eventually get high visibility once they do publish their work, probably somewhere really good. Often, that risk is mitigated by the person already having high visibility from their grad school work or their (typically very visible) postdoc mentor advertising their work and promoting them heavily. Incidentally, this will cross a fair number of people off of your list.

    I agree that it's a bad thing for science that we covet publications in a few journals run by a small set of people, and I'm glad that people can get hired without these glam pubs. But it is simply naive to think that where you publish doesn't matter. In the long run, there are other ways to gauge the impact of a paper, but for postdocs, there is only the short run.

    Arjun

    ReplyDelete
    Replies
    1. Hi Arjun,


      You are making the reasonable argument in terms of visibility. Visibility is clearly very important. Visibility, however, certainly cannot explain all worship of prestigious magazines and journals. For one, a tenured professor at a top university is highly unlikely to lose visibility by publishing in PLOS Genetics, and yet this professor is afraid that such a publication will hurt their reputation.


      My points here are: (i) I agree, as a statement of fact, that visibility is currently closely associated with the publication venue; (ii) I am cautious of using visibility as a justification since at least in some cases, it does not seem to be the only factor. Ultimately, in the age of google we the people (we the scientists) determine what we read; yes, you can block that email with table of contents and featured advertised and look at other channels for new relevant research. It is up to us whether we are ruled by advertisement or we take seriously our scholarly duty to find and reference the most relevant and well substantiated research. For me, it is often not in the magazines.

      Delete
  2. I am not saying where you published "doesn't matter". I am saying it matters much less than most people think. And most mistake the correlation with causation in this case.

    Not too long ago, there was no Cell journal. Not too long ago, professors were hired without a postdoc or after a 1- or 2-year quick postdoc in which none of them published. Most scientists know the corrupting influence of the glam chase. They also know that most of their papers don't get into Cell. The professors on the hiring committees know that. And when you hire you are betting that the person will succeed as a scientist, not that he or she will get their postdoc research into Cell.

    I don't know how much chasing the glam pub hurts or helps. And I have yet to see any data that informs this. What is clear is that the correlation can be a simple byproduct of the way hiring committees and high-impact journals select, but it doesn't mean one is a prerequisite for the other.

    ReplyDelete
  3. I would be willing to spend the time to compile the data and analyse them if I could come up with a sensible methodology. The problem is that I do not know of a good quantitative metric for research assessment. So we are left with the outliers (exceptionally significant papers) that almost all agree are great but not suitable for this analysis because, well they are outliers, not obeying conventional rules. The vast majority of papers, on the other hand, are very hard to assess objectively and quantitatively to the satisfaction of all, especially if we strip away the hype. So we are left in the position that we cannot truly evaluate a claim but somehow this claim is very influential among scientists. Isn't this the antithesis of the scientific method?

    ReplyDelete