Thursday, November 20, 2014

Startups are children; please be kind to the parents

There is a reason why you often hear entrepreneurs say something like, "I have three kids - two daughters and a startup." It's because parenting and being a founder have countless similarities. They are not identical since a startup, unlike your toddler, isn't going to bite you. And despite the biting, you don't really want to sell your child, but eventually hope to do that with the startup. Still, the comparison resonates because kids and startups are both rewarding, exhausting, exhilarating and life-changing. You never stop thinking about them, you worry ALL the time, you panic, you don't sleep - and yet, despite it all, you still have another kid and start another company.

What puzzles me is how often people lash out at founders and criticize and question our motives and decisions. Why? I have thought for a long time of writing a blog about this, and a major Twitter storm yesterday that accused us of being EVIL finally gave me the needed push.

We have given up our jobs and careers, have made enormous sacrifices, have subjected our families to financial difficulties and worries because of the crazy drive to create protocols.io. Of course I want to make money on this. But the real promise here is not the lucrative exit but the opportunity to change science communication. It's the opportunity to save society billions of wasted dollars every year and to speed up research.

So when someone accuses us of trying to stifle innovation and science communication, it hurts much more than any random mean Twitter comment ever should. It reminds me of the person at the park who criticizes a mother giving a bottle to her infant, "Don't you know that breastfeeding is so much better for your kid?" Why the assumption that the mother is a bad parent who doesn't care about the kid? Maybe the mother's milk never came in or she had a mastectomy. Maybe she tried her best and after two months of excruciating pain had a nervous breakdown and switched to formula.

Parents and founders need help and support, not vitriolic criticism. And keep in mind that neither parents nor founders are experts. At ZappyLab, we have degrees in math, molecular biology, computer science and business, but none in parenting or startups. That's because there is no PhD for these. We are all amateurs who learn on the job. That means we are all trying our best and we are all making mistakes.

If you disagree with what we are doing, just consider that there may be valid non-nefarious reasons for our decisions. But if you are sure that what we are doing is a mistake due to lack of experience, please do reach out and advise us. We crave feedback and help. Most parents and founders will welcome the advice.

Monday, November 10, 2014

Yes, papers are important for getting faculty positions

Arjun Raj has written a good post on why publishing papers is important for getting a faculty job. It is in response to my argument that having a Science/Nature/Cell (SNC) publication is not nearly as important for getting a professor job as most people think (I published a list of professors who got job offers before publishing their main postdoctoral work).

I am in violent agreement with Arjun on much of what he wrote, and disagree just as strongly with other parts of it.

Agree
"I definitely feel like my job search might have been easier with a published paper, especially in biology/medical departments. And I have definitely heard of places, for example in other countries, in which applicants have been explicitly told that the job is theirs if and only if their postdoc paper is accepted."
I never meant to suggest that publishing does not help to get a faculty job. Reading your manuscripts allows search committees to assess how you communicate your science, how you write, and how you think - all important for trying to predict whether you will be able to do good science, get funding, teach, and get others interested in your research - factors crucial for the hiring decisions, as Arjun points out. A university would be a crazy risk-taker to hire someone who has never published at all.

Disagree

"If the search committee understands the work and the researcher and believes in them both, then why does the existence of an accepted high profile paper matter so much in and of itself? A big part of the answer is that visibility matters... Having a high profile paper when you start is undeniably a part of the answer to these questions. And it’s also a simple metric of success that is readily interpreted by people across disciplines."

While publishing your work is clearly important and helpful for assessing you as a researcher, using the journal/impact factor as a proxy is the opposite of good assessment. If instead of reading your papers, the committee invites you based on the name of the journal where you published, they are using a wrong and lazy metric. Your paper in Nature may be great or terrible, and it's impossible to say which one it is, without reading the paper itself.

Faculty searches are complicated with a million factors that play a role. Hope Jahren wrote a terrific comment on how faculty searches work, describing how unique each one is and that there is no formula or method that all universities or committees use. The list of faculty who got hired prior to publishing their postdoctoral discoveries makes it clear that having a paper in a fancy journal is not a prerequisite for getting a job.

Do publications help to get a job offer? Without a doubt. Does having your paper in a glam journal help? Yes, for some search committees it increases the chances you will be invited. But the correlation between hired faculty and glam pubs is driven by the quality of their work, as my list illustrates, rather than the name of the journal being the causal factor.

As I have commented already, "What I am arguing is that it's unclear if chasing the SNC paper helps or hurts your chances of becoming faculty, all other factors being equal. That chase - the rejections, rebuttals, resubmissions, followed by more rejections, and resubmitting to another glam journal - carries a huge cost. And if that costs you advances in your research, it's not inconceivable that this very chase will hurt your chances of getting a faculty job."

I'd love to see data on whether chasing the high-impact-factor publication helps or hurts postdocs.

Saturday, August 30, 2014

PhD is perfect for a startup

In my 2.5 years as a co-founder of ZappyLab, the most surprising aspect of entrepreneurship for me has been the realization that graduate academic training is just perfect for founding a startup. I can’t think of any other training program that could have better prepared me for the challenges of creating a company. The skills I acquired in the 6 years of PhD work are precisely the ones essential for a startup.

I will discuss in detail below the specific similarities of academia and startups; however, I first want to repost my answer to a postdoctoral researcher who asked if an MBA or some business classes could help to transition from academia to industry.

As a sixth-year postdoc, you have over a decade of training. You are more trained than effectively the entire world labor force.  

The last thing you need now is additional classes.You are an extremely valuable employee because you are a scientist. It’s not the pipetting skills or the protein you have been studying – it’s the fact that you are a scientist that makes you so valuable.  You know how to research, form hypotheses, test them, analyze and evaluate results, make quantitative conclusions, and communicate the results to others.

The problem we have is that we have not been exposed to non-academic careers. While good mentors are supportive of all career plans (see here), good mentors are rare.  We are trained to be professors, even though only a small fraction of the PhDs in life sciences will actually become faculty. Non-academic careers are called “alternative”, even though the numbers make it clear that it’s the professor job that is “alternative”.

Scientists simply don’t know that they are valuable outside of academia. How many know that intellectual property (patent) law firms will hire PhDs and will pay for them to go to law school? How many know that life science venture capital firms hire PhDs to evaluate proposals? How many scientists know that graduate training is PERFECT for founding a startup? Think about lab meetings and the qualifying exam – it’s all about presenting, communicating, and defending your ideas. That’s exactly the set of skills key in pitching to investors. And running a startup is all about experimenting, evaluating, and forming new hypotheses. There are so many levels on which academia and the startup world are similar, I could give a 2-hour talk on this! For the sake of brevity, I’ll just say that I know many startup founders, and not a single one of them has an MBA. Conversely, I know many MBAs, and not one of them has co-founded a startup.

And now for the concrete similarities.

Lab Meetings and Pitching to Investors

This may be the most valuable commonality. The qualifying exam, thesis committee meetings, and lab meetings sharpen the communication skills spectacularly. You learn to make effective presentation slides and to present to a small and hyper-critical audience. You need to master the non-aggressive control. Answer all questions and the lab meeting just got away from you; you did not present the key data and did not get advice on the parts you really care about. But you can’t be dismissive or aggressive. Same thing with the qual – answer everything without any control of the flow, and you never get past Aim 1. Most of all, these meetings are all about fielding questions. Just as with control of the tempo, this is an art. You need to convey confidence and defend your ideas and positions. Yet, you also need to be open to advice. If you are too dismissive, your colleagues will stop asking anything at your meetings, and that defeats the purpose of the entire exercise.

The above perfectly describes a meeting with an investor. These are strong and sharp personalities, just as the researchers you are used to. They want confident answers and a founder who has obviously thought through all contingencies. At the same time, they don’t want to be cut off and don’t like feeling stupid if you belittle their questions. They also want to make sure that if they invest, you are a type of founder who is open to advice and suggestions. Investors don’t like to think of themselves as just wallets; many will be getting a board seat, and if you don’t listen to anyone, they won’t want to be on your board and that means won’t want to invest in your company.


Handling Rejection - the Manuscript Reviewers and Venture Capitalists

Good results, getting your PhD, publication, fellowships, funding, faculty position, and tenure all have something in common – unpredictability. The only thing that is guaranteed in academia is Rejection. Getting your first manuscript rejection letter is a stunning and long-lasting level of pain. Pain killers don’t help, and the only remedy I know of is to lose yourself in reading famous rejection letters to Nobel Prize winners and authors like Nabokov

I never thought that I would one day view my rejections from every single postdoctoral fellowship in a positive light (interestingly, my postdoc proposals were really good scientifically but not funded; meanwhile, my graduate NSF proposal was terrible but I got it). The academic rejections seriously thickened my skin. Without this, I could not have handled the constant stream of “NOs” that every startup experiences (if you think getting a “yes” to a paper from an academic journal is hard, try getting a “yes” to a huge sum of money in an investment or a business deal).

And I certainly never thought that I would view my two manuscript rejections in a positive light. Well, I do. It gave me the opportunity to learn how to write a rebuttal. The rebuttals worked both times. The key with rebuttals, just as with presenting and fielding questions, is to strike the right tone. The rebuttal has to be strong, but it can’t be angry. You have to use the reviews and investor rejections to improve your manuscript/slide presentation/offer. Need to identify the weak parts of your communication, strengthen them, and eloquently and diplomatically explain why the reviewer/investor is wrong and why your paper deserves to be in the journal and why investing in your startup is in fact an opportunity the investor cannot miss.

Risk

As you embark on the PhD/startup, you are assuming a crazy level of risk. Years and years of effort, with hope but no guarantee of success. Miniscule financial compensation. Stress, self-doubt, and burnout. The one thing that drives you is the passion and belief in your project (passion and belief that is far from constant and often hits such lows that even you don’t understand why you are not dropping it).

Perseverance

Einstein, Ben Franklin, and Mark Twain have all been credited with the quote, “The definition of insanity is doing something over and over again and expecting a different result.” Regardless of the source, this remark brilliantly catches what my experimental research was like for a decade.
Whether you are in science or humanities, getting a PhD is extraordinarily hard. It requires years of a super-human level of commitment and perseverance. So does a startup.

Self-doubt

Whether male or female, it seems that almost all scientists suffer from the impostor syndrome. The self-doubt is persistent throughout the graduate, postdoctoral, and faculty appointments. But with each stage, you get better at taming this doubt and have a track record of overcoming challenges and gargantuan tasks like thesis-writing.

If you don’t learn to control the impostor syndrome, you can’t succeed in academia, and you certainly can’t succeed with a startup.

Research Proposals, business plans, and experiments

I often see people comparing writing a research proposal to a business plan. It is true that in both you make up rosy projections and the people reviewing summarily dismiss them. Beyond that, I disagree that this is a valid comparison.

I personally think business plans are a waste of time. Not a single savvy investor has asked to see our business plan. Y-combinator explicitly says it never reads them. Many startups, including ours, do end up writing one, but usually that’s because one of your angel investors or their spouse went to business school and asks for it, for no good reason.

A research proposal is very different from a business plan in that you outline a series of experiments. You have to formulate the experiments in a way that leads from one question to the next, keeping in mind that for each experiment, the outcome may be A or B. You can’t plan on “A” because “B” is just as likely. What if the hypothesis in the first section is wrong? How do you move on to the next section? This is the tricky and valuable part of writing a research grant proposal. And this is the part that is relevant for a startup – the setup of the experiments and the evaluation of your hypothesis.

One of the main things you learn as a scientist is how to properly test a hypothesis. This is key for startups. A startup itself is a research project. You are experimenting, adjusting based on user feedback and data, devising new experiments, and switching to a new project/pivoting if the first one does not work. Research is just as unpredictable as startups. And it is also important to know when to ignore your research plan. Some of the most interesting and important discoveries happen accidentally, just as many startups are founded to do one thing but end up switching and succeeding with an entirely different product.

Your PI, thesis committee, and the startup advisory board

When I give talks and advice on founding a startup, I think my most valuable suggestion is to get good advisors. Don’t worry, there is plenty of trial and error and you definitely will learn from your mistakes. But if you don’t get advice and try to make all the mistakes you are fated to make, instead of learning, you’ll just fail. Similarly, the role of the PI and the thesis committee and your labmates is to limit the number of mistakes you make, so that you graduate in 6 instead of 66 years. That is also why I have always stressed to students picking a graduate or postdoc lab not to underestimate the importance of a good advisor (see my recent "Your mentor can make or brake your academic career").


The second-most valuable suggestion I offer is to learn when to ignore the advice. Part of maturing as a PhD student involves becoming the expert on your thesis project. And at some point, instead of asking for guidance from your advisor, your job becomes guiding your advisor to understand and support your experiments and plans. As important as good mentors are to your success, it is equally important to learn that advisors like to advise and a huge portion of the advice they give you is wrong. At some point in your startup, you suddenly start drowning in conflicting advice from your investors, directors, advisors, friends, and users. If you listen to everything, you’ll just fail. Again, filtering this is an art that you pick up with time, and academia provides you with plenty of time and opportunity to become a master at this.

Thursday, August 7, 2014

The line between scientific discussion and a witch hunt.



In the past few days, I have received multiple emails arguing that the debate around STAP was normal discussion. On the other hand, PubPeer and Retraction Watch have come under attack as the agitators facilitating the witch hunts. Retraction Watch just responded. And another terrific post on this issue "It isn't the fraud witchhunt, it's the Glamour culture of science."

There is a difference between “peer review” and a “witch hunt”, and I think it is important to clarify this.

To me, it’s a simple line. Discussion of the science on Retraction Watch and PubPeer is not just "okay". It is essential for the progress of science. I am an ardent fan of post-publication peer review and particularly PubPeer. I wholeheartedly support investigations of misconduct and in no way consider them witch hunts. I found it important to connect PubChase and RetractionWatch.

If you cannot tolerate criticism and post-publication scrutiny or criticism of your work, science may not be the right place for you.

What unnerved me over the last few months about the STAP discussions is not the efforts to reproduce or the investigation. I want more rather than less post-publication discussion. What I am calling for is caution in assigning blame, guilt, and allegations of misconduct and fraud in the court of public opinion. Media, blogs, Twitter, and private discussions of the STAP work have been full of heated rhetoric, mostly based on assumptions, and often without realizing the damaging consequences of the vitriol. Out of sensitivity, in light of Dr. Sasai’s death, I will not mention any specific posts or individuals.

I draw the line at discussions of Dr. Sasai that focus on “he was full of himself” or “was not a researcher any more and only thought about fame” or “did not have the time to supervise his lab, so of course this happened” and so on. These assign blame. They say that he could have and should have detected problems and is therefore responsible. If you don’t have internal knowledge of RIKEN CDB and Dr. Sasai, if you are not on the investigating committee – don’t blog these thoughts. You don’t know what happened.

I draw the line at blog posts arguing that Dr. Obokata must have done this intentionally; at blog posts saying that her citation count is low and Dr. Sasai should not have promoted her. At articles focusing on Dr. Obokata’s “lawyering up”. Do you know Dr. Obokata? If not, you don’t know why she did what she did, you don’t know whether it was misconduct or genuine error. Have you been in her shoes? If not, you have no idea why she has an attorney.

My line is very simple – focus on the figures and the methods. Not on the intentions and accusations of misconduct and assigning blame. Leave that to the investigating committees.
------------------
And a few post scriptum thoughts.

1. Retraction Watch has been accused of witch hunts mostly because of the comments that appear under their posts. People, it’s the internet! Have you seen comments on YouTube? If we argue against RW because of the comments, we need to shut the internet down.

2. (This is from my e-mail to a professor who argues that Sasai should have never published it and deserves the blame for missing such poor science.)

On a personal note, I should say that I came across blatant fabrication once in my 13 years in science. It was another student in the lab who had incredible imaging results with a paper that would win all top prizes. Then the hard drive was stolen and all of the images were on it without backup. Then the student lost interest in the project. 

Of course, the professor, couldn't let it go. He put another student on the project. The student couldn't reproduce anything. He put a postdoc on it. Postdoc couldn't reproduce. It took a year. The original student kept claiming it is reproducible. Every time the person did the imaging, it would be at midnight with no one else nearby. Refused to have anyone watch.

I observed this from the side, incredulous. Finally, I felt so bad for everyone else, I walked into the professor’s office and asked him how certain he was that the data were real. I made an explicit accusation of fraud. I said I did not think there ever were any results. The professor was stunned. Couldn't believe it. 

Soon after, the student got on a plane with the PhD in hand and went back to the home country. My professor persisted for a few more months, wasting time of yet another postdoc. Finally gave up. It's been years, and no more talk of this research.

This was a very eye-opening experience. The professor is an amazing scientist. Simply brilliant. Yet, somehow, he just couldn't see the blatant fraud in front of him. It was so obvious. Red flags waving as if it was Soviet Union. But he was as colorblind as it gets. 

Luckily, this student pulled back. But had the student persisted and finished the story, I am 100% sure it would have been published. As long as you trust the people in your lab, you just can't detect fraud.

Not being from the Sasai lab, I would hesitate before making any statements of what he should or should not have done/detected.


Tuesday, July 15, 2014

Who will train tomorrow's scientists?

 In 2009, when I was graduating and about to embark on my postdoc, I was concerned about the research funding lottery and worried how I could continue on the academic track with such prospects. My co-adviser Jasper Rine confirmed that we were in a crisis, but he said, “NIH only acts in times of crisis.” It was a good time to do a postdoc, and by the time I would embark on my faculty career, things would have improved.

Of course, in retrospect, 2009 feels like a pre-problem era. The funding situation today is best summed up by yesterday's: "Congress: NIH Funding To Be Distributed Via Cage Fighting" with this brilliant nugget:


"My fellow Americans, today marks an important milestone in our collective history," President Obama began, "with the enacting of this bill, federal funding for health-related research will be distributed in the fairest way possible -- by physical combat."

For as long as I remember from the start of graduate school in 2003, depressing reports on the prospects for PhDs kept surfacing (good examples: TheScientist 2006Science 2008Economist 2010, an entire special issue in Nature 2011, and a famous blogger in 2013 saying that it's immoral to hire PhDs). In February, my post “Goodbye Academia”, warning of a crisis in academic biomedical research, drew over 100,000 readers. Countless scientists commented, confirming the concerns, and I learned of a whole genre called QuitLit of researchers publicly leaving academia (ever-growing list of the stories here). A few days after my post, Keith Yamamoto, Vice Chancellor of Research at UCSF published a video, arguing that we should not be sending graduates to postdoc positions. Then in April, the titans of the US bioscience world published “Rescuing US biomedical research from its systemic flaws”.

I was somewhat encouraged, hoping that NIH would finally recognize the extent of the crisis and avert disaster. Alas, it is clear to me today that despite all the warnings, the vast majority of academics, and certainly the head of NIH, are going to pretend as though there is no problem, for as long as they can. Francis Collins, the director of NIH, just stated that he does not believe we are training too many PhDs.

Figure 1: New faculty positions versus new PhDs.

 Schillebeeckx M., et al. Nature Biotech. 31:938 (2013)

In my conversations with scientists, the waving away of the problem is consistently justified with the following two answers:

1. Everything is uncertain. No employment guarantees in industry either.

2. There are plenty of non-academic jobs, and training PhDs is not vocational training for professorship.

Everything is uncertain. You can lose your job in industry too.

This is not a valid comparison. Of course no employment is guaranteed. However, if I lose my job at Novartis, I can probably find another one at Merck. That’s not the same as losing funding for your lab – you lose funding at MIT, you don’t just move to Stanford. You lose your funding, and you lose your lab. It’s not a job; it’s your entire career that is at risk.

There are plenty of non-academic jobs, and training PhDs is not vocational training for professorship.

I applaud efforts to expose graduate students to non-academic careers. I personally tell students and postdocs all the time that they are valuable to the society because of their science training, across a wide spectrum of research and non-research positions. Yet, I don’t sense that there are enough fulfilling positions for the scientists we train (will write a separate post on this). And I certainly don’t believe that the programs to facilitate non-academic career exploration are in any way addressing the central problem of academia. (By the way, telling students that there are lots of non-academic jobs is also the solution to the crisis that Francis Collins advocates.)

The problem – academia is no longer competitive for the best and most talented researchers.

This is the part that worries me the most. We have too many PhDs competing for postdocs. We have too many postdocs competing for fellowships and faculty positions. And we have too many professors, competing for grants. Combined, the low pay, hyper-competition, and the guaranteed uncertainty at every step make the academic track a bad career choice today.

So what? As many have replied to my Goodbye Academia post, “There is nothing wrong with getting out of academia. We, taxpayers, fund your training so you can join the industry, start biotechs, and not just sit in the ivory tower.”

I love my role at ZappyLab. I do not regret getting a PhD or doing my postdoc – no way I could have founded ZappyLab otherwise. But my question to the taxpayer is, “Who do you want as professors, training the next generation of scientists for the biotech/industry jobs?”

The NIH alone spends $30b each year of taxpayer money, supporting 50,000 grants across laboratories at 2,500 institutes. Who do we want heading the laboratories under these grants? Who do we want training the future researchers? If the best scientists aren’t the ones training, what does it mean for the quality of scientists and science that will be in the industry? 

We have a bubble in academia. Like all bubbles, if ignored, it will pop. There will be a natural adjustment. We are beginning to see it already. As I was inviting mentors to participate in our Career Forum, several warned me that they regretted getting a PhD. They said that they would not be able to give rosy encouragement to students and postdocs. These are folks from the very top research universities. Folks who graduated 5-10 years ago when the situation was far better than it is now. This is phase 1 of the bubble-bursting. The next and most painful phase will be a decrease in the number of talented students going into biomedical PhDs.

The system will naturally self-adjust if we take no action. Eventually, the number of PhDs we train will decrease simply because few will want these degrees. But what will be the cost to science, progress, and society if this happens? The quality of academic research is already suffering with pervasive irreproducible results and outright fraud, as a consequence of the hyper-competition. We are already starting to lose the scientists with the stronger moral compass and ethical standards. If we don’t fix academia and make it an attractive career choice again, we will all pay a devastating price.

-----------------------------------------------------

Some statistics on percent of biomedical PhDs getting faculty positions:
1963: 61%
1983: 38%
1993: 25%
2003: 15%
2014: <10%
(numbers for 1963-2003 from the Scientist and 2014 from ASCB.org)