Biomedical funding is broken; crowdfunding is not the fix.

Last week, I wrote Calibrating crowdfunding expectations – a post based on our experience running the protocols.io Kickstarter campaign. I shared that pre-launch, we had wildly incorrect expectations about the average backing amount. We expected $70-$100, but the real mean turned out to be $39. So we had to lean on our close relatives for help, and of the $54K that we raised, $28K came from just four relatives. In this post, I want to focus on the implications of our experience for research crowdfunding.

Crowdfunding platforms can work very well for pre-sales of popular items. Even in the case of our protocols.io, while not great for funding, it was very useful to give visibility to our effort. However, while projects on Kickstarter need marketing, science projects need funding rather than advertising. Moreover, unlike ZappyLab with thousands of existing users that we asked to back our Kickstarter campaign, most scientists do not have an easy network to appeal to for crowdfunding.

Based on our experience, I was concerned that researchers turning to crowdfunding were setting themselves up for a harsh month of crowdfunding education, with very little chance of successfully raising the research dollars that they so desperately need. I know that biomedical science funding is broken, but I don’t see how crowdfunding is fixing it.

Of course, our crowdfunding was aimed at scientists to support protocols.io (a communication platform), which is very different from the research projects posted on experiment.com (the leading site for research crowdfunding) and aimed at a general audience. Therefore, to check if our experience is consistent with that of the scientists running projects on experiment.com, I took a quick look at all biology projects on the site (84 projects funded at $3K or above; spreadsheet here). The average contribution for all of them was $94, very close to our protocols.io average of $107.

The problem is that most of the 50 projects had very few backers, and that means mostly funded by close friends and relatives, which strongly skews the average towards higher numbers. I plotted the average contribution as a function of the number of backers. Figure 1 below shows that the more backers a project has, the lower the average. For the most popular projects with more than 100 backers, the averages hover around $50 – much closer to the $39 that we saw when excluding the relatives.

Figure 1. Average contribution as a function of the number of backers of the project. Numbers are from all 84  biology projects successfully funded on experiment.com at $3,000 or above.

The projects on experiment.com, just as on Kickstarter, are all-or-nothing. That is, if a project does not reach its funding goal, it gets none of the contributions. As I wrote in my previous post, there are good reasons to structure funding campaigns this way, but it also creates an extraordinary pressure on the scientist running the campaign to somehow get it to 100%. I have spoken to a number of people who ran campaigns on experiment.com, and several indicated that they had to use their own credit cards at the end to make the campaign successful. Based on this, I hypothesized that projects raising close to 100% of their total will often be rescued by relatives as was our protocols.io or will be partially self-funded by the scientist. Consistent with this prediction, graphing the average contribution as a function of the percent of the raised funding target shows a strong bias towards inflated averages around 100%-105%.

Figure 2. Average contribution as a function of the percent of funding target. Numbers are from all 84  biology projects successfully funded on experiment.com at $3,000 or above.

I know personally Cindy Wu and Denny Luan, the co-founders of experiment.com. I like and respect both of them and know that they truly have noble goals and intentions. I am publishing this post because I think transparency and correct expectations are important.

------------------
Note 1. There are also philosophical reasons to question crowdfunding for research. The whole point of NIH grant committees reviewing and scoring the proposals, the point of HHMI funding innovators -- is to support good research and researchers. There are many problems with NIH funding and the selections, but as imperfect as these processes are, they aim to select good science. But crowdfunding cannot judge the merit of the proposal or the proposing scientist. Crowdfunding selects for scientists who are good at convincing the public to support them. Crowdfunding selects for projects that resonate with the public. As such, crowdfunding isn't well suited for supporting high quality research.

Note 2. If you are planning to launch a crowdfunding campaign, take a look at the experience of Jacquelyn Gill, which is remarkably similar to our experience.

Calibrating crowdfunding expectations

[TL;DR summary – we raised $54,600 from 506 backers in our Kickstarter campaign. It literally saved our startup and was amazing for visibility and marketing. However, our expectations of how much we could raise were wildly off. Of the total, $28,300 came from just four relatives. The all-or-nothing model at Kickstarter misleadingly skews the “average backing” to high amounts.]

A year ago, our startup ZappyLab ran a Kickstarter campaign for the creation of protocols.io – a platform for sharing and discovering up-to-date science methods. This was the first successful crowdfunding project aimed at researchers. This Kickstarter also saved our startup. We were bankrupt, deep in the “death valley” of startup funding, with no one willing to back us. The response to our project was overwhelming; that gave us the much-needed visibility, led to our first major contract with a reagent vendor and helped to convince investors to support us.

I want to emphasize the above again – the Kickstarter saved us. I am deeply grateful to the hundreds of people who contributed, to the bloggers who wrote about it, to the countless people who spread the word. I think Kickstarter is an amazing platform and am writing this post simply to help those who plan crowdfunding campaigns. My goal is to share so that others can better calibrate their expectations and run more successful campaigns.

Before launching our project, I did a lot of research to determine the feasible funding goal. Kickstarter stats show an average pledge is $70. There is variation by category, with technology projects at an average backing of $107. Several crowdfunding gurus told us to expect a mean at or above $100 because a lot of our support would be coming from our existing users who already love and value our company and tools. We thought we could get 400-500 backers, and so we set the funding total at $50,000. We ended up raising $54.6K from 506 backers; an average of $108 per person, right around the expected range. But averages are wildly misleading. The median contribution to the protocols.io Kickstarter was $20, and the true average closer to the median than it may seem.

Within two weeks of launching our campaign, it became painfully clear that even with 500 backers, we would be nowhere near the $50K mark. Plotting our curve, we were on the path to $20,000 total. Our estimate that we could get about 500 backers was on target, but the average pledge amount was less than half of what we expected.

The tricky part is that if didn’t get to $50,000, we would get nothing and everyone watching would see a failed campaign on Kickstarter. We panicked. That didn’t help. We went to our closest relatives and begged for help. By begging for help, I don’t mean the expected leaning on friends and family to back the crowdfunded campaign. I mean an “SOS”. In the end, four relatives contributed $28,300 of the total that we raised.

Of course, our campaign is unusual for Kickstarter. The true reward we promised was better and faster science; we did not have a physical product to ship in the end. It is possible that our experience is an outlier. On the other hand, 4/6 people who ran crowdfunding campaigns told me privately that they also had the exact same experience and were forced to beg parents and other relatives to get them to the target in the end. Given our experience, I expect that this is rather common.

Kickstarter’s all-or-nothing model means that a campaign that raises $48,000 out of the target $50,000 will get zero. I understand why Kickstarter does this. This protects the backer against a campaign that raises 10% of the required total, with no product ever delivered, but the money gone. I personally would not have backed the crowdfunded projects that I did without this safety. On the other hand, this also creates an extraordinary pressure on the people running the campaign to come up with the funds by whatever means are possible. Especially for startups like ours, when investors warn that not reaching your goal is a red flag, failure is not an option.

I doubt that we are the only ones to naively launch a Kickstarter with wrong expectations. I hope this post doesn’t hurt Kickstarter, and if it helps the people running the campaigns to set better targets, this may actually result in more of the projects succeeding. To that end, more details are below.

------------------------------------------------
[UPDATE March 31, 2015]

To determine if our experience is common, I looked at 30 projects in the Food section on Kickstarter, funded at $10K or above (data here). The hypothesis is that projects raising close to 100% of their funding target will frequently be in the save-our-souls mode as we were, begging friends and relatives to close the gap with enormous contributions. Indeed, projects funded 100%-110% had an average backing of $112 while projects raising 150% or above averaged only $65.

Average contribution as a function of the percent of funding target. Numbers are from 30 projects successfully funded in Kickstarter's food section at $10,000 or above.

------------------------------------------------

1. Distribution of all 506 contributions to protocols.io Kickstarter.

2. Distribution of the 502 contributions to protocols.io Kickstarter, excluding the four relatives who saved the campaign.

3. Distribution of the 486 contributions to protocols.io Kickstarter at or below $200.

4. The average was ($54,600-28,300)/502 = $52 (compared to $108 from all).

5. Of the 502 backers, 138 were friends and relatives. They contributed $12,121. That’s an average of $88. The median contribution from friends and family was $50 (compared to $20 from everyone).

6. Excluding the friends/family contributions, we got $14,014 from 361 backers, for an average of $39 – much closer to the $20 median.

7. Naturally, considering that this Kickstarter helped us survive, we are glad we did it. But if you are planning to run a crowdfunding campaign for marketing and outreach, keep in mind that it is far from trivial to pull this off. See my guide to crowdfunding here for a description of the required effort.

8. This is part 1 of a 2-part post. Next week, I will be sharing thoughts about crowdfunding for research (now published).

Public Service Announcement: John Wiley & Sons, you are in deep deep trouble

A week ago, Wiley published a blog post celebrating 350 years of the academic journal and patting itself on the back for the unparalleled service the publishers have been providing to the scientific community: "Why has the scholarly journal endured?" Four people weighed in with their perspectives on why the academic publishing as conceived hundreds of years ago is so wonderful today. The top two reasons were that the journal provides metrics to judge the scientists and the research published and is a good way to filter out the sea of publications and decide what to read.

The entire post is a perverse inversion of all the problems in academic publishing, presented as good reasons to keep the status quo. It is so out of touch with the conversations and concerns inside the research community, it's jarring. But the real problem for Wiley is that this is not an isolated mistake in PR; this post is representative of the culture inside Wiley and perfectly encapsulates why this corporation is about to march off the cliff.

Everyone knows that past performance is no indication of future results. Many monarchies survived way longer than 350 years, but not because of the extraordinary value they provided to their citizens. And most monarchies saw a day which marked the end of their rule.

I have been interested in science publishing since 2003 when PLOS Biology launched. I have been actively talking to publishers in the past two years. While I give the corporate publishers a hard time and am convinced that it's hard for the publisher to innovate in general, there is something unique about Wiley. Unique in a bad way.

I know many people who are or were in executive positions at all of the major publishers. I know many who were in key positions at Wiley itself. One thing that comes across in conversations with ex-Wiley executives is that this publisher is deep in denial about the looming changes. The culture inside Wiley makes it seem like a Tea Party organization, with Nature and Elsevier as the Green Party radicals. For those who know the corporate publishing world, appearing conservative on the background of the other publishers is not a good distinction.

All my conversations aside, any outside observer can tell that the publishers see the writing on the wall and are desperately trying to figure out how to stay alive and what their role will be in the future. Hence Digital Science from the Nature Publishing Group, acquisition of Mendeley by Elsevier, and the acquisitions of Papers and Biomedcentral by Springer. (UPDATE March 18: Elsevier now refers to itself as a technology company, which is a major shift. From their recent report: "[This] reflects the transformation of the company to a technology, content and analytics driven business while maintaining the link with its proud heritage.")

I would love for someone to point me to any signs of creative thinking and attempts to evolve at Wiley. Alas, I think they are busier congratulating themselves on the great service and the wonders of the impact factor. Good luck Wiley.