Tuesday, July 7, 2015

Please stop referring to growth hacking scams as "data science"

I just bumped into an exposé by Data Science Renee of sleazy practices by DataScienceCentral to attract readers to their site. Apparently they create fake profiles of women and minorities and post articles under these profiles because it brings good traffic to their site.

The person behind DataScienceCentral is Vincent Granville. He obviously doesn't like Renee.



On Vincent's Linkedin page, I saw as key accomplishment of his regarding DataScienceCentral:
Delivered $2,500,000 in sales to one of our clients within a 12-month time period, providing a 1,000% return on their ad spend. All our large clients are consistently renewing and expanding their advertising programs with us, as we find new ways to bring new quality leads all the time. 
Leveraged state-of-the-art data science to grow internal revenue and profits by 500% in less than two years, while maintaining profit margins above 65%. Grew traffic and membership by 300% in two years. Introduced highly innovative, efficient, and scalable advertising products for our clients. Used proprietary growth science techniques to outperform competition.
And this language was so familiar and reminded me of the scam artists behind the Web Summit, Collission, Rise conferences (really same scam conference, cloned to different places). I wrote about them before here and here. They pretend to offer free venue to startups, pretend to filter and select you, and then try to squeeze thousands of dollars out of the people who can least afford it. Sleazy folks who also pretend to be data science geniuses:
It’s taken us 4 years to scale Web Summit from 400 attendees to 20,000 and a bunch of physicists have played a big part...
With each passing year an increasing number of people ask how has it grown so fast? When I share the answer it’s almost never what people expect, so here it is:
Our growth has been largely propelled by data science. Or more correctly, in my view, network science. While conference companies hire event managers, we hire physicists with PHDs in areas like complex systems and network analysis...
 If your growth hacking stands for spam, scam, sleaze - please don't run around talking about the physics and data science behind your remarkable growth.

Tuesday, June 23, 2015

Milo Yiannopoulos destroyed by Emily Grossman in a debate

I just watched the 10-minute debate between Milo Yiannopoulos and Emily Grossman on the scandal surrounding Tim Hunt. It was stunning. Yiannopoulos uttered a stream of unsupported and inflamattory nonsense, and Grossman just dismantled him with facts. I know I would have lost my composure and temper in a debate with him, but Grossman magically remained professional and unfazed by his debate theatrics.

To be honest, I felt slightly bad for Yiannopoulos. Grossman so eloquently and forcefully showed him to be a fool and a clown, it's impossible to watch without sympathy for him. I even heard a rumor that Yiannopoulos cried after the debate and asked for it not to be aired. (Can anyone confirm if this is true or not?)

However, after the debate, in an attempt to recover and spin his poor performance for personal benefit, he wrote Why do feminists cook up stories about misogyny when they lose debates? This is the part where my sympathy for Yiannopoulos evaporates. When you lose a debate, admit it and move on. Write a blog post apologizing for stupid and offensive stuff you said, admit that Grossman destroyed you, and try to do better next time.

Friday, June 19, 2015

An argument FOR electronic lab notebooks

Most existing electronic lab notebooks (ELNs) are terrible. There is a good reason why the vast majority of scientists still use paper. But these are not arguments against ELNs - these are arguments against poorly-designed ELNs.

Below is my e-mail to a professor from a few months ago on this topic.
------------------
From: ProfessorX
Date: Sun, Jan 4, 2015 at 9:51 PM
To: Lenny, ProfessorLab

The attached article [How My Mom Got Hacked] is about the latest form of malware.   There is no way in the world that I could ever get authorization to pay the ransom to get any of out data back for reasons you will see.   So beware…..and keep a real note book.
ProfessorX


-----------------------
From: Lenny
Date: Sun, Jan 4, 2015 at 11:24 PM
To: ProfessorX, ProfessorLab

Dear ProfessorX and lab,

I think this is a fascinating article and is in fact an argument for a digital lab notebook as Protocols.io envisions it.


ProfessorX, we have repeatedly discussed electronic lab notebooks (ELNs) over the past two years. You argue that the paper notebook has reliability and persistence that is not matched by the ELNs. You cannot access anything from the digital files on the floppies from the 90s, but you can still read your paper notebook from the 70s. You've also told me that the ELN business is the Afghanistan of startups. 


I agree with everything you point out, and these considerations have influenced our approach to the digital notebook since the birth of Protocols.io. I agree with everything you point out, but not with your final conclusion that the paper notebook is the best way of keeping records. It is indeed today, but that's just because the existing ELNs are shitty, not because the ELN concept is flawed.


Now for the details:


1. Formats change and startups fail, so you can lose all of your data.


Agreed. That's why we made sure that our protocols.io journal can be exported as .txt and PDF. You can export the whole thing, print it, put it on dropbox, thumb drive, and anywhere else. And it's good practice to do so periodically, as we do with all of the data that we generate. The important aspect is that your records belong to you and not to Protocols.io.


2. ELNs have not been adopted by biomedical scientists because the paper notebook is best.


Agreed that ELNs have not been adopted and paper is best today. However, that's because most ELN companies have been solving the wrong problem. Labguru, eCAT and LabArchives all think they are competing with each other. They are not; their real competition is the paper notebook, and paper wins by a long shot against them because all of them have a terrible product. The existing and dead ELNs all thought the problem they need to solve is the retrieval of information, which is infinitely better in digital format. No. The problem is the entry of the information in the first place. That's where the paper notebook shines in contrast to websites and desktop software. My takeaway here is that as a mobile company, if we really do this right, we can come close to and possibly beat the paper notebook for the entry of the information (e.g. ability to write over the gel image, directly on the iPad). If we do that, we have a shot at adoption.


3. Hard drives can die and computers can become infected, as outlined in this article.


Agreed. This is an argument for backups. Hence we have a daily snapshot of all user data going back a month and a monthly snapshot beyond that. As the article itself points out, this is also an argument for cloud storage. Chances of losing your notebook on a laptop or single hard drive are high, but chances of losing it entirely on our Amazon server or on Dropbox are lower than the chance of losing or burning your paper notebook (doesn't have to burn as the drowned notebooks during Sandy at NYU showed us).


4. The paper notebook stays in the lab and can be easily read by anyone forever.


I agree it stays in the lab forever, but disagree it can be easily read by anyone. Most notebooks can only be parsed by the person who wrote them. So when that student moves on, her inability to physically access the notebookeffectively makes the entire record useless. My PhD notes from 2003-2009 may be physically in the Berkeley Lab, but when I am in Boston for my postdoc, good luck trying to answer a reader's question without me, just from flipping through my notebook. In fact, a paper notebook can't easily be used by anyone, including the author. That's because looking for a primer number or some note, without the ability to search digitally, requires hours of flipping through the notebook.
When students or postdocs leave the lab, they should re-assign their digital lab notebook to the professor. At the discretion of the professor, the student continues to have access to the notebook after leaving the lab. So when ProfessorX gets a question about a 2007 paper from a previous student, he forwards it to that student, student logs in, does a quick keyword search, reads the relevant page, and answer the question precisely. That's how the protocols.io journal will work.
----------------------------------------


The inefficiency of the paper notebook, in the electronic age, is staggering. That's why both Novartis and Merck moved last year to ELNs for their chemists and are currently in the process of moving the biologists to ELNs as well. The arguments against the existing ELNs are arguments for a better digital lab notebook product, but they are not arguments that the paper is best. After all, we don't transcribe RNA-Seq or ChIP-Seq data into our notebooks. Nor do we transcribe our qPCR readouts from Excel. We are firmly in the digital age. We need a good ELN, be it from Protocols.io or someone else, and we need good practices to make sure the electronic records are accessible long into the future, without wasting trees.


Best,

Lenny

P.S. I'd love to hear more arguments from everyone, for and against the ELN.

Saturday, May 23, 2015

The PhD Misconception

[This is an anonymous submission]

Currently, I'm nearing the end of my undergraduate career in Biology/Environmental Sciences. From the start, I thought I was doing everything right. I was studying a STEM major and getting involved in the environmental field, a supposedly "booming" part of the economy. I started research my first semester of freshman year, got to know all of my professors, participated in four internships, led tutoring sessions, and kept on the President's List, all while being extremely active with organizations on campus. I always thought I would stay in academia and become a professor - or that is what all of my professors assured me would be the best route. I wanted to do something important that bettered the earth we live on for future generations. 

But, what they didn't tell me about were the messed up hours, extremely low wages, and countless postdoc years. I started to observe all of the graduate students in my department. Most of them did really cool stuff while they were here - ecological restorations, public health microbiology, etc. - but most either ended up chasing postdoc after postdoc, waiting for a faculty position to open up, dropping out, or just...disappearing after graduation, never to be brought up in alumni news or updates.

Now with graduation nearing, I've almost entirely ruled out attending graduate school for the sake of working in academia. While I believe the work I'd most likely do to be extremely important, especially considering the environmental challenges we face now, I also want to be able to live my life fully, and be compensated accordingly for the effort I put into my work. 

Thursday, April 16, 2015

Last post on Collision Conference and Keiretsu Forum

Yesterday I wrote Hey startup parasites! We don't have time for you. In that post, I explicitly pointed my finger at the deceptive tactics used by Kieretsu Forum and Collision Conference to make money on startups. The people behind both of the organizations have contacted me to apologize and to defend their fees and legitimacy. Paddy G wrote lengthy replies here and here.

So to both, I want to be very clear - the big problem I have is not that you charge, but the way you mislead startups in the way you approach us, waste our time, create a false sense of exclusivity and conceal the fee in the initial conversations. If you were to make the fees you charge clear and explicit from the first contact, I wouldn't be blogging about either of you.

1. Keiretsu

You have told me today that the month of conversations I had, with not a single mention of the fact that you charge $4,500 for the right to pitch your angel investors, was just an honest mistake. You said that it never happens and I am just unlucky. You argue that the $4,500 fees that you charge are legitimate and provide value to the startup.

If that is the case, why not be upfront about these fees? There isn't a single mention of the exorbitant fee on your main Entrepreneur Page. Nor do you mention it on the application page. Another simple mistake? Or do you think it is not important? Or perhaps you realize how totally insane this fee is and how much of an outlier you are with this charge? Just glance at this nascent information sheet that I started; you are in the first place and that's not a good place on this sheet.

Considering the scathing coverage you got in Business Insider in 2009, claiming that you forgot to mention the fees on your website is stretching my very flexible imagination beyond its limits.

2. Collision Conference

You say that your website is very clear on the fees with:

“Each week we selected 25 early stage startups from around the world to exhibit for free as part of our Collide Program. The bigger tech companies pay $9,950 to exhibit, meaning exciting, disruptive, early-stage startups can afford to attend no matter what. All they will pay is a discounted price for tickets and Collide registration, and we’ll give them a free exhibition stand.”
No, that isn't clear at all. Let me highlight the clear parts to you "Each week we selected 25 early stage startups from around the world to exhibit for free as part of our Collide Program." To me, that says "free." Then you say "The bigger tech companies pay $9,950 to exhibit, meaning exciting, disruptive, early-stage startups can afford to attend no matter what." To me, that again says "free." Then you say "All they will pay is a discounted price for tickets and Collide registration, and we’ll give them a free exhibition stand" and this part is San Francisco fog. I see the "free" in there again. I don't see any clear fees you will charge. How am I supposed to know that this is you saying, "Friend, we have a bargain here - buy this shirt and the sleeves are FREE!"

Your other conference in Asia, Rise has this beautiful image:


And in the fine print underneath, the same text you quote above. No, this isn't how clarity is defined.
I am not a communications major, but you can simplify your messaging and get the pricing across more effectively if you get rid of the misleading "FREE" and instead say, "$1,450 if we invite you and you are selected."

And I need to remind you - I didn't come to you via the website. You reached out and said that you are interested in protocols.io. You had me present, asked questions, and told me that we have a chance to be selected to present because what we are doing is important and unusual. When I asksed how you heard of us, you said, "You were referred. We reach out to companies based on the high quality referrals from our trusted network of advisors." The gullible schmuck in me thought that it may be on the heels of our FREE appearance at LeWeb. Of course, once you "selected" us in your rigorous judging, we got the e-mail letting us know about the great honor of being selected, invited, and having the right to get the free exhibit table if we pay you a mere $1,450 instead of $10K.
-----------------------------------------------------------
In summary, you both practice deceiving and misleading tactics. I have wasted enough time on your organizations, and I have no further plans to engage in a discussion. It is entirely possible that I am an idiot and simply didn't understand your words or didn't look in the right places. Instead of playing "he said, she said," I will just point people to the countless comments about Kieretsu (here, here and here) and Collision Conference (here and here). There are certainly lots of folks out there feeling the exact same way as me.
------------------------------------------------------------
[UPDATE 4/20/15. There are 34 chapters of Keiretsu. If each one screens 5 companies per month, as does the South California Chapter. That's 5*34*12 = 2,040 companies per year. Let's be conservative and say 1K screened companies. Yet, only 34 companies are listed in the 2014 portfolio. So the startups might have paid 1,000 * $4,500 = $4.5m to Keiretsu with over 90% of them losing time and getting nothing in return.]

Tuesday, April 14, 2015

Hey startup parasites! We don't have time for you.

I am a good bullshit detector. Apparently, not good enough to have sensed that Keiretsu Forum and Collision Conference are parasites. Congratulations to both of you for wasting my time! Well done! You both reached out, pretended to screen our startup, asked good questions, made me present, and then kindly gave me the honor and opportunity to spend thousands of dollars for no good reason.

About once every day or two, I get a spam from some pay-to-pitch event. They are usually easy to spot and trash. But the two above were special. The woman from Collision Conference interviewed me, asked about our business model and strategy, then told me she would encourage their screening committee to invite protocols.io because what we are doing is unusual and has huge social impact. A week passed, and we got the great news that we are indeed getting invited! Not only were we selected after their rigorous application round, but because we are invited, the cost of attending this conference would be reduced  from $9,950 to a mere $1,450.

I curtly replied that when we were invited to present at LeWeb, our $1,400 entrance fee was waived entirely. And the many other conferences that I attend when invited as a speaker, not only waive registration but also typically pay for the lodging and travel. I was going to let it go at that, but today's Keiretsu experience made me write this blog because these folks waste time that startups do not have.

The Keiretsu Forum contacted us a month ago to learn more about protocols.io. Then a phone call for a quick interview. A request to enter our information into their platform. Then a screening conference call were I presented our deck to two people. A week later, the next round where I presented to a bigger screening committee with 4-5 people on the call. Then a request to address some of their concerns before we move to the actual pitch to their investors. And finally, a "by the way" note that they charge $4,500 to pitch their angels. The full egregious exchange is below. 

Of course, as soon as I got this, I searched for "Keiretsu scam" on google and found two excellent articles on BusinessInsider: My Latest War: Angels Who Charge Startups To PitchRead and The Amazing Response To My War Against Sleazebags Who Charge Startups To Pitch. Yes, I should have done this search before taking the call, but I don't have the time. 

The irony is that my lack of time as a startup co-founder is what allows these parasites to waste more of my time. I am not going to get into a discussion of whether it makes sense to pay $500 to attend  an event where some VCs will speak for 20 minutes and then quickly escape, or pay to pitch angels who will most likely not invest in your startup, or pay an unknown conference to present your startup. Instead, I just made a very simple Google Spreadsheet to crowdsource information on which events charge and how much. I hope this will help us and others to avoid wasting time on the parasites.

[UPDATE 1: The comments on Hacker News in response to this post are very revealing.]
[UPDATE 2: Several people from Keiretsu Forum have contacted me apologizing and saying this was a mistake. Apparently they always warn about the fee from the first phone call. Except, they don't mention any fees on their main Entrepreneurs page or the Application page. If you think this is a legitimate reasonable charge, why not be upfront about it?]
[UPDATE 3: Keiretsu and Collision have contacted me to apologize and defend their organizations. My last post in response.]

---------------------------------------------------------------
[Update April 16, 2015]
Collision Conference claims that they are very transparent that they charge startups $1450-$1950 after inviting and screening/selecting. Here is how their clear pricing page looks:

The only clear thing about it is "FREE."

---------------------------------------------------------------
[UPDATE 4/20/15. There are 34 chapters of Keiretsu. If each one screens 5 companies per month, as does the South California Chapter. That's 5*34*12 = 2,040 companies per year. Let's be conservative and say 1K screened companies. Yet, only 34 companies are listed in the 2014 portfolio. So the startups might have paid 1,000 * $4,500 = $4.5m to Keiretsu with over 90% of them losing time and getting nothing in return.]]
---------------------------------------------------------------


Forwarded conversation
Subject: Keiretsu Forum - ZappyLab
------------------------
From: <zzz>

Date: Tue, Apr 14, 2015 at 10:52 AM
To: Lenny Teytelman <lenny@zappylab.com>

Hello Lenny,

Hope you had a great weekend. After reviewing the presentation I had the following comments.

- "Both have high valuations, especially Zappy. Hard to determine market size for Zappy but I think if they get those niche eyeballs they are valuable to companies that sell them"


- "There was no talk about how much they expect to be able to charge. What they expect as far as timing of revenues, break even plan, revenues at exit, etc. they really need to explain and show proforma expectations."
If we can work on this, you are ready to present on our April forums.


Best,ZZZ


Keiretsu Forum Southern California
270 Bristol, Suite 200
Costa Mesa, CA 92626


A region within the worlds largest network of Angel Investors.  Keiretsu Forum Global has 34 chapters on 3 continents with 1400 accredited investor members. Our founding team of angel investor members, along with our partners and sponsors, are committed to growing the community of investors who are interested in funding entrepreneurial opportunities and providing valuable resources for their growth. Our goal is to grow our community of investors by providing an international platform to showcase the most promising opportunities, ultimately creating the right environment, a Keiretsu, for everyone's potential success with ROI.
Since Keiretsu Forum's founding in 2000, its members have invested over $460m in companies in technology, consumer products, healthcare/life sciences, real estate and other segments with high growth potential. Forum members collaborate in the due diligence, but make individual investment decisions, with rounds in the range of $250k-$2m. Keiretsu Forum's community is strengthened through education on angel investing, as well as charitable giving.   

----------
From: Lenny Teytelman <lenny@zappylab.com>
Date: Tue, Apr 14, 2015 at 4:30 PM

Thank you ZZZ.


Had a full day of meetings and will respond tonight/tomorrow.
To clarify, the first question is about the market size?


Best,
Lenny



----------
From: <zzz>
Date: Tue, Apr 14, 2015 at 4:34 PM

The first statement is from the valuation aspect and the second is the addressable market size.

Just to clarify, because I think I didn't write it down on any email, we do charge a $4,500 fee for the whole process of presenting and joining the network.

Best,
ZZZ



----------
From: Lenny Teytelman <lenny@zappylab.com>
Date: Tue, Apr 14, 2015 at 5:08 PM

ZZZ,


Could you kindly clarify? It is $4,500 to be paid by the startup?


Thanks,
Lenny

----------
From: <zzz>
Date: Tue, Apr 14, 2015 at 5:13 PM

Yes Lenny.
Best,
ZZZ

----------
From: Lenny Teytelman <lenny@zappylab.com>
Date: Tue, Apr 14, 2015 at 5:20 PM
To: ZZZ

I am baffled. How could you not mention it on the first phone call? How could you not mention it in the past month? That is highly inappropriate.


Lenny
----------
From: ZZZ
Date: Tue, Apr 14, 2015 at 5:27 PM
To: Lenny Teytelman <lenny@zappylab.com>

Lenny,

It was my bad, thought I've sent, but I've rechecked the emails and I didn't.

We are not charging you any fee right now, this is only if you want to move forward to the forums.

Best,
ZZZ


----------
From: Lenny Teytelman <lenny@zappylab.com>
Date: Tue, Apr 14, 2015 at 5:29 PM
To: ZZZ

I am glad you are not charging us now. Thank you for that.

And I don't think you will be charging us in the future. We are not going to proceed.
Regards,

Lenny

----------
From: ZZZ
Date: Tue, Apr 14, 2015 at 5:33 PM
To: Lenny Teytelman <lenny@zappylab.com>

Wish you the best luck.

Best,
ZZZ

Monday, March 30, 2015

Biomedical funding is broken; crowdfunding is not the fix.

Last week, I wrote Calibrating crowdfunding expectations – a post based on our experience running the protocols.io Kickstarter campaign. I shared that pre-launch, we had wildly incorrect expectations about the average backing amount. We expected $70-$100, but the real mean turned out to be $39. So we had to lean on our close relatives for help, and of the $54K that we raised, $28K came from just four relatives. In this post, I want to focus on the implications of our experience for research crowdfunding.

Crowdfunding platforms can work very well for pre-sales of popular items. Even in the case of our protocols.io, while not great for funding, it was very useful to give visibility to our effort. However, while projects on Kickstarter need marketing, science projects need funding rather than advertising. Moreover, unlike ZappyLab with thousands of existing users that we asked to back our Kickstarter campaign, most scientists do not have an easy network to appeal to for crowdfunding.

Based on our experience, I was concerned that researchers turning to crowdfunding were setting themselves up for a harsh month of crowdfunding education, with very little chance of successfully raising the research dollars that they so desperately need. I know that biomedical science funding is broken, but I don’t see how crowdfunding is fixing it.

Of course, our crowdfunding was aimed at scientists to support protocols.io (a communication platform), which is very different from the research projects posted on experiment.com (the leading site for research crowdfunding) and aimed at a general audience. Therefore, to check if our experience is consistent with that of the scientists running projects on experiment.com, I took a quick look at all biology projects on the site (84 projects funded at $3K or above; spreadsheet here). The average contribution for all of them was $94, very close to our protocols.io average of $107.

The problem is that most of the 50 projects had very few backers, and that means mostly funded by close friends and relatives, which strongly skews the average towards higher numbers. I plotted the average contribution as a function of the number of backers. Figure 1 below shows that the more backers a project has, the lower the average. For the most popular projects with more than 100 backers, the averages hover around $50 – much closer to the $39 that we saw when excluding the relatives.

Figure 1. Average contribution as a function of the number of backers of the project. Numbers are from all 84  biology projects successfully funded on experiment.com at $3,000 or above.

The projects on experiment.com, just as on Kickstarter, are all-or-nothing. That is, if a project does not reach its funding goal, it gets none of the contributions. As I wrote in my previous post, there are good reasons to structure funding campaigns this way, but it also creates an extraordinary pressure on the scientist running the campaign to somehow get it to 100%. I have spoken to a number of people who ran campaigns on experiment.com, and several indicated that they had to use their own credit cards at the end to make the campaign successful. Based on this, I hypothesized that projects raising close to 100% of their total will often be rescued by relatives as was our protocols.io or will be partially self-funded by the scientist. Consistent with this prediction, graphing the average contribution as a function of the percent of the raised funding target shows a strong bias towards inflated averages around 100%-105%.

Figure 2. Average contribution as a function of the percent of funding targetNumbers are from all 84  biology projects successfully funded on experiment.com at $3,000 or above.

I know personally Cindy Wu and Denny Luan, the co-founders of experiment.com. I like and respect both of them and know that they truly have noble goals and intentions. I am publishing this post because I think transparency and correct expectations are important.

------------------
Note 1. There are also philosophical reasons to question crowdfunding for research. The whole point of NIH grant committees reviewing and scoring the proposals, the point of HHMI funding innovators -- is to support good research and researchers. There are many problems with NIH funding and the selections, but as imperfect as these processes are, they aim to select good science. But crowdfunding cannot judge the merit of the proposal or the proposing scientist. Crowdfunding selects for scientists who are good at convincing the public to support them. Crowdfunding selects for projects that resonate with the public. As such, crowdfunding isn't well suited for supporting high quality research.

Note 2. If you are planning to launch a crowdfunding campaign, take a look at the experience of Jacquelyn Gill, which is remarkably similar to our experience.