The chemfp project: problems selling free software

https://jcheminf.biomedcentral.com/articles/10.1186/s13321-019-0398-8#Sec24

by dalke • 6 years ago

66 46 6 years ago

jcheminf.biomedcentral.com

dalke20• 6 years ago

I started the chemfp project in part to see if I could develop a self-funded free/open source product in my field, cheminformatics. (In short, storing and searching chemical information on a computer. Chemfp does very fast Jaccard-Tanimoto similarity search for "short"/O(1024 bit) bitstrings.)

The answer: no.

The section I linked to highlights some of the problems I had selling software under the principles of free software. For examples: How do I provide a demo if I always provide MIT licensed source code? Academics expect discounts, but they are also the ones most likely to redistribute the code. Which is not a wrong thing to do! But it affects the economics in a way I could never resolve, compared to proprietary/"software hoarding" licensing models.

As an HN note, I contracted a couple people to help improve the popcount implementations. HN user nkurz developed and tweaked the AVX2 implementation, and proof-read the paper. Thanks nkurz! As a result, chemfp is, I believe, the fastest single-threaded Tanimoto search implementation for CPUs available, and most likely memory bandwidth limited, not CPU limited.

(Note: the mods asked me to repost. My earlier post is at https://news.ycombinator.com/item?id=23598470 .)

ssivark• 6 years ago

Very interesting experiment.

Did you consider having academics cite your software as an academic work, and then monetizing those citations to get grants from funding agencies who were funding the research that used your software?

Given this knowledge, can you think of other similar experiments worth performing? Alternatively, are there any likely changes that might lead to such approaches becoming more feasible?

I’m quite interested in this question and appreciate your comments. In case you’ve already answered these in the article, I apologize — the article is long and the HN thread will likely expire before I have the chance to peruse it thoroughly.

dalke20• 6 years ago

I tried talking with funding agencies. As a self-employed/sole-proprietership, it proved to be difficult.

I conducted the "mmpdb crowdfunding project" at http://mmpdb.dalkescientific.com/ . Unfortunately, it's been on delay because of major changes to my work schedule once the coronovirus came. My conclusion from that was it's still easier for me to make money as a consultant than from selling a product.

One thing I didn't mention in the paper was that being a consultant on different projects makes it easy to present new talks at conferences, which leads to new work. Continued development of a single project with one focus leads to variations on a theme, feeling sometimes like "have you ever looked at your hand. I mean, really looked at your hand" introspection.

You can always email me as follow up. dalke at dalkescientific.

maximente• 6 years ago

in your future ideas section you have listed a few features or directions that the software could go. what about selling those?

in my mind, in order to monetize a greenfields FLOSS project, it seems you need to basically create software so impactful that users are willing to pay for particular features or bug fixes that they need, and that don't already exist in the software. so basically, you have to not only get people to use this new thing, but also get them to be willing to pay for particular improvements to it! quite a task.

this first occurred to me when i heard Stallman talk about how one can monetize FLOSS projects by selling bugfixes or improvements. contrast the typical developer-driven setup where a team implements the feature then sells it by selling a new version, upgrade fee, or what have you.

so it's a tricky situation, in that one needs to offer a compelling product, but not something so good that individuals can take it from the shelf and never have to do anything to it. however, this is sort of helpful insofar as it acts as a way for you to ship something and then - hopefully - let users drive the product (we'd pay $X for a UI on top).

for cheminformatics, i'm not really sure what some killer features are, but you have at least some ideas as potential things to sell. perhaps the community would be interested in pooling together funds for some of those ideas, or a UI/UX, or whatever - but yeah, definitely seems more customer-driven then traditional software.

dalke20• 6 years ago

Yes, those future ideas are also possible salable products, thought the market is smaller. I had hoped that the surplus from the core bitstring search would provide the seed money for their development.

Chemfp is (or rather was) at least an order of magnitude faster than its competitors. One of my early customers went from taking 5 days to cluster 1M fingerprints using their internal tool to several hours with a re-writes based on a chemfp core.

I figured that was sufficiently impactful. But that was in the early stage when people did fund me to add new features as an open source project. I think that once it got "good enough", people didn't want to fund the incrementally smaller improvements. The commercial version is only about 2x faster than the no-cost version. I would talk with potential customers and they already had the no-cost version installed in-house.

Now, there are a lot of other improvements in the commercial version, but it's a variation on the "offer a compelling product" - I was my own competition. That was the primary failure in the original business model, based on Ghostscript. (I didn't realize that hardware companies paid Aladdin to keep up with Adobe Postscript, which provided the reason to pay for the latest version rather a no-cost older version.)

I tried crowdfunding a project, ie, "pooling together funds", for another project. It worked, in that it made enough money to fund the software development and the marketing, but it also required a lot of time for the marketing (which I'm not good at), and I would have made more money doing straight consulting work instead of product development.

BTW, I focus on "classic" cheminformatics. Similarity search is from the 1980s, with roots in the 1970s. My thesis is that there people haven't looked into these topics in years and there might be orders of magnitude gains by developing better fits to modern architecture, that is, improve existing methods enough that UI/UX changes. Most other people's definition of "killer feature" is "an improveed method predict how a molecule will work in the body."

alicemaz• 6 years ago

have you considered offering it as a cloud platform? we're doing something along these lines, niche scientific software (biological modeling, bioinformatics) as a paid hosted service. still at the prototype stage! so I can't comment on how well the business model actually works yet lol

but the idea is our mathematician will be able to publish whatever novel math she develops, and we may eventually open source the math core as a reference impl, but we'll keep all the cluster management and other supporting infrastructure code proprietary. sort of a "if you want to run it on your desktop, go ahead! if you want to actually scale this up for big jobs, we've done all the legwork already so it's really in your best interests to just pay us." I think open source ideals are good and worthy but from a business perspective, you capture value by providing value that can't be got without you. relying on customer goodwill is particularly difficult because any large org, the people who will feel goodwill toward you and the people who can authorize purchases are in two different departments

also fwiw I think if you wanted to do the model you described in the paper unchanged, gpl is a much better choice than mit. copyleft actually serves as a wonderful poison pill: you can try us out for free, but if you want to ship us, you need to pay for a proprietary license or legal will nail you to the wall. whereas mit, there's no stick. I've seen affero used by several projects for this express purpose: you have to buy a proprietary license because agpl is so onerous you just can't use the code for commercial purposes at all

interesting project btw, I love seeing stuff like this!

dalke20• 6 years ago

Thanks!

Yes, I've considered cloud platform. There are several big difficulties with that.

First, data. It's easy to grab public data from PubChem, ChEMBL, and a few other projects, and make a service. But why would anyone pay for it given that PubChem, ChEMBL, ChemSpider, and others already provide free search services of that data?

There's search-as-improved-sales, like how Sigma-Aldrich lets people do a substructure search to find chemicals available for sale.

There's value-add data. eMolecules includes data from multiple vendors, to help those who want to purchase compounds more cheaply.

Or there's ZINC, which already provides search for their data.

So you can see there's plenty of competition for no-cost search. I don't have the ability to add significantly new abilities that people are willing to pay for.

Note also there's a non-trivial maintenance cost to keep the data sets up-to-date.

Second, the queries themselves may be proprietary. I talked with one of the eMolecules people. Pharmaceutical companies will block network access to a public services to reduce the temptation of internal users to do a query using a potential $1 billion molecular structure (or potential $0 structure). eMolecules instead has NDAs with many pharmas which legal bind them. Managing these negotiations takes experience I don't have, and neither do I have the right contacts at those pharmas.

Sequences don't have quite the same connection between sequence and profit as molecules do.

BTW, part of the conclusion of my work is that people don't need a cluster for search - they can handle nearly all data sets on their laptop, so there shouldn't be a need to scale up any more. And small molecule data has a much smaller growth curve than sequence data, so Moore's Law is keeping up.

My first customer, who continues to be a customer, said outright that they would not buy if it were under GPL.

Since my paying customers are pharmaceutical companies who, as a near-rule, don't redistribute software, it doesn't really matter if they don't redistribute under MIT or don't redistribute under GPL.

I came into the project in part to see if FOSS could be self-supporting on it own. AGPL is often used as a stick to try to get people to use a commercial license - the implicit view of the two-license model is that FOSS is not sustainable. Which is now my conclusion, for this project and field.

fock• 6 years ago

not really into industry, but a) the pharma-companies using it are probably reluctant to give you their data and b) uni researchers are not overly fond of high-fee services and labor is cheap there.

zokier• 6 years ago

I think to truly appreciate FOSS as a model, one needs to shift away from thinking software as an asset to be monetized to more of a liability that needs to be managed and maintained. Then the benefit of FOSS becomes clear: by publishing your software there is possibility of sharing that burden with others instead of carrying it alone yourself.

taneq4• 6 years ago

I agree. I can't see how you can 'sell free software' as a standalone product. You build and evangelise free software while selling feature requests, services and support to the users.

dalke20• 6 years ago

You can sell a proprietary product, right? With a restrictive software license?

That means customers are willing to pay $base + $yearly renewal for the product.

Why aren't they willing to pay the same price for the same product but with an open source license?

I really don't understand why they don't.

I'll go one further - how much will people pay for an open source license over a source available license with a right to modify, no time limits/renewal requirement, but no distribution right?

Answer: all but one of my customers jumped at the chance to reduce the cost by switching from the MIT license to a not-quite-open-source license.

Which means they don't really value the redistribution right.

And I saw this at one small conference about industry use of open source. The organizers - who use chemfp! - stated at the start that the biggest reason they love open source is because it's "free" (meaning no cost), not the principles of software freedom nor the improved development methodology of open source.

I tried selling feature requests, services and support to the users. That was my original plan, and it worked so long as those feature requests were easy and there were enough of them.

But consider that the upgrade to Python 3 took two months. Who pays for that? The first customer who wants Python 3 support 5 years ago, who pays $20K for a feature request which everyone else gets for free? Then there's inventive to wait for a feature request in hopes that someone else will pay it. While the sales model - even as free software - lets me split the cost among multiple customers who need that feature, and across a few years.

I also pointed out that selling services is a disincentive to developing good document and good APIs. I feel like there's a sweet spot where if I were to skimp on the documentation some then there's an increased chance of getting consulting work.

EvanAnderson3• 6 years ago

I agree too. Free software definitely doesn't work as a "product".

Software has always been a service. Software as a "product" was an aberration. That model isn't sustainable. The fact the largest software companies are moving to subscription-based models is proof-enough that software as a product is unsustainable.

dalke20• 6 years ago

As a revenue-generating project I was able to share the burden by paying people to work on parts of it. I think that was a more effective way than waiting for others to join in.

Note that my paying commercial customers are pharmaceutical companies, where employees must present their presentation materials to their legal department before they give a talk.

In some cases I've heard that that extends to non-trivial code changes, which reduces the number of people who can help out.

Even setting that aside, I've been working on FOSS project for 20+ years. Outside code contribution is rare, and substantial code contribution is like hen's teeth. Bear in mind that most people in my field are chemists-who-program, not software developers.

One major exception is the RDKit, which was primarily funded by in-house R&D. Open sourcing it benefited the company because 1) it was primarily pre-competitive (the internal version included competitive features not in the FOSS version), 2) it lowers the price of commercial tools in the area, and 3) the differential cost of organizing the FOSS distribution was a net benefit.

This is similar to other successful FOSS projects. However, this model implies that one must be an employee of a large company in order to work on FOSS projects. Which is not for everyone. Clearly small profitable proprietary development shops exist, so why not FOSS ones?

WJW2• 6 years ago

In this viewpoint it should also become immediately clear why for-profit companies interact so poorly with FOSS communities: they are not interested in taking on unnecessary burdens and would prefer the original developer(s) contributing more free labour over having to pay for it.

dekhn3• 6 years ago

It's funny just how much the implementations described in the paper map to how modern search engines implement retrieval. The same is true for BLAST and other search engines.

(it's a very readable paper and I enjoy the frank expression of view, even if I have a vastly different perspective on how to accelerate problems like this)

nkurz• 6 years ago

even if I have a vastly different perspective on how to accelerate problems like this

Do tell more!

After the initial optimization, I did hint at an approach to Andrew that I thought could get a further large speedup. Essentially, the idea was to "rotate" all the stored data 90 degrees, so that instead of counting the features present in each compound you read lists of compounds that contain a given feature, storing the hits in some very fast custom data structure. He wasn't particularly interested, likely correctly realizing that it would be a lot of work for an uncertain amount of gain. The question wasn't really whether I could achieve further speedup (although there was question as to how much), rather (as alluded to in the paper) whether he would be able to sufficiently increase sales to justify the additional development cost and added complexity of the codebase.

dalke20• 6 years ago

Towards the end of the writing the paper, another paper came out on "RISC: rapid inverted-index based search of chemical fingerprints", https://doi.org/10.1021/acs.jcim.9b00069 which does something along those lines.

It was close enough that I published the pre-print "RISC and dense fingerprints" at https://doi.org/10.26434/chemrxiv.8218517.v1 to examine its claims. I found that their RISC implementation was faster than chemfp for low bit densities (<~5%), which includes the popular 2048-bit ECFP/Morgan fingerprints for smaller radii, and uncommonly high similarity thresholds.

Otherwise, chemfp was faster.

So while there's certainly something to investigate there, I think it's better to focus that effort on truly sparse fingerprints and count fingerprints, rather than nominally dense bit fingerprints.

Just needs money and time. ;)

Plus, part of the focus was on making chemfp a really good baseline for these sorts of timing tests.

dalke20• 6 years ago

There's a deep connection between what I do and text retrieval in general.

Take a look at the early work in IR in the 1940s and 1950s, at https://en.wikipedia.org/wiki/Information_retrieval#Timeline

1947, "Hans Peter Luhn (research engineer at IBM since 1941) began work on a mechanized punch card-based system for searching chemical compounds"

1950s, "invention of citation indexing (Eugene Garfield)" - Garfield's earlier work was with chemical structure data sets, and his PhD work was on the linguists of chemical compound names.

1950: "The term "information retrieval" was coined by Calvin Mooers." - that was presented at an American Chemical Society (ACS) meeting that year, and in the 1940s Mooers developed an early version of what is now called a connection table, hand-waved a substructure search algorithm which was implemented a few years later. (I'm a Mooers fanboy!)

Many of the early IR events were at ACS meetings - the concept of an "inverted index" was presented at one, as I recall.

This is because in the 1940s, chemical search was Big Data, with >1 million records containing many structured data search fields, and demand for chemical record search from many organizations.

So many of the core concepts are the same, though in cheminformatics we've switched to a lossy encoding of molecular features to a bitstring fingerprint since we tend to look more at global similarity than local similarity, and there are a lot of possible features to encode.

Thank you for writing that it was a very readable paper. I have received very little feedback of any sort about the publication, and have been worried that it was too verbose, pedantic, or turgid.

dekhn3• 6 years ago

Its a bit verbose, and I really think it's several papers (the technical details of the package is one, the open source positioning is another). But it's readable- a person outside the field (say, a search engineer at Google) could sit down, read this and immediately recognize what you were trying to achieve ("implement popcnt" used to be a popular question), and then immediately suggest ways to get the output results faster by using a cluster :)

rurban• 6 years ago

Im sceptical that a good single CPU search can compete with massive parallel HW, like this one: https://www.graphcore.ai/posts/introducing-second-generation...

dalke20• 6 years ago

Sure. It can't. Even GPUs will beat a CPU. In my paper I commented:

GPU memory bandwidth is an order of magnitude higher than CPU bandwidth, so a GPU implementation of the Tanimoto search kernel should be about ten times faster. Chemfp has avoided GPU support so far because it’s not clear that the demand for similarity search justifies dedicated hardware, especially if the time to load the data into the GPU is slower than the time to search it on the CPU. GPUs are more likely to be appropriate for clustering mid-sized datasets where the fingerprints fit into GPU memory.

Corporate compound sets have ~5 million records. That can be searched on a laptop in about 50ms.

A large data set containing physically measured properties is ~100M records, which takes a bit over a second. The largest data sets people search, with synthetically generated compounds, is around 1G records. That requires distributed computing. But most people don't work with them.

They say the best camera is the one you have with you. Most people have a CPU with them. Fewer have massive parallel HW with them.

The chemfp project: problems selling free software

Related Stories