Elasticsearch is open source, again

https://www.elastic.co/blog/elasticsearch-is-open-source-again

by dakrone • 2 years ago

759 438 2 years ago

simonw• 2 years ago

The good news is that while it was painful, it worked. 3 years later, Amazon is fully invested in their fork, the market confusion has been (mostly) resolved, and our partnership with AWS is stronger than ever. We were even named AWS partner of the year.

I don't entirely understand this bit.

encoderer• 2 years ago

I think they are saying when somebody looks to buy elastic search on AWS they find the Elastic offering on the market place and are not confused by the AWS offering that’s now called open search.

foxyv4• 2 years ago

It was so that AWS would create their own name for their fork:

we changed the license, knowing it would result in a fork of Elasticsearch with a different name and a different trajectory. It’s a long story.

I think the name of the fork is now OpenSearch.

Raed667• 2 years ago

what is preventing AWS from dropping OpenSearch and going back to just selling Elastic ?

anticorporate• 2 years ago

What could be learned from this example that might enable companies in the future to leverage trademark instead of a license change to accomplish the same thing? So much damage could have been avoided if this had been resolved differently to begin with.

orra5• 2 years ago

I don't understand why they didn't treat Amazon's previous offering as a trademark infringement.

ezekg4• 2 years ago

Am I the only one not buying this reasoning? Seems like there's more than is being said, otherwise they would have said this by now. I'd reckon that ELv2 had friction that couldn't be easily overcome without OSS or at the very least DOSP [0]. I personally experienced said friction with ELv2, so makes me curious.

[0] https://opensource.org/dosp

thih9• 2 years ago

Their post[1] from three years ago explains in more detail the reasons behind the license change and what they consider not ok behavior by AWS. The "it worked" likely means that they consider these problems resolved or otherwise no longer relevant.

[1] https://www.elastic.co/blog/why-license-change-aws

adonese• 2 years ago

I think they were trying to say something along the lines of we wanted amazon to take full responsibility over a fork and to dedicate actual resources for it, such that it will be distinguishable from our product.

giamma2• 2 years ago

My guess: as I wrote in another comment, Elasticsearch had significant evolution in the area of "search engine", while OpenSearch trajectory (to use the wording of the press release) has been in the direction of APM/monitoring.

When Elastic changed the license, all vendors of APM products based on Elasticsearch jumped to OpenSearch.

Maybe, they want to pass the message that the fork, OpenSearch, is enough different to not represent real competition, and in any case, now that it exists, after the investment done, AWS will offer that as an alternative to Elasticsearch and not Elasticsearch itself but managed by AWS as it used to be before the license change.

Now, who wants a managed Elasticsearch service will only buy it from Elastic.

skywhopper5• 2 years ago

The "market confusion" bit has always struck me as disingenuous. The market was never confused about who was offering the old AWS "ElasticSearch Service" or what it was. Elastic's licensing shenanigans and the fork they forced AWS to create have introduced far more long-term confusion. It's certainly not the case that any confusion has been resolved by licensing the ElasticSearch code under three different licenses, all of which are unusual, confusing, and untested in various ways.

unethical_ban• 2 years ago

Okay, cool. No notes, that's neat.

Are there any SMEs that have worked with both OpenSearch (the fork) and ElasticSearch? Are there significant differences?

I know the AWS fork had the big difference back then of having RBAC built into their Kibana portion.

DominoTree• 2 years ago

I've found OpenSearch to be a bit flaky but I haven't worked with it very seriously compared to ElasticSearch

(and before OpenSearch, the AWS-managed ElasticSearch absolutely hurt the ElasticSearch brand because of all of the issues Amazon created - it couldn't even rebalance shards, let alone add new nodes or switch to larger nodes without a blue-green deployment)

everfrustrated3• 2 years ago

This would be the same period when Elastic didn't even have a cloud offering....

candiddevmike2• 2 years ago

OpenSearch has terrible docs.

mentalgear• 2 years ago

That's so quintessential AWS :P. Even only for this Elastic should be compared to not waste 10s of hours on poorly written documentations.

therealdrag02• 2 years ago

Can’t be worse than Solr heh.

coding1232• 2 years ago

The spark components were rushed and the documentation is super crappy.

Elasticsearch hands down for this area.

beardface• 2 years ago

For most use-cases (indexing denormalised data as documents then running searches against them), there's little difference between the two. The mechanics of how the cluster operates are almost identical.

There are some Elasticsearch features that were part of X-Pack (their commercial offering) so aren't included in the OpenSearch fork. Some of those features are really nice to have and make life much easier; the enrich ingest processor is something I really miss in OpenSearch.

The biggest differences are in the tooling around Elasticsearch. All the observability stuff, the SIEM features, various integrations, and now the AI fluff. I've worked with clients in different sectors and - aside from the observability stuff (which is really nice) - none have had an appetite for any of that.

The OpenSearch team is doing some really cool stuff and the project has come a long way. I'm sure it'll continue to improve. It has a very loyal customer base and even has its own annual event; OpenSearchCon 2024 is next month!

maxk42• 2 years ago

Pure hearsay, but apparently ElasticSearch is considerably faster than OpenSearch. Would be good to compare the sources again, but I don't see any links to source on elastic.co yet.

jillesvangurp5• 2 years ago

That would be strange as it's largely based on the same Lucene library.

dinomoe• 2 years ago

https://www.elastic.co/blog/elasticsearch-opensearch-perform... https://www.elastic.co/search-labs/blog/elasticsearch-opense...

[I work for Elastic]

jillesvangurp5• 2 years ago

I support both professionally. The last Apache licensed version of Elasticsearch was a very capable product and Opensearch inherits all of that. In the few years since the license change both products have evolved a little bit but the vast majority of those changes don't really matter to new users. Both products use the same core component, which is Apache Lucene; which powers all of the search features. If you are a new user, there is very little reason to prefer Elasticsearch over Opensearch. And this is confirmed by the fact that most of my new clients default to Opensearch.

The exception to this might be vector search, which is a relatively new feature that was implemented on both sides post fork. Lots of users want/need this. And both Elastic and Opensearch provide independent implementations with very similar feature sets. Both build on what Apache Lucene offers on this front. So there isn't a massive difference between the two. But I would give the advantage to Opensearch here since its implementation offers a bit more beyond just the Lucene support.

Kibana had a lot of closed source components before the fork already and Amazon fork of that is a bit more limited. But notably Amazon indeed re-implemented the security layer (before the fork actually), which on the Elastic side is a bit of a dumpster fire of complexity. In general I'm not impressed with the product from a usability point of view. Either on the Elastic or the Opensearch side. But the Elastic version is arguably a bit richer in features.

Some notable recent changes there include trying to introduce a new query language based on SQL and the whole fleet ecosystem (an agent based system) to get logging and other data into Elasticsearch. I don't think either is getting a lot of traction because of the licensing.

richbell• 2 years ago

[D.N.A.]
[LOVE.]

I don't understand what these are supposed to mean. Is this a new writing style, song references, or just a quirk of the author?

newzisforsukas2• 2 years ago

Seems to be references to Kendrick Lamar songs... for some reason?

samplenoise• 2 years ago

They are Kendrick Lamar song titles. Not clear to me why, though

aabhay• 2 years ago

These are songs by Kendrick Lamar. I guess that these were used to try and add a youthful, lighthearted touch to the article? Didn't work on me though.

29athrowaway2• 2 years ago

[Count me out] No reason to use Elasticsearch in 2024.

NewJazz2• 2 years ago

I'm a fan of that album and I didn't even realize the reference. Is there any other point to it?

busterarm• 2 years ago

I really hate this corporate trend. The soundtracks to all of our company meetings are basically mostly songs about wanting to fuck that girl you met in the club last night and I seem to be the only person who finds it incredibly inappropriate for the setting.

0x5FC3• 2 years ago

All those are Kendrick Lamar tracks. Highly recommend if you're into hip hop.

jcoc6112• 2 years ago

It kind of reads like it was LLM generated..."Write an announcement/apology about ElasticSearch finally being open source again (for now), where each paragraph starts with a relevant title from a Kendrick Lamar song"

shiomiru• 2 years ago

To me, at first, it read as satire - but that doesn't make sense, coming from the official blog. Being LLM-generated is a plausible explanation - considering the circumstances, saying "open source is in our DNA" is right inside the uncanny valley.

reducesuffering• 2 years ago

If Amazon's OpenSearch plateaus, and everyone wants to host ElasticSearch, but on their existing AWS infra/sales business inertia, this issue will fundamentally reoccur. Nothing actually changed except Elastic thinks Amazon will commit to their own fork. If Amazon doesn't, we're back to square one: Amazon hosts open source ElasticSearch, Elastic changes their license, another fork.

"Amazon is fully invested in their fork." Amazon is a cutthroat business that will change strategy if their investment isn't paying off.

That this scenario isn't addressed at the very top of your "addressing the trolls" doesn't bode well at all.

JoshuaRogers• 2 years ago

I felt like that was implied by the usage of AGPL: if Amazon wanted to start using this and apply patches on top of it, the AGPL would require that they share those patches with their customers, which would allow Elastic to integrate them into main again.

quyse• 2 years ago

I don't think that's their intention. Elastic wouldn't be able to integrate Amazon's patches back into their codebase without losing the ability to change the license in the future. Even more, since it's AGPL, they'd have to get rid of their other licenses immediately.

NewJazz2• 2 years ago

Is that important, though?

Isn't the whole point that amazon doesn't care about source availability or openness, so long as they can extract profit from people running it?

chadash3• 2 years ago

I love https://resend.com/ as an example of a website that does a GREAT job explaining what they do. Lots of companies tell you that integrating their product is quick, but with their website I can look at it for 30 seconds and understand what the next steps would be.

zenorocha• 2 years ago

thanks for the kind words, we spent a ton of time on the website ;)

newzisforsukas2• 2 years ago

Great, how long until they change the license again?

troyvit• 2 years ago

Depends on what new tricks ~FAANG~ MAMAA companies come up with to get as much as they can from open source without giving anything back I suppose.

outop7• 2 years ago

Whatever you think of those companies in a wider sense, it's totally inaccurate to suggest that Microsoft, Google or Amazon haven't given anything back to open source.

everfrustrated3• 2 years ago

Too late, the community all moved over to OpenSearch.

I'll never forgive Elastic for locking basic security features behind their paid licence. Over the years probably millions of people had their data compromised due to that (due to people inadvertently leaving instances on the public internet - having auth enabled by default would have helped a lot)

jgb1984• 2 years ago

Source? Unless you're locked into the Amazon flavor of cloud hell I'm pretty sure most people are using the "real" elasticsearch. I know I am.

tapoxi• 2 years ago

We have a vendor that refuses to support ES 8.x due to license fears, so we're just migrating everything over to OpenSearch.

paulddraper2• 2 years ago

Yeah, it hasn't been nearly long enough for "everyone" or even most people to migrate.

sergiotapia• 2 years ago

I think it's pretty sad that Conglomo X can take an open source product, add some tweaks and sell/capture the entire market. Basically taking all the work that open source maintainers have done.

What's the solution here?

echelon3• 2 years ago

Source available + MAU/ARR restriction clauses that prevent hyperscalers from robbing the bank.

jddj• 2 years ago

I've always been a little sceptical as well of the grassroots (maybe, maybe not) resistance on here against any efforts to stop the big 3 from using their network effects to syphon off the revenue from these projects.

The messaging against elastic style licences and even copyleft licenses is too convenient for me to trust as being 100% genuine.

outop7• 2 years ago

This is literally what open source is. If you don't support this, you don't support open source.

There isn't a 'purer' form of open source which does exactly what you want with respect to big companies using the code.

You can be in favour of licensing that restricts Amazon or Microsoft's right to use your work. But that position is detrimental to, not supportive of, open source, since such a license would not be open source.

liveoneggs• 2 years ago

Like elastic did to Apache Lucene, the core of their product?

Or when they infected their previously open source components like logstash?

valyala2• 2 years ago

The main incentive for maintainers of open source software is to see many happy users of this software. Most open source maintainers do not care about monetizing their products (or the monetization has very low priority).

That's why most open source maintainers use truly open-source licenses for their software - BSD, MIT, Apache2, and they will be happy if Amazon, Google or Microsoft builds successful product on top of their open-source software, since this gives them the desired recognition.

echelon3• 2 years ago

Yes, let's praise and reward the hyperscaler and not the small company that is 1% of the size of AWS.

AWS got Elastic's goodies for free. They just came in and gobbled up all of that value for themselves like vultures. Meanwhile the people putting in the work effectively got robbed.

Small companies should stop doing open source and switch to source available + MAU/ARR restriction clauses.

zokier• 2 years ago

The small company with over billion dollar revenue and thousands of employees?

lsh0• 2 years ago

the small company that is 1% of the size of AWS

What on earth? Elastic is a multi-billion dollar company. They are no indie startup, scrappy underdog nor are they victims here.

AWS took the high road during this fiasco despite Elastic's mudslinging and flailing about.

globular-toast• 2 years ago

They shouldn't stop doing open source, they should just make sure they use the correct licence, like AGPL. Businesses will always take as much as they can and give back as little as they can. The "permissive" licences specifically allow them to give back nothing. Cue shocked Pikachu when they do exactly that.

coding1232• 2 years ago

They did? I can't stand all the missing parts around documentation and the lesser known pieces like Spark integration - horrible.

the_duke• 2 years ago

That's hugely overblown.

Maybe 20% of the companies I work with have switched.

jillesvangurp5• 2 years ago

It's the new users where the pain is. A lot of companies that were using Elasticsearch indeed haven't switched because doing so is a bit of work without a lot of advantages. But all my new clients are defaulting to Opensearch. It's not even close.

latchkey• 2 years ago

Maybe 20% of the companies I work with have switched.

Smells like opportunity!

lolinder12• 2 years ago

What is "the community" here? Every metric I'm able to find (commit frequency, gh stars, gh stars gained since the launch of opensearch, stack overflow tags, google search result counts...) suggests that OpenSearch didn't really take much of Elastic's mind share in the end. Each metric I've looked at shows OpenSearch at somewhere between 20%-50% of Elastic's numbers, which isn't nothing but is a far cry from "the community".

I've heard a few anecdotes suggesting some people took it seriously, but while we're sharing those: my company actually adopted ElasticSearch since the license change and never seriously considered OpenSearch.

js4ever4• 2 years ago

Last time I deployed elastic was years ago. In the meantime I deployed hundreds of opensearch instead for customers.

mirashii• 2 years ago

For example, MongoDB used to be AGPL and Grafana is AGPL. It shows that AGPL doesn’t affect usage or popularity.

I take some issue with this characterization. Let's look at Grafana in particular. Grafana was not always AGPL, and much of its popularity came before the license change. I've been in multiple organizations who only purchased a license for Grafana to avoid the AGPL terms because it had gained traction already in the organization and switching away would have been more costly, and AGPL software is still outright banned.

That Grafana is still popular does not show that the AGPL doesn't impact usage or popularity, only that Grafana is still popular.

skrtskrt10• 2 years ago

AGPL software is still outright banned

This is just an unforced error to enforce this for a product that is literally an internal analytics tool. You can even host it and sell it as a service under AGPL! you just have to open source your changes/contributions.

The Grafana AGPL-licensed stuff has massive adoption, the few places where corporate lawyers can't get their heads out of their butts can just keep suffering.

CtrlAltDelete51• 2 years ago

Open source, again … for now.

system22• 2 years ago

Nice, maybe they can hire some humans to write more explanatory website text, documentation, pricing information—better yet, anything on their website.

Elastic.co is the #1 example we use with our clients when we want to show how 'vague websites make you lose clients.' We show them the website and ask, 'What do you think of this company?' and 'What do you think they are providing as a service?' Not a single client, including tech-savvy ones, has been able to answer.

Elastic.co is probably one of the worst websites that somehow gained popularity despite its crappy pricing model and support. Their documentation assumes you already know everything about their weirdly vague services and have in-depth knowledge of server infrastructures.

To anyone who works for them: If you're reading this, know that your website is so terrible that it became our first example of a crappy company.

dboreham2• 2 years ago

I assumed that "website that tells you nothing" was a deliberate thing, presumably because somehow they make more money as a result. It seems to be the rule more than exception.

system22• 2 years ago

I can't think of any company does that and makes money. Imagine Apple doing the same, they'd go bankrupt.

candiddevmike2• 2 years ago

Stock is down almost 25% afterhours (not sure if due to this or their earnings call)

mrcwinn• 2 years ago

"'We had a slower start to the year with the volume of customer commitments impacted by segmentation changes that we made at the beginning of the year, which are taking longer than expected to settle. We have been taking steps to address this, but it will impact our revenue this year,' said Ash Kulkarni, Chief Executive Officer, Elastic."

So, whatever that means, is the answer to your question.

trog• 2 years ago

I guess it's better to have nerds fighting about licensing filling up the Internet instead of people talking about why there's a 25% decline in the share price :)

nprateem• 2 years ago

Segmentation changes probably means more companies are in the "not interested" segment now than we'd like. We're going to throw a load of stuff at the wall this year to see what sticks.

29athrowaway2• 2 years ago

There's already an open source Elasticsearch, it's called OpenSearch and it is pretty much all you need right now.

Users that had to pay the price to migrate to OpenSearch do not have a reason to migrate back to Elasticsearch.

yencabulator2• 2 years ago

Also, OpenSearch is Apache-2 licensed so much more usable.

asjkaehauisa• 2 years ago

Also, opensearch is free, enterprise-ready tool. It's 2024 and you have to pay for things like ldap auth in elasticsearch. https://sso.tax/

rdsubhas• 2 years ago

Lots of negativity in this thread. Can we take just a bit of time to appreciate this. Thank you Elastic folks!

bytearray• 2 years ago

What are we appreciating exactly?

politelemon• 2 years ago

Elastic being flexible

zymhan• 2 years ago

Maybe a ridiculous writing style for a blog post about software licensing?

imp0cat• 2 years ago

We've altered the license terms. Pray we don't alter them any further.

It's hard to not be sceptical about their motivation.

xbar• 2 years ago

Nice! Time to switch back real quick.

chadash3• 2 years ago

I'm pretty happy with this, since they are keeping the option to use the Elastic License. Now everyone can be happy. To me, it's weird that the AGPL is any more "open source" than the Elastic License. The AGPL requires you to publish all of your source code if you make any changes to the product; the Elastic License just says, "don't use our code to make a direct competitor to Elasticsearch". I find the former to be much more restrictive in most practical ways since the majority of companies don't want to open source their code, but very few of them plan to sell hosted search.

Personally, I do wish that there was more broad acceptance of the Elastic License. Who wants to put in years building a business and then have a competitor with better distribution take your code and compete directly with you? For me, the reasons to want open-source code are:

* If a vendor goes under, I can self-host

* If a vendor raises prices too much, I can self-host

* If there's a bug in the code that affects me too much, I can fix it

* If there's a feature I really need, I can add it

The Elastic License allows for all of the above. Seems fair to me.

sofixa2• 2 years ago

Same applies to BSL and similar. Not being able to compete with the project owners is much less restrictive, for me, than AGPL/GPL.

outop7• 2 years ago

But it restricts your ability to use a commodity product based on Elastic, provided by a third party who will compete on price or bundle it with other cloud services.

hamilyon2• 2 years ago

GPL is viral and somewhat compatible with AGPL.

BSL and GPL code are probably never mixing since they prohibit each other. This creates friction in GPL world and tends to produce incidents line this [1] out of thin air.

1. https://github.com/jshint/jshint/issues/1234

ezekg4• 2 years ago

If a vendor goes under, I can self-host.

It's worth mentioning that this is true -- to an extent. Under ELv2, if the vendor goes under, you can self-host, but you will eventually lose access to any features protected by a license key if/when that license expires, since said vendor can no longer renew said license.

This was one of the main drivers for me writing the FCL [0], which undergoes DOSP [1], even for the protected features.

[0] https://fcl.dev

[1] https://opensource.org/dosp

ensignavenger6• 2 years ago

You also can't pay someone else to host it for you. Nor will the community be able to fork it and support development by paying one or more community members to host it for them.

At least with DOSP, eventually the community will be able to do those things.

lolinder12• 2 years ago

to publish all of your source code if you make any changes to the product

Specifically, this is the text [0]:

if you modify the Program, your modified version must prominently offer all users interacting with it remotely through a computer network (if your version supports such interaction) an opportunity to receive the Corresponding Source of your version

There are a few companies who try to make it sound like if you interact with an AGPL program over a network then your client code is now infected with the AGPL, but I'm not at all sure how they arrived at that conclusion unless it was willful misinterpretation.

Under the mainstream view, you only have to publish the source code for the AGPL work that you modified, which for 99.9% of users is fine but isn't great for a reseller.

The main barrier isn't the actual text of the license, it's that AGPL is still untested in court and there are companies who will try to make it mean something different than its apparent meaning, so legal departments are liable to get antsy. But lawyers are likely to get antsy about self-hosting under these other licenses as well.

[0] https://www.gnu.org/licenses/agpl-3.0.en.html

ratorx3• 2 years ago

not great for the re-seller

If the AGPL is exactly as you say, I don’t see why this would be a problem for a re-seller. For a pure re-seller I don’t think the value add is provided by modifying the software.

E.g. take the example that Amazon hosts the service and integrates with their internal services etc for logging, storage, load balancing etc. If they only have to distribute the modified source, then their internal service APIs will be leaked. This is probably fine most of the time, but what if the API reveals too much of the secret sauce (very unlikely, but possible so necessary for legal CYA, or extra approvals every time you want to modify the AGPL code). In a more devil’s advocate reading, the following stands out (IANAL, just conjecturing):

all the source code needed to generate, install, and (for an executable work) run the object code

Do I need to include my super secret storage engine X because my modification requires that I use it? Let’s say I write the code in a way that it is an optional dependency, but because of a programming mistake, a single version goes out where it becomes a non-optional dependency, do I now have to include it (for the users of that version)?

In an even more contrived case, let’s say I integrate it with a vendor closed source program X. The virality is impossible to satisfy, unless I negotiate an AGPL license for X.

Source includes interface definition files associated with source files for the work, and the source code for shared libraries and dynamically linked subprograms that the work is specifically designed to require, such as by intimate data communication

Intimate data communication is pretty vague. Imagine the AGPL is a database, and I write a custom storage engine. Naively that seems pretty intimate to me.

Either correctly or incorrectly (because it’s never been tested), the perceived virality of AGPL is probably a major reason, regardless of the actual intent.

remram6• 2 years ago

They define it earlier in the license:

To "modify" a work means to copy from or adapt all or part of the work in a fashion requiring copyright permission, other than the making of an exact copy. The resulting work is called a "modified version" of the earlier work or a work "based on" the earlier work.

While the AGPL might be untested, copyright isn't, and I don't think any copyright lawyer would say that "calling over the network" is adapting "the work in a fashion requiring copyright permission".

pmontra• 2 years ago

There are a few companies who try to make it sound like if you interact with an AGPL program over a network then your client code is now infected with the AGPL

Of course not, or

1. Any web browser would have to be open sourced, and

2. That would be the consequence of an action taken by a third party (eg: me) and not by the parties that created the web app and the web browser.

tensor• 2 years ago

If the client talks to service A, which talks to AGPL service B, I assume that would count as having to "prominently offer the source code for service B". No? If that's true, then it becomes a real pain to track all the places where an end user could indirectly come in contact with service B.

If that's not how to interpret the license then wouldn't a simple API gateway or proxy circumvent it?

SpicyLemonZest• 2 years ago

The Elastic License prohibits you from moving, changing, or disabling some of the software's functionality. It's a limited compromise, and I understand why it's necessary to achieve their business objective, but it's pretty straightforwardly not compatible with the open source ideal.

Imagine what the web would be like if React users weren't allowed to compete with Meta.

re• 2 years ago

The AGPL requires you to publish all of your source code if you make any changes to the product; the Elastic License just says, "don't use our code to make a direct competitor to Elasticsearch". I find the former to be much more restrictive in most practical ways since the majority of companies don't want to open source their code, but very few of them plan to sell hosted search.

There's two ways that this doesn't seem right to me, though it hinges on the vague term "interacting" and how it's interpreted.

Suppose I use Elasticsearch to power website search on my company's website -- maybe something like a customer support knowledge base of a bunch of FAQs and support articles, and I make some modifications to Elasticsearch to better fit my requirements. My website makes calls to an Elasticsearch service to provide search results.

1. Based on my interpretation of the AGPL, visitors to my site who make searches are not remotely interacting with the Elasticsearch software that I am running; they are not sending requests directly to the Elasticsearch software, and thus they have no rights to its source code under the AGPL. (I'm not suggesting that a proxy server that passes on requests and responses unmodified would be the same situation.)

2. If they do in fact have rights to the source code, it is only to the modified version of Elasticsearch, not "all my source code" (which could include the web server software itself).

Notwithstanding any other provision of this License, if you modify the Program, your modified version must prominently offer all users interacting with it remotely through a computer network (if your version supports such interaction) an opportunity to receive the Corresponding Source of your version by providing access to the Corresponding Source from a network server at no charge, through some standard or customary means of facilitating copying of software. https://www.gnu.org/licenses/agpl-3.0.en.html#section13
In AGPLv3, what counts as “interacting with [the software] remotely through a computer network?” If the program is expressly designed to accept user requests and send responses over a network, then it meets these criteria. https://www.gnu.org/licenses/gpl-faq.html#AGPLv3InteractingR...

SahAssar2• 2 years ago

The AGPL requires you to publish all of your source code if you make any changes to the product; the Elastic License just says, "don't use our code to make a direct competitor to Elasticsearch"

"changes to the product" means changes to the service itself, and "publish all of your source code" means the specific service, not for everything you build. If you patch the ES service you make the patch public, but you don't need to make any service calling ES public. That is pretty static and controlled on your end.

On the other hand "direct competitor" can change over time, so that if elastic buys a competitor to my product or reinterpreters what it means to be a competitor it changes how I can use the software. Say you were early into ML stuff and built a RAG on top of ES, ES will probably offer that soon as a service (if they don't already) so now you are a competitor without any change to your business. Or you want to launch a small project that is ambiguously adjacent to a component (with these non-OSS licenses) that another team within your company uses from a third party. That now becomes a huge legal liability and risk, regardless if you use that exact component to compete with that supplier or are even aware of it.

At least that is the way I've understood the risks of these non-OSS licenses and have gotten similar advice from lawyers at major users of OSS.

chadash3• 2 years ago

The words of the license are "You may not provide the software to third parties as a hosted or managed service, where the service provides users with access to any substantial set of the features or functionality of the software.".

I don't think competing with ElasticSearch is mentioned anywhere within the license. If team 1 uses ElasticSearch and team 2 is developing RAG (without using ES), then that's not an issue (but only if using the most recent version of ES, since the license for the code that I fork today has no bearing on future code that ES hasn't written yet).

jchw2• 2 years ago

For the most part I don't think people are against shared source or closed software existing, being sold, being marketed, etc. There's really only two things people viscerally don't like:

- Marketing a project that isn't open source as open source. Debate about what the "definition" is or why it matters all you want; taking a term and using it in a way that contradicts the vast majority of domain experts is bullshit.

- Taking an open source project, which people adopted on the basis that it was open source, which people contributed issues and pull requests to on the basis that it was open source, which people evangelized and promoted because it was open source, blogged about, built on, and so forth because it was open source... and moving it to a license that isn't open source.

To be clear: yes, the unforced error here in many cases is accepting a CLA. That said, I think it's not even unreasonable that people initially accepted CLAs: many of them presumably believed they would only ever be used in good faith, as a sort-of CYA. But CLAs are now very commonplace, so refusing to contribute to any project with a CLA requirement is hard.

If nobody cared about the benefits of open source, then it would be easier for companies to just start with a closed or shared source offering and call it a day; not much backlash for not changing a license. Clearly, marketing something as open source helps... but once you've gotten what you need out of it, it's easy enough to click a button and change it back to being closed.

In my opinion the big advantage of open source is that everyone is on a level playing field. This isn't "fair", it's balanced, and that matters if you are serious about long-term software. If shared-source software is discontinued, that's probably the end of the road for it. For open source software, it only depends on if there are big enough stakeholders to keep funding development; it never has to stop.

There's ideas like BUSL, which might work better... but it's still awkward and experimental. I don't put much stock into any of the other "shared sorta-like-open source" licenses, they're mostly bullshit and sometimes catastrophically horrible, i.e. much worse than AGPL.

prmoustache• 2 years ago

BUSL is the worse of both world imho. People are not willing to send patch or contribute to something that is not yet open source and vendor thus does not get any benefit of having openly readable sources.

sanderjd5• 2 years ago

I think you make good points here, but it's also annoying that the words "open source" are defined to mean something a lot more specifically detailed than what the words themselves intuitively mean.

For instance, your post calls things "shared source", which, to me, is a lot less clear of a description for the projects you're describing that way. ("Shared" how? Shared ownership? Or what?)

I think "source available" is intuitive and fine (and better than "shared source"), but to me it's still a bit weirder. To me, it sounds like if you send the company an email, they might send you back a zip file with a bunch of source code. But most of these "source available" projects operate just like any other open source project.

But I'm also not unsympathetic to your arguments here at all.

ensignavenger6• 2 years ago

I would rather a software product be eventually open source than use a never open source license, but I still try not to use it if I can choose open source. And I refuse to sign CLA's that require giving more eights than the license grants to me, and won't sponsor projects that require them. (with some limited, carefully considered exceptions for well established open source foundations that require CLAs, but that have sufficient gornance to add trust)

bad_user2• 2 years ago

If you use the Elastic license, legally speaking, you're in hot waters. The biggest problem with software licenses for freemium is that you have no contract with the company, money doesn't change hands, and the license itself can be open to interpretation. What's a competitor anyway? This sounds like that JSON license saying you shouldn't use the software for evil.

The Open Source licenses have been vetted and are time tested. That's one big reason for why Open Source is valuable. When you adopt an OSS project, you know exactly what you're getting, and the legal departments of corporations are prepared for it. Some are banning copyleft licenses, of course, for good reasons, but the knowledge is there.

Personally, I wouldn't touch the Elastic license.

jotaen• 2 years ago

The Elastic license doesn’t use the term “competitor”. To me, the definition of the limitation is actually pretty clear:

You may not provide the software to third parties as a hosted or managed service, where the service provides users with access to any substantial set of the features or functionality of the software.

https://www.elastic.co/licensing/elastic-license

crewdragon• 2 years ago

It feels like Elastic got burnt with the license change, their stock is down 40% since they announced the fork, and they are starting to realize that being open source is important. I don't think AWS would abandon the fork given the amount of efforts they put in, they cannot walk back and re-brand their products.

It's sad to see elastic turning sides for their benefit, and as a contributor I feel betrayed. While OpenSearch on the flip side is more contributor friendly. I honestly feel all energies should be focused on one product to make it better instead of walking in different paths. Amazon has already taken that path and I don't think they will ever walk back, unlike Elastic.

maccard3• 2 years ago

It’s impossible to know what would have happened if they had continued on their existing path. My guess is Amazon would have eaten their lunch and they’d be in a similar situation.

giamma2• 2 years ago

My understanding (after talking to several market analysts) is that OpenSearch is focused on APM/monitoring/log-aggregation, while Elasticsearch has an edge on pure search engine functionality and now AI.

That's because the license change by Elastic impacted not only Amazon, who could not provide Elasticsearch as a service anymore through its administrative consoles, but also all those vendors who were building APM/monitoring/log-aggregation solutions as-a-service on top of Elasticsearch. In fact, such vendors would typically use Elasticsearch as a back-end behind some custom UI.

So those vendors teamed up with AWS to develop OpenSearch.

Now last time I checked the commit history of the two projects, Elasticsearch had 3x more commits and many of them on cool new stuff, while OpenSearch focus seems to have remained on APM/log aggregation.

As someone who needs an actual "search engine", I am glad of the change, as I was worried OpenSearch may not be a viable open source alternative as it could be lagging behind in this domain.

Now I need to check what happens with the clients: will the client remain Apache License or will they change to AGPL? The latter would be a problem for closed source software.

winterowlpigeon• 2 years ago

I think your comment around the dip in their stock price is fairly misleading because it lacks context of the market sector overall. When ETSC announced in January 2021 their stock was around 150 and by November 2021 it was in the 180s, so the change very much was not responsible for crashing their stock - the market was. Their entire industry sector was pounded heading into 2022 and has never recovered. For example Datadog crashed from ~$190 to $80 over the course of 2022.

jraph2• 2 years ago

Your users might want to benefit the same rights that you are listing. Hence the GPL family of licenses.

sneak• 2 years ago

The AGPL is more restrictive than the nonfree license because the AGPL is also a nonfree license.

I await the day that the industry corrects itself and stops calling the AGPL open source/free software. It isn’t. It is very obviously a EULA, despite what the anticapitalist zealots at the FSF wish to claim.

consteval• 2 years ago

It is very obviously a EULA

It's really not, because the end users of your service, that being whoever consumes them, does not care about the AGPL and CAN close source their code.

If I call an AGPL service I can do that from a proprietary application. What I can't do is publish an AGPL service, modify that service, and then hide the modifications. So it works just like GPL in that way except instead of, like, including it's publishing an internet-available service.

Companies are super scared of AGPL but that's just because they're scaredy cats (sorry, "risk averse"). But no, you're free to publish an AGPL service and you can even monetize it, if you want. You're also free on the client-side to do whatever and have whatever license you want for your code.

sanderjd5• 2 years ago

This is exactly where I'm at.

I find what seems to be the prevailing opinion of people here (and in similar places) of passionate opposition to these kinds of licenses to be very mystifying.

It seems to me like they hit a pretty good spot on the continuum of trade-offs here.

I might add one, which is related to your third bullet point, but which I avail myself of far more often:

* If I'm confused by how something seems to work, I can read the implementation.

eadmund2• 2 years ago

To me, it's weird that the AGPL is any more "open source" than the Elastic License. The AGPL requires you to publish all of your source code if you make any changes to the product; the Elastic License just says, "don't use our code to make a direct competitor to Elasticsearch". I find the former to be much more restrictive in most practical ways since the majority of companies don't want to open source their code, but very few of them plan to sell hosted search.

The four freedoms of software are: the freedom to use the software for any purpose; the freedom to study and change the software; the freedom to share the software; and the freedom to share one’s changes. The AGPL permits all four; the Elastic License does not allow using the software to make a competitor; therefor the Elastic License is not a free software license.

More details: https://www.gnu.org/philosophy/free-sw.en.html

Who wants to put in years building a business …

Free software is not about the original author of code; it is about the users of that code and what they do with it. Copyleft ensures that those who build upon a software foundation grant the same freedoms to their users which they themselves received. Free software is about the users.

9dev• 2 years ago

That is a very US-centric perspective on freedom, and one that isn’t actually very helpful in assessing actual real-life restrictions of a license. You may win the philosophical argument on the nature of freedom, but you lose the debate participants. If both the vendor can’t continue working on the software, because they’re unable to monetize it, and the user is unwilling to use it due to concerns over exposing business secrets, the outcome is a net-negative, no matter how venerable your cause might be.

skrtskrt10• 2 years ago

the four freedoms were not written by God, just a bunch of ideological pedants.

it's perfectly valid to have a completely different view of what freedom is for software

philippemnoel• 2 years ago

This is huge. We wrote "Why we picked AGPL" only a few weeks ago, discussing why ParadeDB (an Elastic competitor!) chose the AGPL. Glad to see Elastic is joining forces on the AGPL front.

blendergeek• 2 years ago

Thank you, ElasticSearch!

I know the AGPL may be a terrible license and all, but it is allowed by Free Software purists. I hope more companies follow suit.

winddude• 2 years ago

They've probably been losing market share to all the opensource vector search solutions. But anyways, this is still good.

andrewmutz• 2 years ago

For use by businesses, the AGPL is a nightmare from my perspective. What does it actually require on behalf of a company using AGPL components? If I write my own library and link statically I need to release that? What about if I link it dynamically? What if the library is running on a separate machine and is separated by the network?

I'm sure there will be people commenting in this thread that they understand exactly what the AGPL requires, and it's not that bad, but their opinion matters much less than the opinion of lawyers.

I've never been able to get lawyers in a business setting comfortable with us using AGPL components, for fear that it will be interpreted at some point to require us to release our application source code.

As a result, we've never been able to use anything licensed AGPL in a corporate setting.

Nathanba• 2 years ago

Those are the same reasons for why I avoid AGPL (and GPL frankly, because I dont want to make my frontend code fully open source either). I always thought that when a project is AGPL then I might as well consider it off limits for myself because if any of my code is touching it and then that code indirectly eventually reaches users via the network then wouldn't that also mean that I have to open source everything? I mean otherwise you could just put any AGPL software behind a proxy and you'd be instantly fine which I don't think that's how it works. In an ideal world we would have a license where you're required to open source only the changes you make to the 3rd party code, not everything that surrounds it and without restrictions on linking or network. But I guess that is too hard to enforce.

jen204• 2 years ago

In an ideal world we would have a license where you're required to open source only the changes you make to the 3rd party code, not everything that surrounds it

Good news! That license mostly exists: it’s called AGPL.

The one where you have to open source “everything” (not really everything, but close to it) is SSPL [1], but that requirement only applies in certain circumstances.

[1] https://en.wikipedia.org/wiki/Server_Side_Public_License

remram6• 2 years ago

The AGPL is actually pretty clear:

if you modify the Program, your modified version must prominently offer all users interacting with it remotely through a computer network (if your version supports such interaction) an opportunity to receive the Corresponding Source of your version
To "modify" a work means to copy from or adapt all or part of the work in a fashion requiring copyright permission, other than the making of an exact copy. The resulting work is called a "modified version" of the earlier work or a work "based on" the earlier work.

Calling over the network is definitely not "adapting (...) in a fashion requiring copyright permission". While the AGPL might not be very much tried in court, copyright is.

tyre2• 2 years ago

It’s not the network call, it’s what you do with it.

Let’s take redis, pretending it’s AGPL.

Adding a new command to the source code: definitely a modification.

Adding a new feature to the redis client: also a modification.

Creating a class (RedisPlusClient) that inherits from its client and adds a feature: Maybe a modification? Feels modification-adjacent.

Building a microservice that uses the redis client and exposes the exact Redis API? I don’t know! Maybe? Feels like a technicality that it is “using a network call” so it doesn’t count.

The thing is, I don’t want to think about any of this. I want to build things. We have seen overwhelming evidence that for-profit companies give back to open source, both through contributions and open sourcing their own code.

Are these licenses limiting the actual impact of the projects in exchange for ideological purity? There is consistent feedback from engineers that this is the case.

qaq• 2 years ago

It being clear to eng. and how lawyers evaluate risk are pretty orthogonal.

aljarry• 2 years ago

Maybe the license is pretty clear, but interpretations differ. E.g. Minio provides a very aggressive interpretation of AGPL, equating to "if you use it in closed-source commercial product, it's a violation of AGPL": [0], [1], [2]. For me the whole problem with AGPL is that it's so subjective.

[0] https://github.com/minio/minio/issues/13308#issuecomment-929...

[1] https://github.com/minio/minio/issues/12829#issuecomment-889...

[2] https://github.com/minio/minio/discussions/13571#discussionc...

pluto_modadic• 2 years ago

what.... are those random words... in brackets??

hs86• 2 years ago

Reminds me of HK-47 from KOTOR: https://www.youtube.com/watch?v=UPeI4mX8Nus

gradascent• 2 years ago

They're all names of popular Kendrick Lamar songs. Don't ask me why though

hashtag-til2• 2 years ago

Perhaps AI generated?

therealdrag02• 2 years ago

A bit aside but anyone have experience using elastic managed on Azure with a many terabyte+ cluster for prod? Just curious if having less control etc has ever been problem.

greatgib• 2 years ago

Despite what they are claiming, they are clearly changing back the license because they felt the pain of not being Open Source anymore.

The proof that companies like that publicly under value the success that they owe to being Open Source.

   But being able to use the term Open Source, by using AGPL, an OSI approved license, removes any questions, or fud, people might have.

Also it makes me laugh so much the guy is trying to victimize himself pretending that they are unjustly targeted by FUD when it is not FUD but a real existing problem.

haolez• 2 years ago

Is it still open core? Which parts are only available on the enterprise version? Couldn't easily find in their website (after this announcement).

hintymad• 2 years ago

Could be a defensive move of Elasticsearch to win the mindshare of the community. OpenSearch has been gaining some momentum, albeit far from being comparable to Elasticsearch. See OSS Insight: https://ossinsight.io/analyze/elastic/elasticsearch?vs=opens....

I'm not sure Amazon is willing to go all in to win the war of search services, for that means they need to to handsomely reward the best coders that contribute the most to the OpenSearch project to produce insane amount of high-quality code for better and new features. See the great article Code Hard Or Go Home: https://hypercritical.co/2013/04/12/code-hard-or-go-home. Nonetheless, there's a chance that Amazon may be determined to make OpenSearch catch up with ES, just like Apple has made Apple Maps comparable, even not better than, Google Maps. Therefore, open sourcing ES with AGPL is not a bad choice to retain the talent in the ES community.

And AGPL is kinda restrictive to many large customers too, as the customers do not want to risk being forced to open source their business-critical code. In fact, many companies simply ban the use of licensed software. Therefore, AGPL reflects quite the spirit of OSS while in the meantime will not undermine Elastic's business model.

dblock4• 2 years ago

Note on the ossinsights data point, there are 115 public repos in https://github.com/opensearch-project, including, as an example https://github.com/opensearch-project/k-nn where all vector search is written, or https://github.com/opensearch-project/neural-search.

nerdjon• 2 years ago

Honestly at this point I just see little reason to invest in Elastic Search.

I have to wonder how much what is happening with terraform and tofu is related to this.

While I can understand why they went down this path, it burned a bridge and I just don’t know why I would even bother instead of using opensearch.

9cb14c1ec0• 2 years ago

Too late. We just deployed a new project with OpenSearch after learning from an Elastic salesperson that their licensing fees would eat up significant portions of the projects profits. Also, since it's not spelled out clearly, I'm supposing huge parts of vital functionality are still subject to a paid license, even if the core functionality is technically open source now.

remram6• 2 years ago

Plus they lost all trust now. Who's to say they won't pull the same thing again if developers were to come back? It's not like they stopped requiring a CLA...

KingMob5• 2 years ago

To be fair, though, every project of a certain size requires you to sign away your rights via CLA, so I don't think that can be held against them. (Though I admit, dispensing with a CLA would be an amazing gesture of good will.)

wg03• 2 years ago

Bit hard to trust - plus OpenSearch would continue to go strong because lot many companies have built business on top of it. Most of the centralised logging providers come to mind.

Grafana and Elasticsearch with their AGPL, are not deployable for teams that don't even want to provide a competing service because even basic security features (group membership via external OAuth source for example) are only available in Enterprise Edition (TM)

fsndz• 2 years ago

Elastic trying to reclaim the lost ground in their original territory by re-embracing open source is a bold move. They might be hoping to attract developers and businesses back by offering more transparency and community involvement. Could this have anything to do with vector databases starting to eat into some of their market share, pushing Elastic to innovate and re-establish their relevance?

ppaanngggg• 2 years ago

But es has fallen behind in vector search

ac130kz• 2 years ago

Too late, there are so many competitors that are both faster and provide more features. Same will happen with Redis in 3-4 years.

metadat• 2 years ago

Hasn't everyone already moved on to Open search? At this point it's more stable, which is preferable.

Too little too late. Cannot trust.

sschueller• 2 years ago

I am in the middle of the migration from Elastic Search to Open Search... Not sure what to do right now.

wg03• 2 years ago

Go for OpenSearch. Elastic might flip the license once more depending upon how and what stock market wants.

changexd2• 2 years ago

Petition to make all the official blog post written like this, I love the rap songs reference

mcfedr• 2 years ago

I guess it's true. Except you cannot contribute code under AGPL, so it's only open source in name, there is no room for a community to form around the software, Elastic co remains in sole control

PeterZaitsev• 2 years ago

Looks like Elastic is bowing to OpenSearch threat. This is great news to have more Open Source choices, yet I think for those who care about Open Source the trust have been broken and there is little assurance there will not be a license change one again, serving needs of the moment

valyala2• 2 years ago

IMHO, this is just the beginning of the trend for reverting source-available licenses back to open-source licenses. The reason is that source-available licenses don't help increasing revenue growth rate. https://news.ycombinator.com/item?id=41266819

nullify88• 2 years ago

Since Elasticsearch changed their license, Loki has also appeared as a competitor and the Grafana machine have released a suite of tools that cover the observability categories. It may also be that the license change encouraged users to look for alternatives and there are more now than just being graylog and Elasticsearch.

Clickhouse has proven to also be a very capable database for logs and there are stacks that use it for log storage.

nklmilojevic• 2 years ago

Loki and Clickhouse are not really excelling in what ES is. For example, full text search.

VictoriaLogs is very promising in this regard!

OuchLinux2• 2 years ago

It's fair to say that most elasticsearch use cases aren't search based, they are analytical - which they don't 'excel' at. There are much more compelling products on the market these days.

jraph2• 2 years ago

Congrats on releasing under a free software license again.

We are there to complain when something becomes proprietary, there's no reason we'd not be there when the opposite happens.

ksajadi2• 2 years ago

The change of Elastic license resulted (or perhaps coincided) with the massive growth of other tools like Meilisearch, Quickwit, Loki, typesense, and more.

In a way, their original open source license was suppressing innovation. I know this is not a popular opinion in some circles of pure OSS aficionados but it seems the evidence is to the contrary

raxxorraxor2• 2 years ago

I am biting here because I don't understand the train of thought. How would such a license suppress innovation? What evidence?

ksajadi2• 2 years ago

For all those new search projects named in my OC to be viable, ES had to stop being “open source”. Before that it was not worth trying because there was one single dominant player in the market giving search away for “free”.

The same is happening with Redis after their license change.

jillesvangurp5• 2 years ago

It's an interesting move for sure but I don't think it's enough. A move back to the Apache license would open up the landscape for a reunification with the now dominant fork, which is Opensearch. But that's not going to happen with AGPL + copyright transfers. AGPL + copyright transfers vs. SSPL is a choice between getting stabbed or shot from a legal point of view. It's a hard no either way for a lot of corporate legal departments.

I also don't think this will inspire a lot of companies or developers to start contributing changes to the Elasticsearch code base again; which is something that ground to a halt earlier. I saw my modest contributions under the Apache license being locked up behind this bullshit license and I learned my lesson: I'm never signing another contributor license again. My trust was violated. Not lifting a finger to help them.

Elastic suffered a self inflicted fork of their developer community three years ago and Opensearch has become the default search solution for a lot of developers and companies. Opensearch replaced Elasticsearch as a neutral ground for open source researchers to rally around. I don't see that changing in any material way because of this license change.

It's interesting that they are doing this though because clearly they are feeling the pressure and basically people using the opensource argument was cutting off their stream of new users. I consult in this space and Opensearch has become the default choice for new users. It isn't even close. Why would you pick Elastic as a first time user? They don't even consider Elasticsearch because it's all closed source and proprietary and Opensearch does the job. I don't think this change is enough to change that.

IMHO their next logical step is embracing/acknowledging Opensearch and moving their efforts to join Opensearch and supporting that. That's a huge community of users, developers and companies that's just sitting there without delivering any revenue to Elastic. It's stupid; they are competing with their own product and leaving a lot of money on the table. Elastic has all the core skills to support that community but they are just sitting on their hands now pretending it doesn't exist. They must be starting to feel the pressure to just toss in the towel and grab a chunk of that market. This our way or the highway position sure has resulted in a lot of people choosing the highway.

jprante• 2 years ago

I was scared away as a plugin developer when Elastic changed to closed source, because the plugin that I assembled was based on third-party code and can only licensed under AGPL. Fun fact is, I could now resume my plugin development, and provide the code in the open with future Elasticsearch versions. But will I do it? I am also frustrated and my trust was violated.

The point is, having Opensearch as an alternative is finding answers to my question to resume Elasticsearch development not easier than before. All the positive aspects are still counting for Opensearch. It is true that AGPL is not enough.

While I do not think there will ever be an effort by the Elastic company to join Opensearch or embrace the Opensearch community, it seems to me the license switch to AGPL was driven by the analysis that Elastic's product offering can not be copied any more by hyperscalers. It was a mere question of the exclusiveness of the cloud service offering. That collided with the Apache license once. But now, it is clear that Amazon will never switch back to the Elastic stack as their primary cloud service search product.

openplatypus• 2 years ago

Yes, awesome news.

I made a career with Solr, but building my product, Wide Angle Analytics, on top of Elastic search, made me realize how much more robust and polished ES actually is.

With restoration of Open Source alignment I am confident we will continue building with ES as we are very happy with it.

adrianco4• 2 years ago

Here’s the initial AWS response to the license change that they made in 2018, which I helped write. At the time we didn’t think a new license made sense, as AGPL is sufficient to block AWS from using the code, but the core of the issue was that AWS wanted to contribute security features to the open source project and Elastic wanted to keep security as an enterprise feature, so rejected all the approaches AWS made at the time. https://aws.amazon.com/blogs/opensource/keeping-open-source-...

_pdp_2• 2 years ago

Unfortunately many companies charge extra for security where security should be the default. Truth to be told there some some situations where extra security costs could be justified but there are not many if charge is necessary it should be considered as a temporary measure. My $0.02.

thelittleone2• 2 years ago

Including the recent trend of access to SOC2 reports requiring an "Enterprise" tier subscription.

originalvichy• 2 years ago

I think it's fair to say most security is built-in by your average developer. However, the security side of things needs efforts from a far smaller pool of experts that can make your tool as secure as is possible. I don't imagine it is as cheap to find feature developers as well as security experts or security developers. Practically speaking, cheaping out on security might cost you the reputation of the app, so doing it well will be expensive but worth it, but might never be worth it if you offer everything by default but in the end only a fraction of your customer base uses the features.

cduzz3• 2 years ago

I'm not sure what you mean by "security" in this context.

Identity management, role based access control, useful audit logs; all "enterprise" features, probably are very expensive to implement, and make for obvious "up-charge" product segmentation.

I suspect there's some combination of "the community doesn't add useful implementations of these features" and "we can't possibly risk our reputation based on some community contribution" and "we can use this to segment our product to sell to some and give it away to some."

This set of features seems to always get put in the "enterprise, only for licensed / supported customers" and it stinks.... I can understand why, though, and none of these are strictly speaking "security" as much as "compliance"

perryizgr8• 2 years ago

I don't know if this was part of the issues but adding authentication to Elastic APIs and Kibana is so confusing and complicated that it is almost impossible to do unless you go for a managed solution. I'm sure that one factor alone motivates a lot of users to buy the service instead of hosting their own using the available source.

skywhopper5• 2 years ago

Yeah, this is an underrated aspect of all the managed hosting options out there. If vendors made it easy to deploy their code, folks would be far more willing to run it themselves. But just rolling out a simple production-ready cluster of most software is a nightmare of complexity. (Note that while open-source software is often not great at this, proprietary software is often just as bad or worse. This is not a side-effect of open-source. It's a failure of prioritization of the operator experience.)

weinzierl• 2 years ago

"as AGPL is sufficient to block AWS from using the code"

I have taken this position in another thread a while ago, but the responses seemed to indicate that this is not a clearly cut situation at all. If it was, what is the point of the "source-available" licenses in the first place? I mean, the idea that they were invented to cut out AWS is pretty prevalent, no?

_cenw3• 2 years ago

There's enough legal uncertainty about API calls being considered linking that it keeps coming up. Minio are probably at the forefront of claiming this somewhat implicitly while referring you to your lawyer (or their pricing page, preferably) when asked about how they understand the AGPL.

FSF/GNU have an example of an AGPL proxy becoming compliant by serving it a page with the offer to download source code on the first request, pretty far off from reality if you ask me. That's also the big other issue, AGPL is unclear about conveyance over a network. Does a header work? Does a link to the source repo work or do you need to offer hard copies? What do you do if the "networking" is a highly specific protocol that simply can't make that offer over the wire?

I much prefer the clarity of intent of the EUPL.

skywhopper5• 2 years ago

AGPL doesn't forbid Amazon from providing a competitive service using the software. Elastic License/SSPL/BSL all do. That's the difference.

kikoreis4• 2 years ago

Well, the comment from OP isn't necessarily complete. The AGPL is not about preventing someone from using source code (indeed that would be contrary to the spirit of all liberal and copyleft licenses), but rather the condition under which source code modifications need to be made available.

Specifically, if you offer the software for "Remote Network Interaction" (AGPLv3 section 13), well, "if you modify the Program, your modified version must prominently offer all users interacting with it remotely through a computer network (if your version supports such interaction) an opportunity to receive the Corresponding Source of your version".

I think the original challenge with AGPLv3 vs (to grossly generalize) the VC-backed open source corporate ecosystem was not around source code, but around monetization as SaaS by the hyperscalers. The problem there is even if the hyperscalers publish source code modifications (which they probably have no problem with) they have such sales efficiency and gravitational pull that they will end up eating your business.

skrtskrt10• 2 years ago

translation: AWS refuses to provide unmodified open source hosted software as a service or to open source the changes they make to host it.

It's not a "can't" it's a "won't".

adrianco4• 2 years ago

AWS at the time had AGPL on its list of licenses that couldn’t be used. There were other clouds in China especially ignoring the AGPL provisions and I think SSPL was used to try and be more explicit.

redwood3• 2 years ago

Out of curiosity, did you pursue a rev share model with Elastic (Co) for your Elastic managed service? I guess that's not something thay can be discussed openly but recognizing you probably had 10x their revenue in the managed service and another 10x their revenue in compute behind the OSS, I wonder if there could have been a proactive happy middle ground found years ago.

I suppose that they might not have accepted something that was too small percent wise and hence might have preferred to go head to head no matter where that might have gone.

My real sense for why they've struggled to out maneuver is their lack of execution on their managed service (9 years in market, still minority of their revenue); while you had a head start and I'm sure that's what they point to as preventing execution, if they had really focused there they might be more like Confluent in terms of being considered the well regarded SaaS leader in their segment.

But I do think it'd be a good look for AWS to proactively help these companies. I didn't think the approach taken with Grafana Labs was right... that looked more like a Faustian bargain to an outside observer (e.g. we'll cut you down at your knees and directly compete but offer you their more expensive version on our paper. It looked incredibly humiliating).

sanderjd5• 2 years ago

But I do think it'd be a good look for AWS to proactively help these companies.

But how much value does "a good look" have to AWS?

nijave3• 2 years ago

From previous threads, my understanding is Elastic is a pretty arduous enterprise sales process which turned a lot of small/mid customers away.

High vendor management overhead is a huge pain for smaller companies that don't have robust IT to manage those relationships.

The smaller/mid size startups I've worked at almost never acquire "enterprise" software and always leverage pay-by-credit-card type SaaS

Besides operational overhead there's also a much longer acquisition time (no one wants to spend 3-6 months working with a sales team to sign a contract on a project with 2-3 month timeline)

awwaiid• 2 years ago

... Yes, they submitted labor backed security fixes as their form of rev share.

adrianco4• 2 years ago

There were several proposals made to Elastic at the time, but their attitude was that they controlled the project and didn’t want AWS to make big contributions to the open source distribution that would reduce their differentiation. They were also mixing licenses in the code base and deliberately making it hard for AWS to use.

I was also assisting in the deal with Grafana, which I thought was a good deal on both sides, setting up a framework for AWS and Grafana to work together over a longer timeframe. Ash Manzani who negotiated the deal for AWS later joined Grafana to run Corp Dev for them.

xorcist• 2 years ago

Why? Does Amazon rev share with Red Hat? Does Google rev share with Linux? Or is it the other way around, should perhaps Linux instead pay Android for putting their product in front of billions of eyeballs? I'm sure there's a way of monetizing a billion users even for a kernel.

The answer is no, because Linux is open source. It is a multi stakeholder model where no single actor is allowed to control other actors use of the product. There is ownership, but it is separated from control. This is implemented with the GPL, but the license is only a tool to achieve the outcome, a multi stakeholder product.

This is why Amazon, Red Hat and Google all can justify to employ hundreds of engineers all contributing to a common product. Amazon can work on security functionality with no risk that Red Hat will veto it because it threatens its revenue stream. And while none of the top kernel developers have made billions from their important work, they still earn well, and all the mentioned companies have grown to become billion dollar companies.

Everyone knew this in the 90s, that's why we have the philosophy around open source. Now the discourse has changed. It is suddenly immoral to earn money from someone else's product, because if you start a project then you own it outright. Not only that, but you also have a right to become rich from your work. Discussions how immoral it is for a large company to use a piece of software without paying the original author is a completely normal thing to do, never mind that they would have no problem reinventing that particular wheel in a heartbeat.

Companies like Elastic have latched on to this, and call their product open source, but call foul when other people build products and make a living from their software. They're not actually interested in a multi stakeholder model at all.

How big would Wordpress have been without every cloud actor out there offering to host instances for a cheap fee?

chihwei• 2 years ago

But why couldn't you or AWS donate/pay to Elastic for what they created to get those features in? I understand the security features you mentioned is very necessary, but Elastic will lose revenue because of this, and they are not a trillion dollars cap tech giant like AWS to support the project for free.

pabs3• 2 years ago

AWS could easily comply with the AGPL, why is AWS blocked from providing services using software licensed under the AGPL?

ExoticPearTree• 2 years ago

To super-simplify things: using the AGPL software, can I offer a competing service to Elastic’s?

If yes, then it is opensource, otherwise it is not.

frankjr• 2 years ago

"Changing the license was a mistake, and Elastic now backtracks from it"

This could have been the entire article and it would make more sense than whatever this is.

wg03• 2 years ago

That is great news but I have prepared my stack for OpenSearch which seems even if less feature rich but is more trustworthy.

At least there are some security features built in (OIDC/OAuth/JWT/Proxy etc.) which are dead critical to operate any software stack be it internal or otherwise.

As for centralised logging or building a search functionality, OpenSearch was already good enough back in the day at the start of the fork.

I think both ES and OS would continue to flourish in their own ways.

OuchLinux2• 2 years ago

The software engineers who care enough to have an opinion probably likely don't contribute to the ELK stack, and those who are impacted by this license change are no better off. Ever since Elastic went on the warpath with their serverless cloud and Gen AI the entire "open source" pitch is moot, regardless of the license.

Regardless of the openness of their code - their observability product is grossly bloated and unimpressive, the security product is sideways, fleet is broken by design, the entire database sector is coming after their analytics use cases at much better perf + much lower costs (and winning), management look incompetent, RAG is a big bet - but unlikely to be the saving grace the stack + company needs. It's truly a product on fire. Elasticsearch was interesting 10 years ago - nowadays not so much. This just seems like a "hope for the best" distraction for scarier things to come for Elastic.

jesprenj• 2 years ago

I think they still have a contributor licence agreement, meaning every contributor transfers the copyright of his contribution to the owner of Elasticsearch.

giancarlostoro• 2 years ago

What license are contributions to Elastic given as? I'm very confused how this works with all the licenses, are they just all compatible with AGPL magically? Or are all contributions under their primary private license, I think the last time I saw someone change a license similar to how Elastic did (maybe MongoDB?) I suggested AGPL, I don't really like the license, but it is designed for this type of thing.

I'm guessing these license models are all because they want to just plainly sell their own instances without cloud providers competing with them directly (who typically have unlimited resources to do so!) or keeping any changes or fixes to themselves (I think that was the reasoning with Mongo?).

Offtopic / Meta to the article itself:

What is with the format of this article, like if they meant for the stuff in brackets to be headings, but chose this format instead of making them headings.

rw-access• 2 years ago

The CLA best answers the questions in your first paragraph.

https://www.elastic.co/contributor-agreement

pnt12• 2 years ago

tl;dr: you keep the copyright but give inconditional rights for elastic to distribute your code, eg under different licenses.

hungie4• 2 years ago

I'm increasingly of the opinion that the definition of "open source" that narrowly defines open source is going to be the thing that contributes to the reduction of open source software.

Open source communities are essentially anarchist syndicates, collectively working towards common good. Groups like Amazon coming in and taking their work and selling it, profiting to the tune of millions, and contributing nothing back will fundamentally alter the dynamics of the system.

IMO, we need a definition of open source that permits me to limit users to those who aren't actively exploiting me. GPL flavors get close, but not sufficient.

I believe, genuinely, in wellbeing for all, and that we should be working towards the common good of all. Open source is one such effort, but I'm tired of seeing people say, "it's only open source if the code is also permitted to be used against the common good and for the enrichment of a very select few already wealthy people."

KolmogorovComp• 2 years ago

GPL flavors get close, but not sufficient

What do you think makes it insufficient?

hungie4• 2 years ago

Amazon has the market power, legal strength, and willingness to run at a loss indefinitely, as well as the regulatory capture and connections to ensure they cannot be fairly investigated.

So if I want to ensure my software is used maximally, by the people who need or want it most, I need to ensure there isn't a "gravity distorting player" taking my project and white labeling it to push out all the other people.

The AGPL gets closer, but still doesn't go far enough in defining "modifications to the software" or linking, IMO.

wavemode2• 2 years ago

You can be as tired of it as you want, but tiredness doesn't change the meaning of words.

What you probably want instead is "fair source": https://fair.io/

Like their names imply, open source is about source being open. Whereas fair source is about ensuring that code is used fairly.

I would argue that people contributing to open source shouldn't be putting themselves in a position to be "actively exploited". If that's something you're worried about, then open source was probably the wrong choice. You should have sold the code for a profit instead, or established some revenue-limited source sharing (so that only indie devs can use it for free, like e.g. Unreal Engine). Or use a fair source license. Or a proprietary "source available" license.

hungie4• 2 years ago

You speak as if there is a divinely written definition for the words "open source". There is not, there's a group of people who have said, "this is ok, this is not".

I'm of the opinion that those people have made a mistake that will work against them, and they should consider revising their definition.

aftergibson• 2 years ago

Does anyone else find the wording of this is really strange and mildly unprofessional?

drewda• 2 years ago

our partnership with AWS is stronger than ever. We were even named AWS partner of the year.

This detail in the post made me chuckle. Oftentimes big vendors give out these kinds of marketing awards strategically.

One big firm I know makes it a point to have its CEO present on-stage awards at its annual user conference to customer that have indicated they might not review.

donor20• 2 years ago

This is their somewhat muddy response to the “trolls” who might say

“Changing the license was a mistake, and Elastic now backtracks from it”.

We removed a lot of market confusion when we changed our license 3 years ago. And because of our actions, a lot has changed. It’s an entirely different landscape now. We aren’t living in the past. We want to build a better future for our users. It’s because we took action then, that we are in a position to take action now.

skywhopper5• 2 years ago

This statement is confusing to me. I never found the old situation confusing, until Elastic started adding invented licenses and trying to claim openness while monopolizing the right to host their software. That was confusing.

VeninVidiaVicii• 2 years ago

Does anyone involved with MediaWiki here know how this could affect CirrusSearch? Once ES became non-OS it kinda went to hell. https://www.mediawiki.org/wiki/Extension:CirrusSearch

tccole• 2 years ago

Why are they using Kendrick Lamar songs for each title?

vips7L• 2 years ago

It's honestly really weird and makes reading the post hard for me.

amai• 2 years ago

Confused? Alternatives to Elasticsearch are

- https://www.meilisearch.com/

- https://typesense.org/

- https://manticoresearch.com/

and maybe

- https://github.com/Sygil-Dev/whoosh-reloaded

doublextremevil• 2 years ago

I'm glad the AGPL is getting some attention; it is the ideal open source license in the age of big tech

Elasticsearch is open source, again

Related Stories