Sandstorm Oasis Is Shutting Down

https://sandstorm.io/news/2019-09-15-shutting-down-oasis

by kawera • 7 years ago

225 97 7 years ago

ykevinator• 7 years ago

What is it?

ocdtrekkie5• 7 years ago

Sandstorm is an open source platform for web apps. Basically instead of having to host a bunch of different servers for different web apps you want to run on a server, they're packaged and installed in a way much like you'd install and run apps on your phone. Just, on a web service.

For me it entirely replaced Google Docs (using Etherpad and EtherCalc) and Trello (using Wekan) in particular. I'm also super dependent on it for my RSS reader with Tiny Tiny RSS. Everything on it you could host yourself individually, but it's easier and more secure via Sandstorm.

ChristianBundy• 7 years ago

I read the article, but I'm still unclear on what "Oasis" is. Is that what they call their free offering?

icebraining4• 7 years ago

For those who think this sounds like Docker: in a way, it's similar, but while Docker is mostly designed for packaging and running regular Linux server software, Sandstorm provides a common framework (particularly around authentication, authorization, configuration and upgrading), which makes the whole user experience much more unified, and requires less fiddling with configuration files or terminal commands, while exposing a smaller attack surface, even if the attacker manages to exploit a particular app (there are various levels of isolation, even between different documents of the same app).

https://sandstorm.io/news/2014-08-19-why-not-run-docker-apps

ussrlongbow• 7 years ago

Sounds like univention appliance https://www.univention.com/products/univention-app-center/ap...

mnutt3• 7 years ago

I also like to describe sandstorm as a bit closer to native mobile app development: the developer building the app has to think harder about what permissions (capabilities) their app needs, and in exchange a non-developer can install the app safely and have more confidence that the app isn’t full of security holes or leaking their data, intentionally or unintentionally.

ocdtrekkie5• 7 years ago

As a big Sandstorm user and contributor this is a pretty sad day for me, I've found using Oasis to be pretty darn convenient. But I'll be setting up my selfhosted version this fall. Sandstorm hasn't seen a ton of improvement over the past year but that hasn't been a bad thing: It's been a consistent and rock solid product I can rely on whenever I need it.

sansnomme• 7 years ago

A week ago I mentioned that it was half dead, guess this is the final nail in its coffin.

https://news.ycombinator.com/item?id=20933849

ocdtrekkie5• 7 years ago

My hope is that someday we'll see a resurgence of development for it, or the creation of something new like it. A lot of open source projects have survived on a very slow burn for a very long time, only to show up again later.

What's truly unfortunate, is Sandstorm was massively ahead of it's time: A lot more people are willing to understand the need for Sandstorm today than they were just a few years ago, when Sandstorm was an active development project. I feel like if Sandstorm had launched in 2019, it'd have enjoyed a lot wider support than people were ready for in 2014.

ben_jones• 7 years ago

There will be a continual need to "hold your own corner" of the internet (self-hosting common services) as it becomes even more segmented by Google, Facebook, and advertising companies like them.

AlchemistCamp• 7 years ago

This is sad. But there is still hope. Open source is very hard to kill. Anyone, anywhere can take up the mantle and Sandstorm will go on.

anderspitman9• 7 years ago

Sandstorm is one of the coolest pieces of tech I've come across in the last few years. Not just on a philosophical self-hosting level, but also a technical level (capnproto is awesome). I'm working on a product right now that is heavily inspired by Sandstorm.

In my opinion the one thing that makes it hard for sandstorm to grow is the requirement that apps have to specifically be modified to work on the platform, and it never reached critical mass of developers building in sandstorm support to their apps. So it's essentially a chicken-and-egg problem. I'm not sure what the solution to that is.

I would love to read a pseudo-postmortem (I refuse to believe this heralds the death of sandstorm) by @kentonv about what he thinks the main challenges of getting to a truly decentralized world are, how sandstorm plays in, and where we go from here.

kentonv20• 7 years ago

In my opinion the one thing that makes it hard for sandstorm to grow is the requirement that apps have to specifically be modified to work on the platform

This is true, though of course without that requirement, Sandstorm wouldn't be able to do what it does.

More broadly, I think the challenge we have today is that we've settled into a local maximum way of doing things that we call "Cloud Architecture" and "Software as a Service". The industry has spent many billions of dollars exploring and optimizing this approach. Even open source developers are designing their little apps using hyper-scalable techniques because they think that's what they're "supposed" to do -- never mind that such architecture is actively hostile towards small-scale self-hosting.

I think the Sandstorm approach -- which is essentially "distribute apps to people's personal servers the same way you distribute apps to phones today" -- would be vastly preferable (both to users and developers), if it had the same level of development and investment.

We can't just invest $100B upfront to build the new world, so instead we need to find an incremental strategy to get there. That's the hard part. Sandstorm tried to harness the investment already being made by "indie" / open source developers. We got a long way with not very much money! But the indie motif didn't exactly give us enterprise credibility, or any other way to sustain the company.

My new hope is that "mainstream" cloud infrastructure will push towards being incrementally more and more decentralized on a technical level, because of the technical advantages that brings... If your software is designed to run as a million little servers rather than one huge one, then a company like Cloudflare (my current employer) can go deploy it to literally hundreds of locations around the world for huge savings in latency and long-haul bandwidth, better reliability, etc. Then once apps are designed that way, maybe, just maybe, it'll be easier to flip control of code execution and data away from the vendor towards the consumer? But obviously there's a long way to go to get there.

chuhnk• 7 years ago

We're broadly working towards a lot of what you're saying here at https://micro.mu. It's an incremental process which I fundamentally believe needs to begin with giving developers a new method of development guided by a framework. We're leaning more towards the android model of managing a complex runtime and providing the framework for developing in that environment.

We'll probably leverage cloud, edge, personal servers, mobile and any other device in the long term to move services to the user.

I share a lot of your ideas and philosophy. Shame it didn't come to fruition with sandstorm.

mwcampbell4• 7 years ago

One thing I appreciated about the Sandstorm architecture, and I wonder if you now consider this a mistake, is that each grain holds its own state, using ordinary files and possibly in memory (since IIUC, there's one instance of a grain running at any given time).

But I would have liked to use an architecture like this in a SaaS application. We may wish that applications were self-hosted, but SaaS is what people are used to, and there's a lot of money in it. An architecture for SaaS applications based on many small, stateful units, like Sandstorm grains, would be a refreshing alternative to the current norm of trying to make everything stateless and web-scale. And I think it's something you could have sold. I remember proposing something like this to you on an HN comment thread a few years ago, but maybe I didn't articulate it well.

The reason I wonder if you now see the statefulness of grains as a mistake is that Cloudflare Workers are going even further in the other direction. Whereas Sandstorm grains can be long-running, Cloudflare Workers are very short-lived. Of course, I realize that you aren't explicitly developing Cloudflare Workers as the successor to Sandstorm; it's a new product with its own requirements. Still, I wonder if your thinking on that part of Sandstorm has changed.

EGreg• 7 years ago

Any Sandstorm developers reading this... I would like to invite you to reach out to me. We’re trying to build this decentralized future and we’re making revenues — would be great to explore collaboration.

Username: greg Domain: qbix.com

See https://qbix.com/QBUX/whitepaper.html for the economics of it

sitkack• 7 years ago

There are a bunch of us out there that still believe in the vision. You were just too early. This needs some sort of WRT54G kinda moment, synology or ???, you needed to have a last mile partner or a host (vpn in to services). I thought containers were going to take over cloud in a pervasive way and it hasn't happened yet.

I can't say thank you enough for what you have done. None of this was a failure.

anderspitman9• 7 years ago

Thanks for the thoughts.

My new hope is that "mainstream" cloud infrastructure will push towards being incrementally more and more decentralized on a technical level

That sounds pretty close to what I'm doing with my current project. Start with a centralized platform that is instanced per-user under the hood, provide tangible benefits vs the big guys, and eventually open source it for self-hosting. That's the dream anyway.

canadaduane• 7 years ago

Loved your work & was an early supporter. If you haven't already, I think you should take a look at some really cool new possibilities in the app development space like automerge/hypermerge, braid.id, and gun.eco. These offer a new path forward where a home server is just one possible topology, but the data is freed from servers. See also https://github.com/SmithSamuelM/Papers/blob/master/presentat... (I particularly like how Sam has coined the term "autonomic data", meaning data that embeds its own authorization through encryption--once you separate authorization from distribution, you can do some amazing things).

indigochill• 7 years ago

never mind that such architecture is actively hostile towards small-scale self-hosting

Can you provide some information about why this is? Are you talking about vendor lock-in with certain cloud providers? Because Kubernetes, for instance, provides a layer that lets you move applications around (even locally with Minikube) with no fuss. So I suspect I'm misunderstanding what you're referring to.

snowe2010• 7 years ago

I investigated capnproto for a while, how do you feel about them compared to the alternatives?

anderspitman9• 7 years ago

Disclaimer: I've only looked fairly close at capnproto, RSocket, and gRPC, never used them myself.

It depends on what you need to do. What's your use case? My primary constraint is that I need web browser compatibility, ie websockets currently. If capnproto had a solid browser JS implementation I would said use it hands down if it has the features you need. There's some work being done to shoehorn gRPC into the browser[0], but it seems very complicated to me (requires a proxy server IIRC). If you need a robust browser solution today, take a look at RSocket[1]. I think the reason rsocket isn't talked about more is that it comes across as very "enterprisey", but on a technical level it looks very good to me. You may also be interested in my minimalist approach, omnistreams[2]. It basically adds backpressure and multiplexing on top of websockets. We're running that in production at iobio.io, but the API isn't stable yet (would love external input).

One big caveat of capnproto vs gRPC is that capnproto doesn't really have built-in "stream" mechanics, ie the concept of making a request that you expect to return an unbounded list of elements. I've mentioned this before and kentonv explained a way it could be mimicked though[3].

[0] https://grpc.io/blog/state-of-grpc-web/

[1] http://rsocket.io/

[2] https://github.com/omnistreams/omnistreams-spec

[3] https://news.ycombinator.com/item?id=20041250

mnutt3• 7 years ago

I always thought one of the big barriers was porting existing apps: you need to disable/rip out their auth and user management, and you need to pare them down to operate on only a single unit of whatever thing they manage (a grain). These changes are often hard to upstream, so you’ll be applying upstream updates over and over. I was hoping we’d see a bunch of small purpose-built apps that were perhaps companions to a standalone app that just needed a small internet-accessible component, such as IoT devices or even mobile apps that need a way to host shareable links.

jimmcslim• 7 years ago

I alway felt that Sandstorm would be an ideal project for one of the NAS vendors like Synology, QNAP, or Netgear to get behind.

simplehuman• 7 years ago

as an alternative, cloudron.io is a good way to run apps on your own server. I think one of the main differences is that the app packages are regularly updated and maintained. And it uses docker, so a bit easier to sort out problems.

filmgirlcw• 7 years ago

It also costs $30 a month if you want to use it with more than 2 apps. And I think Cloudron is an excellent project and I would be happy to pay, but paying 3x or 6x the cost of my VPS for an easier way to run apps is just too rich for my blood.

kissgyorgy3• 7 years ago

Wow I started to implement essentially the same thing for small businesses, I did not know about Sandstorm. Does this means this is a bad idea?

dev_dull• 7 years ago

Did you take VC?

kissgyorgy3• 7 years ago

No.

mynegation• 7 years ago

This is subject to the specifics and execution. The likes of BitNami, NextCloud, Cloudron, Yunohost continue to operate, acquired or standalone.

teleclimber2• 7 years ago

I asked Kenton V about the reasons for the failure of the business. You might be interested in his reply[0].

My 0.02: I'd interview your target market to find out if this is a pain point they recognize. Many of these projects are started out of decentralization idealism (for good reason, and it's not a bad thing), but as a business you do not want to be left having to educate your potential buyers about why they need this.

[0] https://twitter.com/teleclimber/status/1173345736013910017

anderspitman9• 7 years ago

Agreed. You shouldn't have to explain someone's problem to them. Ideally when you present a good solution they will immediately match it to a pain point they're already experiencing.

redshirtrob3• 7 years ago

Never heard of them. June 31, 2020? I'm guessing this is a weird leap-year bug.

Dates are hard. Use a library. Always.

kentonv20• 7 years ago

No it's just me, a human, being an idiot and forgetting how many days are in June. Fixed now.

redshirtrob3• 7 years ago

Fair enough. I'm glad you fixed it.

anon1m0us• 7 years ago

There's a not un-serious push to switch to permanent daylight savings in Cascadia (western part of north america). I wonder what this will do to all the date code that doesn't use a library or libraries that aren't updated.

kentonv20• 7 years ago

DST rules change all the time, all over the world. Any software that gets DST right today must be using a library. Even if you only look at the US, ten years ago Congress changed the dates that DST starts and ends and it wasn't a big deal. There's also many places in the US today that use permanent DST or permanent non-DST, so that's not even new, really.

orblivion• 7 years ago

Kenton - You say we shouldn't rely on you to safeguard our data. How does this affect security updates? As it is, I don't put awfully sensitive things on there anyway because Linode still has access to it. But it would be good to know where to draw the line. I'm hoping that since you rely on it for email, you're gonna do basic security hole plugging at least?

Thanks as always in either case. Hoping it somehow gets new life down the line after this!

kentonv20• 7 years ago

Yes, I plan to continue pushing security updates for Sandstorm as I have been for the last five years, and self-hosted Sandstorm is very good about auto-updating.

Meanwhile, with a self-hosted server, you get better security (compared to Oasis) due to the fact that only you and people you authorize can install apps on that server. For even more security you can of course put it on a private network or behind Cloudflare Access[0] for extra defense-in-depth.

Note: I don't rely on Sandstorm for email yet, because Sandstorm doesn't have good email support currently... but I'd like to build that support, because I'm getting really uncomfortable with gmail.

[0] https://www.cloudflare.com/products/cloudflare-access/ -- Disclosure: I work for Cloudflare.

akkartik• 7 years ago

My reaction after reading this is that if it's too much burden for the original author to keep up with updates, it's very hard to make a case for people to self-host their own services using Sandstorm. Upgrade effort will now get multiplied many-fold as everybody has to duplicate the effort that one maintainer was putting in.

Sandstorm is an amazing project, and the people who built it are way better programmers than me. But I'm betting that we need to go even deeper and think about why the maintenance burden for software is so high.

https://github.com/akkartik/mu#readme

This is a very long-term project, and it's still early days. But none of the short-cuts seem to pan out. It makes me sad, and it stiffens my resolve.

kentonv20• 7 years ago

My reaction after reading this is that if it's too much burden for the original author to keep up with updates, it's very hard to make a case for people to self-host their own services using Sandstorm. Upgrade effort will now get multiplied many-fold as everybody has to duplicate the effort that one maintainer was putting in.

That's not correct.

Self-hosted sandstorm is completely auto-updating. You don't have to do anything. I still have to push updates periodically, but I intend to keep doing that. After I push an update, all Sandstorm servers update themselves automatically within 24 hours.

Updating Oasis was, ironically, much more work for me than pushing an update to self-hosted sandstorm. Oasis is built on a much more complicated cluster architecture designed to be scalable -- scale that, sadly, we never actually needed.

garganzol3• 7 years ago

Kenton, pardon my hyperactivity, but here is a couple of observations you may find useful:

Oasis is built on a much more complicated cluster architecture designed to be scalable

You could dog food Oasis on top of Sandstorm. For example, Oasis could be a Sandstorm app and use Sandstorm services. In this way, you could feel very closely where and when Sandstorm is lacking. And once you fix or invent those parts, the results would be available to all of your customers, and not exclusively to Oasis. This would shift the laborious spot from Oasis to Sandstorm, and would allow you to open up the enterprise gates.

The second observation is a model of sales. You sold a "decentralization idealism" as you coined it, but with an optional central place like Oasis. In my opinion, this offering represents a natural conflict and is not destined to work in a sustainable way. Why not sell a good old license for advanced/enterprisey features? Oasis would be a great add-on to that model for those who wanted to rent the resources. And yes, exclude the free tier from Oasis to make things fair.

unityByFreedom• 7 years ago

This is one of the coolest, boldest concepts I ever saw, and put forth by an extremely capable programmer and human. I look forward to hearing more from Kenton in the future.

phoe-krk• 7 years ago

Insightful sentence:

While Sandstorm was popular on Hacker News, that popularity never really converted into paying users.

garganzol3• 7 years ago

Back in the day I approached Sandstorm by email and asked if they could provide an Enterprise/Server edition of Sandstorm (hosted apps with stable/customizable endpoints, authentication and stuff).

I directly communicated to them that we are ready to shell out something like $2,000 per license.

The answer by Kenton surprised me. He told they are aiming to get the consumer market first. Which is a bit odd, given the server is where the money is.

Enterprises of all kinds already have their hierarchies of customers. Moreover, they are in a constant need of a simple but highly efficient server platform (see Kubernetes of a future). It would be a much easier sell to them.

kentonv20• 7 years ago

You must have caught us early on, when our focus was more on attracting developers than attracting users. Later on we were trying very hard to find enterprise customers who would pay us something like $2000.

slezyr• 7 years ago

https://sandstorm.io/install

Alternatively, you can let us run the server for you: Use Sandstorm Oasis

They still have this remark on install page.

ocdtrekkie5• 7 years ago

There's probably several references to signing up for Oasis on the website yet. They'll all get purged with time.

underdeserver• 7 years ago

Sounds like awesome tech with a bad sales plan. Would you consider selling?

std_throwaway• 7 years ago

Selling what?

kentonv20• 7 years ago

The tech is all open source. Anyone is free to try to make a business out of it (under a new name), no need to buy from us.

test1979• 7 years ago

Thats sad news

Sandstorm Oasis Is Shutting Down

Related Stories