Hakuna Cloud – Stop cloud servers when they are not in use

by tarl0s • 7 years ago

124 61 7 years ago

dharmab6• 7 years ago

Efficiency is an area where there can be a lot of cost hiding. We recently saved a lot of money by:

- Using the Kubernetes Vertical Pod Autoscaler (https://github.com/kubernetes/autoscaler/tree/master/vertica...) for CPU and memory scaling, and switching to metrics like requests per second and connection count for horizontal scaling

- Collecting metrics on container CPU and memory allocation vs utilization

- Writing scripts to auto-submit PRs to SREs with better recommended sizing based on actual usage

- Tuning our VM instance sizes and autoscaler configs

A few engineers were able to save the company several times their salary with a few months of work, and plan to 10x the savings over the next year

totomz• 7 years ago

Actually, HakunaCloud works on cloud servers, not containers. With Kubernetes you pay for the worker nodes, not for running containers.

HakuaCloud starto&stop cloud servers, so you actually pay only for the time the servers are needed

dharmab6• 7 years ago

The number of running nodes is a direct function of the number and size of your containers. The cluster autoscaler stops and and start servers based on how many containers are desired, which is in turn based on the Horizontal Pod Autoscaler inputs (e.g. incoming requests).

justicezyx• 7 years ago

Very nice! I suspect the economy of Cloud will soon change in such a way that Cloud providers will be focusing more on Enterprise, which have very little capability of doing similar things. And then, a lot of companies like Hakuna Cloud (reported here) will emerge, just like what happened in the last iteration of IT infrastructure.

najarvg• 7 years ago

Would love to see a more detailed write-up, if one exists, on your experience. We're nowhere near your scale but will probably ramp up in the next year or so.

colechristensen• 7 years ago

During a ramp up the limiting factor is usually engineer hours, new features, and ability to have the load at all and not cost.

avip• 7 years ago

Pre-scale just overprovision and pay $ to Bezos instead of $$ your precious engineers. 96% you won't "ramp up" anyhow.

dharmab6• 7 years ago

I'm going to try to convince the lead on this project to present at an upcoming Kubecon :)

cbluth• 7 years ago

Tuning our VM instance sizes and autoscaler configs

Do you run kubernetes in VMs on baremetal?

dharmab6• 7 years ago

Yes, we own some datacenters as well as use multiple cloud providers. You are still responsible for running your own worker VMs with a managed Kubernetes anyway.

qubit23• 7 years ago

If you ever open-source your rightsizing scripts, I think a lot of people (including myself) would find that useful. While the vertical pod autoscaler attempts to solve this problem, it's incompatible with horizontal pod autoscalers based on CPU and memory, and thus unusable in a lot of cases. Going with a semi-automated solution might be ideal.

dharmab6• 7 years ago

You can just run vpa-recommender without vpa-updater to get VPA recommendations without the autoscaling. That should work for most people.

ma2rten• 7 years ago

I am confused by your use of the word cost hiding. Do you mean cost saving?

dharmab6• 7 years ago

as in hidden costs- costs that the organization is unaware of because it's wrapped up in r&d/goods delivered on the accounting sheets. Its='s not until you pull in the utilization metrics that it reveals itself as waste.

gramakri• 7 years ago

Interesting idea. Almost bring the idea of lambda to VMs, doesn't it?

Also, how does hakuna work with DO? I thought DO still charges when VMs are powered off?

tarl0s5• 7 years ago

Hi, I'm a developer at Hakuna. Thank you for the feedback.

With DO, we destroy the instance so that nothing is charged for it, and save a snapshot to be used to restart the instance; you pay only for the snapshot storage - $0.05/GB per month.

brianwawok• 7 years ago

What is the cold start time of a DO instance image to working VM? Isn't it like 2 minutes?

If a DO droplet for my marketing site cost me $5 per month to run.. I could use your service and save a buck or two, at the risk of a $5000 client timing out on my page as it tries to spin up?

Scaling based on demand is important, but it seems you are better using k8s with some metric based scaling, over trying to hack it on DO?

nine_k• 7 years ago

Now that we have CGI scripts in the cloud (lambdas), there ought to appear an implementation of inetd, too!

Jokes aside, I wonder when cloud providers will add something like this as a native feature.

bifrost• 7 years ago

^^ Winning joke of the year!

cristyansv• 7 years ago

google Kubernetes engine, alerts you when a cluster is under used and shows the amount you could save if you reduce the number of machines.

gregmac2• 7 years ago

Hakuna Cloud is a software-as-a-service HTTPS proxy. You don't need to change existing software or infrastructure, and you don't need to install additional tools on your servers.
Each cloud server must have an FQDN/DNS name configured as a CNAME to our load balancers.
When your server stops receiving requests, it will be stopped. As soon as a new request arrives, Hakuna will start it.

Interesting idea. It's like a proxy that kind of makes an instance/vm-based service act like a serverless service, without moving to containers or rewriting.

Seems kind of niche but I can see the use: there's a lot of services that have a time-based usage pattern (during working hours, or used interactively for a few minutes/hours sparsely through the day).

What are the cold start times like with this (at least for a typical, simple app - say on asp.net on Windows or something hosted via nginx on Linux)? What happens if an instance is being stopped and a new request comes in - does the request have to wait for shutdown plus startup?

tarl0s5• 7 years ago

At Hakuna we use our own service to start and stop our Jira and NPM servers; the instances are automatically stopped at night or whenever nobody needs to do `npm install`.

In our use case, the EC2 instance starts in about 50 seconds, with another minute needed to start the Jira service.

We have a demo, deployable directly from our CLI, that starts a Nginx server on an Oracle Cloud instance in less than 40 seconds.

If the instance is being stopped and a new request arrives, it will have to wait shutdown + startup, yes.

gregmac2• 7 years ago

Thanks.

Is there UI feedback to the user during the wait, or does the browser just show "waiting for response" for the whole time? If a user refreshes the browser a bunch of times during the wait, will the Hakuna proxy give up on those requests or still pass all of them through to the target server?

mkl• 7 years ago

You should probably check your website on mobile. For me on Firefox Android, the unmoving header takes up 60-70% of the screen and the actual scrolling content is crammed into the bottom fraction.

scjody2• 7 years ago

For time-based server start/stop, we use https://www.parkmycloud.com/ .

I'm looking forward to checking out Hakuna Cloud - looks like the two services are pretty complimentary.

alanbernstein• 7 years ago

I just had a conversation about this product idea a couple days ago. How long before Amazon acquires it and cripples the functionality? Could be a good exit strategy, at least.

eloff2• 7 years ago

I don't think they'd cripple it. Amazon takes a pretty long term view of things. They might well introduce something like this into their load balancer or api gateway though. It'd be nice to have in front of fargate because of the faster cold start times.

comeonman1• 7 years ago

2 minute cold start is pretty slow for a lot of use cases... Amazon already built Lambda which is more widely applicable.

entscheidungsP• 7 years ago

True, I think lambda would be a better choice for new projects. However, for old monolith projects, this sounds like a great way to cut costs on dev/staging servers.

tlarkworthy• 7 years ago

It's pretty cool that utility computing is large enough to spawn 3rd party companies that further increase effeciency of the pool.

It's layers upon layers of technical progress in parallel.

grepthisab2• 7 years ago

Maybe. This is a bit of a cottage industry that shouldn't need to exist. It's so hard to get data and visibility into these things in AWS right now. Billing is hard, it's impossible to easily tell what is costing me what in every region, etc. AWS should do all this natively as part of their service, but everything is so poorly designed from a user standpoint that it's very time consuming to do, so third-party services like Hakuna pop up to take advantage of the inefficiency by saving some money. I'm halfway convinced AWS does this on purpose so you have to pay them for support or training for people new to cloud service providers, or AWS specifically, increasing their revenue.

rednerrus• 7 years ago

https://www.cloudhealthtech.com/

SpicyLemonZest• 7 years ago

I think you’re underestimating how hard of a problem it is to provide continuous billing information. I’ve seen SAAS companies with an engineering team dedicated just to the monthly reports, handling things like pipeline delays and data cleanliness and conversion to both a clean UI and fiddly excel spreadsheets.

sharadov• 7 years ago

Of course they do this on purpose, it does not require a lot to give you metrics on usage and tell you if you are over provisioned, when mint.com can suggest a lower interest credit card or a higher yielding account based on my financial history, AWS cab easily do the same. But, why would they. They rather laugh all the way to the bank!

reilly3000• 7 years ago

Tag every resource you provision (probably on 3-5) dimensions and you’ll never wonder where your money is going. It’s a great system, but most people (including me for the first 3 years) skip the tags screen by default.

ssttevee• 7 years ago

From the FAQ:

The HTTPS trigger is intercepting all my traffic? No, your data are safe if your server support HTTPS protocol. All the data exchanged between your server and your clients is encrypted and not accessible by us.

Unless there's an IP allocated to each user, I don't think this is accurate. With SSL, the HTTP headers are encrypted, so there would be no way to know where know where to route the request without first decrypting the data, and thus having access to the data.

tarl0s5• 7 years ago

In HTTPS, all the headers are encrypted except the hostname:

https://en.wikipedia.org/wiki/Server_Name_Indication

SSL termination is always done on our customers' servers.

p-schultz• 7 years ago

The whole header is encrypted, including the Host header and request line. Certificate selection and routing relies on SNI[1]. The server name in the TLS client hello message is almost always, but not necessarily, a copy of the value of the Host header field.

[1] https://en.m.wikipedia.org/wiki/Server_Name_Indication

grepthisab2• 7 years ago

SNI was added on top of TLS to allow one server to serve multiple certificates. SNI is different than the hostname header, and in fact no headers are available until the traffic is decrypted post-handshake, which Hakuna can't see if it's not terminating TLS. But SNI lets the hostnames be available outside the encrypted TLS stream on hello.

From a comment above, it seems like Hakuna requires a FQDN of each AWS server it's serving traffic to, so if you're not MITM'ing traffic, this FQDN I'm guessing sits in with SNI and is used for routing rather than serving certificates. I don't think I've personally dealt with this use case on SNI, but it makes sense.

jke• 7 years ago

hakuna = there is no (Kiswahili)

what a great name!

robmccoll• 7 years ago

The header of the website occupies >50% of the screen on Chrome on Android and stays there on scroll. It is very frustrating.

dvtrn• 7 years ago

"Hero images". Done well it's an informative "here's what we do in a nutshell". Done wrong, it's an annoying UI.

wheelerwj• 7 years ago

You don't need to change existing software or infrastructure, and you don't need to install additional tools on your servers.

then

Install Hakuna CLI

and

Update the DNS

That certainly sounds like installing things and making changes to your infrastructure...

It sounds like a cool idea for sure and can be really helpful for a lot of companies but this seems like an outright lie.

SparkyMcUnicorn• 7 years ago

Installing the CLI is optional.

"npm install @hakuna.cloud/cli -g and you are ready to go. Or use the web dashboard!"

tarl0s5• 7 years ago

Hi, I work at Hakuna Cloud as a developer. Thanks for your feedback.

If you don't like the idea of updating your DNS records, Hakuna provides a hostname creation feature for trying the product (*.demo.hakuna.cloud CNAME records that point to our load balancers).

You can install the Hakuna CLI on your laptop, no need to have it on the cloud instances. Also, most of the features you can find on the CLI are also available on the web dashboard.

tasssko• 7 years ago

We do this with custom autoscaling with aws cloudwatch, lambda, alarms though we usually have something online all the time for uptime monitoring.

gingerlime2• 7 years ago

Interesting. Why proxying though rather than monitoring DNS queries on the CNAME? And updating to point to the right ip when the server is live (This could maybe help with the 10gb base limit + $0.08/gb?)

Not trying to be an armchair coach but rather understand the architecture decisions and trade-offs that I must have missed

manigandham2• 7 years ago

What happens where there are no servers live? Should the request fail?

gingerlime2• 7 years ago

I think they can point to some kind of a "Loading..." page that they can host? (SSL can be managed with Let's encrypt, since the CNAME is effectively controlled by them)

lone_haxx0r• 7 years ago

[Bikeshedding]

Why Hakunacloud?

Having #000 headings on blue background while using #fff text without shadows or borders on that same background looks really amateur.

And the "read more!" is blue text on a blue background. Barely readable.

mixmastamyk• 7 years ago

I like the colors and find it readable.

simonebrunozzi• 7 years ago

The image in the front page reminds me of Drupal's logo [0]. Not necessarily an infringement, just saying.

[0] https://www.drupal.org/about/media-kit/logos

scjody2• 7 years ago

Related: if you're looking for a service that starts/stops instances on a schedule (we find this really good for QA and development instances), check out https://www.parkmycloud.com/ . You can also set an instance to "always parked" and unpark it for a certain number of hours or until a certain date/time.

(No affiliation, just a satisfied customer.)

opportune• 7 years ago

It seems like it does exactly (or a subset) of what Google Cloud Run already does. Just shove an application into a container and scale up/down depending on use. Other cloud providers probably have this too. So is the value add that this is less expensive or what?

sytse• 7 years ago

With GitLab Auto DevOps we are using Knative to scale things to zero when not in use.

mnutt• 7 years ago

I had long wondered if it would be possible to have a custom autoscaler that just stopped/started instances rather than terminating and re-creating, in order to respond to load increases more quickly than amazon's autoscaling groups. You still pay for the EBS even when it's stopped and deploys involve briefly starting all of the stopped instances, but EBS is a fraction of the overall spend and maybe in some cases the complexity is worth it?

manigandham2• 7 years ago

Interesting model, although I suggest looking at the GCP Cloud Run approach using Knative to start and server containers on demand.

That's the next generation of Lambda that all clouds and vendors are moving towards, and increases developer agility with much faster cold-start times. If we could have Cloud Run today across multiple clouds and locations with geo-loadbalancing stitched together automatically, that would be valuable.

koolba• 7 years ago

So is this the bring your own server version of “inactive app hibernation” that you see in the free tiers of PaaS providers like Heroku? If so, that’s neat!

Hakuna Cloud – Stop cloud servers when they are not in use

Related Stories