Hi,thinking of creating an app to limit AWS account spend, need honest opinions

https://news.ycombinator.com/item?id=24128235

by HiDavidDay • 6 years ago

4 15 6 years ago

news.ycombinator.com

I am new here so please forgive me if I'm stepping out of line here.

Long story short, I recently got an aws account for side projects but I am scared that someday someone's going to go crazy with one of my public websites and I'd end up with a large $XXX bill. I have also heard horror stories from this sub about people trying something or forgetting to turn resources off and ending up with a large budget. I know we can set budgets and all but it doesn't work for people who want to cap the total $ spent.

I am a little surprised aws hasen't introduced this feature despite many requests (although when I think from a business pov, it makes sense).

Basically I am want to create an app where you could say something like

"I ONLY want to spend $XX per month, if my bill is more than that, shutdown my resources"

My idea is to create a website where you can set a budget amount (monthly?) for your account and setup rules for different resources ( like shutdown EC2, make S3 bucket private etc)

I am pretty good with creating backend tools and worked a lot with aws lambdas and s3.

I'd love to get some ideas from you guys.

Also if you don't think its gonna be helpful, let me know that too.

thanks

markorbida• 6 years ago

I think it's useful. Im not sure if any third party tools has a similar feature but definitely would be helpful specifically to those who are new to the product.

So you will have the option to auto stop services to minimize spend or terminate services if stopping will not reduce the bill (rds is an example).

User probably needs to be able to set priorities, locks, etc to prevent disruption of services in case they are using it for production.

HiDavidDay5• 6 years ago

agree. I could not find any third party tool that does that. Can you please tell me more about setting priorities, locks?

solidasparagus4• 6 years ago

Very helpful. Pretty hard to do comprehensively without (but less comprehensive solutions could still be useful).

The biggest question is "where are you going to get your data about current spend?. AFAIK the budget data lags behind usage (which is what I assume is the reason people haven't done this before). You can do this at a per-service level (e.g. EC2) by periodically polling EC2 for active resources and keeping a running tally of cost by yourself. This works well for some services like EC2, RDS (although serverless Aurora gets tricky), etc. However for other managed services like S3 or Lambda, it's harder to keep track of how much you have spent.

Cloudtrail could be useful here, but it probably doesn't have enough information to calculate actual cost (e.g. Lambda charging based on invocation time or Athena charging based on the volume of data scanned).

But even something as simple as a tool that keeps track of just your EC2 instance spend and shuts them down if it gets too high would be useful to developers who have AWS accounts for their side projects.

HiDavidDay5• 6 years ago

Yes I think less comprehensive solution could cover major services (EC2, lambda etc)

The biggest question is "where are you going to get your data about current spend?. AFAIK the budget data lags behind usage

interesting, I was thinking I could poll billing metrics (total charges etc) to figure out account level charges.

In your experience, do you care more about EC2 costs or that was just an example?

solidasparagus4• 6 years ago

interesting, I was thinking I could poll billing metrics (total charges etc) to figure out account level charges.

If it were that easy, this would already exist ¯\_(ツ)_/¯

I think many people use AWS primarily for s3 and EC2 (and s3 costs are negligible for most side projects) so if you were picking a single service, EC2 would probably be the most broadly applicable (and IMO the most realistic one to actually build). On the other hand I would imagine that for many side project developers, their EC2 costs are the easiest to control - even if you do autoscaling, you can cap your max nodes.

What is most useful is tracking the total AWS spend because what really gets you is the service you weren't expecting to cost as much as it did, but that is a very very very hard problem.

foreigner• 6 years ago

I would love this! I agree it seems like a missing feature of AWS. However as someone else commented I'm not sure how you could practically go about implementing it.

HiDavidDay5• 6 years ago

thanks, yea I'll try to figure that out

adventured• 6 years ago

Since the available billing information lags actual usage, track the cost data over time and interpolate it forward in time to be a projected / estimated real-time figure, and act according to that projection. Be sure to let users know how your system works and that it's a calculation you're running based on a smoothed cost (you can make that as rigid or not as makes sense, you could look for variances in times of day to the cost for example and adjust your real-time estimates accordingly, and so on).

It's entirely unnecessary to be exact, you need to be close enough. This is especially true since it's a feature a lot of people want, they'll accept some weakness in the system within reason.

You could offer a flex range the customer can set, a tolerance. If you can't reliably get near that (eg if you can't get within N% when back-checking your estimates versus what ends up being the actual billing), then you let the customer know with a status update and the customer can make a decision as to what to do at that point (they could stop using your service, they could adjust the tolerance, and so on).

Require a minimum number of polled data points before the system comes alive and becomes capable of shutting down services. You need a certain number of records before you can reasonably extrapolate anything.

This system won't work well in cases of normal usage with a sudden extreme burst that blows out the customer's budget (where the customer is hoping to prevent that type of unexpected one-off surge). You could do an accelerated estimate guess if you see a very large, very unusual spike, and jump ahead as a means to over-protect (another setting the customer could choose, they could set a caution level that guides how much you should accelerate ahead in cases of huge surges).

If you're successful, Amazon will kill you or buy you (possibly ask to buy you for cheap, then kill you if you refuse). As with similar hyper platforms, the platform eats its own ecosystem inhabitants in the name of growth. And this is a blatantly obvious feature that is widely demanded, so if you figure out a way to do it even modestly well, they'll knock you out eventually. Print money for a while if you can, charge as much as you can, prepare for the inevitable; consider selling to a cloud service company if you can find the right offer.

solidasparagus4• 6 years ago

You need to be very careful with projecting, particularly in a production environment. Costs are not evenly distributed across the month (e.g. IIRC reserved instances hit on the first of the month with a potentially very large cost).

HiDavidDay5• 6 years ago

Thanks for the detailed writeup. I think projecting future costs would be hard but someone pointed out in previous comments that service specific rules ( EC2, S3) etc could be more realistic.

Also what do you think about AWS Cost Explorer API? It looks like it can provide monthly/daily charges and we can build rules for customer tolerance, like you mentioned.

laurentl• 6 years ago

Unless I misunderstand your concept, using your website would entail granting it rights to my AWS account to access billing info and shutdown pretty much any service. So, something close to the Administrator policy. I guess you could do something more granular e.g. stop but not start, specify resources, etc but in any case I need to grant you enough rights to shut down my account, which is the entire premise of your service. What are the chances that I’ll grant this type of access to a 3rd party I don’t know about? Snowball, meet Hell.

Also, I don’t want my production account to be shut down. Notified, sure. But if my spending takes off because Elon Musk tweeted about my website and I’m getting a lot of hits (and hopefully a lot of new paying customers) I definitely do not want my service to go down.

In other words, your idea isn’t for me, but it might well work for side projects where you don’t want to spend too much and don’t care about the service going down. In any case the security aspect is going to be tough to manage properly, and is going to make your service a juicy target for hackers.

gtsteve• 6 years ago

Also, I don’t want my production account to be shut down. Notified, sure.

This is already an AWS feature: https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitori...

borplk• 6 years ago

I'd advise against trying to turn this into a product. Companies will be hesitant to grant a third-party that level of privilege on their infrastructure. And sophisticated companies already have the means to manage their costs. For sophisticated deployments a third-party is unlikely to be in a good position to make those decisions. It's not as simple as "just turn it off". Unsophisticated companies tend to have a simple and predictable architecture with a (largely) fixed monthly cost. The product will be too complex for those simple companies. So I suspect a product like that will be "not good enough" for the serious companies, and too dangerous/difficult for simple companies.

Hi,thinking of creating an app to limit AWS account spend, need honest opinions

Related Stories