tgeek

Hi there, Hey there, I lead Developer Advocacy at AWS for Serverless (https://twitter.com/chrismunns).

I'll give you that this 80% number seems pretty out there. I don't know how that is measured or what it would be referencing.

If you step back and remove all the commercial software from the argument (something like 50%+ of enterprise workloads, the kind of things you buy from a 3rd party and just run it, like Sharepoint, SAP, or similar) and then look at how many business applications take on a trivial amount of load over time, then the author's post becomes more of an outlier. Few folks have apps that do 100rps realistically. And so for data processing/streaming/batch or web/api workloads serverless actually does work out pretty well. Is this 80%, I am not sure.

There is 100% an inflection point where if your operator cost is low enough(human work+3p tools+process+care and feeding) then the "metal to metal" costs can be comparable. Even the author admits that's leaving something on the floor and so it really comes down to what your organization values most.

I would love for most of our serverless app workloads to be top-down organizationally driven but the reality of it is that it comes often from developers themselves and/or line of business organizations with skin in the game of seeing things move faster in most organizations. This will then typically require buy in from security and ops groups. If these folks you know have the trick to driving top down incompetent strategic management towards serverless I'd buy in on that newsletter.

In terms of HN sentiment and in being a member of this community for almost a decade, I don't know if I'd say it widely represents most of the dev world as it tends to lean way more open-source and less enterprisey. I think there's also a larger number of people that represent IT vendors that would love to see AWS fail here :)

Thanks, - Chris Munns - AWS - Serverless - https://twitter.com/chrismunns

Hey there, I lead Developer Advocacy at AWS for Serverless (https://twitter.com/chrismunns) and have been involved in this space since pretty much the start. Thought I'd toss my hat in the ring there as there seem to be some fairly passionate responses across the board here.

This post is accurate. It's accurate in the new dev experience of someone picking up a framework tool and following the standard walkthroughs/examples that folks like my team and I have been creating the past few years. As far as I'm concerned Einar (the author of the post) did everything I would have done in their position with their experience with this tech. The math Einar did also seems to be accurate given a 1gb RAM and <100ms function duration as well as the API-GW costs, which don't fluctuate based on request duration or Lambda configuration.

Einar got feedback later on that they should have looked at ALB (Application Load Balancer) which does provide a different HTTP request interface back to a Lambda invoke. ALB is the right choice for dead simple proxy requests where you aren't using any of the more advanced features of API-GW. It would lower the cost significantly as well as increase the performance due to a slightly lesser overhead in request path. It would change both the numbers in the blogs title though I can't say it would be 1:1.

We've done a shit job of making this clear and thanks to Einar's post lighting the fire under our butts we'll make work to correct that.

What I am happy about in Einar's post is that they were able to get up and running in almost no time, explore, poke, test, and write this post about their experience very very quickly.

The other thing to highlight here is that I am happy to lose a battle to Elastic Beanstalk, or ECS, or Fargate pretty much any day of the week. All of these exist to significantly reduce the operator burden which compared to running your own container management can be incredibly high. Most companies I've run into in the past 7 years that I've been at AWS have struggled to really measure operator cost but when they compare it to just raw time difference both the technologies I just mentioned and "Serverless" tech like Lambda shift the time spent factor so greatly that cost savings becomes painfully obvious.

Anyway, we're here, we're listening, there's a lot going on behind the scenes (see recent announcement about how VPC cold-start pains will be gone soon).

Thanks, - Chris Munns - AWS - Serverless - https://twitter.com/chrismunns

"AWS costs almost twice more than analogous DO servers, but AWS has way more features."

When you us AWS you are paying for more than just a cheap VPS, which essentially is all DO is. It's comparing fast food to a fancy steak house. You can get meat at both, but at one side its microwaved. Not to say DO isn't great, it gets the job done and provides a valued service, but your money gets you what your money gets you.

I find the better pattern here is to limit and discourage SSH and then monitor and log the hell out of it. There are numerous tools out there that can centralize any actions being taken on a host and sending it to a log that can be centralized. Outright removing all SSH puts you in a rough spot if things go south with some piece of software that your system monitoring/centralized logging don't cover 100%, and makes it way hard to do things like strace on a process.

AZ's are typically 5-15miles apart. Needs to be a pretty large hurricane that far in land to take out all the various datacenters.

Also, just to clarify, you can do multi-region RDS read-replicas for MySQL: https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/USER_...

I've always wondered why this type of information hasn't been attached to a driver's license much like organ donorship is in many states. Seems like a prime system to keep such a thing organized and available.

More likely this. AWS has at numerous talks mentioned a slightly franken-xen fork of Xen.

"The price premium of using AWS is high enough that it's trivial to afford leasing tons of excess capacity to handle failures and still save tons of money."

"AWS certainly is convenient, but it's also so expensive I'm charging my highest day rates ever for projects to help clients move off AWS these days."

This is FUD and nothing more than FUD. As bad as the article the other day that said a company saved 50% moving off AWS. Show me a company that can save 50% by moving off AWS, and I'll show you a company that isn't using AWS properly at all.

When you compare a single piece of hardware that you can buy and run yourself to an instance in EC2, you are leaving a lot off the equation.

Such as:

Deployment tools (CloudFormation, CodeDeploy, EB, OpsWorks)

Monitoring tools (CloudWatch (sure it leaves some to be desired)

Machine Image building/tracking tools

Hardware provisioning tools

Security tools ( Security Groups, NACL's in VPC, plus the stuff you don't see in the infrastructure)

More varied hardware than you'd have in house.(different amounts of ram/cpu/storage).

More hardware than you'd have in house. (no need for spare parts cabinets, waiting on vendors for replacements).

Storage (EBS, S3, local).

When you pay for EC2, you get all of this. Doing this yourself isn't free in any way shape or form, even with opensource tools. A company with a good ops team that is cloud savy is going to be several times more effective at a smaller size than a team that has to manage a datacenter, all the hardware, and all these other bits. Folks discredit all of these when they do apples to apples comparison of hardware you can buy to a service that you use.

Let alone that most people don't understand per-server network, space, and power cost over its lifecycle. (I've spent months with companies doing datacenter ROI analysis and having no idea what things cost).

And going to a managed hosting provider, you're either locking yourself into the frozen tech world they are in, small development resources, and typically constrained resources. You don't hear of managed hosting providers building any of the above themselves, so you continue to have a higher management overhead than a true cloud provider like GOOG, AWS, MSFT.

I sincerely hope you are joking here.

np! at the end of the day we all hate on call.

cheers.

You say LUN, are these SAN devices, or is it direct attached storage? Was the replication, load balancing, and snapshotting all something that you set up and manage yourselves?

--edit-- Ahh you've been editing your comments so the thread is a bit out of wack! (no problemo)

Newegg: http://www.newegg.com/Product/Product.aspx?Item=N82E16820167... ;) (yes its not the same as some of the much higher end stuff).

Fair enough, but again, your comment is about hardware that you are managing, that you've built, thats glued together from a lot of different components, both software and hardware, and this post is about a cloud service that doesn't even compare. So your initial post comes off a bit as trolling for the sake of trolling.

I've done my fair share of rack-n-stack, and I've now spent the past few years "in the clouds" as it were. Wouldn't go back for anything, but I dont think this makes me a fanboy. Sure there is kit that you'd only ever be able to build/buy yourself (for now at least), but most ppl will never need more than 100k IOPs, let alone 500k+.

--edit again-- In regards to security, if you think you are a capable of running an infrastructure more secure in a datacenter yourself, than on one of the major 3 cloud provider's infrastructure ( AWS, GOOG, MSFT ) where they have some of the best sec teams in the world, then you are probably not as deeply aware of whats possible in cloud from a security standpoint. Banks, Medical institutions, government agencies, and so forth are all trusting their infrastructure on the cloud, across many countries in the world.

Calling Postgres the leading open source DB is probably a bit more than a stretch these days, but yes its quite popular with startups(though one would argue nosql DBs are #1 in the eyes of most startups sadly). Unfortunately one of the places its not popular in, is Amazon and the majority of their business customers. MySQL, Oracle, and SQL Server are all much more popular than Postgres, which is why they have probably focused on it more over the years. That said, the features have seemed to be coming pretty regularly.

Can you completely snapshot those volumes at any time, recreate them and attach them to new servers? Could you take these snapshots and easily copy them around the world?(again assuming you could snapshot). Are those SSD's automatically replicated to 2 different storage devices behind the scenes to give you near-instant failover? When they go boom are you then driving out to the datacenter to replace them (assuming you have replacements and don't need to wait for them to arrive). Can you do all of this without any upfront cost or excess in capacity??

Probably not. At all.

EBS is NOT harddisks inside a server. Comparing them to such is missing out on all the things that makes it a SERVICE and not disks you buy from Newegg/PCmall/<insert vendor here>. Yes there are disks you can buy to physically put in a server and they are super blazing fast. In fact AWS has those in their i2 instances and they get hundreds of thousands of IOPs as well.

This isn't even comparing apples to oranges, its apples to space monkeys.

there are languages inside of Amazon that are first class citizens (java & ruby mostly, node.js more now). Go is too new, moving too fast, and hasn't seen the same kind of adoption yet to warrant an SDK.(despite what the folks here might think..)

AWS has Amazon Linux: https://aws.amazon.com/amazon-linux-ami/

Since its the default OS for Linux on AWS, it probably has a pretty large install base. More than most would probably assume.

AWS doesn't oversubscribe at all. This is a pretty baseless rumor.

US-East isn't a single building. Per AWS its over 10 datacenters that make up US-East(from a post they had in 2012). You aren't in the same building as their availability zones, you are probably talking about their Direct Connect POPs which is something completely different.

--(updated with links)--

https://aws.amazon.com/message/67457/

https://aws.amazon.com/directconnect/details/

10Gen is based in NYC, so yeah they get around.

This isn't ARP per se as one would traditionally expect it. This is the networking in VPC handling the movement of the ENI.

"That being said, opsworks is still based on chef, which is still cumbersome."

re:chef

I don't think this is necessarily a fair assessment and true for all. There are a ton of community cookbooks out there, and a very large community now online to help people with Chef. I've been using Chef for years and like anything else once you get over the learning period, it becomes second nature. Any tool that you use will have some learning curve to it, and so the trade offs that make it cumbersome for some, make it easy for others.

Also, you could technically Chef in Ansible in OpsWorks(if OpsWorks met other needs for you), but thats just crazy talk haha

"Except every other provider enables swap by default."

This is false and 100% incorrect.

Most AMI's on AWS don't enable SWAP by default whether supplied by a 3rd party or AWS themselves. Its not a forced thing either way.

In October Rackspace announced that they would be getting rid of SWAP by default: http://www.rackspace.com/knowledge_center/article/changes-to...

Linux VM's in Azure don't have Swap by default: http://blogs.msdn.com/b/windowsazure/archive/2013/07/29/swap...

On any host, physical or not, with more than 2GB of RAM people haven't run SWAP for years now. On cloud providers, even hosts with less than 2GB it is rarely worth it. It has horrible performance ramifications even when backed by SSD for the local host and any other tenants on that shared host.

This isn't a scam or them trying to upsell you. This is smart people who know systems operations making the right call for their customers.

I lived in a building where the landlord stopped renting out apartments normally and started instead putting them on Airbnb. He charged ~30% for doing it via Airbnb than the normal rents would have been. Then they raised rents more on apartments in the building. When I left ~4 out of 50 or so units were on Airbnb. Apartment next to me was a continuous ramble of people who left trash around and threw late parties(more than the normal residents).

I'm all for the idea of Airbnb, but there needs to be something in place that prevents this type of thing. Finding an apartment in NYC is rough enough, but artificially reducing supply via this type of thing causes the rent increases even more.

Taxing it seems like the only way to put the reverse pressure on landlords doing this, but I'm doubtful it will do enough.

everyone saying they can't hit certain sites(images on Wordpress/amazon/etc) is actually seeing the affect of their ISP's networks and the CDNs those sites use. Lotta Level3 issues right now. CDNs are often peered as close to the major ISP hubs as possible for speed reasons, but it can unfortunately result in these kind of chained outages. Since a lot of companies pump full site traffic through CDNs, it can appear as them being down.

There seems to be a lot of wasted breath in these comments about IPv6. Given that its unlikely that your provider even supports IPv6 on their networks, and that globally the traffic via IPv6 is nearly non-existent, I wouldn't spend too too much time caring about it.

Less than 2% of traffic to Google is IPv6: http://www.google.com/ipv6/statistics.html Even the attacks CloudFlare sees are mostly DDOS: http://blog.cloudflare.com/ipv6-day-usage-attacks-rise Traffic through Akamai is minimal: http://www.akamai.com/ipv6

Maybe someday it makes sense to spend a lot of time around IPv6 defenses, but today is not that day.

I built 2 CloudFormation templates to allow you to easily spin up a ton of these things across multiple availability zones. It uses the Amazon Linux AMI that exists in each region instead of the ones listed, and builds up the dependencies and the application on the fly. You can run either of these two templates in any region and it should just work.

Download one of these files:

With a keypair ( so you can login to the host) http://files.wordsaboutbytes.com/yahoo-messages-save.cf.txt

Without a keypair ( can’t log in locally, but it will run) http://files.wordsaboutbytes.com/yahoo-messages-save-nokeypa...

Then:

1. Open the console ( https://console.aws.amazon.com )

2. Go to CloudFormation

3. Give your stack a name

4. Browse and select the file you downloaded from above

5. Click Next.

6. Fill in the parameters here ( # of instances, The nick you want to be tracked with at the archive team site, the spot price you are willing to pay, and optionally a keypair if you selected that file).

7. Check the box at the bottom acknowledging that the template will create IAM resources ( used by the host to bootstrap )

8. Click Continue.

9. Tags if you want, or click continue.

10. Review. Click Continue.

11. Close.

This will launch however many instances you told it to, as t1.micro’s, as the spot price you set it to. When you want to stop, you just go and delete the stack in this console and everything should go away.

Running this right now in US-West-2, spread across all 3 AZ's there, about 90 instances total, and cranking through things.

Given that TOS's are often written by Lawyers it seems that their reading like a legal document has a high probability.

Fmr employee as well. I'm gonna go out on a limb here and guess you've never been involved in running/managing a huge online community(ies) with in person social interactions as the primary goal. I might be wrong, so apologies ahead of time if so.

Anywho -

It actually isn't unprofessional when the goal is to protect the members from some organizer who might be malicious in nature. There are a lot of rules and guidance around how organizers should do things, and Support's main goal is making sure the general members of Meetup aren't preyed upon. You're idea of giving a heads up, what are the parameters around before action is taken? 24 hours? 48 hours? A week? What if this post went "i was on vacation for a week so didn't read the email, and they deleted my group, how could they!!". It wouldn't matter. TOS are TOS.

If this person was malicious how much negative should the community accept? In that time how much spamming of a product the creator is trying to sell happening? How much misleading around the member base is there? You need to deal with these things as quick as possible, and the TOS exists to give people the framework of usage. A violation of a TOS(on any site) is just that, and it needs to be dealt with equally across violations. Support can and will make mistakes. That's human nature. THis isn't one of those. The poster in this case was even told they could re-create the group because at this point the old one is considered poisoned, and not to be trusted. Given Meetup has been doing this for over 10 years now, I'd like to think they have these policies pretty well grounded in reality and experience.

People can't reach out to you unless you are part of a group or via a direct "friend" relationship. Sounds like you joined a group that has someone spamming the rest of the members. Meetup themselves won't spam you like that unless you ask to be told when new groups are formed around certain topics (at least it used to be this way )

Fmr Meetup employee, for about 2.5 years, several years ago now. On the tech side, but the company is/was small enough that everyone got some voice about how things worked, which was fantastic.(note i say a voice, but not necessarily a vote)

This post reads a lot like "I violated TOS, and got shutdown for it, and now I'm going to complain because that isn't "fair"." WAAAAA.

The person who wrote this post used Meetup in the way Meetup doesn't want it's service to be used. What this person was looking for, was EventBrite, which is a fantastic site also, but geared to the idea of singular events that happen, and then go away. If the poster was trying to create a group of regularly meeting/communicating folks, to foster a true community, then Meetup would have most likely not shut them down. This is a lot different than the Vim London story, which it seems Meetup rectified after gathering better understanding ( seems like someone from community support responded to internal tooling flagging what looked like a violation and didn't understand what Vim was. Honestly, does your company's support people know what Vim is??). Meetup has a right to defend the use of their platform as they see fit, and when they do things like this its not for the one organizer who did the wrong thing, its for the X number of members who are part of groups who Meetup protects like a guard dog.

Meetup's support org is top notch, and they spend day and night watching an ever growing online community. When I was there we dealt with everything from Kiddie Porn Groups, Hate groups, illegal prescription sellers, pushy marketers, SPAMmers from around the world, and people who just generally wanted to abuse the trust that the platform tried to foster. The poster here hasn't posted the contents of what his Meetup group was defined as, nor has he posted the wording of the event. Note event in the singular sense, as opposed to events, or community, which is Meetup's purpose.

How about if you want to fling shit at a great company that serves a large user base with respect and honesty, you do the same before flinging out a "poor me/evil company" post without any basis for proof of your case. I would imagine a "12 year veteran" would get how this should work.