Amazon API Gateway – Build and Run Scalable Application Backends

https://aws.amazon.com/blogs/aws/amazon-api-gateway-build-and-run-scalable-application-backends/

by strzalek • 11 years ago

403 189 11 years ago

aws.amazon.com

ecopoesis2• 11 years ago

This looks incredibly slick. Speaking as someone who is implementing all the ceremony (security, logging, etc) around a new API right now I would use this in a heartbeat.

Of course, in a couple years, assuming success, the AWS lockin will suck. But given the odds of success I think I'd take the chance.

jively12• 11 years ago

You could always use an open source gateway like Tyk (https://tyk.io) or API Umbrella (full disclosure: I run Tyk.io, and this news is... disheartening).

herval• 11 years ago

Think of it as Amazon validating your business! It's yet another selling point (specially for orgs that can't or don't want to use AWS mind-boggling myriad of products).

angersock• 11 years ago

Wow, Tyk looks kinda exactly like something we'd use. Thank you so much!

profinger• 11 years ago

HackerNews effect :-( Site is down.

vidarh• 11 years ago

Consider that AWS has always been incredibly expensive. If you can't compete with AWS on price, then sure, it should be disheartening, but then you should take a long hard look at why.

Otherwise let AWS grow the market, and look for opportunities to serve people who don't want to pay the AWS premium and/or to plug feature holes.

alexbecker• 11 years ago

The analytics on Tyk look amazing, definitely a step up from the internal tools I've built.

Is Tyk HIPAA compliant? I'd definitely like to use something like Tyk, but I manage an API that transmits health data, so lots of tools have turned out to be off-limits because of HIPAA issues.

breakingcups• 11 years ago

Thanks for posting this, Tyk actually looks great (and easier to understand than Kong or Amazon API Gateway).

We've been considering building something like this ourselves or looking for a good existing solution, but hadn't found anything yet. I'm definitely checking this out.

vosper2• 11 years ago

Why do you think the lock-in will suck? Lack of flexibility or that you'll eventually want, pricing, or something else?

ecopoesis2• 11 years ago

Flexibility mostly. Pricing you can just bake into you're business model.

Flexibility however is always something you run into with platforms. And while this is a bit different then what folks typically think of as a platform (Django, Rails, Play, Spring) it looks like it would suffer the same problems. As soon as you need to do something weird, you're stuck fighting the platform or porting out instead of making your project better. This is why I prefer platforms that are more a collection of libraries with a preferred way of doing then as opposed to rigid box box style things.

kordless2• 11 years ago

Welcome to cognitive dissonance.

culo13• 11 years ago

Did you guys tried the open source api gateway KONG? (https://github.com/mashape/kong)

dangrossman• 11 years ago

API is an acronym, and the product is called "Amazon API Gateway". This submission title is bugging me more than it should. Sorry for the meta-comment.

Edit: The submission title has been changed since this comment was written.

dragonwriter• 11 years ago

API is an acronym

Actually, its an initialism (an abbreviation from first letters where each letter is pronounced as a letter) rather than acronym (an abbreviation from first letters pronounced as if it were a word.) Were it an acronym, many style guides (particularly British ones, IIRC) would support presenting it as "Api" rather than "API".

bdcravens• 11 years ago

Kinda like "Ask HN" right?

jakozaur• 11 years ago

Isn't it yet another case of AWS doing a cheap replacement of existing company: https://apigee.com

I doen't have experience with their product, but on surface they look similar.

mavus2• 11 years ago

I have used Apigee a bit. Amazon's product does seem to me a direct competitor to Apigee. For anyone already on Amazon's infrastructure it's a logical step to use another of their products than buy a service from a third party provider. Amazon's pay-as-you-go service is also very appealing as you might pay for a lot of features you don't use with a provider like Apigee.

daigoba66• 11 years ago

They is also http://www.mashery.com/. I've run across a lot of companies using Mashery to manage their API.

acyacy• 11 years ago

I wish Lambda would allow listening to a socket [it helps binaries communicate with node]. This would move our team to use this without any further doubt.

fooshint• 11 years ago

For me, I need either socket support or go support.

IanCal2• 11 years ago

Go support? You can run binaries on lambda, as long as they're small enough.

http://blog.0x82.com/2014/11/24/aws-lambda-functions-in-go/

athrun2• 11 years ago

What would prevent you from using this API Gateway as a proxy to your Lambda functions? It would work just fine for request/response use-cases.

mpdehaan22• 11 years ago

I get the web UI for understanding it, but this is often not how people want to work...

What tools are there to allow me to keep my code/API layouts in source control when uploading to this?

I'm sure they exist somewhere, so mostly curious about pointers. (A sample node project using this would probably go a long way)

BinaryIdiot• 11 years ago

I see this more of a replacement for parse which does a similar thing with cloud code both as far as I can tell there isn't a good place to keep the code / API layouts elsewhere.

athrun2• 11 years ago

The blog post mentions that you can describe your API using Swagger and then import that. I suppose it means you can keep your Swagger definitions under version control.

mpdehaan22• 11 years ago

thanks!

pea2• 11 years ago

Yo! We're building this at https://stackhut.com . You can see what an example API looks like @ https://github.com/StackHut/pdf-tools . You `stackhut deploy` this and it pushes it to an endpoint you can call over JSON-RPC or HTTP. Would love to hear your feedback (or ping me at leo@stackhut.com if you want me to walk you through anything!)

andybak• 11 years ago

I've not tried very hard but I'm not sure I get it.

I've got an API already running. What does this buy me?

Caching? I can see some benefit there it's read heavy.

Auth and access control? Feels like that's part of my app code but maybe there's a benefit I'm missing

A lot of the other benefits also feel like it would be hard to cleanly separate them from my app code.

What's the elevator pitch and who's the target market?

jively12• 11 years ago

API Managmeent market is quite big, makes sense for Amazon to get on board.

Imagine this: You are BigCorp USA Inc. and have a bunch of internal SOAP-based web services that are 10 years old and cost a fortune to maintain.

You hear someone will pay you to use them, but SOAP sucks. So you could either go and change your APIs (and probably break a bunch of internal processes) OR you could use a gateway to sit in front of them and translate in bound / outbound requests. Now you can charge people for your APIs!

Scenario 2: If you're an indie dev/ startup but dont want to run EC2 infrastructure, Lambda + Gateway = Cloud service, without any heavy lifting, servers and cost effective. It's development lego. Kind of like Parse.

Scenario 3: You have an API for your app, now you wnat third party devs to use it.Your API is not set up for that kind of access (multi-user security, throttling, quotas, analytics etc.) - solution: shove a gateway in front of it.

WaxProlix6• 11 years ago

#1 is exactly what I'm working on right now; turns out there's a huge market for this sort of thing. We're using an API gateway (Axway/Oracle) to build an "API Mullet" (all SOAP in the back, all REST in the front), and while I have some qualms with the tooling, the idea of an API gateway is very sound.

culo13• 11 years ago

API management is not big. Is under $500M/year [1] - plus most of the features are becoming commodities. Take a look at the open source KONG API gateway [2]

[1]https://www.forrester.com/Sizing+The+Market+For+API+Manageme...

[2] https://github.com/mashape/kong

mavus2• 11 years ago

Let's say you had 10 or so APIs that you wanted to monetise. You might want to bundle these up into groups that you sell. You might want institute different pricing tiers for different rates. To have to bundle all that into each application (along with communicating things like rate usage between them) might not be very appealing.

admiun• 11 years ago

This removes infrastructure from the equation entirely. You already had AWS Lambda which could run small pieces of Java or Javascript code when certain events were fired from other services (all without running a server yourself). The missing piece was something that catches HTTP requests and passes them to a Lambda function. This service seems to fill that gap.

It means you do not need to think about x instances that serve your API anymore. You just upload your Lambda function, wire some API path to it and pay per request. For smaller APIs this will be probably be negligible. Could be a gamechanger!

rowofpixels• 11 years ago

Yeah that's why I'm excited about this. I have a small instance that just runs a sort of "node proxy" to lambda, it's cheap ($11 / month or something), but annoying to run just to be able to connect to lambda. Now I can get rid of that.

_pius• 11 years ago

This is a pretty big deal, even if just for the fact that this plus Lambda eliminates fixed costs.

Rather than running an API server 24/7, you can write your API as a set of functions that cost you money only when they're actually invoked.

oblio7• 11 years ago

Let's just say that I know of a big cloud company who now has 1 major internal project which has been obsoleted by what Amazon releases now.

If you have a large cloud platform, an API gateway is priceless.

Especially if you have single sign on and therefore can use auth tokens for everything.

atonse• 11 years ago

I don't see why they'd be obsoleted. If I have infrastructure already setup on say, Azure, and Azure comes out with something like this, I'm going to use that instead of AWS, simply because it's familiar territory (not to mention LAN-level latencies between the gateway and my other API servers)

culo13• 11 years ago

https://github.com/Mashape/kong#architecture

Natsu3• 11 years ago

It depends on the features of the gateway, but you can add or centralize various functions and not have to make them a part of every API ever. So you save in not having to duplicate that code as well as being able to change it everywhere rather than having to fix one API at a time. Okay, sure, maybe you had a library or something that each API used, but this does make managing it very simple and simple is good, especially if you have hundreds or thousands of APIs.

In the API Gateway I work with allows you to do authentication within the gateway and then delegate responses to the back end API (which can still have its own security, if you really want it, but usually is just hidden on an internal network). It works pretty much the same for other things like rate limiting. It also gives you monitoring for each API, again cutting down on the things you have to add to every single API by encapsulating the function in the gateway.

Also, you can separate your internal API and your external APIs. Maybe you have to always support certain operations to the outside world, but internally you'd really like to change something. By having them separate, you can create a policy within the gateway to change how the internal calls are done without changing the interface as far as the outside world is concerned.

You can also do various types of conversion (e.g. REST->SOAP) or even more exotic things so that outside devs don't have to care just how weird your damned mainframe is or other aspects of your internal architecture that they shouldn't know about.

The target market is, effectively, anyone who wants to expose internal APIs to the outside world.

Source: I work on an API Gateway and this is how our product works.

joeyspn2• 11 years ago

Seems like a great product to quickly get started with a mBaaS in a powerful cloud like AWS. The concept looks really similar to StrongLoop's loopback [0] with a big difference: vendor lock-in. I like the openness that StrongLoop is bringing on this front... IMO the best solution is one that allows you to move your containerised API from cloud to another cloud.

That being said, having this as an option in AWS is pretty cool and potentially time-saving. I'll probably give it a shot soon.

[0] https://strongloop.com/node-js/api-platform/

culo13• 11 years ago

There is also api umbrella or KONG which is the most popular OSS API gateway: https://github.com/mashape/kong

cdnsteve• 11 years ago

There goes the rest of my workday :D

I was looking for something like this. Lambda functions are amazing but restricted because they weren't easily consumable externally. This is they key.

pw• 11 years ago

Totally. This takes Lambda to a whole 'nother level.

cddotdotslash2• 11 years ago

This is definitely a game changer. Probably 80% of our apps can move to API Gateway > Lambda and we can get rid of scores of EC2 servers.

Marazan• 11 years ago

In onl fly in the ultra-low cost ointment is caching

To put the minimum cache in front of your API costs more than running an EC2.micro

For my projected use where my volume are going to be so low that I will fall under the Free lamda tier and the very first API tier I don't need to worry about caching but for those that do the price will start to rack up.

kordless2• 11 years ago

Given the current market direction with containerization and decentralization, I think using something that is vendor specific is probably a bad idea.

traek• 11 years ago

Google has a similar offering for apps running on App Engine, called Cloud Endpoints[1].

[1] https://cloud.google.com/endpoints/

culo13• 11 years ago

that's not similar. there is no API management in there.

TheIronYuppie• 11 years ago

What are you looking for in API management? There's key management, DDos protection, etc.

dougcorrea• 11 years ago

https://console.aws.amazon.com/apigateway is down for me

iamflimflam1• 11 years ago

Me too - 404 page

jonahx• 11 years ago

Would someone knowledgeable mind answering a few questions:

1. What are the differences between this + AWS lambda and parse? Is there additional functionality or freedom with this route? Is it cheaper?

2. What kind of savings could one expect hosting an API with this vs a heroku standard dyno?

jamiesonbecker• 11 years ago

AMAZON DEPRECATES EC2

November 3, 2017, SEATTLE

At the AWS Re:Invent in Las Vegas today, Amazon Web Services today announced the deprecation of Elastic Compute Cloud as it shifts toward lighter-weight, more horizontally scalable services. Amazon announced that it was giving customers the opportunity to migrate toward what it claims are lower cost "containers" and "Lambda processes".

"We believe that the cloud has shifted and customers are demanding more flexibility," tweeted Jeff Barr, AWS Spokesperson. "Customers don't want to deal with the complexities of security and scaling on-instance environments, and are willing to sacrifice controls and cost management in order to take advantage of the great scale we offer in the largest public cloud."

Barr went on to add that since their acqui-hire of Heroku last year, AWS has decided that the future of the cloud was in Platform as a Service (PaaS) and is now turning its focus to user-centric SSH key and user management services like https://Userify.com.

Amazon (AMZN) stock was up $0.02 on the latest announcements.

hitekker• 11 years ago

A little anal: Heroku is already owned by Salesforce.

Kudos• 11 years ago

November 3, 2017, SEATTLE

sneak• 11 years ago

Skype used to be owned by eBay.

shankysingh• 11 years ago

I almost got an heart-attack because I skimmed over the date :)

graycat9• 11 years ago

Okay, at Google I found that IAM abbreviates Amazon's Identity and Access Management.

So, the OP has an undefined three letter acronym.

Suspicions confirmed: The OP is an example of poor technical writing.

Why do I care? I could be interested in Amazon Web Services (AWS) for my startup. But so far by a very wide margin what has been the worst problem in my startup? And the candidates are:

(1) Getting a clear statement of the business idea.

(2) Inventing the crucial, defensible, core technology.

(3) Learning the necessary programming languages.

(4) Installing software.

(5) System backup and restore.

(6) Making sense out of documentation about computer software.

May I have the envelope please (drum roll)? And the winner is,

(6) Making sense out of documentation about computer software.

And the judges have decided that uniquely in the history of this competition, this selection deserves the Grand Prize, never before given and to be retired on this first award, for the widest margin of victory ever.

No joke, guys: The poorly written documentation, stupid words for really simple ideas, has cost me literally years of my effort. No joke. Years. No exaggeration. Did I mention years?

At this point, "I'm reticent. Yes, I'm reticent." Maybe Amazon has the greatest stuff since the discovery and explanation of the 3 degree K background radiation, supersonic flight, atomic power, the microbe theory of disease, electric power, mechanized agriculture, and sex, but if Amazon can't do a good job, and now I insist on a very good job, documenting their work, which is to be yet another layer of documentation between me and some microprocessors, then, no, no thanks, no way, not a chance, not even zip, zilch, zero.

What might it take me to cut through bad Amazon documentation of AWS, hours, days, weeks, months, years, then from time to time, more hours, days, or weeks, and then as AWS evolves, more such time? What would I need to keep my startup progress on track, 500 hours a day? More? All just to cut through badly written documentation for simple ideas, and worse documentation for complicated ideas?

First test: Any cases of undefined terms or acronyms?

Result: About three such cases, and out'a here. Gone. Done. Kaput. I don't know what AWS has, but I don't need it.

Sorry, AWS, to get me and my startup as a user/customer, you have to up your game by several notches. The first thing I will look at is the quality of the technical writing in your documentation. And, I have some benchmarks for comparison from J. von Neumann, P. Halmos, I. Herstein, W. Rudin, L. Breiman, J. Neveu, D. Knuth.

Amazon, for now, with me, from your example of writing here, you lose. Don't want it. Can't use it. Not even for free. I'm not going to invest my time and effort trying to cut through your poor technical writing. And, the next time I look at anything from AWS, the first undefined term and I'm out'a here again.

Yes, I'm hyper sensitive about bad technical writing -- couldn't be more sensitive if my fingers and arms were burned off. Whenever possible, I will be avoiding any chance of encountering bad technical writing, ever again. Period. Clear enough?

More generally, my view is that bad technical writing is the worst bottleneck for progress in computing. Come on, AWS, up your game.

I don't want to run a server farm and would like you to do that for me, but neither do I want to cut through more bad technical writing -- for that work, my project is already years over budget.

prof_hobart6• 11 years ago

OP is writing a technical blog on AWS, primarily aimed at existing AWS users. Defining what IAM means in there is about as pointless as you needing to define what OP meant in your post.

graycat9• 11 years ago

primarily aimed at existing AWS users.

Either that, that is, really poor business thinking, or, as I concluded, some poor technical writing. I don't believe the first.

NathanKP3• 11 years ago

When you were a child you started out learning simple one syllable words, and speaking one word sentences.

When you were toddler you started stringing those words together, and learning how to form extremely basic sentences.

And eventually you learned to form complex sentences with complex ideas.

The same thing applies to learning computing terms. AWS is now 9 years old, and has 9 years of development and features to catch up on if you are just starting. Yes, it will take some effort to learn everything, and if you don't see the benefit in doing so then you don't have to.

But if you want to be an "adult" engineer rather than a "toddler" engineer it is necessary to be willing to learn some things, even if it isn't easy.

graycat9• 11 years ago

I strongly disagree: I'm under no obligation to have followed AWS for the past nine years. Instead, in each field, we should write in ways as self contained, and make the minimum assumptions, possible. In particular, in technical writing, we should not have undefined terms.

The research libraries are awash with single texts that start with meager prerequisites and get to some quite advanced and challenging material. AWS can do so, also.

I've been in computing for a long time, but there's no hope for me to know all the jargon of all parts of computing. Mostly jargon is not conceptually difficult, and the solution to the challenge of jargon is simple -- just define it.

Parts of computing have long enjoyed having a closed priesthood that spoke in three letter acronym gibberish. IBM used to do this with PCP, MFT, MVT, MVS, JCL, DCB, IMS, DB2, CMS, SQL, and many more. And there were some four letter acronyms -- CICS, DASD, REXX.

The concepts were mostly simple; the gibberish was a secret language. Once I gave a grad seminar on computing and had a lot of guests from industry. IBM talked in their gibberish, but none of the other vendors did. The students caught on: IBM wanted to intimidate people and otherwise invite people into their priesthood where they, too, would understand the gibberish. IBM was the least popular of the vendors.

sgustard• 11 years ago

From Google I just learned that API stands for "American Petroleum Institute", so that has really cleared up a lot of confusion for me too.

marincounty• 11 years ago

I am not a Programmer. I put up simple websites, and like technology, and enjoy this field. That said, I probally shouldn't comment, but from day one of my computing journey; My biggest gripe with this field has been terrible technical writing. (I put up with it because it was all so new, and figured it will eventually get better.)

I can still remember picking up those 500 page computing books. Flipping through the phone book and putting it back on the shelf.

As to the overuse of acronyms; I think they are overused in this profession. It almost seems fashionable?

Jump to today, things have gotten better. I'm not sure the writing has gotten better, or I just have more presenters of the information(Internet)?

I don't know why you got down voted? The level of technical writing in this field is atrocious? I understand "these are tough concepts!". Yes, in many ways it's like learning a language! I get it. In most cases I'm glad someone took the time to write something down. Yes--I'm grateful.

I do have a problem with entities, with a lot of money/resources, whom publish terrible technical writting. It does seem like a big company, like Amazon, could have done a little better job conveying this product?

selimthegrim2• 11 years ago

Much like for Multics, Algol, Smalltalk, Unisys, Plan-9, Modula-3, various LISPs, etc, the kinds of people who care about bad technical writing probably have a budget that allows them to pay for more than AWS.

adamkittelson• 11 years ago

It'd be cool if they'd use this to wrap their own XML-only APIs to provide JSON wrappers.

culo13• 11 years ago

Smart move for AWS, but they are not innovating nothing here, just following. Late.

Companies like Apigee/Mashape/3scale/Mulesoft have been doing cloud API management in various forms since 2008. Even Microsoft Azure has an API management offering since two years.

Nowadays all those API gateway features are commodities and doesn't make sense to pay for it anymore. Indeed Open Source projects such as KONG [1] are getting tremendous tractions. Same things happened in search with all cloud solutions and then ElasticSearch came out and was game over.

[1] https://github.com/mashape/kong

oblio7• 11 years ago

AWS has a lot of pull. And they iterate quickly. Plus AWS is probably still the most complete cloud offerings, and integration is always great.

culo13• 11 years ago

the more complete is the integration, the more locked in you are.

fosk5• 11 years ago

This is very interesting, and I am surprised it didn't happen a long time ago. The Lambda function integration opens up lots of new ideas when building API backends ready to be consumed by clients apps like, for example, a client-side Javascript application.

On the other side it seems like other extra functionality is limited and very AWS-oriented. If you are looking for an open source gateway that can be expanded with extra functionality, and potentially sit on top of the AWS-> Lambda integration, take a look at https://github.com/Mashape/kong (I am a core maintainer too, so feel free to ask me any question). Kong is accepting plugins from the community (http://getkong.org/plugins), which can be used to replace or extend functionality beyond any other gateway including AWS Gateway.

jively12• 11 years ago

I think AWS is going after the Parse / Heroku market here, the combination of Lambda and this gateway make it more suited for that kind of implementation, they are missing out on the larger features that enterprise needs - the blog post doesn't really go into detail on how SOAP conversion works, and I doubt it's a cakewalk.

It's in legacy API services that places like Mashery and Apigee make all their real money, and I can't see large enterprises shifting their legacy services to EC2. It's quite forward looking. The satire on how EC2 may be deprecated is really not far off from where things are heading.

I always find it quite funny when only single OSS projects are mentioned as alternatives, there's so many more full-featured projects out there: Tyk (https://tyk.io) API Umbrella (http://apiumbrella.io/) and ApiAxle (http://apiaxle.com/) to name some. For more fine grained approaches there's also Vulcan (http://vulcanproxy.com/)

culo13• 11 years ago

apiaxle and apiumbrella have few features actually. vulcan is not for api management and beside that quality > quantity.

__Joker• 11 years ago

I have worked with L7 for a client which offers something similar. The whole thing was painful in a way.

The SOAP to JSON conversion is easy for baby SOAP or JSON requests using transformation. For larger ones they turned out to be complex.

Once you start with client started with using the "SOAP" converted to JSON apis, there were new feature kind of requests. Now the problem is somebody needs to be dig into that old code mess of SOAP web services. Or write new services and split requests between old and new services.

There will be performance issues because of the no of transformation doing on top of the request. Ultimately has to write plugins on top write performant transformation in a single piece.

Personally SOAP to JSON conversion will not be as simple as move to AWS, although I will assume Amazon will do far far better job on this than others.

getriver• 11 years ago

kong looks pretty cool!

any plans to add pluggable backend ie not use cassandra? i've no experience managing cassandra and would rather not worry about it.

fosk5• 11 years ago

Yes, Kong plans to add support for PostgreSQL (https://github.com/Mashape/kong/issues/331). In the meanwhile you can outsource Cassandra to cloud providers like https://www.instaclustr.com

For testing check out http://kongdb.org/, but don't use it in production.

bbq• 11 years ago

Interesting to note that Jeff Bezos is an investor in Mashape: https://www.crunchbase.com/organization/mashape

sinzone2• 11 years ago

If you guys want to learn more live and are based in SF, we're hosting our 2nd KongSF meetup [1] next week, along with the announcement of the v 0.4.

[1] http://www.meetup.com/The-Mashape-API-Developer-Community/ev...

serverholic• 11 years ago

How do you develop for these kinds of services? It seems like you'd need to setup a whole development cluster instead of developing locally.

jively12• 11 years ago

Therein lies the rub - if you are working with AWS' key-value store, and then also their Lambda system, then you don't really develop server side - you develop everything client side (mobile app) and just use the cloud as a dumb logic store.

plicense• 11 years ago

Well, you are still free to write resource consuming operations and deploy it to an EC2 instance and call it from Amazon Gateway. Lambda is just an example.

Moreover I feel Lambda can do pretty good stuff. I am sure it will integrate with other AWS services pretty soon and once there is connectivity to an RDS instance, then it solves the use case for half the developer population out there.

brightball• 11 years ago

One of the other benefits of using Cloudfront based endpoints is that your app servers behind it can avoid the TCP handshakes that add some latency. Amazon did an interesting presentation at re:Invent on the performance improvement from using Cloudfront ahead of dynamic requests that was eye opening.

intrasight• 11 years ago

They say "If you already utilize OAuth tokens or any other authorization mechanism, you can easily setup API Gateway not to require signed API calls and simply forward the token headers to your backend for verification." It would be nice if AWS would stand up an authentication service that could handle oauth. Or do they already have such a thing?

fosk5• 11 years ago

Kong[1] is releasing OAuth 2.0 support in a week or so (when 0.4.0 will be announced). You will see it appear in the plugin gallery[2]

[1] https://github.com/mashape/kong [2] http://getkong.org/plugins/

joeyspn2• 11 years ago

They say we can use Amazon Cognito[0]. I hope they put some examples with ACL and Role management (I wonder if this could live in lambda functions using packages like node_acl)

[0] http://aws.amazon.com/cognito/

estefan• 11 years ago

Please can people give examples of what they're using lambda for. Everything I've seen has been really basic (like image scaling), but most things I think of require a database.

deegles• 11 years ago

I'm writing several apps for the Amazon Echo that are hosted on Lambda. I'm also using DynamoDB.

mattkrea• 11 years ago

I use it with Amazon Kinesis to funnel Kinesis data into a Slack channel after it has been parsed.

trevordixon• 11 years ago

Compiling LilyPond scores (http://lilybin.com). Usually we have at most 1 user at any time, but we can also handle bursts, like when a classroom of 30 students all use it at the same time. Ends up even cheaper than running one VM that can only handle 2–3 simultaneous users.

cddotdotslash2• 11 years ago

Image processing primarily: user uploads an image to S3, a thumbnail version gets created automatically. Now, we'll probably put it behind some of our APIs that see huge swings in traffic. It'll be nice to not pay to be running even a single x-large instance at night when the API is only seeing a few hundred hits per hour.

tootie3• 11 years ago

For heavy users of AWS services (not just EC2, but fancy SaaS/PaaS stuff) do you ever regret being locked in to a hosting provider? Does it restrict your ability to develop locally? Have you been bitten by problems that you can't resolve because you don't own the product? Or do you pretty much just love it?

xrstf• 11 years ago

We had cases where failing MySQL queries[1] would leave behind GigaBytes of garbage data that we (as users of Amazon's RDS) have to pay for and have no option of removing (besides killing the entire database and rebuilding a new one, which would cost us more than just living with the garbage). Having our own small EC2 instance with MySQL on it would have saved us quite a few hours of work/downtime.

[1] Have a database of a few gigabytes of sizes, start an ``ALTER TABLE`` and wait until your pre-allocated disk space is filled up. The query dies and you just fucked yourself. Can happen unexpectedly: We tried to remove a non-existing index on a table and MySQL decided to FIRST copy the entire table and THEN report that there is no such index.

gtaylor• 11 years ago

FWIW, we absolutely loved it at the last place I worked. We were all too happy to let AWS host our Postgres instance (RDS), our static assets (S3 and CloudFront), and a bunch of other things.

We weren't particularly locked in, either. We always had the option to run our own Postgres cluster (it just didn't make sense to spend the manpower on it for us). We could have hacked together our own CDN, but that wasn't our core business focus.

For small companies with low manpower, build on the services and get things done fast and as well as possible. It may cost more upfront than doing it on your own, but you are probably going to come out ahead by minimizing your maintenance/security/administration surface area.

ademarre• 11 years ago

It sounds like you were using relatively loosely coupled AWS services. RDS doesn't change your programming interface to the database, it just hosts it. If you needed to, you could switch to a non-RDS Postgres server with only a configuration change to your core application.

Contrast this with using AWS DynamoDB, which would likely be a major overhaul if you needed to switch to something that could be used outside of AWS.

NathanKP3• 11 years ago

Do you ever regret being locked in to a hosting provider? Does it restrict your ability to develop locally?

No, we use Vagrant so that we can run our docker cluster either locally or on Amazon. We have stubbed alternatives for all AWS services, including SNS and SQS, allowing us to run the entire backend locally even without an internet connection.

Have you been bitten by problems that you can't resolve because you don't own the product?

I'd say its kind of like coding in Ruby on Rails. If you are bit by a problem you can't resolve there is a high chance you are doing something wrong, or at least not the "one true way".

Sometimes we have issues when exploring the edges of what is possible on the AWS ecosystem (we've been using the fresh-out-of-beta ECS, and running into a lot of limits there), but generally the basic stuff like S3, SNS, SQS, Elasticache, etc just works, and doesn't require any attention. It's just a working, hands off gear in your system.

Do you pretty much just love it?

Sure. It's a lot better than racking physical hardware, and the deep integration between things like autoscaling groups, EC2, ELB, etc makes it easy to automate all the things. In the end the more our service is a machine that can run itself and repair itself (replacing its own instances as needed) the more I love it.

eric_h2• 11 years ago

We moved our whole stack from manually managed Linode instances to AWS Opsworks, Elasticache, RDS, etc. We are currently somewhat tied now to the AWS stack, though through a large collection of our own Chef scripts I imagine we could move to a manually setup chef setup ourselves.

Largely we've been very pleased with AWS, but the black box nature of a number of products (particularly Elasticache and RDS), has caused us some minor grief. Also, the higher frequency of single instance failures (which our stack can handle pretty gracefully) is somewhat annoying.

EDIT: We were also able to fairly significantly lower our bill with reserved instances while also significantly increasing our ability to handle peak capacity with AWS.

tootie3• 11 years ago

I would think Elasticache and RDS would be the least of your worries since they're just fully managed versions of readily available software. You can always host your own Redis and SQL database. Something like DynamoDB would be way more complicated.

agentgt• 11 years ago

I can't help but notice that this looks more like an enterprise integration tool (think mulesoft) than API management (think apogee.. or I think that is what they do).

Speaking somewhat from experience (webmethods, caml, spring-integration and various other enterprise integration tools) they always want you to use "their" DSL which is often not even stored on the developers filesystem (ie webmethods... you submit the code to the "broker" or "rules engine" or "router"... lots of words for thing that does the work). Which leads to very awkward development.

Consequently I wonder if they will have git integration because writing code even snippets in a web form no matter how nice gets old fast.

jively12• 11 years ago

Prediction: Amazon buys a cloud IDE platform.

jstoiko• 11 years ago

I am surprised that Amazon did not add support to more API description formats like RAML or apiblueprint. It is such a key feature. If I wanted to use this service in front of existing APIs, even only one API, I would not want to go through the work of having to redefine all my endpoints through a web form!

Shameless plug: after working on several API projects, I have been researching ways to not have to "code" over and over again what goes into creating endpoints, it became so repetitive. Lately, I turned to RAML (Yaml for REST) and, with 4 other developers, we created an opensource project called Ramses. It creates a fully functional API from a RAML file. It is a bit opinionated but having to "just" edit a Yaml file when building a new API simplified my life. As a bonus, I also get a documentation and a javascript client generated from the same file.

EDIT: forgot the url https://github.com/brandicted/ramses and url of a quick start tutorial: https://realpython.com/blog/python/create-a-rest-api-in-minu...

vlad6• 11 years ago

Check out this screenshot, what I may release as ApiEditor.com

http://i.imgur.com/wSEKeVb.png

It would be nice to have one place to edit routes with a good UI, better than what Amazon released today.

sytringy052• 11 years ago

swagger 2.0 spec and RAML are almost the same thing. There's some minor differences but I would guess you could import the RAML into AWS and it would work

kpennell• 11 years ago

For someone who doesn't understand this that well, is this similar to firebase?

jively12• 11 years ago

Not really, firebase is a database that has a RESTful interface (and events and all kinds of other cool stuff). with this you set up endpoints, point them at cloud functions that may or may not interact with AWS' DynamoDB and your own services to provide functionality but I think you need to worr it all up yourself.

vlad6• 11 years ago

I'm working on ApiEditor.com, here's a screenshot:

http://i.imgur.com/wSEKeVb.png

I originally built it for the Swagger Spec 15 months ago, as the first visual Swagger Editor. Let me know if you guys are interested in using something like this.

Notice it has the code side-by-side. Also, it scrolls to and highlights the section of the spec you're editing. Notice the dropdowns are aware of custom models (e.g. "User", "Order") in the documents and suggest them to make it easy to use.

ayushgta• 11 years ago

Also see http://editor.swagger.io

vlad6• 11 years ago

I'm not mentioned on that site or their launch blog post, but my code to ApiEditor was shared privately with the Swagger Editor folks who went on to create that editor, also in AngularJS.

fehguy• 11 years ago

Looks great vlad. It is true, this was the first known visual swagger editor and was a very helpful project (also open sourced). The Apigee folks went on to build the current version, a separate effort from the swagger-ui.

ramon• 11 years ago

https://console.aws.amazon.com/apigateway up and ok here :), great!

zenlambda• 11 years ago

I just tried this out with a Lambda function; I was wondering why you can't serve HTML with this (Yes, I know this product is aimed at enterprise ReST API stuff... one can try at least).

Well, it seems that authentication for the client is mandatory. This makes it unsuitable for rendering markup and serving it directly to clients.

Can anyone confirm this can only serve JSON content? I suspect were anonymous requests allowed, I'd see my markup rendered as a JSON string.

jwatte2• 11 years ago

If you absolutely must build a website on this, you can serve the bootstrap HTML page out of S3 and use the JSON to populate it, single page app style.

pea2• 11 years ago

This looks really interesting -- I think the abstraction of the server away from a bunch of development cases is gonna happen pretty quickly.

We're hacking on something really similar to this at https://stackhut.com, where we are building a platform for Microservices which are powered by Docker, can be run wherever through HTTP/JsonRPC, and are really simple to build/deploy. Think "Microservices as a service"... ;)

To give you an example, here is a task which converts PDFs to images: http://stackhut.com/#/services/pdf-tools, and here is the code powering it https://github.com/StackHut/pdf-tools which we `stackhut deploy`d.

We're about to open up the platform and would love to hear what, if you had a magic wand, you'd wish upon the product.

anoncoder• 11 years ago

Do some math. It's expensive. 100 r/s for one month is about $900. Plus your bandwidth and EC2 charges (unless your using Lambda). For simple Lambda functions, you can get 100 r/s on a micro for $9.

clay_to_n• 11 years ago

For those interested, the creators of Sails.js (a node-encapsulating framework) have created a sorta similar product called Treeline[1].

[1] https://treeline.io/

fiatjaf• 11 years ago

How do you test these things?

jwatte2• 11 years ago

Service as a Service - we've reached Serviceception!

machbio• 11 years ago

I was trying this API gateway out, unfortunately there is no way to delete the API created..

avilay• 11 years ago

I had built a command line tool for a similar purpose to generate REST APIs running on the MEAN stack. It creates "user" and "account" resources by default with user and admin authn/authz built in. It then deploys to heroku - creating a mongodb instance and a web dyno. Putting this out here in case anybody finds it useful.

https://bitbucket.org/avilay/api-labs/

vlad6• 11 years ago

I posted a 10 minute video overview for those who'd rather listen than read:

http://ratemyapp.com/video/VIqZFaU1PhQ/Product-Preview-Amazo...

Amazon API Gateway – Build and Run Scalable Application Backends

Related Stories