Day One with the Amazon API Gateway

https://blog.hiramsoftware.com/blog/day-one-aws-api-gateway/index.html

by ihiram • 11 years ago

99 35 11 years ago

blog.hiramsoftware.com

nolite• 11 years ago

thanks for your sacrifice in investigating this. I will return in 2016 when hopefully this is production-ready

Just a quick nitpick: the second link in the article (the one to AWS Lambda) should have an all-lowercase URL; the URL's case-sensitive and - in its current capitalized form - is resulting in a 404.

impostervt• 11 years ago

Good write-up, interesting find about the lacking features. My take on the API Gateway - https://medium.com/@john.titus/moving-a-simple-api-to-amazon...

cddotdotslash• 11 years ago

My take after trying this out for a few hours mirrors yours: it looks like it was rushed out the door and not 100% ready. It's fun to play around with, but I wouldn't put anything production on it for at least another few months.

lflux2• 11 years ago

I agree - the docs are kinda hard to parse, and after launch it didn't work in us-west-2 for several hours. I've got great ideas for what I can build with it if they fix a few issues with it, but for now it's just a toy.

justonepost• 11 years ago

Isn't lambda suicide for a startup? Think of all the lock-in here. If you go big you could suddenly find yourself really stuck if you're overly dependent on it and the overhead of lambda/gateway is too much.

snikch• 11 years ago

If it enables a start up to grow without having to worry about scale, I don't think so. You could say the same of a lot of the technologies in a cloud provider. At the end of the day, if you want to take advantage of the speed to market and cost reductions these technologies enable, you're going to have to take a punt.

easong• 11 years ago

It's actually not bad at all - I can't speak for the java side of things, but transitioning a couple random nodejs services to lambda took under ten minutes each, with most of that time being testing (my only complaint is that the logging sucks). Lambda function are so modular that I can't imagine not being able to trivially slot them back in to a standard express endpoint. It might be tricky if you do a bunch of stuff with S3, but even then it shouldn't be terribly difficult to sub in the standard AWS modules.

cbhl• 11 years ago

A startup is more likely to die for another reason first, such as a founder-breakup, or some form of "running out of money".

Google App Engine and Heroku and Azure also have lock-in too, to varying degrees -- but at the start it still makes sense to use these sorts of services rather than worry about lock-in. (And if someone later gives you $100k of compute credits on their platform, and it costs you less than that to rewrite the "lock-in" parts, then you still win.)

joeyspn4• 11 years ago

Isn't lambda suicide for a startup?

With no upfront costs and w/o the need to manage servers I'd say it's actually the other way around! ;)

omgitstom• 11 years ago

I spend some time with API Gateway this weekend. Since I work at a company that has a product that helps customers with authentication and authorization for APIs, I am always curious where everyone else is skating.

Your section about authentication resounds with me loud and clear, I couldn't hack it out in 6 hours. This will be a pain point for anyone using API Gateway for anything other than a toy API.

jessaustin• 11 years ago

Thanks for a clear, sensible investigation. ISTM your auth complaint is well-taken. Either Lambda should be seeing the same role as the gateway automatically, or there ought to be a way to configure that.

However, I have to admit that my first reaction to that difficulty in getting the account ID from the path/query/etc would have been to just shove it in the request body JSON that was already coming through. Then it could probably be taken out of the path completely, which would be a win for url privacy at least. This doesn't seem like the showstopper that the auth issue does.

_Marak_• 11 years ago

The feature-set for this is pretty low. I've built a better solution with much more raw access to HTTP request and response streams.

If anyone is interested in an open-source version of Amazon API / Amazon Lamba, you can check out: http://github.com/bigcompany/hook.io http://hook.io

cariaso• 11 years ago

My major use case is a read only, throttled, and cached front end for the mediawiki api at http://snpedia.com . The ability to monetize the api for higher throughput is an appealing plus. Gateway still seems pretty good for this. I'd welcome input or thoughts from others.

fosk2• 11 years ago

Having the possibility to create an HTTP RESTful interface to many Lambda functions running on AWS is the biggest advantage of the AWS API Gateway.

Technically, stretching the system a bit, one could argue that we don't need backend software running on EC2 anymore, since everything can be a Lambda function with an associated HTTP endpoint to trigger it. I have been thinking about this a lot, and it could work if being locked into AWS is considered an acceptable risk.

For anything else more API-oriented (more authentication schemes, better security, more logging options, etc) I would suggest checking out Kong (https://github.com/Mashape/kong), which is pluggable with extra functionality and can work in front of the AWS API Gateway to enhance the REST-to-Lambda interface (and with any other API).

maratd• 11 years ago

I have been thinking about this a lot, and it could work if being locked into AWS is considered an acceptable risk.

This is a huge risk. If they shut you out, for whatever reason, you're done. You can't port your code to somewhere else. It's Lambda specific.

wnevets2• 11 years ago

You can't port your code to somewhere else. It's Lambda specific.

is that really true tho? Its just a function that does work from a request and send the result off to some where else.

ac3605• 11 years ago

I appreciate Kong. But we are here because we don't want to deal with servers!

sinzone• 11 years ago

You don't. Kong (gateway) + AWS/DigitalOcean (the servers).

silasb• 11 years ago

I've started messing with API Gateway and Lambda there a couple errors.

If you create a lambda function and add a gateway inside the lambda function configuration screen, it will give you a bad API gateway URL. Going into the API gateway page and looking at your stages URL and trying that works.

This just proves that it was rushed out. I do like the idea though and want to use it, but I don't think it'll be ready until Q4.

lflux2• 11 years ago

I've experienced the same frustrations with the API Gateway. Lack of knowing the IAM principal that authenticated, and no clear way to differentiate if a call came from the API gateway or not makes me feel that this needs some work. SNS, for example, will sign it's HTTP requests and that would be a good start IMHO. Ability to use security groups in proxy mode would make this even better.

ac3605• 11 years ago

Amen.

Last Sunday morning, I was so excited to move my APIs over to AWS’s API Gateway. That evening, I was ready to kill.

I spent the entire day (9+ hrs) getting merely two API routes working. A GET and a dreaded POST request.

I couldn’t tell from your post if you eventually figured it out, but it is possible to get the POST request body and url params in your Lambda event as JSON.

In my POST Method’s Integration Request, here’s how I get the query string parameter and the request body. I use this Input Mapping.

#set($inputRoot = $input.path('$')) { "access_token":"$input.params('access_token')", "name" : "$inputRoot.name", “email” : "$inputRoot.email” }

A “glass half full” way to think about all of this is that the AWS API Gateway is ridiculously strict about what data can be passed in to your Lambda functions, and you must explicitly state that data.

Here are other problems I ran into over the weekend:

- The web console is super buggy with broken links and really, really bad UX.

- The documentation is missing for many essential components.

- No support for API Gateway in the AWS SDKs

- I can't figure out how to copy resource methods, so I have to do tons of redundant manual input when creating each resource method, which includes explicitly stating every HTTP Error response code.

- Returning error objects from Lambda functions also needs mapping.

- Out-of-the-box slow response times. My Lambda function with two simple DynamoDB queries often takes 3 seconds to return a result when its route is hit.

Overall, I'm a huge AWS fanboy but the execution of this product is incomplete and out of touch.

C'mon AWS, you can pull through on this. Keep it simple. Build out from there. We all want you to succeed.

ac3605• 11 years ago

UPDATE: They just announced a nice fix for easily getting the request JSON Body into your Lambda functions and some documentation improvements. See jeffbar's post in this thread.

joeyspn4• 11 years ago

Out-of-the-box slow response times. My Lambda function with two simple DynamoDB queries often takes 3 seconds to return a result when its route is hit.

This is something I've also noticed. Even their official examples are really slow [0]

About auth: I've been researching API Gateway + Lambda and auth via Amazon Cognito. As of now the best example for auth I've found is LambdAuth [1]. I'm trying to add the API Gateway in the mix, it looks possible to integrate it.

I also agree that docs and more explicit examples are needed.

[0] https://aws.amazon.com/blogs/compute/the-squirrelbin-archite...

[1] https://github.com/danilop/LambdAuth

simonel• 11 years ago

Apparently, there's no way to use API Gateway with Cognito, it looks (and should be possible) there are so many lacks in the documentation... I'm following this forum thread [0] regarding Cognito integration in JS. [0] https://forums.aws.amazon.com/message.jspa?messageID=645618#...

jeffbarr• 11 years ago

I spent some time reviewing this post with the API Gateway team here at Amazon! Here's what I learned:

1. We are updating the documentation in response to the feedback in this thread and in the blog post. Additional feedback is always welcome.

2. Based on the feedback in the blog post, mappings can now access the JSON body of a POST. See the answer (login required) at https://forums.aws.amazon.com/thread.jspa?messageID=645596&#... .

3. Models are not required for validation. They are simply used to generate the objects in the client SDKs.

danielhunt• 11 years ago

This here? This is why I love HN.

A good, useful, detailed blogpost that highlights a number of important blocker-level issues - and there's an unexpected, speedy response from the previously-anonymous devs announcing improvements as a result of the post and analysis itself.

Fantastic stuff

ac3605• 11 years ago

Thank you Jeff and the API Gateway team for the communication and prompt fixes.

I’ll update my original posts so that they do not misinform others and I look forward to future improvements.

One additional thing that would be incredibly useful is a simple blog post that describes steps/options one could take to increase API Gateway + Lambda response times. I'm fiddling with this now, but I wish I had some guidance as to where to focus my optimizations and how to make them, all in one place and straight from the source (AWS). For example, I'd love some examples on API Gateway caching, making my API accessible in multiple regions, how to make my Lambda functions faster, and why my first requests to my API Gateway + Lambda routes are so slow even after only a few minutes of inactivity. On a first request, my Lambda function will run for 1.2 seconds but the network request as a whole takes 2.8-3.2 seconds, yet every immediate subsequent request comes in much faster (~1.4 seconds)(Everything is in the same us-east region). If there is nothing I can do about this because the server my lambda function is on has been de-prioritized due to minutes of inactivity and is in an idle state, then the guide on optimizing elsewhere is even more necessary. This is the last piece of the story that is missing for me personally, and it causes me to hesitate before deploying more code. Also, if you have this written already and I've missed it, please let me know!

Overall, you've lowered the barriers to entry and increased efficiency so much I plan on deploying much more server side code than ever before.

Thank you,

Austen

joeyspn4• 11 years ago

Hi Jeff, thanks for responding to the feedback. Personally I’m just concerned about two things:

- Execution response time: the responses to API > Lambda > DynamoDB are too slow

- Reference Samples: we need docs and/or code samples with the most common patterns and workflows for mBaaS and IoT (i.e: user authentication with JWT and ACLs via the API > Lambda > Dynamo)

Many of us are stoked about this product. Hope you can iron out the issues that the community is raising and that are preventing wider adoption. The concept of server-less microservices via REST looks like the next big trend after containers. Hope you get it just right soon... I plan to make intensive use of this service in no time! =)

Cheers

ac3605• 11 years ago

Out of curiosity, are you getting slow response times only on the first request, or on all?

I'm also using API Gateway + Lambda + DynamoDB

All of them are very slow on first request, even the DynamoDB queries. After that, they're much quicker, unless there are maybe 5 mins of inactivity.

cargomaster• 11 years ago

The other thing that is terrible is versioning. Lambda does not support versioning. And the resources/methods that you define for your various stages all share the same resources/methods, unless you want to change the signature of your calls. So if I want to change an existing Lambda function, then the minute I change it for my staging/test/qa environment, I have changed it for production as well. So, I can't roll back easily. I can script all this, but ouch. The alternative is to create a whole new API gateway, but that means that once I get my stuff tested, I essentially have to recreate my new environment in production, insteading of just clicking a button.

We have a client who is looking at sending a massive amount of data to a REST API. Looks like we are going back to Elastic Beanstalk, which is pretty good, it just has servers. But it works.

And, I'd say that using Dyanamo as a DB locks you much more securely into AWS than API Gateway and Lambda. Be pretty easy to take those Lambda functions and map them back into Express or Restify routes. Changing the database, arg.

jamesxab• 11 years ago

I'd come across the same issues you are experiencing and had been looking for a solution to the $input.path() issue.

Turns out to retrieve JSON you can use $input.json().

Full details are on the AWS documentation page. Not sure if this has been newly added as I only found it today.

http://docs.aws.amazon.com/apigateway/latest/developerguide/...

Day One with the Amazon API Gateway

Related Stories