Intro to Node on AWS Lambda for S3 and Kinesis

http://eng.localytics.com/taming-aws-lambda-for-s3-and-kinesis-at-localytics/

by nicksergeant • 11 years ago

95 25 11 years ago

eng.localytics.com

icpmacdo3• 11 years ago

Is there a way to automatically kill Lambda if the cost is going out of control because of an infinite loop? As a solo dev I cant be willing to bet hundreds or thousands of dollars potentially lost because of a bad piece of code I wrote. If thats possible it seems like a nice service that will save a lot of time not having to worry about doing server administration.

My ask HN post from earlier today basically asking the same thing https://news.ycombinator.com/item?id=9520964

nicksergeant5• 11 years ago

As far as I know, there's no way you can automatically kill it. Of course, you can setup billing / execution alerts, which would probably be very helpful in at least notifying you that there's a problem (I personally have an alert set up to notify me if my estimated charges are anything out of the ordinary).

A small anecdote: Lambda processes are _very_ cheap most of the time. When I was writing the initial version of this starter module, I accidentally threw the lambda into an infinite loop. It was triggered about 30k times within a minute or so and ended up costing around 13 cents if I remember correctly.

icpmacdo3• 11 years ago

So it seems like Lambda is too risky of an option because of the potential for catastrophic failure. It would probably make more sense to try and work with something like Elastic Beanstalk or a lone EC2 instance Im trying to avoid a large amount of time being taken up by doing server administration. As I mentioned in the other post a new hobby tier Heroku Dyno and DB at around 19$ a month is possible within my budget if they make launching a node backend for my app significantly easier. I know this is getting off topic from the OP but any guidance from HN would be appreciated.

kondro• 11 years ago

Lambda functions can execute for a maximum of 60 seconds.

nicksergeant5• 11 years ago

But, if you have more than one Lambda working on a single S3 bucket, for example, it's possible to trigger executions infinitely.

Edit: my mistake - you still can't have multiple lambdas operating on a single S3 bucket. I had triggered an infinite loop by manipulating files in the same bucket that it was triggered on (triggering another execution, infinitely).

dorfsmay• 11 years ago

Huge drawback... I wanted to use AWS lambda for some long running task and run into that limitation.

robbles2• 11 years ago

Since the Lambda functions are written in Node.js, you can probably take advantage of the way exceptions are handled and give your code a default timeout:

  setTimeout(function() { throw new Error('timeout'); }, 1000)

raingrove• 11 years ago

No. Javascript is single threaded. So if the code is stuck in a synchronous loop, this won't help. (Of course there are ways to make the loop yield and not be completely synchronous.)

cddotdotslash• 11 years ago

I really wish AWS would open up Lambda to more events than just S3 uploads and SQS / Dynamo queues. There are so many workloads I could move to Lambda if I could just invoke it via an API call.

cherioo• 11 years ago

You mean this? http://docs.aws.amazon.com/lambda/latest/dg/walkthrough-cust...

nicksergeant5• 11 years ago

You could just write some code to manually invoke the lambda with specific data using the SDK, no?

kennu• 11 years ago

I wish Lambda could listen to arbitrary TCP ports and respond e.g. to HTTP requests or MQTT messages. In the current form it seems pretty limited for any scenarios where generic clients initiate the requests, such as collecting device data and metrics. As I understand, currently the clients have to speak AWS-specific protocols to submit events / data.

justincormack• 11 years ago

The impression I get is that it is not really realtime, I am guessing all requests get queued and then scheduled on to a suitable host soon, but perhaps not within the time to respond to a TCP request.

(Although custom events http://docs.aws.amazon.com/lambda/latest/dg/walkthrough-cust... suggest its pretty close to real time; should measure the latency).

Intro to Node on AWS Lambda for S3 and Kinesis

Related Stories