Running Fast SQL on DynamoDB Tables

https://rockset.com/blog/running-fast-sql-on-dynamodb-tables/

by vahidfazelrezai • 8 years ago

40 10 8 years ago

rockset.com

coleifer• 8 years ago

Running (expensive) SQL on dynamodb tables.

threeseed• 8 years ago

Need to look at TCO.

Managing a database can be pretty expensive and time consuming.

luhn• 8 years ago

AFAICT, this actually isn't running SQL against data in DynamoDB. Rather, it syncs the the data in DynamoDB with their [Rockset's] own proprietary database, and runs SQL against that data.

It's effectively the same but technically quite different.

dkobia• 8 years ago

Run SQL queries using Athena and a Dynamodb crawler in Glue. It's not that complicated.

wh-uws• 8 years ago

Link to a tutorial that is as simple with code examples / screenshots as this one?

I ask because I have looked and haven't found one that I could follow without having to dedicate several hours to it.

To the point where I want to move off of dynamodb to aurora because I can't believe how hard it is to get simple insights like how many rows exist out of dynamo (I inherited the dynamo choice would not have used it myself)

iamsomewalrus• 8 years ago

I’ll have a blog post out in AWS Big Data in a few weeks but the parent is right in that it’s not terribly difficult.

Should take about an hour with testing to get a Pyspark script together to read in a DynamoDB table and write it out to S3.

https://docs.aws.amazon.com/glue/latest/dg/aws-glue-programm...

You’ll then need to crawl the S3 data to add it to your Glue catalog and then you can query it with Athena.

teej• 8 years ago

How are you getting data from DynamoDB to S3 so Athena can read it?

vonholstein• 8 years ago

ETL job in AWS glue with DynamoDB as source and S3 as sink is one option.

https://aws.amazon.com/about-aws/whats-new/2018/07/aws-glue-...

ralusek• 7 years ago

You could have a lambda hook to trigger on dynamo write, and copy the dynamo change to s3 in the lambda.

Running Fast SQL on DynamoDB Tables

Related Stories