abd12

I always tell people there are two clear areas where DynamoDB has some major benefits:

- Very high scale applications that can be tough for an RDBMS to handle

- Serverless applications (e.g. w/ AWS Lambda) due to how the connection model (and other factors) work better with that model.

Then, for about 80% of OLTP applications, you can choose either DynamoDB or RDBMS, and it really comes down to which tradeoffs you prefer.

DynamoDB will give you consistent, predictable performance basically forever, and there's not the long-term maintenance drag of tuning your database as your usage grows. The downside, as others have mentioned, is more planning upfront and some loss of flexibility.

One difference with DynamoDB is that there's no query planner, so you can have a pretty good sense of how many items you'll hit and how big that read is.

Nice post!

Question for you -- what are the performance implications of "re-keying" my records during the transform? Should I try to keep the same key for my transformed record as from my original record so that they align with the same partitions, or are they likely going to get sent to different partitions across different brokers anyway?

Wow! Congrats to you, Jay and Frank. I've been a fan of your work on both Seed.run & Serverless Stack for a while. Best of luck, and I'm excited to see Seed grow :)

I highly recommend this book and Swyx's other work. He's thorough while also distilling down to the most important bits.

Great book for people that are just starting their coding career but also for those with a good bit of experience. If you're feeling 'stuck', this is a great guide to understand how to advance your career.

'Learn in public' is probably the #1 piece of advice I'd give to anyone in tech. It helps your writing, it builds your network, and it grinds down your ego (because people will certainly let you know when you're wrong). Swyx has been a huge proponent of this, and this whole book is a great kick in the pants to get started.

I'd be shocked if this book doesn't make you >>>10x the amount you spend on it (even counting a healthy hourly rate for reading it).

Yep! There are entire chapters on sorting & filtering. Note: it's different than in a relational database, but it's doable :)

Yep! It comes with PDF, MOBI, and EPUB formats :)

It depends!

For me, I like that 98% of DynamoDB work is frontloaded. I spend the time building the model but once it's done -- set it and forget it.

With RDBMS, it's like there's a hidden 5% tax that's lurking at all times. You have to spend time tuning querying, reshaping data, changing patterns, etc. It can add up to significant drag over time.

Different teams might think the costs are different for their application, or they may be fine with one pattern over the other. Fine with me! I just know which one I choose now :)

Thank you! I really appreciate it :) Hit me up if you have any questions!

Haha, I love that story at the end. I promise not to tell your company that it is a database.

luhn responded to this one pretty well :)

Basically, most of these issues are gone. As long as you don't have extreme skew in your partition keys, you don't need to worry about throughput limits.

I recommend On-Demand pricing 'until it hurts'[0], but that's because a ton of people I talk to are spending <$50/month on DynamoDB. At that point, it really doesn't make sense to spend hours of time optimizing your DynamoDB bill.

If you are at the point where you are spending over thousands of dollars a month on DynamoDB, then it does make sense to review your usage, fine-tune your capacity, set up auto-scaling, buy reserved capacity, etc. But don't waste your time doing that to save $14 a month. There are better things to do.

But it's really nice to have a database where you can set up pay-per-use, don't have to think about exhausting your resources, and have an option to back out into a cheaper billing mode if it does get expensive.

[0] - Hat tip to Jared Short for this advice & phrase

Daniel, I'm a big fan of yours but disagree with this take :).

It's definitely a database. The modeling principles are different, and you won't get some of the niceties you get with a RDBMS, but it still allows for flexible querying and more.

S3 is not a database, but DynamoDB is :).

All the examples are specific to DynamoDB and use DynamoDB features.

That said, the principles apply pretty well to other popular NoSQL databases, especially MongoDB and Cassandra. There will be some slight differences -- MongoDB allows better nesting and querying on nested objects -- but it's broadly the same. If you want to model NoSQL for scale, you need to use these general patterns.

If you want to check it out but find out it doesn't work for you, just let me know. I've got a 100% money-back guarantee with no questions asked if you don't like it.

True! I'm not a huge fan of Aurora Serverless and the Data API. The scaling for Aurora Serverless is slow enough that it's not really serverless, IMO. And the Data API adds a good bit of latency and has a non-standard request & response format, so it's hard to use with existing libraries. But it's definitely an option for those that want Lambda + RDBMS.

The RDS Proxy is _hopefully_ a better option in this regard but still early.

Good point! I would usually not recommend using a database from a different cloud provider just because of different hassles around permissions, connections, etc.

I've never found the speed an issue, but YMMV. To me, the best thing is that you won't see speed degradation as you scale. With a relational database, your joins will get slower and slower as the size of your database grows. With DynamoDB, it's basically the same at 1GB as it is at 10TB.

Awesome! Hit me up if you have any questions :)

I can understand the sentiment and don't fault you for it.

That said, I think you can definitely handle complex, relational patterns in DynamoDB pretty easily. It will take some work to learn new modeling patterns, but it's absolutely doable.

Thank you for your support!

The biggest advice I can give you is not about any specific tool, it's about an approach. You need to think about how you will market the book if you're self-publishing.

Engage with the community that will be interested in the book. Write articles, help out on Twitter, write code libraries, etc.

For me, I wrote DynamoDBGuide.com two and a half years ago over Christmas break. I wanted to just make an easier introduction to DynamoDB after I watched Rick Houlihan's talk at re:Invent (which is awesome).

That led to other opportunities and to me being seen as an 'expert' (even when I wasn't!). I got more questions and spent more time on DynamoDB to the point where I started to know more. I gave a few talks, etc.

I finally decided to do a book and set up a landing page and mailing list. I basically followed the playbook that Adam Wathan described for his first book launch.[0] Write in public, release sample chapters, engage with people, etc.

In terms of tooling, I used AsciiDoc to generate the book and Gumroad to sell. On a 1-10 scale, I'd give AsciiDoc a 5 and Gumroad an 8. But the tooling barely matters -- think about how to find the people that are interested :)

Happy to answer any other questions, either in public or via email.

[0] https://adamwathan.me/the-book-launch-that-let-me-quit-my-jo...

Thanks for your support! I'm grateful for the fix you suggested as well :)

I'd much rather pay for reads & writes directly rather than guessing at how my CPU and RAM will translate to the reads and writes that I need.

RDBMS capacity planning basically goes:

1. How much traffic will I get? 2. How much RAM & CPU will I need to handle the traffic from (1).

With DynamoDB, you can skip the second question.

In a nutshell:

- It was designed for super high scale use cases (think Amazon.com retail on Cyber Monday). It has decent adoption there. Competes mostly with Cassandra or other similar tools.

- With the introduction of AWS Lambda, it got more adoption in the 'serverless' ecosystem because of how well its connection model, provisioning model, and billing model works with Lambda. RDBMS doesn't work as well here.

A lot of people find 'problems' with it because they try to use it like a relational database, which it most certainly isn't. You have to model differently and think about it differently. The book helps here :).

Thank you for the kind words! :) Glad you're liking it.

My contention is that it's much easier to have an access pattern that won't scale in a relational database than in DynamoDB. DynamoDB basically removes all the things that can prevent you from scaling (JOINs, large aggregations, unbounded queries, fuzzy-search).

This is underrated, but it's really helpful. So many times w/ a relational database, I've had to tweak queries or access patterns over time as response times degrade. DynamoDB basically doesn't have that unless you really screw something up.

Fair enough! I think that's a reasonable position.

IMO, there are two times you should absolutely default to DynamoDB:

- Very high scale workloads, due to its scaling characteristics

- Workloads w/ serverless compute (aka Lambda) due to how well it fits with the connection model, provisioning model, etc.

You can use DynamoDB for almost all OLTP workloads, but outside of those two categories, I won't fault you for choosing an RDBMS.

Agree that DynamoDB isn't _blazing_ fast. It's more that it's extremely consistent. You're going to get ~10 millisecond response times when you have 1GB of data or when you have 10 TB of data, and that's pretty attractive.

Thanks, Matthew! Appreciate it, and I agree with you :)

Email me, and I'm happy to discuss :). alex@alexdebrie.com

Fair enough! IMO, it's worth it :). You could spend a bunch of time cobbling together free resources, and you'd still only get about 30% of what's in the book. How much is your time worth as a software engineer?

That said, a few notes:

1. I added a coupon code ('HACKERNEWS') to knock $20 off Basic, $30 off Plus, and $50 off Premium.

2. If you're from a country where PPP makes this pretty expensive, hit me up. I'm happy to help.

3. If you're facing income challenges due to COVID-19, hit me up, I'm happy to help.

4. If this is unaffordable for any reason, hit me up, I'm happy to help. :)

Waves Author here. Happy to answer any questions folks have about the book, about DynamoDB, or about self-publishing.

NoSQL modeling is waaay different than relational modeling. I think a lot of NoSQL advice out there is pretty bad, which results in people dismissing the technology altogether. I've been working with DynamoDB for a few years now, and there's no way I'll go back.

The book has been available for about a month now, and I've been pretty happy with the reception. Strong support from Rick Houlihan (AWS DynamoDB wizard) and a lot of other folks at AWS.

You can get a free preview by signing up at the landing page. If you buy and don't like it, there's a full money-back guarantee with no questions asked. Also, if you're having income problems due to COVID, hit me up and we'll make something work :)

Anyhow, hit me up with questions!

EDIT: Added a coupon code for folks hearing about the book here. Use the code "HACKERNEWS" to save $20 on Basic, $30 on Plus, or $50 on Premium. :)

One note on this -- if you have an LSI, you can't have an item collection larger than 10GB, where an item collection refers to all the items with the same partition key in your main table and your LSI.

A DynamoDB table with an LSI can scale far beyond 10GB. That said, I would avoid LSIs in almost all circumstances. Just go with a GSI.