Flask-SQLAlchemy Caching

http://www.debrice.com/flask-sqlalchemy-caching/

by debrice • 12 years ago

1 2 12 years ago

I think people often too quickly turn to caching when their problem is in the ORM/db.

I generally like having models, but I'm not a fan of ORMs (ie, OM without the R).

Where this often gets you is lazily loading attributes in loops -- it kills performance and it's non-obvious that it's happening when just looking at code. Caching helps this problem by either putting the related attribute in-process memory or in memcached which is relatively faster than hitting the database. But, to me, this seems like an anti-pattern -- caching instead of optimizing the underlying ORM/db access patterns or preferring a style in which database accesses become apparent in application logic instead of being masked by the ORM.

That being said, I think this is a very good article showing how to do caching when you really need it - I'm just trying to say that you probably don't for generic database accesses.

And if you're going to do it, I'd always just start out with redis/memcached to begin with - even on a single box, you're probably going to want a process per core, and having every process keep a copy gets expensive fast.

debrice• 12 years ago

I agree, with the exception in the case of a single web process, I wouldn't use memcached but just rely on dogpile memory backend, which is about the same (unless I'm proven wrong :p). I'd use memcached otherwise to share the benefit of caching poll. Caching in redis, I'm not fan, since redis will write to your drive at one point (a large cache would impact your drive space and slow down redis start dramatically).

I share your opinion on lazy loading, whenever possible you should know in advance if you're gonna need that related information before you pull your data: it's much faster to chain 2 select in one call (even with a connection pool).

That being said, I love redis and wish I could use it for all my objects (I could, since all my objects have a pk) but it lacks some fondamental real world features (relationships and transactions) that make business query/analytic a nightmare (who connect to your website, at what rate...) while Relational database does an amazing job at it.

Flask-SQLAlchemy Caching

Related Stories