HNGlance

Top Best New Show

Performance optimizations for storing web server access logs in ClickHouse

https://clickhouse.com/blog/log-compression-170x

by krizhanovsky • 9 months ago

2 1 9 months ago

CL

Related Stories

Loading related stories...

Comments

1

krizhanovsky• 9 months ago

It's useful to store a web server access logs in an analytics database, e.g. to fight against bot attacks. We store structured access logs in Clickhouse, which is already good, but compression and data ordering from the post may improve performance even more - we'll try this.

The thing is that a web server, especially under DDoS, may produce much more records than Clickhouse can ingest. But there is good news: for Nginx, if you build a fast pipeline to feed access logs to Clickhouse, you can increase performance, I'd say up to x2, thanks to faster access logging.

Source preview

clickhouse.com

HNGlance

© 2024-2026 HN Glance. Story content belongs to its original sources.

Privacy
Terms