Adaptive rate limiting is a game-changer

https://blog.fluxninja.com/blog/why-adaptive-rate-limiting-is-a-game-changer

by KaranSohi • 3 years ago

48 44 3 years ago

blog.fluxninja.com

TekMol4• 3 years ago

I don't believe that Twitter has to limit requests to avoid bots.

They would just have to limit requests by IP and thats it. Say 10 tweets per IP per day. For requests which do not come from logged in users.

I can't see how scrapers would be able to come up with enough IPs to make a dent in Twitters costs.

malfist• 3 years ago

IPv6 makes it trivial to come up with a mind boggling number of ip addresses. You don't even have to leave your network

jsnell• 3 years ago

Nobody does rate limiting or other abuse work on a full /128. Typically a /64 is treated as the equivalent of an IPv4 address.

TekMol4• 3 years ago

First of all, Twitter does not support IPv6.

Secondly, if they would, they could simply rate limit by the first x chars of the address.

berkle4455• 3 years ago

So your solution is to just make it a walled garden. What about the scraping bots who simply log-in?

TekMol4• 3 years ago

My solution is to make it less of a walled garden. At the moment, they often refuse even reading a single tweet when not logged in.

And for the "simply log-in": It is even more expensive (way more expensive) to get good email adresses as it is to get good IPs. Email providers have very tough checks these days before they give you one.

garciasn• 3 years ago

It has to be behavioral, not IP based for the most part because of the ways these networks operate—primarily on personal computers/routers on US-based residential connections.

I deal with these constantly for our clients. It’s amazing how smart they function to get around limits (thousands of IPs and valid recent user agents) but how they miss easy things like using the same cookie to submit 1000s of different entries with obvious purchased lists of names/emails, many of whom are deceased.

TekMol4• 3 years ago

"Thousands of IPs" can not hurt Twitters bottom line in any way. With a "read 10 tweets per IP per day" limit, that is 10k tweets. Delivering 10k tweets does not even cost a penny.

38• 3 years ago

I dont know what the hell Twitter is doing, but I simply dont use it anymore. I cannot access this page without logging in:

https://twitter.com/SarahKSilverman

I check it ONCE per day, thats it. and for weeks every time it just prompts me to login. no thanks, done with Twitter.

pabs3• 3 years ago

You can use nitter to access Twitter content anonymously at the moment:

https://nitter.net/SarahKSilverman

jokoon2• 3 years ago

No surprise that it's not already implemented because we just throw more cheap hosting and bandwidth at this problem until it's fixed.

The "premature optimization is the root of all evil" misquote really caused a lot of damage to favor growth and profit making instead of better software. It seems programmers have a general habit of just implementing things in the most naive and cheap way and just solve any problem with expensive hardware.

We really need a new trend of fast and minimal software because climate change and a finite planet will not allow us to follow Wirth's law for very long.

andrewstuart2• 3 years ago

The misquote, in my experience, is abusing the "premature" aspect. It's often thrown around as time-based but my understanding of the original quote is that it's all about assumptions rather than data driving complexity in the name of optimization.

Optimization should have measurable improvement and the need for it should be demonstrable. Neither of these is harder than writing a benchmark and/or performing some simple math on data measured for that purpose before going about the optimization.

mcpackieh• 3 years ago

The quotes from wikiquote:

"The real problem is that programmers have spent far too much time worrying about efficiency in the wrong places and at the wrong times; premature optimization is the root of all evil (or at least most of it) in programming."

and another instance of the same:

"Programmers waste enormous amounts of time thinking about, or worrying about, the speed of noncritical parts of their programs, and these attempts at efficiency actually have a strong negative impact when debugging and maintenance are considered. We should forget about small efficiencies, say about 97% of the time: premature optimization is the root of all evil. Yet we should not pass up our opportunities in that critical 3%."

https://en.wikiquote.org/wiki/Donald_Knuth#Computer_Programm...

I'm not really seeing much of a "misquote" to be honest. I have always understood this quote to mean that spending too much time micro-optimizing things unnecessarily will waste your time and become a burden to you later. Instead, optimizations should only be performed when the need for optimization becomes manifest.

BodyCulture• 3 years ago

K8S idling will burn much more CPU than any plain OS, still masses of engineers are following the cult of “burn planet, burn“ like in a zombie distopia. Is this a collective brain failure? Bandy Lee should look into destructive tech trends.

jokoon2• 3 years ago

I have never really made extensive use of either docker or k8s, but those things always made me feel weird and I always saw those as some kind of shotgun solution to nightmare-ish software who are so complex, badly written and designed that they requires some kind of hazmat suit to work properly.

ComputerGuru• 3 years ago

It was really quite the let down of an article. All that build up, then basically nothing about how adaptive rate limiting would be implemented in practice, how expensive it would be, what it requires, the challenges it involves, etc.

nisa• 3 years ago

in theory (and also probably in practice) it's a control theory problem and there are already algorithms and ideas out there - here is an article not exactly about rate limiting but implementing something based on this might work really well: https://www.eecs.umich.edu/courses/eecs571/reading/control-t...

gillh3• 3 years ago

Aperture project's policy language is indeed inspired by Control Systems, especially PID Control.

See: https://docs.fluxninja.com/concepts/advanced/circuit

ryantgtg• 3 years ago

This article was crossposted to hell and back last week on reddit. And it has an AI vibe…

gillh3• 3 years ago

Thanks for the feedback - This blog could have been better and less aggressively distributed. At the same time, we should consider the credible body of work behind this. It's not AI-generated spam but backed by actual code that provides a reference implementation behind the idea.

See: https://github.com/fluxninja/aperture

gillh3• 3 years ago

It's a blog on an open-source project that precisely tells you how to implement adaptive rate limiting.

Just click around a bit:

- https://github.com/fluxninja/aperture

- https://docs.fluxninja.com/use-cases/adaptive-service-protec...

Note: I am one of the authors' of this project.

Sudhanshu4562• 3 years ago

Thanks for the feedback. I agree I could've gone into more technical depth and implementation details. So, taking your feedback in account, I've modified and added more details in the blog. But, I already have a next blog in planning covering the topic in-depth, along the similar lines of your feedback, because of that I didn't intend to make a long one, decided to split the blog :)

Dwedit• 3 years ago

Article is broken if you have JS from cloudfront denied.

dadie• 3 years ago

I do not know why, but by default the body element has an overflow of hidden set via css. There is probably a good reason to do this that I do not know about. Would love to know the reason behind that from a WebDev as it strikes me as unnecessary having seen the page with JS and without JS and not being able to see much of a difference.

jazzyjackson• 3 years ago

Might be to avoid a visual reflow - otherwise the content will render, then the webfont will load and the whole page will reflow and jump around.

Google amp used to have a ridiculous 4 second timeout waiting for the font

thih9• 3 years ago

I feel this has potential to bring new levels of painful ux.

They might prioritize serving the VIPs and slow down the pace when there is an influx of newcomers.

Or they might prioritize people who are about to convert and deprioritize them once they pay.

mgaunard• 3 years ago

As a user, I paid for a bandwidth and I want that bandwidth.

The rest is the service provider's problem.

michaelcampbell2• 3 years ago

Not sure it "changed my game" but I have a process that I run on many files at once. In a *nix I would run the processes like so:

    for i in list_of_files; do
      run_intensive_process &
      sleep $(awk '{print $3}' < /proc/loadavg)
    done

As more processes start upping my load average, it sleeps longer before running each one in the background. I guess GNU parallel can do this sort of thing with a load-sensitive command line arg, but I find this pattern easy to remember, and parallel's decidedly not.

Works for me.

twic3• 3 years ago

You could also hand all the invocations of run_intensive_process to at, and let it take care of running them when the load average permits:

https://linux.die.net/man/1/at

I don't think at is very heavily used on modern systems, so it's quite possible that it doesn't actually have very useful behaviour here.

dredmorbius2• 3 years ago

More specifically, batch(1).

at(1) runs jobs after a specified time. (A common mistake is to presume that "at" means "will run when specified" when in fact it means "will run no earlier than specified", and depending on system characteristics, could run substantially later.)

batch runs jobs ... as system load permits, without any time specification, in a FIFO queue.

There's also cron, the third Unix scheduler, which executes a set of specified tasks on a regular schedule. All three are related.

caro11ne• 3 years ago

Stop starting more jobs if load > number of cpus:

    cat list_of_file | parallel --load 100% run_intensive_process

mjb• 3 years ago

The biggest problem with adaptive rate limiting is that, in many contexts, clients value predictability more than they do peak performance. Adaptive rate limits change client behavior in ways that clients can't predict or control.

Even in networking, the classic application domain of adaptive rate limits (like TCP's algorithms), core networks are built to offer predictable performance. Edge networks, like public WiFi, that offers best-effort performance with adaptive rate limits, are often met with frustration and even anger the moment those limits kick in.

twic3• 3 years ago

Easy enough to implement on top of a token bucket filter - just adjust the rate at which tokens are added.

Adaptive rate limiting is a game-changer

Related Stories