HN user

samcolvin

24 karma
Posts0
Comments3
View on HN
No posts found.
[GET] "/api/user/samcolvin/stories?hitsPerPage=30&page=0": 500 Failed to fetch user stories

Victor Stinner (Python core developer focused on performance) has a great speech about this question:

There are a lot of reasons actually

- Performance limited by old CPython design. If you fork it you have to deal with all the legacy code.

- CPython is limited to 1 thread because of the GIL.

- Specific memory allocators, C structures, reference counting, specific garbage collector etc.

You can find that video in here: https://youtu.be/TXRPCZ7Nmh4

I can confirm that, scraping Google instantly needs huge effort and money. In our best we can scrape 2500 SERP per IP.

But i must say using proxy services and other things did not helped us much. Because most of them were banned before we use.