HNGlance

Top Best New Show

Using Proxies in Web Scraping

https://stackabuse.com/using-proxies-in-web-scraping-all-you-need-to-know/

by ScottWRobinson • 2 years ago

2 2 2 years ago

ST

Related Stories

Loading related stories...

Comments

2

zero-sharp• 2 years ago

The issue that I struggled with is browser fingerprinting. Servers can detect if you're using a browser. If you are, then there are a number of flags, or parameters, in the browser that give away your identity. You can cycle through user agents and proxies. I don't think these things help. If you're not loading the page through a browser or if you try to hide information, you're already flagged. Scraping is a cat and mouse game now.

ScottWRobinson• 2 years ago

IIRC there are ways to give an authentic-looking fingerprint to browsers used for scraping. But as you said, it's a cat and mouse game

Source preview

stackabuse.com

HNGlance

© 2024-2026 HN Glance. Story content belongs to its original sources.

Privacy
Terms