Ask HN: What set up do you use to scrape web pages

I am writing code to get URL data and realized because of JS using a regular requests module in python doesn't cut it. I need to actually render the page in a browser engine and get the page from there. Googling did not bring up a fixed/common way to do it. I was wondering this must be something a good number of start ups should be doing, so wanted to pick some brains. Thanks for any suggestions.