nachivpn

HTML of a page is received in response to a HTTP request. A page load might involve multiple HTTP requests. Often parts of a web page that interest you might be loaded specifically by a request. By using firebug, you could examine which HTTP request fetched you the content you need and what parameters (headers, cookies, etc) where associated with it. Using this information, you could simulate a HTTP request of your own using cURL (in fact, firebug provides a "copy as cURL command" feature which can be very useful). Firebug is used only in the initial stages to understand. Post which, cURL can be used in your shell script (or a suitable language of choice) to implement your bot.

Interestingly I just came across this - http://torrentfreak.com/google-asked-remove-345-million-pira...

Yes, it is very true that many fail to realize that they are getting scraped simply because there aren't many tools which show the traffic classified among humans and bots. This surely is a problem. Thanks for leaving a comment!

Yes, it definitely isn't the right way to handle ambiguous grammars. My assumption was caused by extensive usage of YACC for writing parsers in the past. To avoid it's default action, generally ambiguous grammars are avoided (since it is used only to write programming language parsers anyway)

Like in my other comment, I agree with this. I missed mentioning mentioning my assumption of a forceful shift or reduce action in case of an ambiguous grammar.

I totally agree with this. I was assuming the case of a default behavior like shift or reduce force fully for unambiguous grammars, which I should have mentioned. Thanks.

Bottom up parsing - "If not, it's necessary to backtrack and try combining tokens in different ways" I feel the way it is put along with shift reduce parsing is misleading. Backtracking is essentially an aspect avoided (more like solved) by shift-reduce parsing. They don't go together in bottom up parsing. Shift reduce parsers posses the potential to predict the handle to use by looking at the contents on top of the stack.

Good job BTW, there are very few people who write about compiler/language theory :)

We moved from Azure to Gcloud. We are currently using Gcloud's compute engine as we didn't want to get too dependent on the App engine. In comparison to Azure, gcloud VMs are much faster in terms of performance. Managing google VMs are much more easier. For example, migrating a gcloud VM would take only a few seconds even across data centers. Google's support team in India didn't really help, so we designed our own architecture. While google has many interesting features like autoscaling API, global load balancer, cloud storage etc, of which most of the services are in beta stage. We use it anyway, and haven't faced too many issues.

The only place where we had to look for an alternative cloud service was when we attempted to set up our own CDN on https. Google currently supports it's global load balancer only for port 80 and 8080 and they suggested we use DNS load balancer which we didn't like the idea of. P.s. we have not worked a lot with AWS. So I do not know much about AWS. From what I hear, AWS is currently much better and gcloud is good upcoming competitor. Overall, we serve 20 million API requests per day on google cloud without any infrstructure issues so far.