geo_mer

Rendora uses internally cdp https://github.com/mafredri/cdp, a golang client implementation for "Chrome Devtools Protocols" which works on top of websockets, when Rendora detects a witelisted request, it instructs the headless Chrome instance using cdp to render the corresponding page and waits for the DOM load event, after that the content of the page is copied to rendora and then returned back to the client while preserving HTTP headers and status code.

It's all about using the chrome-devtools-protocol, if you are interested, you may want to read about it in https://chromedevtools.github.io/devtools-protocol/1-2

If the “initial render” of client side code involves JavaScript mounting nodes on the DOM

That's the raison d'être of rendora, if your website doesn't add DOM nodes using some javascript framework (e.g. React, Vue, etc..)in client, basically the HTML will be exactly the same and thus you don't need rendora in that case

google analytics is the only 3rd party script I am okay with. Maybe you should add it sometime in the future to get a rough idea about your users, I was talking mainly about the more intrusive analytics and ads scripts that plagued the web. It's rare to see anybody who respect his users to that extent these days :D respect again!

This is actually a very neat and useful website. Also respect for making the website fast and not adding any 3rd party scripts.

yes exactly :), as if crawlers see exactly the same HTML of what browsers see after DOM load not the initial incomplete HTML sent by servers

and return less assets (no CSS, fonts etc)

no, it returns the whole page, it's just equivalent to rendering pages on your browser, everything in the initial HTML is the same in addition to HTML generated by javascript frameworks added to the DOM after the webapp scripts are loaded.

What's the disadvantage or advantage of headless chrome over chrome with head intact?

I am not aware of any. You just don't use headless Chrome for typical browsing :D

Is headless chrome faster?

I guess it slightly is, because it doesn't need to paint the rendered page but tbh Chrome/Chromium is an enormous project so I can't comment in details.

Can we control popup windows in headless one? Does it support proxies?

I believe they are not possible because chrome-devtools-protocol doesn't support that yet, but you can add http/socks proxies inside Chrome and then run it as headless to use a proxy if you insist

Is it possible to control tabs? yes

if you're that interested about headless Chrome, I recommend you read about chrome devtools protocol https://chromedevtools.github.io/devtools-protocol/1-2

you can add your user agent to whitlested user agents or (less recommended) you can even connect to the headless chrome instance (usually using http://localhost:9222 but in that case I remind you that some assets are blocked from being fetched by default such as images, fonts and CSS to speed up the DOM load. This doesn't affect the resulting HTML.)

* prerender.io is a paid service; Rendora is FOSS and self hosted

* prerender.io needs additional code in your backend to filter requests and asks for SSR from a remote server then returns back the resulting HTML; Rendora doesn't because it does all that automatically by being a lightweight reverse HTTP proxy server in front of your backend

* with Rendora you can control caching (using an internal local store or Redis), prerender.io claims caching on their servers I guess but then you can't control it espcially if you have fast chaing pages

* Rendora provides Prometheus metrics to measure your SSR latencies, count of cached and uncached requests and an API rendering endpoint

* Rendora supports asynchronous pages where content is loaded after the initial DOM load

* you can simply choose which Chrome version you want to use with Rendora

* Rendora instructs headless Chrome to skip fetching unncessary assets (e.g. fonts, CSS, images, etc...) which makes the DOM load much faster then when loaded and rendered by default

No, both receive HTML, the real problem is that websites built with modern javascript frameworks (e.g. react, vue, angular, etc...) render most of the content using the javascript engine in browsers, search engines don't execute javascript and thus they see almost nothing but a header and almost an empty body (of course it depends on your use case to be more accurate). Rendora solves the SEO problem for such websites, by being a lightweight reverse HTTP proxy in front of your backend server, it detects crawlers by checking whitelisted user agents and paths, if it finds one, it instructs a headless Chrome instance to request and render the corresponding page and then returns the final server-side rendered HTML back to the crawler (or according to whatever whitelisted in your config file). This is called dynamic rendering and has been recommended by Google and Bing very recently (see the links in https://github.com/rendora/rendora#what-is-dynamic-rendering)

Thank you for the kind response

was wondering if there is any other existing product in the market

yes, there is rendertron; here is the comparison between rendora and rendertron and why I think rendora is better https://github.com/rendora/rendora#what-is-the-difference-be...

also there is prerender.io but this is a commercial paid product and also needs change in your backend; and I guess they didn't use headless Chrome until recently

Wish the crawlers incorporate something like this into their crawling logic so that we really don't need to worry about server rendering anymore

Except for Google, probably all other search engines don't even execute javascript to render pages (Bing claimed to be rendering js very recently but I haven't seen it in my production website :D), they claim that the web is too big and the computational complexity of executing javascript in every page is unreasonable. Even Google can't render pages correctly if you have asynchronous content from my own experience.

Thank you! I actually developed it because I have developed a somewhat complex website using vue.js and it was too complex/too late for my architecture to do the native Vue SSR using Node.js so I built rendora instead. I have some really complex pages that are rendered below 150ms (uncached SSR requests). I also tested it on some simple React pages and got as low as 70ms! so I now guess headless Chrome isn't as slow as you might think. Also some other optimizations are made to help lower latencies (skip fetching&rendering CSS, images, fonts, etc..) so that the initial DOM load can be faster.

​

I also added Prometheus metrics and a rendering API endpoint so that you can get an SSR latency histograms with buckets 50ms, 100ms, ..., 500ms and see the avg latencies for your use case.

​

TLDR; headless Chrome is actually faster than you might initially think. It might be 200-300ms at maximum to get the initial DOM load assuming that you have complex enough pages.

Hello HN, I've been developing and testing rendora for some time now and decided to put it publicly today on github, I would love to take feedbacks or answer any question!

You are a unique innovator Mike, I am not really into visualization but I am a big fan of yours.

Is QUIC only built for HTTP or can it be also used as a secure infrastructure for any L7 protocol?

Native async/await support is really great. I haven't gotten very far with Rust but it already made me rethink many things that I thought I had understood.

Kubernetes may be overkill for small projects and it's actually hard to setup for a single-machine cluster, but the idea of container orchestrators (k8s, docker swarm, nomad, etc...) is extremely useful, I understand that some abuse the word "scale", but for me container orchestration is far bigger than just scaling, these features include:

1. rolling updates

2. decoupling configs and secrets from code and mounting/changing config files easily

3. robust and predictable testing/production environments

4. centralized logging

Also microservices's goal isn't really about just "scaling" in my opinion, there are other important advantages even if you have no intention to scale, aspects like modularity, separation of concerns, robustness and lowering the technical debt are still as important whether your app serves 1 or a 10000 users at the same time. Of course you can pull your python app from your repo or even rsync it (just like you can just develop any software without using git or any revision control) and just execute it might work very well, but sooner or later you are going to regret it if you're a business