Leaving JSPs in the dust: moving LinkedIn to dust.js client-side templates

https://engineering.linkedin.com/frontend/leaving-jsps-dust-moving-linkedin-dustjs-client-side-templates

by eneveu • 15 years ago

134 41 15 years ago

engineering.linkedin.com

kls• 15 years ago

I am surprised that many of the big sites are just now getting around to moving away from the server side frameworks for UI. The benefits of doing so are just so great that I am surprised that they ignored it for so long. Flattening the web stack makes development rapid again, UI engineers can stub out JSON files to emulate the REST back end and iterate through bugs so much faster. At many of the companies that I have implemented new style web front ends out TCO dropped significantly. I guess it's telling of the times when you are more shocked to find out that people are just now doing as opposed to the fact that they are doing it.

As well I enjoyed the article, I have not used dust.js yet so it was interesting to see how it works. Dojo has a good templating model and I tend to prefer it over some of the other offerings, it is good to see that others are getting into the space. Though I do prefer Dojo's widget based templating over standard tiles type templateing, I find that it makes the code more compartmentalized and reusable. More of a black box, where code and template can just be added to a page and it works.

rhizome• 15 years ago

JSP, I never knew. That certainly explains a lot, particularly how it meshes with my sense of anything-JSP being a couple of years behind, featurewise. I don't know why this is, if it's a language non-feature, or a benefit of open-source libs, Java coding culture, or what, but I've thought that LinkedIn was making some silly site decisions and being inexplicably behind the times for some time now.

luser001• 15 years ago

I did something similar for a recent project. My backend basically only provides an HTTP API. All rendering is client-side.

It felt natural as a programmer to develop the backend as a "library" that was "called" by the client side.

Immmmmense productivity. All pages require auth and shouldn't be indexed, so I don't worry about excluding search engines etc.

jrockway• 15 years ago

Not sure why you're being downvoted, but I like this too. Your application should just be an API and pages should be tiny programs that call into this API. That way you can easily test your application and easily test your pages. And when you want to release "an API", well, you already designed, implemented, and tested it.

rjurney2• 15 years ago

Here here. Long live the one page web app.

reinhardt• 15 years ago

"All told, we evaluated 26 different templating technologies, scoring them on a variety of factors: DRY, JS library agnostic, mature, open source, easy to learn, documentation, flexibility, performance, and so on."

If anyone from Linkedin is reading - any chance of getting this evaluation posted somewhere? Would be useful even if it's just a "tech x factor" matrix without commentary.

vybs3• 15 years ago

Sure, we should be able to make it available on the blog site.

toonketels• 15 years ago

That would be great. I also would love to see the see the matrix.

Any change you are going to discuss in more detail how to use the js templates on client and server side?

prodigal_erik2• 15 years ago

> Once written, the same dust.js template can be rendered not only in the browser, but also on the server using node.js or Rhino.

I'm relieved to see their staff know what they're doing, and even curl http://www.linkedin.com/in/example still yields content. An alarming fraction of devs and toolsets out there would have screwed this up so badly that their resources would be cut off from reuse by the rest of the web, trapped behind an unstable single-site API.

johnx123-up2• 15 years ago

Can you please elaborate on this?

simonw• 15 years ago

I imagine the parent means that building single-page apps that require a full DOM+JS implementation is bad for the Web, because it cuts content off from being accessed by anything other than a full modern browser running several million lines of C++.

See also my answer to this question about single-page apps on Quora: http://www.quora.com/What-exactly-is-a-single-page-applicati...

ctek• 15 years ago

http://www.linkedin.com/in/example doesn't use dust.js if it did you would see it imported when viewing source. Am I missing something?

sandGorgon• 15 years ago

Doesnt this play havoc with caching - I mean if you had a single UI webapp (that in turn queries all other webservices to render a page), then you could simply heavily cache your UI webapp. But when your client side JS is directly querying individual web services, in that case you would have to cache each individual web service (not sure if this introduces inconsistencies vis-a-vis different web services).

Is this how Amazon - who reputedly has everything as services - renders its webpages as well ?

bigiain4• 15 years ago

Not only caching, but right now I suspect this approach could have _serious_ SEO implications. It seems to run directly against the guidelines against cloaking here: http://support.google.com/webmasters/bin/answer.py?hl=en&...

(Not that I suppose LinkedIn are too worried about search engine discoverability of a lot of their "dynamic, login required" content, but I'll bet they're _not_ using this technique on all those highly optimised pages full of people's names - like the 4th Google hit for my name: http://au.linkedin.com/in/iaindchalmers )

keltex• 15 years ago

It wouldn't have SEO implications because using javascript to load content isn't considered cloaking. (Cloaking is where you present something different to googlebot vs. a live user). It's well known that googlebot does run javascript now:

https://twitter.com/#!/mattcutts/status/131425949597179904

There's also some speculation that googlebot actually runs Chrome and can render anything that Chrome does:

http://ipullrank.com/googlebot-is-chrome/

akavlie3• 15 years ago

JSP is an archaic template language. I'm on a project with JSP on the front end; compared to Django, Jinja, or any of the other Python template languages it's really limited and verbose.

It's interesting that LinkedIn chose dust.js as it doesn't get much attention compared to some of the other JS template languages (e.g. Mustache & Handlebars.js).

strlen2• 15 years ago

Do keep in mind that LinkedIn was founded in 2003, before many of the web frameworks were released. LinkedIn also uses a custom JSP compiler, as at the time the official implementation lacked many feature.

akavlie3• 15 years ago

That wasn't meant as a criticism of LinkedIn's adoption of it, more my own circumstances and the fact that frameworks are still using JSP by default in 2011.

eneveu• 15 years ago

I wonder what's the best alternative for server-side template generation on the JVM.

I used FreeMarker on a previous project, it's pretty cool ( http://freemarker.sourceforge.net/ ). There is also mustache.java ( https://github.com/spullara/mustache.java ). I heard about Velocity and StringTemplate, but never tried them out... I guess Groovy Server Pages could also be considered if one's company is open to Grails...

Anything else I should be aware of?

tunggad2• 15 years ago

If you need pure Speed + compile-time checked Typesafe, then Japid is the nr.1. Very popular under Play! Framework community. The performance is superior, 2x - 3x better than FreeMarker.

efsavage• 15 years ago

Are you using some old version of JSP? Not unlike many other mature technologies, many times people are just victims of some arbitrary corporate standard and not the tech itself.

As far as limited, I looked at Django strictly for it's vaunted templating prowess and didn't see anything that isn't easily reproducible in JSP. There could be some magic there but I missed it.

For verbosity, sure, fn:toLowerCase(string) is technically more verbose than string|lower, but marginally so.

akavlie3• 15 years ago

Version 2.0 (current version is 2.1). Apparently upgrading is not trivial (due to Spring framework or whatever... I don't know, I'm not familiar with the Java stack).

You mention an example where JSP fares best; there are a lot of examples that don't come out looking too great though, such as time/date/decimal conversion, the obnoxious syntax for if/else, etc. A lot of Django's other filters just don't exist in JSP.

But that aside, there are larger scale issues -- composition of templates via blocks in Django/Jinja, which is a lot easier and more flexible than Apache tiles; the lack of auto-escaping in JSP; and the forced XML syntax, even at the expense of valid HTML (see what it does to script includes or empty divs, without a  in between).

sonnym• 15 years ago

It should be noted that dustjs is not yet updated for node v0.6.x. I submitted a rather trivial pull request toward the end of October fixing the problem and another user has submitted one since then, but the author has not responded to either.

This makes me question how much the project will be supported in the future - but I suppose having a large company like LinkedIn invested in the technology is a good sign.

See: https://github.com/akdubya/dustjs/pull/36

bigiain4• 15 years ago

Curiously, we tried this and backed away to using AJAX calls returning pre-rendered html fragments in a recent project.

If you're building a mobile site, and want to target low-end Android devices (our "crappiest-phone-test-device" was a Huawei U8180), you really don't get a lot in the way of JSON parsing or DOM manipulation performance.

I wonder what (if anything) LinkedIn are doing for mobile?

vybs3• 15 years ago

http://engineering.linkedin.com/nodejs/blazing-fast-nodejs-1...

bigiain4• 15 years ago

Nice link, thanks.

It doesn't address the problem we bumped into though. When a page needs a significant amount of content updated, sending that as structured JSON to a low powered Android device (or sometimes even an iPhone2 or a non S 3G) and then parsing that JSON and updatng the DOM was _way_ slower than sending back HTML and just replacing an elements innerHtml. For small tasks (like the personalisation {'name':'John'} example) we could make the JSON approach work, but updating a ~25row 4column list? Not on the cheap Android devices...

tunggad2• 15 years ago

Does any know how is dust.js compared to Beebole PURE, an JS template engine too. They claim, PURE should be fastest JS template. ^^

bvincent2• 15 years ago

As vybs mentioned we'll publish more detail on our evaluation process, for which performance was one of many features we looked at. It may be useful to start by asking what's most important for you/your organization (performance, productivity, ramp-up time, server-side rendering support, etc) since these priorities will likely be different from ours.

mikehal• 15 years ago

What about SEO and accessibility. If the content is sent via JSON it's relying on javascript, no?

bvincent2• 15 years ago

Correct, the template can only be rendered on the "client" in cases where JavaScript is available. In other cases (including for the small percentage of users who have turned off JavaScript and mobile devices without JS support) the solution is to render the templates (turn them into HTML) on the server.

cuginoAle• 15 years ago

bvincent, could you please elaborate on this? How do you tell javascript enabled users apart from disabled ones? Is there any website discussing this you can point me to? thanks

Tautologistics• 15 years ago

Unless I am missing something, the bullet point near the end, "Progressive rendering", is not correct when it states that template rendering is asynchronous and parallel. Fetching of data, sure, but unless they are using web workers to execute the templates client side, and I do not see workers mentioned anywhere, this is not accurate. When a template is rendering, all other JS and UI will stop.

Having applied the same approach as LinkedIn on past projects and jobs (e.g. BN.com), there is definitely a lot to be gained if done right, and web workers, when available, ease some of the issues with rendering large templates and/or large datasets.

Leaving JSPs in the dust: moving LinkedIn to dust.js client-side templates

Related Stories