NYTimes Opensources Their Deep Linking JS

http://open.blogs.nytimes.com/2011/01/11/emphasis-update-and-source/#h[WtEIyw,2]

by joeybaker • 16 years ago

169 46 16 years ago

open.blogs.nytimes.com

joeybaker2• 16 years ago

On Github, they say they'll eventually remove the dependency on PrototypeJS. The library is only ~10k now, hopefully that change won't increase the size too much.

emehrkay• 16 years ago

It looks like they are only using a small portion of it -- selectors in init and a few event handlers. They could use one of the many pre-existing selector engines and write a simple cross-browser addEvent function

donohoe11• 16 years ago

That is the plan. The CSS selectors are only required (for now) as NYT has some specific criteria to cover the markup in various Article and Blg Post pages.

For most cases I imagine 'querySelectorAll' or 'getElementsByTagName' would suffice

pjscott• 16 years ago

You know what I would love to have the HTML5 guys add to the spec? Some way of keeping libraries like PrototypeJS and jQuery in the browser cache at all times, so that pages could just use them without worrying about the size.

Perhaps some kind of alternate src attribute on script tags, so you could list a local copy (for reliability) as well as Google and Microsoft CDN URLs, and the browser would go with the first one it had in its cache. And if individual browsers wanted to distribute jQuery (et al.) along with the browser itself, and define an URN for it, so much the better.

This is just something I thought up in the past few minutes, so take it with a grain of salt, but as a web developer, I would love this.

mikeklaas• 16 years ago

Why isn't it sufficient to use the Google-hosted jQuery? It's likely to be cached.

nl• 16 years ago

In HTML5 you can do this by keeping a script in LocalStorage, then executing it on-demand. It could also have a fallback mechanism for network loading (and lazy initialization).

The implementation of this is left as an exercise for the reader.

steveklabnik• 16 years ago

Wouldn't you be able to do this with the cache manifest? http://diveintohtml5.org/offline.html

donohoe11• 16 years ago

The only reason I didn't rip out PrototypeJS was for the CSS selectors and the add/remove classnames. That's what remain that will take time and hopefully not for much longer.

hinathan• 16 years ago

Speaking of NYTimes, Safari user stylesheet has a line to disable the annoying word definition popup when selecting text.

.nytd_selection_button { display:none; }

lukeschlather• 16 years ago

That's why NYT is never getting on my NoScript whitelist. I have a nervous habit of highlighting random text while I read.

r0s• 16 years ago

Me too, and I would like to know if anyone has ever studied the trend so I can better develop my application for these users.

Wikipedia claims it does not exist:

http://en.wikipedia.org/wiki/Impulsive_highlighting

Swizec• 16 years ago

I don't understand what's so awesome about this? Browsers have supported deep linking since forever. You can link to any specific id on a page and the browser will scroll to it when you open the page.

For example, I could have every paragraph an id say id="p5" and then link ad example.com/story#p5 and voila, deep linking.

Hoorah for reinventing the wheel :)

mey• 16 years ago

The problem with this approach is that as the paragraph is moved around, the linear id's become stale. Say the article is edited to move content, your deep link may break, still right page of content, but wrong deep link to content.

This addresses this (as stated in the article) without some elaborate server content management solution that tracks paragraphs and their html id links.

nicpottier• 16 years ago

RTFM? :)

The difference is no 'href' tags. The 'tag' is automatically created based on the words in the paragraph, via Javascript, and decoded appropriately.

It is also slightly neat in that you can highlight a specific sentence (multiple sentences actually, see the little tutorial at the bottom).

I actually kind of like it, it would be a neat way to really highlight what you think is interesting in an article when sending someone a link. But doing it as a per site thing is crazy.. seems like it could be a good browser extension though. People who have it installed would instantly get a more functional linking experience. Imagine linking off to some documentation in a blog post for a coding problem, say to a Django documentation page, and when someone clicks the link they are not only taken to the specific part of the page that you are talking about, but the relevant stuff is actually highlighted. That'd be Neat (tm).

donohoe11• 16 years ago

I thought about a plugin but then you are maintaining several and handling issues from readers who hits walls installed or uninstalling it...

My hope tis that this approach is equally unhelpful to everyone :)

Seriously, my hope is that if an approach like this is going to happen that we can keep the usage (syntax) consistent.

Further down the road I'd like the view to also show you what people in your network have highlighted, or on a more aggregate and subtle level, what everyone has...

bambax• 16 years ago

Highlighting is neat; but concerning deep linking, the parent has a point: since the NYT controls the source, why not simply generate anchors for each paragraph?

The upside would be that there would be much less work needed to find edited paragraphs: a paragraph would be identified by its anchor and it could be changed completely, as long as the anchor is still there the link's fine.

This approach has certainly been envisioned: it would be interesting to know why it was put aside?

wookiehangover2• 16 years ago

Enterprise JS is 400 lines of code... 0 lines of tests. Kudos, NYT.

wookiehangover2• 16 years ago

Downvote this if you hate writing unit tests for code you release publicly.

jedsmith3• 16 years ago

I'm suspicious of anyone who spends their time critiquing others' work so harshly instead of innovating on their own.

justicefries• 16 years ago

I'm suspicious of anyone who obviously doesn't think innovation is innovation without making sure your innovation actually works (with tests).

goldenthunder• 16 years ago

You clearly don't understand the importance of TDD like many others (count the downvotes).

If you cannot stay with the curve because of your lack of knowledge, maybe you should spend less time flaming on Hackernews and more time studying up, sir.

barredo• 16 years ago

I made something similar to this NYTimes.com feature, a few days ago: https://github.com/alexbarredo/insidelink

It's simpler, of course. It's in plain JS and as jQuery plugin

donohoe11• 16 years ago

It looks like your version uses a numeric value as determined by the order of the P tag. This is what I had before.

The second version generates a Key for the paragraph so it can be moved anywhere in the page, and survive slight modifications if the text changes too.

Nice work though.

chapel• 16 years ago

I played around with this and got it so you can both highlight and have it move the view to the paragraph of choice. Seems pretty interesting.

An issue with it though, when sending the link to someone, and then they click it, browsers (Chrome here) turn the #h[TArTWw,1] into #h%5BTArTWw,1%5D which then seems to be ignored by the script.

donohoe11• 16 years ago

I noticed that just now too. I'll investigate and see if I can get a fix in the next update.

donohoe11• 16 years ago

Fixed it for now.

omouse2• 16 years ago

Awesome, they just invented a small part of Ted Nelson's Xanadu project. Say what you will about Xanadu being vapour-ware but at least Mr Nelson designed it properly to handle problems like deep-linking and track-backs which the Web has to employ workarounds for.

donohoe11• 16 years ago

I'm surprised to find this posted here. I developed this and given the community here I'd appreciate any feedback for future iterations...

joeybaker2• 16 years ago

Many thanks! This is very nifty.

Since you asked…

1) Change tracking. Technical hurdles (and they're huge) aside, it would be great not to have these links break when text is added/removed.

2) Better onsite instructions. I knew how it was supposed to work, and it took me a bit to figure out that clicking a sentence highlighted it. Perhaps a tooltip?

3) A wordpress plugin seems easy/obvious :)

Many, many thanks for the good work!

dev_jim• 16 years ago

Nice library. It looks like a great candidate for one of our upcoming projects.

qixxiq• 16 years ago

I would add a convenient way for this to interact with already existing hashed links that are used for navigation.

Rather than the #p[...],h[...] how about #p=...&h=...?

EDIT: Or even #deeplink=p[...],h[...]

donohoe11• 16 years ago

Its a fair point, and there were a number of ways it could have been done.

Given that there could be a long number of paramaters included for Highlighting I felt the [] gave a sense of belonging.

I like the #deeplink suggestion too, but in the end being as concise as possible was a factor.

siculars• 16 years ago

Grats donohoe, well done. What is of particular interest to me here is the use of the Levenshtein distance algorithm. The reason this works well here is because you are comparing your supplied key against a constrained set. Applying the Levenshtein distance algorithm (or its variants) against a constrained set of small size in this fashion has virtually no performance impact as the time to complete is entirely based on the size of the set you are matching against. On the other hand, matching against a set of millions of records does get costly.

stdbrouw• 16 years ago

I think v2 is trying to solve a problem it shouldn't have to solve. If the NYT made available previous (published!) revisions of their content, there would be no need to assign paragraphs with special IDs or the like. You'd simply say, "get me p2 for the story as it was on Jan 11 15:33". When you link to a specific piece of content, it's in the hopes that people will read or see the same thing you saw when you made the link. You don't want to be talking about different things, so revision-awareness would actually make more sense overall.

donohoe11• 16 years ago

Absolutely.

But is that going to happen? No. Why? Many reasons come from an editorial perspective (which I'm not knowledgeable enough to get into - but a simple one is: corrections), however the big one is also technical:

Providing previous revisions is not trivial feature. It would be a huge effort. There is no way to justify it from the perspective of deep-linking.

omouse2• 16 years ago

Except that would be another hack for the Web. As I said in another comment here, Ted Nelson's Xanadu project already figured this stuff out, we just have to employ some hacks on the Web to make it work ;/

jrockway• 16 years ago

Pretty good. Did they do an analysis of the NYT archives to ensure that their First Three Words Last Three Words technique is "good enough"?

donohoe11• 16 years ago

Yes. Also the 12+ page long articles you'd see in the Magazine section from time to time proved very helpful in this.

NYTimes Opensources Their Deep Linking JS

Related Stories