How we built the new BBC Homepage

http://www.bbc.co.uk/blogs/internet/entries/47a96d23-ae04-444e-808f-678e6809765d

by achalkley • 11 years ago

154 94 11 years ago

bbc.co.uk

rentamir• 11 years ago

Thanks for sharing the tech stack. Would love to read more about the curation kit and the microservice approach.

defenestration• 11 years ago

I agree. More corporates should give these insights and don't care for a bit of bashing when they do.

Domenic_S• 11 years ago

Tons of tech buzzwords, yet the blog intercepts middle-clicks. I want to open your links in a new tab so I can read them when I'm done with your blog!

mattbessey• 11 years ago

Its not intercepting Cmnd+Click (mac)

thejosh• 11 years ago

Linux / Chrome middle clicking doesn't work.

How can they break something so basic?

clarkdave• 11 years ago

It's not just the blog - news.bbc.co.uk also intercepts middle clicks to external sites. Middle clicks on links to other bbc.co.uk pages work fine though.

Is it an accident or intentional? Anyone know?

CatsoCatsoCatso• 11 years ago

I've noted this to them in their feedback forms before. It's been like that for over a few months. If I recall correctly when the Guardian (uk) was developing their current version under Beta the same problem came about for a short while.

simons• 11 years ago

I believe it's unintentional - I expect it stems from JavaScript event tracking (e.preventDefault(); logEvent(e); window.location=e.target.href; Or something to that effect).

marknutter• 11 years ago

Tons of tech buzzwords, yet the blog intercepts middle-clicks

They broke the middle-click? Good god, shut it down!

super-serial• 11 years ago

It's like they monitored the front-page of HN everyday for a year, made a list of the top mentioned technologies and then gave that list to their management.

Their manager said "Your goal is to write a blog post mentioning each one of these technologies, and add links so it appears in dark-bold-blue text. If we don't have a project using a trendy tech-stack... you bust your ass and get something up and running."

The engineers balked... "but why?"

Then the boss said, "I'm tired of being ignored in 'Who's Hiring' on Hacker News. We make this article and they'll all come begging us for jobs. At the BBC we don't wait for news to come to us, we make the news. And THAT'S what we call journalism kids."

pgl• 11 years ago

It's like they monitored the front-page of HN everyday for a year, made a list of the top mentioned technologies and then gave that list to their management

Or perhaps those technologies were actually useful for them? I don't know, just a thought.

ricardobeat• 11 years ago

This is extremely offensive to the team that built this product. Why the hate?

davb• 11 years ago

I've been using the new design on mobile for a while here and the user experience is awful compared to the old design. Although the parent comment might seem a bit patronising, I think he's right - they've focussed too much on the technology and design fads.

Compared to the old site, there's a lot of "big text", too much of a focus on images (when you drill down into the site) and less on text, and low information density.

Worst of all (for me) - when loading on latest stable Chrome on Android, with a very fast internet connection, the page appears loaded but as soon as I scroll a number of the images disappear and the layout changes. I have to scroll back to where I wanted to be. Lots of scripts, client-side rendering, etc. It's overkill for a site whose focus should be content, not interaction.

While I'm sure the team built a technically impressive website, I don't think they've built the "right thing" and I certainly don't think it's better than the old one (from a user perspective).

jwmoz• 11 years ago

Don't invite Ricardo to parties.

andyjohnson0• 11 years ago

If you believe that the technologies they selected are inappropriate then maybe you could explain why?

kokey• 11 years ago

I love your comment, but I think the reality is more that they already had all the hipsters on board, the technology choices just followed naturally from that.

Numberwang• 11 years ago

I have not yet tested the new mobile site, but if the design of the normal site is anything to go by it will have been designed in hell by some teen demon reading too much Kafka.

Twirrim2• 11 years ago

Hmm. While it's nice to push rendering out to clients for saving your server CPU, that can lead to a suboptimal mobile experience, requiring more CPU and battery power on the mobile device to work on the javascript and render the page. There are other factors, for sure, but you want to be keeping client side javascript down to a minimum.

Taking a quick spin through yslow in the mobile browser suggests they've got a number of areas to improve on to make the time to screen significantly better on mobile devices (even on a fast connection here it took several seconds to even start showing me content, and several more before it had finally loaded everything)

Given the world wide reach of the BBC, expecting high speed and low latency networks seems like a bad idea. In the US, 3G & 4G typically see 90-100ms latency per request. Mobile Yslow is reporting that they've got 21 javascript scripts alone on the page. IIRC The android browser will limit itself to 4 threads retrieving content typically so that's (21/4 * 100ms) 525ms just lost in latency requesting the javascript, let alone actually downloading it and the overhead of the javascript renderer. It's also pulling in content from 21 different websites, so at the bare minimum that's 21 DNS calls being made (with the same latency penalty!) A bunch of those are being done just to load a single piece of content too, which is a little crazy.

Don't get me wrong, the site looks good.. it's just for a 'mobile-first' experience, they seem to be missing the all important time-to-screen and giving the mobile user a lot of work to do.

A useful tool from google for analysing the site for both mobile and desktop: https://developers.google.com/speed/pagespeed/insights/?url=...

and a good talk from last year's Google I/O conference on optimising the mobile experience: https://www.youtube.com/watch?v=WrA85a4ZIaM

notahacker• 11 years ago

Plus this is the only time I've ever seen "mobile first" interpreted quite so literally (i.e. on a desktop with a reasonable connection the mobile stylesheet loads before the main styles and content and is visible for just long enough for the eyes to register the layout, producing a horrible "flash of unstyled content" effect).

n.b. plenty of people looking at the BBC site are still being shown the old style.

smcl• 11 years ago

I've been wondering about that unstyled "flash" on BBC myself and whether it's just down to some chrome extension or something I had installed interfering with things. Glad to hear it's not just me.

Twirrim2• 11 years ago

The main sites from my last job load the mobile view/content first, then load in the desktop experience as appropriate (some are still in the process of transitioning to the newer design). However they were done in such a way that it shouldn't result in any jarring transitions. A couple of examples:

https://portal.ehawaii.gov/ http://energy.hawaii.gov/

Cthulhu_3• 11 years ago

How true is that though? Given that native apps render client-side too. In a much more optimized, device-specific fasion, I admit, but still - that too is client-side rendering.

acdha• 11 years ago

Your comment inspired me to use the great webpagetest.org service to collect a trace on iOS using a simulated 3G network condition:

http://www.webpagetest.org/result/150217_BB_Q71/1/details/

There's definitely plenty of room to optimize the time-to-first-render – in particular, switching to an async model would make a huge win since rendering is currently blocked by 6 stylesheets and 7 JavaScript files, most of which appear to be specific to certain content sections.

jordanlev• 11 years ago

Totally agree that it's not ideal to push rendering out to clients -- and this article specifically says they're using React.js and rendering the pages server-side, so they are sending a rendered page to the client (along with the js code to perform further front-end behavior if js is enabled, but content is still there without js).

kirkus• 11 years ago

Amazing insight into how large organisations manage projects.

sklogic10• 11 years ago

The new BBC page is barely usable. Not to mention that the new BBC android app is thoroughly disgusting. Bring back the old one, please!!!

blowski5• 11 years ago

I believe there is a bot that monitors websites for changes, and whenever they change enough it posts randomly generated strings along the lines of:

The new <sitename> design is terrible! Bring back the old one!

Whether it recognises the irony that it also moaned about the 'old one' when that came out, I'm not sure.

sklogic10• 11 years ago

Have you used the previous BBC android app? It worked. The new one is totally useless.

MartynX• 11 years ago

What do you not like about their new Android app? I use it daily and greatly prefer it to the old one.

Nexxxeh• 11 years ago

Obviously the tone wouldn't do at HN, but this Reddit post in /r/unitedkingdom mentions a lot of the problems with the new mobile site and the Android app.

http://www.reddit.com/r/unitedkingdom/comments/2vymm2/does_a...

Re: Android:

Jackal___ 121 points 1 day ago Their new Android app is very very slow to load the news and uses a fuck load of background data.
Quagers 72 points 1 day ago Also it doesn't download the stories when you hit refresh, just the headlines and then downloads the story when you click the headline. Completely bloody useless on the tube!

sklogic10• 11 years ago

1. Crashes about twice in 5 minutes.

2. Tiny cache, compared to the previous one, so totally useless offline

3. Intrusive UI

4. Much slower

weavie2• 11 years ago

They mention using BEM (https://bem.info/) Any idea what it is? The site is not at all clear.

yalooze• 11 years ago

CSSWizardy say it better than me: "BEM – meaning block, element, modifier – is a front-end naming methodology thought up by the guys at Yandex. It is a smart way of naming your CSS classes to give them more transparency and meaning to other developers. They are far more strict and informative, which makes the BEM naming convention ideal for teams of developers on larger projects that might last a while." http://csswizardry.com/2013/01/mindbemding-getting-your-head...

But basically: agreed rules for how to name and format css to make it maintainable and performant in big projects.

weavie2• 11 years ago

Thanks. I dont know why they couldnt just say that on their homepage!

conradfr• 11 years ago

I worked on a project that used that and while I understand the reasoning I hate the ugliness of it.

confiscate• 11 years ago

wow you use nodejs. how amazing that is

alexcason• 11 years ago

There seems to be some kind of bird theme going on with their CSS classes.

  class="distinct-component-group container-buzzard"
  class="distinct-component-group container-pigeon"
  class="distinct-component-group container-macaw"
  class="robin sparrow-container"
  class="sparrow-container sparrow-columns"

smegel• 11 years ago

I wonder to what extent they are still using Perl.

draegtun• 11 years ago

The days of the BBC being a "Perl shop" have long gone.

But they do still use Perl, for e.g. BBC Genome is a new project written in Perl - http://www.bbc.co.uk/blogs/internet/entries/cedd1f30-ac95-32...

_b8r0• 11 years ago

For those that aren't aware, the BBC has one of the worst cases of "Not invented here" syndrome you'll ever see.

Cthulhu_3• 11 years ago

And yet all I see in the article is off-the-shelf libraries and frameworks, and they came from a commercial CMS - where's the NIH?

bshimmin• 11 years ago

You won't find any in that article, but the BBC did build their own jQuery once upon a time (now long-since abandoned, of course, as many "NIH" projects tend to be): http://www.bbc.co.uk/glow/

mikecmpbll• 11 years ago

Christ, so much hatred.

I like the desktop site, I like the mobile site, and I like and appreciate that you're giving us an insight in to your tech stack and how content is published.

Cheers beeb.

legoisbest3• 11 years ago

Does anyone know how much of this development is paid for by the UK tax payer vs BBC worldwide profits from DVDs etc?

I'm not sure website development should be paid by UK tax payers TBH...

Maybe at least some of the reason for hate.

My main hate is the idiotic cookie policy which makes everything look ugly, but I guess the EU is to blame there.

fredkingham• 11 years ago

you mean by license fee payers.

Given the bbc is the 7th most popular site in the UK, and a clear leader over other news sites, I don't see why it shouldn't be paid for by the license. It fulfils its requirements outstandingly well.

(There are questions about whether given the move away from a tv based model the bbc should be paid for by tax payers, and that's fair)

jpatokal• 11 years ago

Per their 2013-14 financial report, the BBC's income was 3.7b pounds from license fees and 1.3b pounds from commercial activities (= largely BBC worldwide profits).

http://downloads.bbc.co.uk/annualreport/pdf/2013-14/BBC_Fina...

And the BBC's mission is "to enrich people's lives with programmes and services that inform, educate and entertain". Building a website to better distribute the content they've already paid for seems like a no-brainer.

Disclaimer: I used to work for a subsidiary of BBC Worldwide. Plenty of bureaucratic nuttiness in there, but this seems like one of their more sensible moves.

vertex-four3• 11 years ago

I'm not sure website development should be paid by UK tax payers TBH...

We pay for it mostly via the TV license fee. The BBC is not allowed to make money off UK citizens for any of its core services (TV, radio, news) - hence why some BBC links for the rest of the world are blocked for us, as they contain advertisements.

BBC News is primarily intended for a UK audience, so it makes sense that it's paid for by UK taxpayers. It's distribution infrastructure, just like the Freeview transmitters and the Freesat satellite - why wouldn't it be paid for by the taxpayer, who overwhelmingly demands online access to news and TV catch-up?

sklogic10• 11 years ago

And I totally hate that they're wasting taxpayers money on all this crap.

Cthulhu_3• 11 years ago

Subtle. The alternative is biased commercial TV which prefers ad-spattered sensation over what the BBC produces. But I'm sure you prefer that.

Besides, the BBC amasses fortunes selling their programs abroad; Top Gear in its original form and in the formula license is shown in over 150 countries worldwide. And that's just one of their productions.

lurcio• 11 years ago

I for one don't mind that they use tax payers money - because for all its many and varied biases* (national & cultural blindspots, party political) - its output is valuable and often of benefit/interest beyond Blighty.

* BBCs insistence that it is unbiased is unrealistic imo (but not as outlandish as Fox's 'fair and balanced'). Its more of an ideal rather than something anyone/org can achieve. I admire their attempts to achieve some kind of balance, although sometimes the BBC can filter too much thats not 'centre-ground' (but less so than US networks).

carsie• 11 years ago

There are always mudslingers following change - its so easy to hate.

I for one think it is fantastic the BBC are sharing this information and am liking the direction they are taking. It might not be perfect - I too have experienced some issues with the site over the past few months - but I'd much rather a BBC that looks to refresh itself and attempt to improve the experience for their users. Glitches happen - as long as they are fixed then it is all good with me.

sklogic10• 11 years ago

Of course it's easy to hate, when something that used to work is now broken and useless. It's not about glitches, it's a thoroughly broken design.

rashthedude• 11 years ago

Pleasantly surprised how 'up-to-date'the beeb is.

thesehands• 11 years ago

The BBC produce quite a few intereting write ups of their R&D process alongside odd things they are hacking on here: http://www.bbc.co.uk/rd

Well worth a read.

Fastidious• 11 years ago

I love the BBC. Very interesting news, fairly balanced.

I hope I can play their videos on the new website, as before they all used to require Flash.

esalman• 11 years ago

Reading the article reminded me of this: http://motherfuckingwebsite.com/

bencollier49• 11 years ago

The new "BBC News (Beta)" website is bloody ugly. They've gone for a "flat UI" thing, but it looks so clumsy that it reminds me of the internet from before people used gradients.

coob• 11 years ago

Haha has the internet groupthink gone full circle on 'flat' now?

Tech is fashion whether we admit it or not.

ggitau• 11 years ago

Home page displays a bunch of javascript before rendering the rest of the page. Doesn't happen all the time though.Maybe you guys might want to look into that.

ndreckshage• 11 years ago

This mentions using Node/Express to help with concurrency. Ok. But this also mentions using 'isomorphic' javascript with React, and also not completely reliant on cache. React.renderToString (and all other isomorphic options -- Rendr, etc) are synchronous and slow. (rendering a large page in React w/ static data would take ~600ms; whereas the same page with with Mustache in Go for example would take ~30ms). End result, decreasing concurrency, and hurting overall performance.

OP - can you shed any light on how this is actually impacting your performance? Or maybe things that you had to do to get around the problem (ex - details of 'module level' cache with Redis etc).

nerdy• 11 years ago

Most of their tests take half a second each to run? Some examples: "The first story displays an image and not an alt text attr if one is defined. (793ms)" "The module banner color is BBC News Red (552ms)"

The 13 tests shown in the screenshot take an average of 577ms each, a total of over 7.5 seconds. 13 simple tests with wild variance in execution time. Checking a module banner color? 42ms. Checking a module background color? 552ms. So a 12x increase for checking a different color within a module?

Those tests are going to rot because of the expense of executing them and they'll be discarded. Over a 500ms average per test just isn't sustainable, particularly at the scale required for checking every conceivable kind of background color.

meesterdude2• 11 years ago

I work on a rails codebase with some tests that take 20 seconds to run. The entire suite is normally 5 minutes. I made a ruby gem to take the entire thing down to 20 seconds, which is much better for flow.

But I agree those are definitely smelly tests with questionable value return, and would not be surprised if they do indeed rot.

noso• 11 years ago

It nice to know what is happening in the BBC and the tech they are using. Additionally, It is appreciated they spend the time writing up what they have done/achieved.

Rock on!

meesterdude2• 11 years ago

I do love when companies share this stuff. Thanks.

Rather than using PHP or Java (that was the requisite of the Forge platform), we have chosen a non-blocking framework, NodeJS with the Express framework . This allows us to serve more simultaneous requests, increasing the performance of the application.

I don't doubt this is true, but its worth noting you can get good performance out of a "blocking" framework too. Node.js does better than others in some situations, but in this its not a snowflake, and is in some regards worse.

But I will criticize priorities: I think this is too much fad, not enough practical. the experience is notably worse than the old site, and it seems like they just threw buzzwords at their problems instead of really crafting a solid solution.

I think where they're coming from, maybe this makes sense - they needed to overshoot from their previous platform. But I think they'll find it problematic in the long run and change some of their approaches and release a new site, and that will be a good platform and serve their needs for a good while.

blowski5• 11 years ago

I agree that it sounds very faddish, but that might be the whole point of the blog post - attracting talented developers. Manchester has an up and coming startup scene, which presumably the BBC is competing with in the hiring pool, and brandishing their hipster credentials may be one way of doing that.

mercurial2• 11 years ago

I must confess I'm quite surprised that they would get more performance out of NodeJS than with a decent Java framework. Java has its problems, but it's usually fast.

hokay• 11 years ago

They do mention prototyping with Scala and Clojure. I also doubt NodeJS beat the JVM out and out in performance. Maybe they went with Node because it is easier to hire web developers that know javascript than it is to hire web developers with knowledge of the JVM

Bahamut• 11 years ago

When you rewrite sites, that is often what happens. It will almost always be buggy at first, then it stabilizes as those bugs are fixed.

collyw• 11 years ago

Actually it doesn't look too bad. Its similar enough to the old one it doesn't seem like when Facebook upgrades and you feel lost. Shame the quality reporting has gone down so much over the years.

bruceboughton• 11 years ago

In Firefox 29.0.1, on the new BBC News homepage, wherever I click, it takes me to same story, even if I click on whitespace.

How we built the new BBC Homepage

Related Stories