How SoundCloud built its new single-page main website using Backbone.js

http://backstage.soundcloud.com/2012/06/building-the-next-soundcloud/

by steren • 14 years ago

198 65 14 years ago

backstage.soundcloud.com

JonnieCache2• 14 years ago

It's been a long time coming, the dashboard has been slow as shit both to download and render for a long time now.

I've been using the desktop client exclusively because the website is often so painful on my 2008 macbook, but it only shows new tracks, not all the other interactions that show up on the dashboard, which means I'm missing out on a lot of good tunes.

alttab2• 14 years ago

And advertising! Just kidding, but really some companies make the website fully featured while apps and other things don't get the updates because those are peripheries for converted users like yourself, and many models require ad based revenue that doesn't translate well to client apps or mobile platforms.

JonnieCache2• 14 years ago

There aren't any ads on SC though. They make their money from premium accounts, of which they apparently sell plenty. Their business model is excellent all round IMO.

(You probably knew that but I thought it should be pointed out.)

flat• 14 years ago

i wonder whether "letting go" describes all of their memory management concerns

i found javascripts lack of weak references extremely problematic when trying to develop a larger backbone app

vitorbal• 14 years ago

you also have to somehow manage the lifecycle of your views, cleaning up after their event bindings and such. See this stackoverflow question[1] for more about this.

[1] http://stackoverflow.com/questions/7379263/disposing-of-view...

bricestacey• 14 years ago

Thoughtbot wrote a lib that they bundled up as a gem that does this for you called Backbone Support. It's an easy way to get started, but you'll likely want to tweak it later :)

https://github.com/thoughtbot/backbone-support

alttab2• 14 years ago

I always get nervous when I see tons of JavaScript. Frankly it's hard enough to deliver a consistent experience in the browser just from a style and layout perspective.

I haven't dug into how it's done, but I feel like JavaScript apps are harder to test and verify. It's not like running unit tests on your server code. To truly verify you need to test the entire application in each supported browser.

mark_story• 14 years ago

Someday soon Javascript will have WeakMaps. They're already in Firefox.

http://wiki.ecmascript.org/doku.php?id=harmony:weak_maps

swah2• 14 years ago

I love the idea of the Handlebars 'view' helper.

I'm not sure if "insertion of subviews" only happens once or on every render()... ideally only the subviews have to be rerendered - not the parent view which is "heavy".

nickfisher11• 14 years ago

yeah, that was something we struggled with when designing this architecture. The problem is that you don't necessarily know which subviews can be held onto and which are no longer needed, so the only way to generically handle the situation is to destroy all subviews when a parent view needs to be rerendered. This sound horrible and inefficient, I know, but our way to deal with it has been to be very careful about what triggers a rerender.

Views state exactly which attributes on their model should trigger the rerender when they change, and no others. If you have a very high level view which has a sizeable tree of subviews underneath it, then its subviews will probably be the ones actually displaying data and that it will essentially just be a 'composite' view and will never rerender.

Views which bind to collections (eg: Lists) have special logic for adding and removing subviews without rerendering all of them.

pxlpshr• 14 years ago

You know, we solved persistent background sound with hidden frames and MIDI tracks back in the 1990s. :)

Jokes aside, I wonder how on-going development and maintenance is for a "single file" website...

nickfisher11• 14 years ago

The source is in sanely separated and named modules in individual files which are concatentated into one for production. We actually concatenate into 4 different files according to how often the source changes (to work best with caching), but yeah, the concept is the same.

robmcm3• 14 years ago

Google Closure and require.js take care of this I believe.

cmicali2• 14 years ago

I'm excited about this as I am a heavy SoundCloud user but I can't help but feel this post is a bit premature. The existing state of the 'next' site is really early.. there are many bugs, most features are not implemented, and the sound detail page is almost completely broken.

I'd must rather see these kinds of posts after a big rewrite has proven the new technologies were the right choice!

nickfisher11• 14 years ago

You're quite right -- Next is nowhere near complete, and still very much a beta. We're working on the "release early, iterate often" approach. (See http://www.codinghorror.com/blog/2009/12/version-1-sucks-but... http://successfulsoftware.net/2007/08/07/if-you-arent-embarr...)

The main reason is quite obviously for feedback. SoundCloud is a community-driven site, and we want to get really early feedback about features, design, etc. Another part of the feedback is exactly what's happening here with sharing our techniques and seeing the results. Hopefully the outcome of this blog post is that some other people will learn something, but also that some people will point out things so that we ourselves can learn.

> there are many bugs, most features are not implemented,

We're definitely aware of the features not yet implemented, but if you find bugs (or are really missing a particular feature), please do use the feedback form to let us know!

Cheers, Nick

cmicali2• 14 years ago

I've been using the feedback feature early and often (nice job putting it so front and center also!)

I think the new design and architecture is coming out great and I don't mean to discourage experience sharing. If anything, better way to have said would be:

I hope that you will post a few follow-ups to this as you get closer to release what/if anything has changed in your stack, approach, and gotchas found building this complex of a single page/backbone app.

Thanks!

tocomment• 14 years ago

So when does it make sense to make a single page web application? Are there any guidelines you folks like to follow?

minikomi• 14 years ago

When there is music playing for a start!

swah2• 14 years ago

    Wikipedia                               Gmail
       <------------------------------------->
    Traditional                              SPA

Those two I'm pretty sure what I would use, but the other applications in between those extremes I'm not that sure...

romaniv• 14 years ago

I wouldn't mind at all if Gmail was a traditional multipage app that works without JavaScript.

Google Maps is probably a much better example.

WiseWeasel• 14 years ago

Whenever you can. It allows you to animate transitions during navigation, and to give your website more of a dynamic application feel instead of a static site. If done well, you encourage deeper interaction, as navigation feels more intuitive with smooth loading performance, without a jarring page reload, and with animations indicating what's going on.

mxxx• 14 years ago

Well, for a start, when you've got a fully developed API. We recently built a project that used a Backbone SPA for _some_ views, but was primarily server side.

Now we're building a mobile version, and having built the API in the first phase, we've got enough to go on to do it all via Backbone.

Additionally, when developing for mobile you're much more aware of bandwidth restrictions, so the ability to load only what you need becomes useful. When you're flicking through a bunch of pages that use the same view but with different data, an SPA lets you load only what you need.

rudasn4• 14 years ago

This is a very interesting read with a lot of interesting topics being discussed.

On "Views as components"

> Each view can include other ‘sub’ views, which can themselves include subviews and so on.

This is something that is missing from Backbone and it's something that always comes up, even when doing something as simple as a Todo list. In the Backbone Todo list example[1] you there are methods on the "parent view" (eg, the List view or the App view) to addOne and addAll (rendering the Item views).

So the concept of "sub views" is definitely needed but what I'd like to know, from SoundCloud's perspective, what were the arguments for/against implementing this functionality from within the templates and not from some place else, eg. the parent view's or the sub view's constructor (as a parent/child attribute) or initialize method.

> Each view is responsible for its own setup, events, data, and clean up.

Having already implemented the "sub views" concept it seems perfectly logical to allow a parent view to inherit DOM events from its children. Considering the Todo list example, and without using Backbone, how would you implement DOM events on the Todo items? You wouldn't bind events on each item's DOM element - that's just crazy. You would bind the events on the list (just once, for all items) and find a way to know on which item the event was triggered. In other words, you would use $('parent').on('event', 'child' ... ) and not $('child').bind.

It seems that with Backbone people have forgotten this practice, in a way similar to how people started using inefficient DOM selectors when jQuery came along.

So, when implementing the "sub view"/"parent view" concept, what would make perfect sense is to specify if the sub view's DOM events should be bounded on the parent view's DOM element. What I'd like to know is if SoundCloud has considered doing this and if yes why they have decided against it.

[1] http://documentcloud.github.com/backbone/examples/todos/todo...

crescentfresh• 14 years ago

> It seems that with Backbone people have forgotten this practice

To be fair, Backbone's events hash idiom is implemented using event delegation straight out of the box. It's not forgotten at all.

Although to your point: since Backbone offers no OOTB mechanism for sub-views, it offers no OOTB support for event delegation on a parent view. I just wanted to clarify where that line is drawn wrt event delegation being "forgotten".

rudasn4• 14 years ago

> I just wanted to clarify where that line is drawn wrt event delegation being "forgotten".

Yes, I should have been more clear on that. I mentioned that because SC says that they may have a view (DOM element) that contains a single button (or whatever). This means they bind events on each one of those buttons, even if they have 100 of those at any given time on the DOM. Maybe they have some other form of event delegation but they haven't mentioned it.

nickfisher11• 14 years ago

Hi Rudas,

>what were the arguments for/against implementing this functionality from within the templates and not from some place else

It is still quite possible for parent views to construct subviews and insert them into its own DOM at any time, but due to the nature of our views, many of them are UI components, and so it made sense to be able to define both the subview as well as its position at the same time. It makes writing a view with subviews very easy. You simply use the Handlebars helper exactly where you want that view and the rest is taken care of for you.

>You would bind the events on the list (just once, for all items) and find a way to know on which item the event was triggered.

Yes, that's a very common approach to handing DOM events, but it ran against our belief that views should be independent. If a view can only work in a particular situation (eg: nested inside a list which handles its events), then that creates hard dependencies between those two views and you can quickly get into a mess. Event delegation is definitely used in Next, but only at a per-view level. It is a sacrifice (I would posit that it's a minor sacrifice), but the benefits are highly independent and reusable views. We do take advantage of this, too: for example, if you look at the waveform in the player, and the miniature waveform in the header which shows your currently-playing sound (visible in the screenshot), that is the exact same view. Because they handle everything themselves, there was absolute minimum work needed to add that feature.

Thanks for the feedback and the interest, Nick.

Note that I'm just talking about our particular project: YMMV, and using event delegation in higher level views for your own project might provide bigger wins in terms of performance or maintainability.

rudasn4• 14 years ago

> It makes writing a view with subviews very easy. You simply use the Handlebars helper exactly where you want that view and the rest is taken care of for you.

It does indeed look very simple and easy but what I would be concerned about is that you have logic in your views that perhaps shouldn't be there. Your views are now dependent on the template language/parser you are using. Have you tried any other approaches that didn't work so well/easy before coming to this solution?

> If a view can only work in a particular situation (eg: nested inside a list which handles its events), then that creates hard dependencies between those two views and you can quickly get into a mess.

Agree. But how about having per view instance dependency instead of per view constructor. For example,

    <div class="listenNetwork__creator">
        {{view "views/user/user-badge" resource_id=user.id parent_view=this }}
    </div>

So your user-badge view can bind its events on the parent_view.el (if available), otherwise on itself. Backbone does this in a way on its delegateEvents method. If there is no selector specified in your event it uses bind on this.$el, otherwise it uses this.$el.delegate.

Thanks

jashkenas2• 14 years ago

    > This is something that is missing from Backbone 
    > and it's something that always comes up

... not so much missing as agnostic. You're quite right that subviews always exist in complex applications, but they're best handled in different ways depending on your templating / HTML-generation library of choice, and your app's architecture. Backbone doesn't want to force you to necessarily have a complete static hierarchy of View objects from the root DOM element down to the bottom ... many applications don't need or benefit from it. That said, if you've got a piece of functionality that you think would help with subviews in most Backbone projects, it would make for a great ticket or pull request.

    > You would bind the events on the list (just once, 
    > for all items) and find a way to know on which 
    > item the event was triggered.

Yep, and Backbone encourages that kind of delegation by default at the view level ... mostly for reasons of statelessness rather than performance. It's awfully nice to know that all of your events are always bound and ready on any given view, regardless of whether or not that view has any HTML yet, or has even been inserted into the DOM yet.

That said, binding to individual elements isn't problematic unless you're truly doing too much of it. 20 songs each with their own bound callbacks would be fine, but 2,000 songs wouldn't be. When to delegate for performance follows the same reasoning as it would in a plain 'ol jQuery app, and is touched on briefly in the FAQ: http://backbonejs.org/#FAQ-tim-toady

For what it's worth, here's an example of a Backbone view that's using delegation and a single event listener as a callback for all of the individual blocks in the charts: http://blog.documentcloud.org/blog/2011/10/entity-charts/

carlosedp3• 14 years ago

How this goes against the recent Twitter move to return rendering to the server? How to find the right balance of client and server functionality?

Pewpewarrows• 14 years ago

I've found that the best solution is to _not_ make your website "just another client" of your API. On the initial page load, render the page normally using your server-side templating. Then have your JavaScript introspect the existing DOM and enhance it into a very interactive close to single-page app, using client-side rendering from that point forward. Yes, there's code duplication with templates, and you have to deal with certain peculiarities like implementing a user event queue between the time the page loads and the JavaScript is done hooking into everything so you don't lose user interaction. But those are pains I'm willing to deal with.

ajacksified• 14 years ago

That's a similar technique to what I'm doing; I use language-agnostic templates (mustache) so that I can do a full server side render, and then my Backbone app initializes with bootstrapped data. This way, we can progressively enhance the site and have a quick page load and at the same time build a single page Backbone app.

nickfisher11• 14 years ago

Hi Carlos,

It was definitely something we thought about, and even discussed with the devs from Twitter. Twitter has a very different use-case to SoundCloud. When you follow a link to Twitter, it's usually to read a single tweet (or maybe a handful), and that's it. SoundCloud is visited by someone who is already willing to invest at least a couple of minutes to listen to a tune, and is much more likely to explore the site. Therefore, the value of making further navigation of the site fast (via client-side rendering, etc) is weighted differently at SoundCloud than at Twitter.

robmcm3• 14 years ago

Yes, I don't think instant loading is as high a priority for soundcloud as it is for twitter. Twitter shouldn't be a SPA I don't think, or at least not initially.

stusmith1977• 14 years ago

> Duplication of module dependencies is also tedious and error-prone.

You can combine these with RequireJS as well. Instead of:

  define(['a', 'b', 'c'], function () {
    var a = require('a');
    var b = require('b');
    var c = require('c');
    ...
  });

You can instead just do:

  define(['a', 'b', 'c'], function (a, b, c) {
    ...
  });

base6982• 14 years ago

Or:

  define(function(require) {
     var a = require('a');
     var b = require('b');
     var c = require('c');
     ...
  });

nickfisher11• 14 years ago

This approach does actually address a lot of the complaints:

- Boilerplate is greatly reduced (though still present) - Way less error-prone since you're not relying on keeping two separate lists of dependencies in same the order

But, it still does require rewriting before use (to add the module name), so I don't see a huge win here either way.

I'll also point out that this "sugared" syntax was added to requirejs after we started developing. If it were there at the start, perhaps we would have used that instead.

jscheel• 14 years ago

I'm glad he covers the commonjs / amd issue. We use a commonjs implementation as well, but it feels like more and more libraries are going to amd. Like soundcloud, the amd approach just isn't ideal when you get to a certain number of components. Seeing that they convert to amd on the fly, and use almond for production is really interesting.

tdskate• 14 years ago

Ember.js FTW !

neotorama• 14 years ago

The new home needs a search form. I always go to your home page just to discover new music.

nickfisher11• 14 years ago

Search is in the header. Is that not what you are looking for?

emw• 14 years ago

What support does SoundCloud's new single-page interface have for IE8 and IE9? Does the new interface use the HTML5 History API to enable state/page transitions? If so, what fallback technique was used for non-HTML5 browsers? Do these browsers get a single-page interface enabled via URL fragment identifiers (#'s), or do they fall back to a multi-page interface where page transitions are handled as they traditionally have been on the web?

Is there any data on how much SoundCloud's transition to a single-page interface decreased latency during navigation relative to the traditional page-loading model?

Apologies if these questions have obvious answers. I tried to sign up for the beta but the party was full, and I'm not familiar with SoundCloud's interface.

nickfisher11• 14 years ago

>What support does SoundCloud's new single-page interface have for IE8 and IE9?

For IE9: some. Backbone automatically detects pushState availability and fallsback to a hashbang system allowing the SPA to work.

For IE <= 8: none.

> Is there any data on how much SoundCloud's transition to a single-page interface decreased latency during navigation relative to the traditional page-loading model?

Not yet, but that would definitely be a metric we'll be collecting. We're still working very hard on increasing the performance, so it'd be a moving value right now.

> I tried to sign up for the beta but the party was full

No worries -- you're in the queue and we're gradually expanding the rollout, so you'll get an email soon. For everyone else, you can join the beta by signing in at http://next.soundcloud.com

spullara• 14 years ago

I think when you start reimplementing garbage collection in your application, you are probably going down the wrong path and should think about a better way to do things.

zwigby• 14 years ago

Any custom built caching layer is normally going to have to handle this situation.

martian• 14 years ago

What's your evidence of this? Have you built a large Backbone application? As it stands, your comment comes across as snide and indefensible.

base6982• 14 years ago

http://en.wikipedia.org/wiki/Abstraction_inversion

It's a little bit like this anti-pattern.

Axsuul• 14 years ago

Backbone.js is minimal on purpose so that you may implement these types of optimizations. That's why there's many ways to do things in Backbone.js but overall makes the application more efficient.

drivebyacct22• 14 years ago

Another blog that doesn't link to their actual site.

dchuk2• 14 years ago

The link to the actual site (webengage.com) is in the header nagivation on the right with an arrow pointing at it

drivebyacct22• 14 years ago

What? There's one link, the "SoundCloud" logo and it doesn't take you to SoundCloud's website, it takes you to the root of their/this blog.

I don't see webengage.com or a link to it anywhere on this link. Did you post on the wrong link by chance?

edit: There are also no arrows and the text 'webengage' doesn't appear in the blogpost page source...

dreamdu5t• 14 years ago

I would be very concerned about performance with a SPA built like this.

Do you have any benchmarks of Next SoundCloud's performance compared to the existing SoundCloud?

jarcoal• 14 years ago

I don't have any numbers, but I switched back to the old version because "next" was so slow.

nickfisher11• 14 years ago

Performance is definitely the biggest problem with fat client applications. As Twitter found out, by crowdsourcing your hardware, you lose a lot of control of the user experience. If you have a 'traditional' web application and it's running slow, it's really easy to by a couple of new servers and the problem is fixed. We can't buy everyone a new computer (sorry!) and force them to use a recent version of Chrome, so it's a challenge.

In the places where we can control the performance, we do that very carefully: CDN loading of all assets, intelligent caching techniques, and of course, finding and removing bottlenecks in the code.

@jarcoal, would you mind letting me know some of your details which might affect performance (which country you're in, what browser, what speed is your computer)? Email me directly at fisher at soundcloud if you'd prefer not to share here. Thanks.

antidaily• 14 years ago

And then you use a popup for the signup box?

bascule• 14 years ago

So they used Ember's templating language and did a crappy, more verbose reimplementation of Ember's subviews? Why did they choose Backbone over Ember, exactly?

Can I propose a corrolary to Greenspun's Tenth Rule?

"Any sufficiently complicated Backbone program contains an ad hoc, informally-specified, bug-ridden, slow implementation of half of Ember.js"

jashkenas2• 14 years ago

Nice corollary. Sounds about as accurate as Greenspun's Tenth as well. In good fun, mind if I propose one back 'atcha?

"Inside of every Ember.js application is a leaner, meaner, far faster, built-with-less-headaches Backbone.js app trying to get out."

How SoundCloud built its new single-page main website using Backbone.js

Related Stories