Rolling your own CDN for $25 in 1 hour

http://www.scalescale.com/rolling-your-own-cdn-build-a-3-continent-cdn-for-25-in-1-hour/

by mxpxrocks10 • 12 years ago

324 89 12 years ago

kbar13• 12 years ago

rolling your CDN on VPS instances isn't exactly cost-effective, and is probably more of an exercise for fun than anything else. Unless you need features out of your CDN that you can't get from existing CDN providers, I don't really see the point.

also kind of unusual that this blog post is by the president of MaxCDN.

mxpxrocks1017• 12 years ago

hey. thanks for the feedback. Yeah, it's more of an exercise for fun but it can also be done w/ high bandwidth dedicated servers as well. It's good as well if it's heavily dynamic stuff that needs really low latency and you need to execute something custom on the edge. Otherwise MaxCDN.com or a similar service scales better. I'm looking forward to building more custom stuff into our edges at MaxCDN.com so feel free to throw out any ideas. This was used to create a presentation for a docker meetup.

pushrax3• 12 years ago

Actually, with DigitalOcean, you get quite a lot of bandwidth for the price. Additionally, their transfer limits are high (1TB for the $5 instances), and overage is only $0.02/GB. If you combine this with something like Route 53's anycast latency-routed DNS, it can work very well.

michaelmior• 12 years ago

Maybe it can work. I'm still a little skeptical. But I don't see why you would ever want to deal with managing all this infrastructure yourself when there are many established providers that can likely do a much better job at a comparable price.

eli• 12 years ago

If you have realtively low usage, Amazon CloudFront is already pretty cheap.

chrisan2• 12 years ago

with DigitalOcean they ask you do not run a CDN: https://www.digitalocean.com/community/questions/do-you-have...

Oculus2• 12 years ago

kind of unusual that this blog post is by the president of MaxCDN

It makes a lot of sense for the president of MaxCDN to make a blog post like this. Bob the developer decides to learn about how CDN's work and happens upon this blog post. Cool, Bob just built his own mini CDN network for fun. Now Bob understands that his CDN, though a fun weekend project, is definitely not production ready. What does Bob do when he needs a CDN he can put into production? He goes to the guys that established themselves as authorities on the topic: MaxCDN.

Content Marketing 101: http://www.kalzumeus.com/2012/02/09/why-i-dont-host-my-own-b...

corobo• 12 years ago

Exactly this. I'm likely to need a CDN in the near-to-distant future. MaxCDN is now on my list because of this post

fmela• 12 years ago

Thanks for sharing. Note that there's a typo in the link to Ewan's github profile (should start with https, not ttps).

mxpxrocks1017• 12 years ago

thanks. will get it fixed shortly. this wasn't really scheduled for prime time but started taking off. appreciate the feedback!

TheBiv2• 12 years ago

"This wasn't really scheduled for prime time, but started taking off"

:) You submitted the link to HN and that comment makes it seem like the post accidentally took off!

petercooper• 12 years ago

Also wanted to note the post doesn't have enough Ron Smooth

mxpxrocks1017• 12 years ago

got this fixed. thanks for the heads up.

akerl_• 12 years ago

I'm more than a little bit concerned that the article doesn't point out the serious risk of running a single DNS server for the records.

I get that they're trying to keep things simple for their demo, but unsuspecting folks following this guide are in for quite a shock when their GeoDNS is taken out by a single physical server fault.

mxpxrocks1017• 12 years ago

Good point - it seemed obvious. We'll cover different options for that in a future blog post.

mxpxrocks1017• 12 years ago

added this to the article in the "Todo" section. thanks for the feedback.

Lennie• 12 years ago

In similar efforts, I also really enjoyed the slides from talk at DNS OARC 2014 about Anycast on a shoe string:

http://www.slideshare.net/natmorris/anycast-on-a-shoe-string

Oculus2• 12 years ago

Cool article for understanding the basics of a CDN. What sort of page load improvement can one get by adding geographically distributed servers?

mxpxrocks1017• 12 years ago

it can be quite a lot. you can test using a CDN or a setup like this and the tools over at www.webpagetest.org (from Patrick Meehan - who is awesome) from different locations.

xxdesmus5• 12 years ago

I'd recommend CloudFlare if you want AnyCast geo-DNS, and a free CDN. We don't charge for bandwidth or our DNS service.

Disclosure: yes, I work at CloudFlare but the previous details are simply facts.

kawsper• 12 years ago

Do you have a link to CloudFlare geo-DNS? I have multiple servers in multiple parts of the world, and didn't know that Cloudflare supported that.

xxdesmus5• 12 years ago

CloudFlare uses AnyCast. We'll route any incoming requests to the nearest data center automatically. http://blog.cloudflare.com/a-brief-anycast-primer No additional configuration required.

If you're referring to making different origin calls based on the geographic region -- such as when we go back to your origin for a request hitting our LAX data center if you happen to have an origin near LAX we'd use that origin specifically versus your other origin which might be located in let's say LHR. We currently only support that functionality for our higher end enterprise contracts. The functionality will likely to be available for other plan levels in the future though.

pyvpx3• 12 years ago

before you gleefully skip into the panacea geo-DNS has been marketed as, please take a quick skim of this 2009 paper[1] from microsoft research about mobile clients and geolocation.

has it gotten better in five years? is an exercise left for the reader :)

[1] http://research.microsoft.com/en-us/um/people/maheshba/paper...

Goopplesoft• 12 years ago

This geo-DNS is related to finding Cloudflare's servers, not finding your servers. I love and use Cloudflare but their support for Route 53 like multi-host configurations (health checks, latency, etc) is pretty bad (I think they only support roundrobin cnames).

tszming• 12 years ago

Hi, I heard that CloudFlare is now blocked in China, do you have more information?

xxdesmus5• 12 years ago

CloudFlare is not blocked in China. A handful of IPs could be potentially, but if you're seeing something indicating that I'd recommend contacting our support team.

The GFC (Great Firewall of China) also changes daily (or sometimes hourly) so it's entirely possible whatever may have been an issue at one point in time has already been addressed.

fakeyfake• 12 years ago

Reminds me of this article about hosting a CDN on digitalocean for the dash documentation software.

http://blog.kapeli.com/a-poor-mans-cdn

mxpxrocks1017• 12 years ago

cool - thanks

lingben2• 12 years ago

or you know, if your concern is cost, go with cdn.net and pay as you go for a kickass global cdn network

full disclosure, I have no business or personal connection to them

Kudos2• 12 years ago

There's 6TB of bandwidth built into this home rolled solution. That's going to cost at least $350 at cdn.net.

lingben2• 12 years ago

you're comparing apples to oranges

natch• 12 years ago

Nice. For tutorials like this it would be nice to see more details on the "how" of each step. For example:

"We setup DNS info to point here (both NS1 and NS2..."

That's great, but it would be nice to see:

"Using the geodns web-based configuration tool, we setup DNS info to point here (both NS1 and NS2..."

"Running <insert full command line here including tool and all options used here> on <insert environment here>, we setup DNS info to point here (both NS1 and NS2..."

"By editing /etc/<file> on the <some environment>, we setup DNS info to point here (both NS1 and NS2..."

In other words specifics are always really great to include.

But still this was a really great writeup, not just for the overview it gives of one lightweight approach but for the comments it has kicked off.

mxpxrocks1017• 12 years ago

great points. happy to take this approach on future posts and would love to collaborate in the future.

akg_672• 12 years ago

I really liked your write-up. As natch mentioned, it will be great to include more details in the article. I like the detail level of Digital Ocean articles.

It seems based on my recent trial experience with CloudFlare that both CDN startups might have this issue of incomplete/insufficient instructions and information for their users/customers to properly implement the solution and lot of assumptions about the skill level of users implementing the solution.

Actually, I landed up providing detailed instructions to CloudFlare on configuration and testing of their service. And a list of questions, answers to which should be readily available if CF is targeting enterprise customers.

joesavage3• 12 years ago

I was planning to publish a post with literally the same title as this one later this week (annoying coincidence as I guess mine will now be largely ignored - maybe I should wait a few weeks to let this space calm down, I'm not sure) with some in-depth step-by-step explanation - so watch out for that :)

wzy2• 12 years ago

How would I go about getting a link for the site you are going to publish article to?

This article title piqued my interest but in the end it lack some key pieces of info I would need to try.

acoyfellow• 12 years ago

I think Genius and DigitalOcean = a match made in heaven. The quality would go from "the best" to "untouchable", if the comments were value adding. Imagine being able to dive deeper into any piece of the tutorial you wanted.

tmaly2• 12 years ago

not to sound like a n00b, but what is Genius in your context?

fred_durst• 12 years ago

This comment is a model of quality feedback. It shows appreciation, not just shallow bookending, and gives detailed improvement examples. I know this is off topic a bit, sorry.

davidcelis• 12 years ago

Step 1: Order a DNS instance from Digital Ocean.

This also feels lacking to me. How does one order a DNS instance from DO? I can tell from the icon that it has something to do with CentOS but I don't see any sort of DNS server under their available applications or images for a one-click install.

jpmonette• 12 years ago

Definitely nice to see a real-world example of how to use Docker.

mxpxrocks1017• 12 years ago

woot woot.

ColinCera• 12 years ago

I'm wondering if it would be useful to plug groupcache into this. It's what Google uses for serving downloads (e.g., Chrome downloads). Inherently distributed and self-balancing. I might give that a try.

mxpxrocks1017• 12 years ago

great idea.

xin3td2• 12 years ago

Looks promising, but might need to add Doozer to the stack as groupcache has an issue with maintaining a list of peers where the cached data is distributed. Definitely worth looking into though.

https://github.com/ha/doozerd

danielrhodes3• 12 years ago

I would highly recommend Varnish using S3 as a backend origin. Add to that a few 100TB.com dedicated server and you have an extremely cheap CDN with pretty decent bandwidth and minimal maintenance.

CDNs have the advantage that they control more of the stack so can do more precise routing and have more edge nodes in more places. However, under the right circumstances, you can take the above quite far before a real CDN becomes necessary.

mxpxrocks1017• 12 years ago

hey Daniel- thanks for the comment. Do you run any special varnish config settings w/ s3?

danielrhodes3• 12 years ago

Nothing fancy right now. It's all static files. I think if I were doing caching of a dynamic application, some special settings would be far more valuable.

RKearney• 12 years ago

100TB.com specifically forbids using their service to create a CDN in their ToS[0].

    9. Acceptable Use/Illegal Activity

    d. We strive to maintain a high level of service, and a lot of
    customers depend on our high standards of quality. As such, we
    will not provide Services to those that are using our Services
    for:
    
    vii Using the Services for a content delivery network or content
    distribution network (CDN). An authorized CDN network offered
    through 100TB is accepted. Special requests to use the Services
    to run an unauthorized CDN network may be approved on a case-
    by-case basis. Failure to comply with this policy will result
    in termination of this TOS, and you will not receive a refund
    of the Fees.

[0]: http://www.100tb.com/tos.php

ddorian43• 12 years ago

people forget the story of simplecdn

nly2• 12 years ago

There's a fundamental difference between reselling CDN services, and running a caching web proxy as part of your own infrastructure for your own use.

AdamN• 12 years ago

Now with libcloud (http://libcloud.readthedocs.org/en/latest/compute/drivers/) you can use multiple platforms in order to get all continents represented.

Including Africa (http://kili.io) where I'm at.

mxpxrocks1017• 12 years ago

awesome. Thanks for posting. I had that as one of my todo's in the blog post. Feel free to reach out to me if you want to help with a follow up post!

muriithi• 12 years ago

Why Africa and not Kenya?

mxpxrocks1017• 12 years ago

has anyone ever heard about these guys in South Africa: http://www.teraco.co.za/? Any other recommendations for Africa and South Africa?

hiddenpearls• 12 years ago

that's awesome +Chris

edoceo• 12 years ago

This is similar to the process I used to build http://gcdn.org/ which I and a few clients + friends use on multiple projects. We used to use RR but GeoDNS has proven to be awesome and reliable. Full NS control makes it easier to do rolling upgrades or maintenance too. Excellent write up.

More notes on GeoDNS http://edoceo.com/howto/geodns

umsm• 12 years ago

This returns a 404-not found for me:

https://gcdn.org/jquery/1.8.0/jquery.js

adarshr• 12 years ago

Why is this page blocked by my UK work network?

Kudos2• 12 years ago

The people doing the blocking can probably better answer that.

leapius• 12 years ago

Excellent tut!

ksec• 12 years ago

Wouldn't it be better on Linode? Since they have wider selection of PoP. But i wonder if the port speed would be a problem.

Rolling your own CDN for $25 in 1 hour

Related Stories