Open-Sourcing Traffic Server: 700k lines of code, 9 months

http://developer.yahoo.net/blog/archives/2010/02/opensourcing_traffic_server_700k_lines_of_code_9_months.html?utm_source=feedburner&utm_medium=feed&utm_campaign=Feed%3A+YDNBlog+%28Yahoo%21+Developer+Network+Blog%29&utm_content=Google+Reader

by peter123 • 16 years ago

37 16 16 years ago

developer.yahoo.net

vgurgov• 16 years ago

Definitely worth looking at - will spend my weekend on that, but so far not sure if(and where?) this thing can find own niche in popular deployment architectures.

zwoop2• 16 years ago

If you do, come join #traffic-server on freenode, the "installation" is still a little rough in the edges.

superted• 16 years ago

Any takes on how this compares to, for instance, Squid?

tlrobinson• 16 years ago

LOC is not something you should brag about. That said, it looks like the open sourced version (300K) is about on par with Squid and Apache:

nginx 75,214

lighttpd 84,852

apache 294,002

squid 283,417

(according to ohloh.com)

axod2• 16 years ago

It's worrying just how bloated and large many simple web servers get. Those are some seriously large LOC counts.

pquerna• 16 years ago

the vast majority of apache's are in bundled optional modules, not the core.

I am not sure where ohloh is getting its numbers, but here are the directory stats from slocount, for the recent httpd 2.3.5-alpha:

  SLOC    Directory       SLOC-by-Language (Sorted)
  95386   modules         ansic=95095,lex=191,yacc=100
  27977   server          ansic=27977
  11362   build           sh=9980,awk=657,perl=433,pascal=292
  7248    support         ansic=7076,perl=92,sh=80
  3250    include         ansic=3250
  1622    os              ansic=1622
  1094    test            ansic=843,sh=230,perl=21
  159     top_dir         sh=159

If you were to strip out everything 'not needed' for a basic server, I expect you would end up 'running' about 40k LOC, which I think is 'inline' with others like varnish/lighttpd/etc

apache's problem isn't bloat, its architecture and bad sysadmins. it was designed from 1994 for prefork-based systems, and sysadmins have been configuring it since then, or not configuring it. Yes, it will use lots of ram when you embed Python and PHP in it, so don't do that.

mini-rant over, and a disclaimer: i am an httpd committer.

erikeldridge• 16 years ago

the line count wasn't intended as a brag. Yahoo! worked on cleaning up the source code for 9 mo in prep for open source release and in the process reduced the size from 700k to 300k

zwoop2• 16 years ago

Yeah, and we've just begun cleanup :). There's still a bunch that can go, there's plenty of stuff that's no longer necessary or supported, and over time, I'd expect that to go away entirely.

Also, the long term goal is to remove more from the TS "core", and move that out to plugins, so the core will be sleeker. Of course, we'll also add on new features, but hopefully many of those can be implemented as plugins as well.

forkqueue• 16 years ago

Will be interesting to see how this compare to the current king-of-caches, Varnish.

The plugin architecture looks like it'll make it pretty flexible, but it'll have to be pretty special to beat Varnish in terms of performance.

jspaw2• 16 years ago

(see my post above) :)

chokosabe• 16 years ago

They mentioned everything apart from the fact that its a web server. Also, have I got this wrong or did they just hint that it doesn't run on linux?

amock• 16 years ago

What makes you think it doesn't run on Linux? From the article it sounds like it only runs on Linux since they say "contributors outside of Yahoo! can choose to build the code on more modern, non-Linux OSes as they see fit."

jspaw2• 16 years ago

It's multithreaded, so can take advantage of more CPU mores, unlike squid.

It has total disk cache persistence, so it won't lose any cache if the server crashes, traffic server crashes, unlike varnish.

If you've got cache spread over multiple spindles and one of the spindles dies while in production, it'll just keep on going, skipping that spindle. Squid does this, varnish does not.

It does run on linux. Actually it's what it was built on, originally, IIRC.

I have used TrafficServer (at Yahoo) and I'm willing to bet that it will overtake both squid and varnish as time goes on.

siculars• 16 years ago

Kind of hard to find information out there... The OP link and links off that page don't lead anywhere useful. Googled around and came up with this:

http://cwiki.apache.org/confluence/display/TS/Traffic+Server

Which also yields:

http://wiki.apache.org/incubator/TrafficServerProposal

http://incubator.apache.org/projects/trafficserver.html

The Apache "champion" is Doug Cutting, of Hadoop fame, so it seems to have street cred.

Open-Sourcing Traffic Server: 700k lines of code, 9 months

Related Stories