Facebook rewrites PHP runtime, will open source on Tuesday

http://www.sdtimes.com/blog/post/2010/01/30/Facebook-rewrites-PHP-runtime.aspx

by VonGuard • 16 years ago

163 98 16 years ago

Facebook has been rewriting the PHP virtual machine for the past 2 years in an effort to make it faster. On Tuesday, they will announce the release of this software as a new open source project.

senko• 16 years ago

> That team were forced to sign NDA's, and taken to a very quiet, secluded meeting room where some cool new Facebook-backed open source project was described.

This caught my eye - an interesting use of the term "open source" that I haven't previously been aware of. This is only a single datapoint (and the article states the project is going to be opened up anyways), but I do have a feeling the term has been diluted and joined the buzzword ranks.

jrockway5• 16 years ago

It's "open source" as in, "we'll release it as open source when we are good and ready". So far, that's been never.

VonGuard2• 16 years ago

Yeah, open source is kinda a verb now. Anyway, Facebook has other open source projects: see Hive http://www.facebook.com/pages/Hive/43928506208

Kinda silly to have a facebook page for an open source project, though.

matthew-wegner• 16 years ago

This was mentioned in that anonymous Facebook employee interview: http://news.ycombinator.com/item?id=1089800

Specifically:

Rumpus: So tell me about the engineers.

Employee: They’re weird, and smart as balls. For example, this guy right now is single-handedly rewriting, essentially, the entire site. Our site is coded, I’d say, 90% in PHP. All the front end — everything you see — is generated via a language called PHP. He is creating HPHP, Hyper-PHP, which means he’s literally rewriting the entire language. There’s this distinction in coding between a scripted language and a compiled language. PHP is an example of a scripted language. The computer or browser reads the program like a script, from top to bottom, and executes it in that order: anything you declare at the bottom cannot be referenced at the top. But with a compiled language, the program you write is compiled into an executable file. It doesn’t have to read the program from beginning to end in order to execute commands. It’s much faster that way. So this engineer is converting the site from one that runs on a scripted language to one that runs on a compiled language. However, if you went to go talk to him about basketball, you would probably have the most awkward conversation you’d have with a human being in your entire life. You just can’t talk to these people on a normal level. If you wanted to talk about basketball, talk about graph theory. Then he’d get it. And there’s a lot of people like that. But by golly, they can do their jobs.

jrockway5• 16 years ago

The computer or browser reads the program like a script, from top to bottom, and executes it in that order: anything you declare at the bottom cannot be referenced at the top. But with a compiled language, the program you write is compiled into an executable file. It doesn’t have to read the program from beginning to end in order to execute commands. It’s much faster that way.

I lost count of the number of untrue statements there after about ten.

olalonde3• 16 years ago

Wanna talk basketball?

pbiggar15• 16 years ago

Obviously, this isn't the traditional way of looking at compilation, but it does make sense.

I guess the problem being described, somewhat obtusely, is that massive amounts of dependencies (in Facebook's case million of lines of code) are loaded in order to execute a relatively small page.

I'll admit that to a CS person who understand compilers, their description makes it look like they don know what they're talking about. But if you explained Facebook's interpretation/compilation problems to a non-techy, it would probably come out sounding similar.

(Note comment reuse: http://www.reddit.com/r/programming/comments/aodzg/one_faceb...)

pbiggar15• 16 years ago

Hmmmm. There's no way you can name ten. I'd wager you can't name five, and I'd be surprised if you could name two.

sounddust• 16 years ago

The person being interviewed was obviously a non-technical person who was trying to provide information as he/she understood. It's easy to just take every statement literally and proclaim the person an idiot, but it's almost as easy to translate these statements into their (obvious) accurate equivalents and get some value from this interview. I don't understand the point of the former approach.

w1ntermute• 16 years ago

Your link is for this story, not for the quote. Here's the correct source: http://therumpus.net/2010/01/conversations-about-the-interne...

quickpost• 16 years ago

Thanks! I kept clicking the above link and ending up right back in the same place!

Time for a break.

carson• 16 years ago

If this is true and it is just a compiled version of PHP how much better is that going to be than APC or eAccelerator? There must be more to it than just compiled PHP.

wvenable• 16 years ago

The idea is probably to compile PHP into machine code rather than byte code.

tlrobinson• 16 years ago

[cringe]

This person knows just enough to be dangerous.

scorxn• 16 years ago

Is there an implication that all this optimization could be merged into PHP core? PHP's appeal is ubiquitous support. Even if it is open-sourced, I can't imagine a bunch of hosting companies suddenly serving Facebook-flavored PHP.

pbiggar15• 16 years ago

The PHP internals developers heaped scorn on compilation when I talked to them about phc (http://phpcompiler.org). I think it might be different with a Facebook seal of approval though.

jerf• 16 years ago

Should the current internal developers object to the idea even when implemented, well, "a new set of PHP internals developers" would not be the worst thing that ever happened to PHP....

jganetsk• 16 years ago

Wouldn't it have been easier to have moved away from PHP? From what I understand, Facebook only uses PHP for the most front of ends. Business logic is all in other languages. Isn't it easy to port the front-end into something else?

olalonde3• 16 years ago

What other languages? (genuine question)

Zev• 16 years ago

Erlang, C++, Java, Python and possibly others that I can't quite remember. See http://www.infoq.com/presentations/Facebook-Software-Stack for actual details.

adrianwaj• 16 years ago

Reddit was rewritten from Lisp to Python: http://www.aaronsw.com/weblog/rewritingreddit

Twitter from Ruby into Scala and JVM: http://www.artima.com/scalazine/articles/twitter_on_scala.ht...

I think there are two aspects of a language (correct me if I am wrong) -- it's elegance/simplicity/breadth of code produced by programmers that write with it --- and then the efficiency/effectiveness/HW-optimized executable code produced by its compiler once it's run: so how does PHP and Facebook fit in with all this?

dylanz• 16 years ago

Yeah, I had the same question. The PHP syntax definitely puts me off. That said, it's just the front-end. Might as well keep on keepin' on with what you're using, and since it's open source, just rewrite the thing.

Deep down inside I wish they were using ERB (or HAML) however, and he was writing HERB (or HHAML). I'm still enamored with Ruby syntax, and wish it would get some more big business love.

indigoviolet2• 16 years ago

No.

varaon• 16 years ago

Note: This user has worked for Facebook for two years.

n8agrin• 16 years ago

The article is completely unsubstantiated.

Well, I was able to put all the pieces together on this one, finally, and I now understand exactly what is up: Facebook has rewritten the PHP runtime from scratch.

Look if it's true, this is cool, it will no doubt be a great contribution to the OS world, but let's wait until Tuesday or until we have more concrete info beyond this author's guess that FB has completely rewritten the PHP runtime. I realize the author has little to gain from making this up, except for maybe 15 min of fame on HN, but still, don't believe everything you read.

timdorr3• 16 years ago

This is particularly interesting because PHP's runtime was rewritten for 5.1 back in late-2005 (on top of 5.0's Zend Engine 2.0 improvements in mid-2004).

And is PHP really considered "pokey"? Sounds like this guy is making stuff up because I get execution times of >0.01 seconds on my micro-framework.

nir2• 16 years ago

Well, with quotes like "it is simply not a language designed for the sorts of workloads that Java and .NET are" I'm not sure of the author's technical level.

Also, his comparison of Zend's "folks" (which ones? Founders? Execs? Sales people?) to "gestapo officer looking for a spy: "What? Who said that? Who said it was slow? Tell us their name!"" isn't too apt - they'd probably like to know who complained about PHP performance, so they can offer them their products/services, rather than silence them somehow.

lanstein2• 16 years ago

Probably the same people who decided the backslash was the correct character to use as a namespace separator.

tapostrophemo• 16 years ago

I'm curious; is this (http://timdorr.com/archives/2005/12/pgf-the-ease-of.php) the micro-framework of which you speak?

timdorr3• 16 years ago

Nope, that's an older version of it: http://github.com/timdorr/asoworx

pbiggar15• 16 years ago

PHP is dirt slow. When you look at the implementations of Lua or Python (which have approximately the same design as PHP), its about 4-5 times slower. Note this is only in the interpreter -- lots of the library code is written in C, which makes the difference somewhat less relevant.

The implication is that writing code that uses PHP's built-in libraries is pretty fast, but the more you write in PHP itself, the slower it gets. For example, my impression is that Yahoo isnt really written in PHP - its written in C patched together using PHP.

The Zend engine was "rewritten" for PHP 4, PHP 5, and to a certain extent for PHP 5.1. I guess it wasn't a complete rewrite because the legacy code from about 10 years ago is still in there. Anyway, its still dirty, badly written, slow, and very very badly commented. Hacks abound (and not the good kind).

zmimon• 16 years ago

This is a point I always have trouble impressing on people. They do a simple benchmarks on a tiny code base that pulls some data from the database, spits out some data, and they compare and PHP seems to be lightning fast.

The problem is that because it's interpreted at some point PHP slows down in proportion to the size of your code base. A small app runs really fast, a big app with 100,000 lines of code will kill your server unless you modularize it really really well - which harder than it seems, because the more modularized you make it the more separate "includes" you end up with in different files and then you come to realize that including a lot of files itself is a problem. And the nature of PHP's very loose coupling tends to lead to code that is nearly impossible to do large scale refactoring on once you have gone too far down the path.

I work on an application that has a very thin PHP layer that performs some simple web services that are the back end for a pure Java web app. Amazingly, when we load test it, the PHP part is the bottleneck, burning CPU like crazy just parsing all our files ... over ... and over ... and over. The java code meanwhile, while theoretically doing far more "work", is completely bored. We will probably look at using an accelerator of some kind or maybe just rewriting all the PHP in another language.

rgrove• 16 years ago

"For example, my impression is that Yahoo isnt really written in PHP - its written in C patched together using PHP."

Yahoo! is a company, not a single application. Not everything at Yahoo! is written in PHP, but the vast majority of Yahoo! properties do use PHP heavily on the frontend, and not just as a way of patching together C extensions.

pvg4• 16 years ago

its about 4-5 times slower.

Really? In what actual benchmarks, real world situations?

timdorr3• 16 years ago

Well, if you use APC then that's all irrelevant. It's the equivalent of creating .pyc files. I never got the impression it was slower in actual execution, though.

jws• 16 years ago

A quick stumble through the Language Shootout suggests using PHP instead of C might make you need, oh say 10 to 100 times as many servers. So there is room for improvement.

Of course if your workload is not dominated by script running CPU time then it doesn't really matter. Even then, until the cost for extra servers and their management exceeds the engineering cost to recode, add servers.

I get execution times of >0.01 seconds – hope you aren't planning to handle more than 50 transactions per second.

It all depends what kind of world you live in. I know people that use their hard disks for booting and loading caches and that's it. If one query in a hundred requires a seek they will not keep up. Most of the rest of us could happily fork a CGI PHP for each request and not notice.

igouy3• 16 years ago

I guess you stumbled past the places where the benchmarks game website links to "Overall Performance: PHP is rarely the bottleneck (HTML slides)" http://talks.php.net/show/drupal08/7

kaens• 16 years ago

I think that someone reimplementing PHP and cleaning up a lot of it's . . . quirks . . . could be a very good thing for the web. Someone who didn't care about maintaining backwards compatibility with code that relies on those . . . quirks . . . and who did care about language design, and clean implementation.

Unfortunately, I have this sinking feeling that this is not going to be that.

Facebook rewrites PHP runtime, will open source on Tuesday

Related Stories