PhantomJS - minimalistic headless WebKit

http://www.phantomjs.org/

by yakto • 15 years ago

96 20 15 years ago

phantomjs.org

chadcf• 15 years ago

A capybara driver for this would be fantastic...

vitovito7• 15 years ago

Note: not genuinely headless. It still requires you run an X server, such as Xvfb.

As far as I know, there's no genuine headless Webkit.

For a genuinely headless system, consider Chris Lord's "offscreen" branch of Gecko, discussed here: http://blog.mozilla.com/ted/2010/07/29/moz-headless-screensh...

I'd be interested in sponsoring someone who wanted to bring the offscreen branch up to mainline, and Chris would be willing to mentor you in such an effort.

newhouseb• 15 years ago

Yeah, I wanted something like this a couple years ago and was disappointed to find no true headless library. I spent a lot of time looking at WebCore, but eventually gave up after a while.

I wonder if there's a way to extricate some of Chrome's rendering logic to do genuinely headless stuff. I know that things are rendered in separate processes and then piped to the browser window process and I have to wonder if there's a way to launch one without the other, and thus, without any requirements for a display server like X.

vitovito7• 15 years ago

If you're okay with using Gecko instead of Webkit, the headless branch does compile into a standalone executable you can fork out to, it's what we do.

The issue with Gecko was less about rendering offscreen and more about rendering widgets (dropdowns, input boxes, etc.) which are normally handled by the windowing system.

wslh2• 15 years ago

I use htmlunit for scraping, it's a headless browser although using WebKit will be far better.

It's sad that WebKit lacks some easier integration (good COM/.NET object in Windows).

vitovito7• 15 years ago

Crowbar is a Gecko-based scraper, if you're interested in using a real browser: http://simile.mit.edu/wiki/Crowbar

ojbyrne• 15 years ago

I've used htmlunit as well, basically because it seems to be the most mature. I find it chokes on some google maps js.

maximilianburke• 15 years ago

EA's branches of WebKit, available at http://gpl.ea.com, are designed for integration into console video games and don't require any sort of X server or OS interface. It's not the most up-to-date but it's sufficiently modularized to adhere to the LGPL without violating the Sony/Microsoft developer agreements.

vitovito7• 15 years ago

This might be interesting. Remember to check all of the products listed, as we only release our latest EA WebKit version when a game ships with it.

"EA WebKit supports Win32, Win64, Playstation 3, and Xbox 360 platforms" according to the documentation, but perhaps the modularization and the EA Raster primitives gets you further along than some of the other forks.

If anyone has any questions about EA WebKit, let me know and I can pass them along.

EDIT: ...Or you can ask the parent poster, who actually works on them. :)

adamkhrona2• 15 years ago

We created a genuinely windowless branch of Chromium two years ago named Awesomium; it's free for non-commercial use and is used in several major games and commercial projects: http://www.awesomium.com

vitovito7• 15 years ago

"Runs on Windows and Mac OSX Intel platforms."

If you only run on Windows or OS X, it's windowless, but it's not headless.

Headless means something running on a command-line console, and you can't run either of those operating systems that way.

joshu2• 15 years ago

Can it be used to render pages? That's still something I desire.

yakto• 15 years ago

Yes. Either as screenshot images, or as full dom with readable text, even if text was js-written. That's the use-case that gets me hot-n-bothered, too.

Now if Google would use such a system for all their crawling, instead of forcing us ajax-app authors to implement their hackish hash-bang "solution." I'm guessing they would if they could, but the cpu cycles are just too expensive still.

vitovito7• 15 years ago

The offscreen Mozilla branch I linked to above can. We use it for our private bookmarking/archiving site. Our headless Gecko drives the thumbnails and previews on qumbler.com.

(I originally wrote "delicious/pinboard clone" instead of "bookmarking/archiving site" but I realized we've had it since before del.icio.us launched.)

joshu2• 15 years ago

really? delicious was 2003; muxway, the predecessor was in 2001.

giberson• 15 years ago

This is brilliant.

Not exactly an award winning comment, but it's literally what I thought when I was reading the quick start and got to the rendering section.

ubasu• 15 years ago

Previous discussion:

http://news.ycombinator.com/item?id=2142104

PhantomJS - minimalistic headless WebKit

Related Stories