Huxley, an app that helps fight CSS and UI regression

https://github.com/facebook/huxley

by kyledreger • 13 years ago

94 20 13 years ago

github.com

achalv• 13 years ago

Pretty awesome. I could definitely see myself using this.

faramarz2• 13 years ago

Is there a visual output? I'm not sure i understand this project

peterhunt5• 13 years ago

It writes a .huxley dir that contains screenshots and a log of user actions. It can replay these user actions programmatically and if the screen shots have changed it will save new ones to disk. You can use a tool like Kaleidoscope to compare the old ones with the new ones (we use git to keep the old ones around).

When you commit, the tests will be updated and the new screenshots will be part of the code review.

Additionally, you can run it in continuous integration mode which will break the build if screenshots are out of date.

abritishguy3• 13 years ago

Just to clarify, if I want to run this as part of travis then I should have the screenshots under source control and when I intentionally change the UI I should run it locally and push the new screenshot as part of the commit but when I don't intend to change the UI it will fail the build if something changes.

faramarz2• 13 years ago

That's quite amazing! Thanks for the write up! Will look into it

taitems• 13 years ago

For anyone looking for a non-Python way of doing this, PhantomCSS will achieve nearly all of this. The main difference between the two is that Huxley seems to listen and re-perform actions, instead of needing them to be scripted.

We've had PhantomCSS running on a living style-guide/style tile on check in and can even fail the build on TeamCity. It's resemble.js under the hood to do the pixel-by-pixel comparison and will spit out a percentage difference, fail screenshots etc.

EDIT: Link added https://github.com/Huddle/PhantomCSS

peterhunt5• 13 years ago

Just to clarify, it's written in Python but you don't have to write any Python code to use it. Just a teeny config file that contains a test name and a URL.

sync• 13 years ago

Looks quite awesome! It would be helpful to include some installation instructions for those of us unfamiliar with python libraries... do I use pip? do I use easy_install? do I git clone and then do something with setup.py?

peterhunt5• 13 years ago

I fixed this and actually put it on PyPI :) pip install huxley should do the trick.

nirvdrum• 13 years ago

Nifty project. It'd be nice to add in automated analysis and make it a full Web Consistency Testing tool (disclaimer: a project I work on a lot): http://webconsistencytesting.com/

The three major problems with screenshot based tools are: 1) hard to adapt to acceptable levels of element shifting; 2) hard to extract semantic understanding of what broke; and 3) if it's not automated and you test a lot of pages in a lot of browsers, they simply won't get compared. But adapting this shouldn't be too hard.

hkmurakami• 13 years ago

Totally thought this would be named after Aldous Huxley, the author of Brave New World, but the answer was quite cool anyways:

So we named it after the guy who coined the term "agnostic", Thomas Henry Huxley.

aphexairlines• 13 years ago

Why Resemble.js instead of http://pdiff.sourceforge.net/ ? Here's a presentation from a Google team using pdiff for front-end continuous deployment: http://www.youtube.com/watch?v=UMnZiTL0tUc

bslatkin• 13 years ago

The project for that is here: https://github.com/bslatkin/dpxdt

cburgmer• 13 years ago

Here is a project that gathers all available solutions around CSS testing: http://csste.st

abritishguy3• 13 years ago

Is there a way to test responsive designs with this? Can you set the device metrics?

peterhunt5• 13 years ago

Like window size? You can from the Python API, I just didn't include the option in the Huxleyfile. I've opened an issue here https://github.com/facebook/huxley/issues/2

Pixel ratio, I'm afraid, would be quite difficult.

ds_• 13 years ago

Would anyone pay for this as a service? i.e. you enter a bunch of urls and screen sizes, then get sent nice emails with request stats, screenshots and pdiffs with easy integration into your deployment workflow.

abritishguy3• 13 years ago

I'd just integrate it into travis.

andreasklinger• 13 years ago

You could try to address "non-dev people" with this kind of tool.

"Alerts when your shop breaks" Basically CI but on the live system (with a testuser) and daily report emails and instant alerts.

Huxley, an app that helps fight CSS and UI regression

Related Stories