Announcing cucumber-electron

https://cucumber.io/blog/2017/01/23/announcing-cucumber-electron

by theoengland • 9 years ago

76 29 9 years ago

yebyen6• 9 years ago

This sounds very interesting. I have been using cucumber in my development with Selenium WebDriver (cucumber-rails) and the author here has promised things that were missing in PhantomJS and Capybara-WebKit that drove me toward using Selenium and Chrome instead. Namely the option of running in either Headed mode with the inspector available, or Headless.

Does this project require your system-under-test to be a pure Node.Js application to work with it? (I am not sure if I had heard of electron, but it looks to be the case.) Is there any possibility of porting this to be a driver for test systems that use Cucumber for systems built in other languages?

joshski2• 9 years ago

Yes you can use it to describe and automate systems built in other technologies and I've done that on a couple of projects. It's a good fit for any web app with a lot of client-side code. But its sweet spot is definitely node.js apps.

You could certainly use electron as a driver for tests written in other tech, but there is no capybara electron driver that I'm aware of. Using cucumber-electron means you write your step definitions (automation code) in JavaScript.

(I'm the author of cucumber-electron)

yebyen6• 9 years ago

Thanks for showing up and answering questions!

I've gotten used to having my rails application (server) running in the same thread as my cucumber (client-side) steps, thanks to cucumber-rails. So I could see why this would be especially useful for testing node.js applications, with the same advantages. (A fault detected anywhere on either side of the test execution, client or server, can be escalated and reported to stop the entire test, delivering a really good, clear idea of where to look in the code for the violator!)

This looks like a really neat project.

stu_k• 9 years ago

The title is slightly misleading. This just looks like an alternative way to run Cucumber, not a complete alternative to Selenium.

yebyen6• 9 years ago

It didn't even occur to me that there were other ways of using Selenium than Selenium-WebDriver!

jgraham• 9 years ago

Well a key property of Selenium is that it's cross-browser. Whilst it's nice to imagine that the web is in a position where running all your tests in some version of Blink means that your site will work on Firefox/Edge/Safari, I fear that the reality is still rather different.

joshski2• 9 years ago

This is not an alternative way of using selenium. It's a way to avoid using selenium. The advantage over selenium is that tests and application code are run in the same process, so they tend to be faster, less brittle and easier to debug. The downside is that they are not cross-browser.

payne92• 9 years ago

Since you work for cucumber, a pro tip: put a summary of what the project is right up front on the Web page, assuming visitors know nothing.

What is cucumber? What is Electron?

For example: "cucumber-electron is a Javascript module (available via npm) that combines cucumber, a framework for writing Web page tests in plain language, with Electron, a framework for building desktop apps using Web technologies. ..."

cookiesboxcar• 9 years ago

whats the best headless browser for CI/CD? I've always used Phantom, but would be keen to explore alternatives (especially if it could quiet the "nobody uses phantom in real life")

cyrusroshan• 9 years ago

I write tests using chromedriver. Ends up being really simple to write, and if I want to debug something, it's just a chrome instance that I can use normally if I make the tests wait.

Only downside is you have to make it run with xvfb on the CI server, but that's pretty easy to set up in your config file usually.

SigmundA• 9 years ago

Is headless Chrome working yet?

https://chromium.googlesource.com/chromium/src/+/lkgr/headle...

mattfrasernz• 9 years ago

You can also run Chrome inside a docker container and drive it with the remote driver. Handy if you are running on an aws Linux instance with no X

mnutt• 9 years ago

While not actually a real-life browser, Phantom 2.5 is _much_ closer to a real-life browser engine than it was previously.

networked• 9 years ago

For me so far the most pleasant to work with has (contrary to its name) been Nightmare (http://www.nightmarejs.org/), which is a Node.js library that drives Electron. It has a really pleasant API and its developers claim that it is faster than PhantomJS. Unlike PhantomJS, however, it is not truly headless; it needs an X server like xvfb to run.

ape4• 9 years ago

Pretty soon no word will have its original meaning. An electron is not a subatomic particle. A cucumber isn't a veggie. Rust isn't...

rjeli• 9 years ago

Perhaps we should take a hint from Urbit and start naming our projects Zog-wip-let-bon, etc.

pjc50• 9 years ago

The pharmaceutical industry has an entire sub-industry of coming up with names - I believe there's an algorithm for producing phonetically reasonable names that are unique.

http://www.popsci.com/science/article/2013-04/fyi-how-does-d...

M_Grey• 9 years ago

I came here assuming some novel electron orbital geometry had been discovered. It's always something to do with physics and/or food... always.

seanwilson• 9 years ago

Can anyone explain why you would want to use Cucumber? I heard it enables non-programmers to write the tests themselves but I don't see that as realistic as non-programmers don't have the training to specify requirements rigourously. At least with coded tests you can use IDEs to validate and refactor your tests.

rukenshia• 9 years ago

We use cucumber at work for a few weeks now. Our product owner actually writes the initial feature files for our end-to-end tests and we refine them. We keep sending him back on what we changed and why, and the product owner learns from that and tries to do it the next time. We started doing this around 6 weeks ago and it is helping us developers a lot in my opinion, as well as getting some work off of us. Sure, we need to refine the POs feature definitions, but it's better than nothing.

jedifans3• 9 years ago

Before the feature files get written, have a conversation with the PO and other stakeholders.

Involve a QA person and relevant developers.

Catch the changes early, ask for more examples of the feature or goal, "have you thought about ...", "What happens in this case...?".

You will get better quality features with less rework, because you are writing the thing the PO actually wants and cover off edge cases before they crop up in testing.

yebyen6• 9 years ago

I can tell you why I'm using it, and it's not to enable non-programmers to write tests. (Not anymore, it was at first...)

The number one advantage is that using Cucumber produces human-readable documentation that is also executable, and through the magic of CI is prevented from becoming stale.

I use Cucumber with Rails, and I've actually heard a lot of really good arguments not to use Cucumber. The one thing these all seem to have in common is, they almost universally do NOT completely rebuke TDD at-large. Nobody (competent) is suggesting you should develop by the seat of your pants. They have issues with Gherkin or with Cucumber's implementation specifically, but if your team had success with Cucumber, you would not give it up without planning to have a replacement.

You may choose to eschew Gherkin style entirely, and you can certainly accomplish much the same thing using feature testing in modern RSpec and scenarios. This may be easier to refactor for you, but for me a well-organized vocabulary of step definitions makes it so easy to copy a scenario, reorganize the steps, and add some new things to create a new feature, adding almost no new test code.

There is also Spinach, which rips out all of Cucumber's regex silliness and declines to enforce a global vocabulary, instead encouraging modular design of scenarios and step definitions (since you might want to say the same thing in two different places and mean two different things.)

There is also Turnip, which integrates Gherkin into RSpec and also has many of the same advantages of Spinach, but includes other options (cleaner options) besides the Regex method for building parameterized step definitions. I think this one is my favorite, but my small project still uses Cucumber proper.

In the beginning of our project, it was very helpful that my PM was writing out requirements in Cucumber style. It meant that while someone (not a coder, but not an inexperienced PM either) had gone to the trouble to specify requirements with rigor, I could focus on coding, just jump in and start writing features, with a clear target of limited scope, that I could easily follow to avoid straying off into the weeds.

Before long my PM got busy with other projects and I took over writing the cucumber stories. Not just me, but the other dev too -- and that was just as well. I often found myself rewriting PM's scenarios anyway, to make them easier to map directly to step definitions, more clear in a context-free sense, or generally less ambiguously specified. These enhancements also made them less human-friendly.

Both approaches to writing Cucumber stories definitely have merit. The one thing you need to have is a strong set of test scenarios (however you've implemented them) with good total coverage of the features and the requirements, so that new or changed requirements can be evaluated in the context of the whole program: what features are broken by your change?

Two things I have found are also critical when using ATDD: your whole team (at least all of the devs) need to be able to run the tests, and (maybe even more important) you need to have a CI solution in place so that no individual needs to be responsible for running all of the tests after each change when your suite gets too heavy for a pre-commit hook. A regression suite is like a pager: when nobody is on-call then everybody is on-call.

jedifans3• 9 years ago

Try not to see a feature file as a way to automate tests but to capture the requirements in a language that isn't technical.

Business rules change far less often than implementations, so the feature files shouldn't be tied to the implementation - that tying together can happen in the ruby code that implement the steps. You can refactor that code easily but the feature is denoting the business rule.

jedifans3• 9 years ago

Cucumber and other tools such as Behat allow you to capture conversations you have with your client, product owner and stakeholders.

They enable you to build the right thing - the thing that helps the users or meets the business goals.

Have a look for BDD, Behaviour Driven Development.

The main thing is to have the conversations. The automated tests are a lovely side effect!

imranq• 9 years ago

wow I actually thought someone had discovered some ridiculous orbital pathway

twolfson• 9 years ago

I wrote something similar to this but 1 level higher by using Karma. It's called `karma-electron` and it allows you to use any test framework inside of Electron as well as any other launcher Karma supports (e.g. Firefox, Chrome, PhantomJS):

https://github.com/twolfson/karma-electron

Announcing cucumber-electron

Related Stories