Salad? Gimme Steak. (Moving off Cucumber for acceptance tests)

http://cmdrkeene.tumblr.com/post/1042767842/group-me-steak

by nakajima • 16 years ago

21 14 16 years ago

cmdrkeene.tumblr.com

smartocci• 16 years ago

yum... steak

etagwerker• 16 years ago

It's interesting. Are there any performance improvements?

nakajima• 16 years ago

It's definitely faster, though I don't have any actual numbers (sorry about that). Developer performance has increased significantly though, since we we're no longer stuck in "step-state" hell.

oomkiller• 16 years ago

I like Steak, but the main reason for using Cucumber in the first place is to interface with your customers, or at least with people that understand the business side of things on a piece of software. Steak doesn't aim to, and won't replace that.

benbjohnson• 16 years ago

I have a couple issues with Steak:

1. It seems to take something that's already pretty straightfoward (RSpec) and just adds a couple keywords.

2. You're writing code when you should be describing features. Trying to remember where to put an underscore or what should be a symbol makes it harder to focus on describing the feature.

3. It greek to any non-developer. If you tried e-mailing this to a product owner or a user they would ask why you're sending code samples.

I know you addressed #3 in your blog post but why did you decide to use Steak over RSpec alone? Do you have any issue ramping up new developers with the Steak syntax?

cmdrkeene3• 16 years ago

To your points:

1) Agreed. It's simple. But, it does some nice things like 1) hook up capybara 2) create a /spec/acceptance folder and suite hook and 3) add some "behavior-oriented" keywords like feature, background and scenario. Of course, these are just dumb blocks and yes you could use the provided RSpec DSL (describe, it, before, etc) but it helps (if only mildly) to indicate that this is an acceptance test and to maybe put the programmer more in that mindset. This isn't groundbreaking, it's just a nice convention in my opinion.

2) This is a fair criticism. It does put the programmer more in a "code-centric" mindset than a "behavior-centric" one.

To address this I usually write out the feature in comments exactly as I would in cucumber. This might sound like an argument against steak, but it's not. The most useful thing about cucumber to me is the pattern of thinking: "Given this, When I do something, Then I should see that". The trouble is in the details: the maintenance, the awkward step gymnastics, etc. If you write out a pseudo-feature in comments then under each line of comments write the code, it totally remove the layer of feature vs. step definition indirection that plagues large cucumber projects.

3) Totally agreed. Though, very few projects I've used cucumber on ever let customers see feature files, let alone write them. Steak is for programmers, not customers.

There is no issue ramping up because it's just rspec and capybara. This is exactly what's under the sheets of most cucumber (for rails) installs. Anyone who implements cucumber step definitions should be well versed.

ihumanable• 16 years ago

How much longer until Steak gets a nifty DSL so it looks like Cucumber again, maybe they'll call it Zucchini this time.

Ruby seems to be playing with fire a little bit here, similar to the insanity that happened with Java and XML.

Java programs can't handle some edge case, program becomes configurable with XML, huge win. Then XML goes way too far, people start disliking XML, Java falls from grace.

Ruby programs are somewhat cumbersome, but Ruby, being fairly awesome, allows you to define a DSL. People write DSLs that make tasks easier, huge win. Then there are too many DSLs, people get tired of trying to figure out _another_ (groan) new DSL to do something they now know how to do in ruby.

Both start with this simple idea, flavor of the month type thing, that in the beginning is a huge win. After a while though both burn out. I think the saving grace for ruby is that people will just reimplement the DSL as a more straightforward ruby library.

cmdrkeene3• 16 years ago

Steak doesn't want to be cucumber at all. It's plain ruby. The "DSL" aspects of the library are purely for legibility and structure. The keywords feature, background, scenario perform the exact same function as the RSpec keywords describe, before, and it respectively.

All steak does is provide a structure for acceptance tests and loop in a web automation driver (capybara).

Far less magical than parsing a the magic language of "Gherkin"

jamesbritt2• 16 years ago

"Ruby programs are somewhat cumbersome"

How so?

cageface2• 16 years ago

My strategy for dealing with all these ruby testing fads is to just keep on plugging away with good old Test::Unit and avoid the churn all together. It takes about two minutes to teach somebody how to read asserts and then you're done.

jamesbritt2• 16 years ago

" ... all these ruby testing fads "

What makes something a fad or not?

cageface2• 16 years ago

When everybody claiming it's the next big thing decides six months later that it's too cumbersome and complicated and must immediately be replaced with the next shiny new thing. The ruby community seems to suffer from an acute case of magpie syndrome.

momoro• 16 years ago

I've used cucumber for several years and love it. The author raises these issues:

a) Organization of steps in a huge project is hard.

b) Complex quoting is hard

c) Token links in e-mail are hard

d) Manual poking of background jobs is hard

As a Cucumber user I've run into some of these.

a) Step Organization: In large projects, this is indeed difficult. I would love an os x app that could search step definitions. On the other hand, Ack makes it fairly simple to find steps. Furthermore, you can always organize them into folders.

If you see cucumber steps as methods, the author is basically saying "having methods makes things hard because I don't know if someone has already written the method. Therefore, I have given up on methods and now write everything inline."

b) Quoting: Cucumber allows for multi-line step definitions

c): Token links in e-mail are hard: Not sure exactly what he's talking about. But check out http://github.com/bmabey/email-spec which I helped write / solves e-mail issues for cucumber.

d) I have a step that just says "And the system processes jobs." In this step, I just run all the delayed jobs. It's not that hard.

To be fair, I haven't looked at steak very much, and am just going off of this blog post.

cmdrkeene3• 16 years ago

To your issues:

1) Yeah, organization is crazy. I think the method comparison is a bit unfair though. In theory, each step should be easily identifiable (like a method) but this is rarely the case given that most steps have complex regex matching (sometimes for grammatical niceness - e.g. (a vs. an), sometimes for other crazy town reasons). This makes not only automated indexing difficult (RubyMine tries...) but trying to keep hundreds of steps in your head is pretty hard. Especially one's you probably didn't write.

The cool thing about steak is that helpers and "reusable steps" are actually just methods! Hello ctags :D

b) Our case was:

  And the user "212-555-1234" should receive a text with:
    | Some "double quoted" and 'single quoted' text |

Definitely possible in cucumber, but I prefer a string equality assertion:

  text.should == "..." (you have to do this anyway in the step_definition)

c) Ben Maeby's email_spec is awesome and provides some steps for clicking on links in email. I still use the library in steak (just the raw rspec matchers though).

The trouble is you have a url:

  http://coolsite.org/something/abcde22424aw3324234

The matcher provided says:

I click on the link "Link Title or URL". Since the token is random every time (unless you stub it, which is kinda out of the spirit of acceptance testing, but that's another debate) it's hard to say "Click on the link with the big ass token in it".

You can definitely write a step like:

  And I visit the account confirmation link

Again, a matter of preference for abstracting the step with one-time use, highly coupled text to code vs. a comment and the code together. The actual mechanics are the same in both systems, it's just that cucumber hides them in a one-time use step away from the actual feature whereas the steak is inline.

d) Similar to the above, the catch all step can work in cucumber. Though, in a few scenarios, it's desirable to run only specific jobs or more often in our case, to check if jobs are scheduled in a specific way. It avoids the weird scenario of writing a step definition like "And I run only the blah jobs" or "And a blah job should be scheduled with ...". Ideally the job system would be decoupled entirely from the acceptance test, but we can't all do cool stuff like this all the time: http://corner.squareup.com/2010/08/cucumber-and-resque.html

Salad? Gimme Steak. (Moving off Cucumber for acceptance tests)

Related Stories