The Siren Call of Automated Browser Testing

https://cugablog.wordpress.com/2016/01/21/the-siren-call-of-automated-browser-testing/

by jcuga • 11 years ago

18 6 11 years ago

cugablog.wordpress.com

jakozaur• 11 years ago

How can we solve that? Perhaps we need a automated browser testing that can be used by non-developers?

TheAndruu• 11 years ago

Selenium provides this capability with a tool that records where you click on your website and then it records that as a macro which acts as an automated test.

However, the issues remain the same as with any other ABT test. An async element might take a while to load due to latency, and isn't there by the time your test says to click a button, or browsers were updated and now render your CSS just differently enough to break the test.

Perhaps part of the challenge is there are so many things that can change and break the tests outside of the actual code.

lotyrin• 11 years ago

I wish more teams could get past the split between roles with QA and DEV, that the idea was as evangelized as DevOps is.

IF your browser test isn't written in high-level concepts (page objects at the very least), and if your developer making UI changes isn't getting immediate feedback to also update the library of test code he's affecting, yes, it'll be fragile and will suck up too much developer effort. (I'll stop short of asserting the opinion that the UI developer should be changing the tests' reflections of his UI components BEFORE making the changes themselves, but I don't think that position is entirely unreasonable)

It should not be hard to find a balance of engineering practice (reuse, modularity, etc.) that gets you a degree of maintainability that is economical (utility provided by the system exceeds that consumed) -- just like any other kind of code.

Anytime someone messes with an interface, they should have roughly one place to adjust things for their changes. Dashboard controller and views get completely rewritten app side? Dashboard page object (maybe) gets rewritten test side.

Things should be written in the first place for maintainability -- XPaths/selectors chosen carefully, using user-facing labels if possible, WITH collaboration from the UI side -- no stupid duct-tape shit because you don't have a good way to get at something (like "the fourth input following the second instance of the word 'Account'").

Like any kind of code subject to change, you must invest in engineering for robustness in the face of that change.

No, you can't have a mandate from pointy-hair types to pour unskilled, unplanned effort into a system or practice because it sounds nice and expect to break even, but no shit -- that has nothing at all to do with browser automation.

Depending on the economics of the team, product, even the code-illiterate guys running the selenium recorder in another timezone and sending in bug reports any time it complains have their place (for instance, in a year-long release cycle of mostly stable line of business software where there are few UI changes because they'd require big change-orders and rewriting all the documentation) -- you have to design your strategy for your product.

crdoconnor2• 11 years ago

I wish more teams could get past the split between roles with QA and DEV

QA and Product, too. A really good QA is the best person to write user stories with a customer because they have a natural ability to sniff out edge cases which they can then walk the customer through.

crdoconnor2• 11 years ago

The problem lies in the fact that ABTs are highly coupled with the UI’s HTML/DOM. ABTs can range from fairly brittle to extremely brittle. The less brittle ABTs typically use some sort of DOM traversal like XPath or CSS-style selectors which can tolerate some UI changes like colors and minor layout changes as long as the changes are via CSS only and don’t dramatically affect the DOM.

This is the reason I follow this pattern for automated testing - only use readable selector IDs (for individual elements) and only use readable CSS selectors (for items in a group):

https://hitchtest.readthedocs.org/en/latest/faq/why_just_htm...

This ends up leading to tests that look a bit like this:

  - Click: view-button
  - Verify text:
      item: first first_name
      text: Sanders

Where such readable ids or classes are missing I either add them myself or ask the front end to add them.

I've tried most other methods of automated testing too (selenium IDE, page object), and this one is the most straightforward and sensible IMO.

I would probably use this for BDD + ATDD too although I've yet to work on a project where that's been possible.

The time it takes to run the tests is often exacerbated by a lot of explicit delay statements in the test cases.

This is an antipattern. You shouldn't do it:

https://hitchtest.readthedocs.org/en/latest/glossary/sleep_o...

Unfortunately, avoiding in most frameworks requires the tester to write asynchronous multithreaded code (which is beyond most QA developers).

_greim_• 11 years ago

A less sensational title for this would be:

Thinking realistically about automated browser testing.

But great article none the less.

The Siren Call of Automated Browser Testing

Related Stories