HN user

invertednz

2 karma

jamesfarrier@gmail.com

Posts2
Comments8
View on HN

I used to work at a company with over 10,000 tests where we weren't able to get more than an 80% pass rate due to flaky tests. This article is great and covers a lot of the options for handling flaky tests. I founded Appsurify to make it easy for companies to handle flaky tests, with minimal effort.

First, don't delete them, flaky tests are still valuable and can still find bugs. We also had the challenge where a lot of the 'flakiness' was not the test or the application's fault but was caused by 3rd party providers. Even at Google "Almost 16% of our tests have some level of flakiness associated with them!" - John Micco, so just writing tests that aren't flaky isn't always possible.

Appsurify automatically raises defects when tests fail, and if the failure reason looks to be 'flakiness' (based on failure type, when the failure occurred, the change being made, previous known flaky failures) then we raise the defect as a "flaky" defect. Teams can then have the build fail based only on new defects and prevent it from failing when there are flaky test results.

We also prioritize the tests, which causes fewer tests to be run which are more likely to fail due to a real defect, which also reduces the number of flaky test results.

Appsurify | Sales Co-founder | Remote

I have just launched a startup which uses AI to predict software defects, improve testing and reduce test automation failures. I'm looking for a co-founder who is sales focused. We have 1 customer (almost 6 figure contract), 2 others in sales negotiations and about 10 in the pipeline. We are also at the interview stage for a couple of well-regarded incubators. Appsurify.com or reach out to me at jamesfarrier@appsurify.com. Remote Ok.

Looking for a SDET/Test Engineer at Fiserv working on mobile development. Located in Auckland New Zealand, but our HR team will work with you to sort any visa requirements.

Also looking for any mobile or c# devs.

You can reach me at jamesfarrier@gmail.com

Thanks, James