Introduction

We will agree that flaky tests make software development a challenging affair. Such tests add non-determinism and uncertainty to our test suite.

“Non-deterministic tests have two problems, firstly they are useless, secondly they are a virulent infection that can completely ruin your entire test suite.” — Martin Fowler Eradicating Non-Determinism in Tests

Although we find it frustrating, it is inevitable that as a codebase grows and becomes more complicated, we get some flaky tests. Be it because of simple reasons as bad setup/teardown or dependency on third party components to difficult to avoid situations such as infrastructure or concurrency.

Shoutout to Azure advent calendar

This blog post is published as part of Azure advent calendar. I would highly recommend a visit to the advent calendar and check out the other amazing topics published there. All videos on the calendar are at Azure Advent Calendar Youtube channel

Flaky test detection on Azure DevOps

If you have got the same issue as Fry then here’s a video explanation on how to use Azure DevOps to assist in identifying and reporting flaky tests.

Conclusion

If you have any thoughts or comments please do get in touch with me on twitter @rubberduckdev. Or use the disqus plugin below.

Credits

Flaky test banner by Bailey McGinn