A large number of organizations that I talk to are giving this as THE goal when hiring Quality Assurance engineers.
This is a fantastic advance in the recognition and value of automated testing in comparison to just a few years ago.
My experience from working in several organizations is that this is typically interpreted, not surprisingly, as simply "write more tests".
The situation I have observed over and over again is that just 'write more tests" does not address the underlying issues. Looking at large and successful organization that release features and changes on a daily basis we can see that massive and slow test suites are not the way of success.
Using root cause analysis we can see that "more tests" is an approach to try and help with issues but it does not not tell us anything about the root issues we are trying to fix - which are always about bugs and the speed of making feature changes. Every organization I talk to at the moment says "we want to release software faster and with less issues than we are currently experiencing". In other words "more speed, less bugs".
It is important to look at the root cause of software with these problems that is being created without adequate test coverage. Without this, adding "the missing tests" without changing the development approach that created them, will lead to the automation falling further and further behind future development features being created without adequate test coverage and test coverage levels never rise. It will not fix the bugs being made or the speed to make feature changes.
It is also important to recognize that a mandate for "100% test coverage" does not resolve the problem because this will lead to test run times increasing over time and their usefulness as feedback to the developer writing the feature diminishes. This is addressed by having the right tests at the right level and curating them constantly. This requires a QA architect who is fully empowered in the organization as well as strong development leaders to guide junior engineers.
Finally tests themselves should be carefully crafted so that they themselves have high quality code (modular, English readable, DRY as needed, etc). Poorly written tests with high dependencies can themselves become "the problem" if they are not written and maintained with care by a person that cares.
THIS CAN BE FIXED!
In order to address the above issues the approach needed is to change how software is written. Instead of treating "QA" as "testing at the end", a new mindset is needed that focuses on "automated testing at all levels as the application code is written".
Under this model, when a feature is imagined, specified, designed and coded, testing is considered at all these stages, by all participants, and the phase where it is "coded" includes writing unit tests, integration tests and end-to-end tests. The very use of the word "testing" changes dramatically. Instead of "the thing that the QA group does that catches bugs" testing now becomes "the thing that developers create to make sure the software works today, tomorrow and in the future.". This is very hard. It can be helpful to use software techniques such as TDD and BDD but without the context that the role of testing is changing from a gating model to a dynamic feedback model, the organization will likely not make changes to how development operates within the organization. This will lead to a cycle of adding tests but then discovering different issues not tested for as the future cannot be predicted.
HERE ARE THE PLAN SPECIFICS TO INCREASE TEST COVERAGE!
1. Start measuring test coverage at the unit level.
Recommended by LinkedIn
Unit tests must be at the unit level itself. They must mock and stub ALL external resources including file, screen and database. Add test coverage minimum limits and raise them over time
2. Fast unit tests for application developers.
Unit tests for a piece of code being worked on should run in 1-3 seconds. Screen, file and database MUST be mocked and stubbed.
3. Extensive use of CI to run test suites.
Tests that are limited to run on one developers machine cannot scale and remain as fast feedback.
4. Address the current test debt.
There will be a backlog. A separate effort should address this. Write and fix the unit tests. Write e2e tests and make them reliable and fast. This requires Quality Automation Engineers with considerable experience such as myself
5. Pay Equity.
Organizations typically pay 'QA' folks at a lower rate. This is true across the industry. It leads to a lot of low quality code that quickly becomes a blocker and not a help. If you want the quality you seek, pay the people who do the quality work well.
6. Testing Resource Equity.
Many organization have development groups that consist of a Project Manager, several developers and one QA person. Given that QA automation frequently takes longer to write (especially initially) this sets up a 'guaranteed to fail' situation where the QA person remains a manual tester at the end of the process. The main fix to this is not to hire more QA folks but instead to retrain application engineers on new roles which place testing at the heart of application development.
