10 Do’s & Don’ts of Test Automation

Our experience - compactly summarized

Mag. Siegfried Ollinger, representative of our customer Wiener Städtische Versicherung, has summarized what you should consider not only in setting up, but also in the further development of your software test automation. Enriched with the experience our experts at Qualysoft, we present you the 10 Do's and Don’t’s of automated testing.

The quality assurance of complex IT systems is very important in the insurance industry - not only must existing systems be kept fault-free, but must also be further developed in ever-faster cycles to support new business models. For this purpose, Wiener Städtische Versicherung is using test automation and, together with Qualysoft exoerts, implemented the Tricentis Tosca Testsuite in a pilot project for the claims department.

In his best practice lecture at the Business Breakfast powerd by Gartner, Mag. Siegfried Ollinger, Project Manager at Wiener Städtische Versicherung, presented his findings with the title "10 Dos and Don'ts of Test Automation". Enriched with the experience of our testing experts, we have summarized the most important facts for you:

1. Do: Know what you get!

In practice, companies that measure the success of their test automation only in terms of the highest possible number of test cases, are often confronted with redundancies of more than 60 % in their test portfolio and a risk coverage of less than 40 %. Thus, time and resources are bound without any significant success being recorded.

Plan the use of automated testing strategically and set measurable targets that are geared towards risk coverage. Even with a small number of properly installed test cases, you can achieve a very high risk level.

2. Don’t: Raise unrealistic expectations

Even after your test architecture and cases are established, automated testing is not running itself. Your portfolio needs to be maintained, new requirements have to be included and implemented. At Wiener Städtische, the number of test cases were reduced, while at the same time risk coverage increased. The customer benefits from the use of more Tosca functions: with Tosca's module-based approach, maintenance can be kept to a minimum.

3. Do: Set priorities – have the right test cases!

In practice, functions that are easy to test are often covered by a large number of tests without taking the contribution to quality assurance into account. This leads to a high test number in certain areas, while other, possibly more important functions, are badly tested - so there is an overall low risk coverage.

Also, make sure that you do not create redundancies in your portfolio and that you use a sophisticated methodology to define test cases. The combinational method of linear expansion supports the generation of test cases at the push of a button. They set "smooth runners" and then test them with all possible variants. To maximize the risk profile of each test case.

4. Don’t: Use sprawling expenses for test data generation

The basis for successful automated testing is a professional test data management, in which data required for tests are provided and managed by a tool. At Wiener Städtische Versicherung, the effort to prepare test cases was massively reduced by consistently planning, generating, providing and managing test data.

5. Do: Employ specialists and use appropriate tools

Establishing knowledge and skills in one's own company is essential to remain successful in the long term. Wiener Städtische decided to evaluate testing with professional support from Qualysoft and Tricentis and to train their own team with the help of experts in practice. The teams developed sets of machine test cases that were immediately applied. In doing so, the insurance team benefited from the know-how transfer and the transfer of input-output ratio "benchmarks" and will be able to support, maintain and further develop the project in the future.

6. Don’t: Make maintenance complicated

To ensure that the risk coverage does not decrease over time, and that the results are of high significance, the test cases must be kept up to date and maintained. Factors such as the neglect of test portfolio management, the uneven distribution of test cases via processes and applications as well as redundancies make maintenance more difficult and time-consuming. If your software testing is already professional from the outset, you can concentrate on the essentials in daily operation.

7. Do: Support learning and networking

Using the selected methodology, business users can be integrated into the definition of test cases in a time-saving and resource-conserving manner. In the course of the test definition, business processes are presented transparently across all areas and clearly separated between specialist and technical topics. This allows IT and technical departments to work together more efficiently and quickly.

8. Don’t: Set areas of application too restrictive

Think beyond individual applications and test end-to-end processes. Modern test methods allow business processes to be examined beyond application limits – not only to test an isolate single application but to implement a whole test case across all applications.

9. Do: Communicate results and successes

Automatic software testing can be compared with insurances - if everything works smoothly, people don’t talk about it – only when a mistake has happened, the issue draws the attention. Therefore it is important to present meaningful key figures in the company and to communicate successes proactively in order to keep the commitment high.

10. Don’t: Neglect further development

You should constantly develop and prepare your test case portfolio for new requirements. So you can ensure that the risk coverage remains high and that the functions of your applications work smoothly in the long term – even if you want to develop and implement innovations quickly.

The result: Professional test automation
for agile software development

With this professional approach, Wiener Städtische Versicherung has achieved remarkable success:

  • All test cases were re-established together with the department and a design coverage of 97 percent was achieved,
  • the automation coverage of the test case sets is now 82 percent,
  • the test cases are spread across processes and applications and are based on the highest possible risk contribution,

just to name a few examples.

With a strategic setup for automated software testing, the right tool and well-trained employees, Wiener Städtische updates features quickly and ensures a consistent quality assurance in IT-delivery.

Benefit from test automation and adapt your IT systems flexible to new business processes so that you can move with the high speed of digital transformation. Our experts will be glad to provide you with a detailed, non-binding consultation!

Contact us and talk to our experts!

Contact us and discuss our strategy for automated software testing in a free consultation with our specialists.

Read more

Related News

Qualysoft is ACT Framework-Partner

15.12.2019 Wien
Qualysoft is ACT Framework-Partner
Qualysoft is partner of the ACT framework. The network aims to close the gap between procedural models such as SAFe or LESS with special regard to continuous testing. Based on the experience of the cooperating companies, guidelines for successful…

bpm'online is now Creatio

30.10.2019 Wien
bpm'online is now Creatio
Since October 30th 2019, bpm’online is renamed to Creatio. With the change in the name the company is following its vision to enable everyone to become a developer, building his own unique creations with the help of its products. The renaming was…