Avaliação empírica da geração automatizada de testes de software sob a perspectiva de test smells

Abstract

The constant search for quality is always highlighted in the area of Software Engineering. Among the various disciplines dedicated to this topic, software testing has established itself as one of the most important, given its effectiveness in identifying defects, prior to the release of software systems to the market. Software testing is a key activity for developing quality software. However, developing tests is as costly or more expensive than developing production code. An alternative to reducing costs associated with software testing is the intensive use of test automation tools. The purpose of these tools is to reduce production time without affecting code quality. Despite this premise, it is not common to find approaches that include a quality check layer for automatically generated tests, which can reduce the reliability of the effectiveness of these tests. In this scenario, the purpose of this dissertation is to empirically analyze masses of test data, from the perspective of test smells, in order to evaluate the quality of tests produced by automated software test generation tools. Test smells are bad choices in test design and have symptomatic characteristics and can lead to a decrease in the quality of systems. Considering test smells in test code, the study analyzes tests generated by two tools widely accepted by the testing community: Evosuite and Randoop. A set of twenty-one open source software projects available on the Github platform were considered in the study. The analysis considered the dispersion of test smells in the test code of these projects, as well as the existence of potential correlations between test smells and relationships with structural metrics. As main results, we found strong correlations between test smells and code coverage metrics, significant differences between the data found in the automatically generated test suites and with pre-existing tests in the evaluated projects.