Combining minimization and generation for combinatorial testing

Itai Segall, Rachel Tzoref-Brill, Aviad Zlotnick

Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review

Abstract

Combinatorial Test Design (CTD) is an effective test planning technique that reveals faults resulting from feature interactions in a system. The standard application of CTD requires manual modeling of the test space, including a precise definition of restrictions between the test space parameters, and produces a test suite that corresponds to new test cases to be implemented from scratch. Interaction-based Test-Suite Minimization (ITSM) is a complementary approach to standard CTD, which reduces a given test suite without impacting its coverage of feature interactions. ITSM requires much less modeling effort, and does not require a definition of restrictions or generation of new test data. On the other hand, it does not improve the coverage obtained by the given test suite. In this work, we introduce Minimization Generation CTD (MG-CTD). MG-CTD is a combination of CTD with ITSM for addressing situations in which CTD is impractical, and ITSM is insufficient. In MG-CTD, one can define a subset of the parameters that can be freely assigned, as in CTD. The other parameter combinations must be selected from an existing set, as in ITSM. MG-CTD is suitable when for some parts of the test space it is easy to specify restrictions and generate new test data, while for others it is not. MG-CTD can be viewed as an enhancement of ITSM, and always achieves better interaction coverage than ITSM. We discuss the trade-offs between CTD, ITSM and MG-CTD, and present an efficient implementation which is based on binary decision diagrams. We then present some of the measures that one should take when implementing such an approach, in order to achieve the best possible coverage in the final result. Finally, we demonstrate MG-CTD on three real-life case studies.

Original languageEnglish
Title of host publication2015 IEEE 8th International Conference on Software Testing, Verification and Validation Workshops, ICSTW 2015 - Proceedings
PublisherInstitute of Electrical and Electronics Engineers Inc.
ISBN (Electronic)9781479918850
DOIs
StatePublished - 13 May 2015
Externally publishedYes
Event2015 8th IEEE International Conference on Software Testing, Verification and Validation Workshops, ICSTW 2015 - Graz, Austria
Duration: 13 Apr 201517 Apr 2015

Publication series

Name2015 IEEE 8th International Conference on Software Testing, Verification and Validation Workshops, ICSTW 2015 - Proceedings

Conference

Conference2015 8th IEEE International Conference on Software Testing, Verification and Validation Workshops, ICSTW 2015
Country/TerritoryAustria
CityGraz
Period13/04/1517/04/15

Fingerprint

Dive into the research topics of 'Combining minimization and generation for combinatorial testing'. Together they form a unique fingerprint.

Cite this