Reproducing Test-Comp Results

A description on how we make the results of the competition reproducible can be found in the competition report; the components that are listed below in the table are described in Sect. 4 "Reproducibility".

Here is the list of all the components that were used for executing the competition:


Component Repository Release Tag
Test-Generator Archives testcomp21
Test-Generation Tasks testcomp21
Benchmark Definitions testcomp21
Tool-Info Modules 3.6
Test-Suite Format testcomp21
BenchExec 3.6

Reproduction Archives

Content Reproduction Archives
Test-Generation Tasks doi:10.5281/zenodo.4459132
Competition Results doi:10.5281/zenodo.4459470
Test Suites (Witnesses) doi:10.5281/zenodo.4459466
BenchExec doi:10.5281/zenodo.4317433

Participating Teams

Tool Lang. Jury member Affiliation Archive Bechmark definition Presentation Publication
Overview Chair Dirk Beyer LMU Munich, Germany
CMA-ES Fuzz C Gidon Ernst LMU Munich, Germany Download cmaesfuzz.xml
CoVeriTest C Marie-Christine Jakobs TU Darmstadt, Germany Download coveritest.xml
FuSeBMC C Kaled Alshmrany University of Manchester, UK Download FuSeBMC.xml
HybridTiger C Sebastian Ruland TU Darmstadt, Germany Download hybridtiger.xml
KLEE C Martin Nowack Imperial College London, UK Download klee.xml
Legion C Dongge Liu University of Melbourne, Australia Download legion.xml
LibKluzzer C Hoang M. Le University of Bremen, Germany Download libkluzzer.xml
PRTest C Thomas Lemberger LMU Munich, Germany Download prtest.xml
Symbiotic C Marek Chalupa Masaryk University, Brno, Czechia Download symbiotic.xml
TracerX C Joxan Jaffar National University of Singapore, Singapore Download tracerx.xml
VeriFuzz C Raveendra Kumar Medicherla Tata Consultancy Services, India Download verifuzz.xml


Validator Contact Affiliation Archive Benchmark definition Presentation Publication
TestVal Test-Suite Validator Thomas Lemberger LMU Munich, Germany Download testcov-validate-test-suites.xml