Reproducing Test-Comp Results

A description on how we make the results of the competition reproducible can be found in the competition report; the components that are listed below in the table are described in Sect. 4 "Reproducibility".

An overview over all components that were uses in the competition can be found on the reproduction page.

In the below listings, we use superscript labels to indicate new tools new, inactive tools , and meta tools meta.

Tools for Test Generation

Tool Lang. Jury member Affiliation Archive Benchmark definition
AFL-to-Test-Casenew C Henrik Wachowitz LMU Munich, Germany 18060896 afltc.xml
cetfuzz C inactive 10207018 cetfuzz.xml
CoVeriTest C Marie-Christine Jakobs LMU Munich, Germany 17732590 coveritest.xml
ESBMC-incr C Chenfeng Wei University of Manchester, UK 17793335 esbmc-incr.xml
ESBMC-kind C Chenfeng Wei University of Manchester, UK 17793335 esbmc-kind.xml
FDSE C Zhenbang Chen National University of Defense Technology, China 17900875 fdse.xml
Fizzer C Marek Trtík Masaryk University, Brno, Czechia 17840184 fizzer.xml
FuSeBMC C Kaled Alshmrany Institute of Public Administration, Saudi Arabia 18051027 fusebmc.xml
FuSeBMC-AI C inactive 10199336 fusebmc-ia.xml
HybridTiger C inactive 10412240 hybridtiger.xml
KLEEF C inactive 14354572 kleef.xml
KLEE C inactive 10412702 klee.xml
Owi C, Wasm inactive 10213245 owic.xml
PRTest C Thomas Lemberger LMU Munich, Germany 17543506 prtest.xml
Rizzer C inactive 10202006 rizzer.xml
Sikraken C Christophe Meudec South East Technological University, Ireland 18062402 sikraken.xml
Symbiotic C Martin Jonáš Masaryk University, Brno, Czechia 17698724 symbiotic.xml
TracerX C inactive 14349664 tracerx.xml
TracerX-WP C inactive 14349699 tracerx-wp.xml
UTestGen C Max Barth LMU Munich, Germany 17792491 utestgen.xml
WASP-C C inactive 10412766 wasp-c.xml

Tools for Validation

Tool Lang. Format Jury member Affiliation Archive Benchmark definition
TestCoCanew C Clang Formatted Marek Trtík Masaryk University, Brno, Czechia 17819536 testcoca.xml
TestCov C Clang Formatted, Clang Unformatted, GCC Formatted, GCC Unformatted Matthias Kettl LMU Munich, Germany 17698774 testcov.xml