Reproducing Test-Comp Results
A description on how we make the results of the competition reproducible can be found in the competition report; the components that are listed below in the table are described in Sect. 4 "Reproducibility".
Here is the list of all the components that were used for executing the competition:
Repositories
| Component | Repository | Release Tag |
| Test-Generator Archives | git@gitlab.com:sosy-lab/test-comp/archives-2021.git | testcomp21 |
| Test-Generation Tasks | git@github.com:sosy-lab/sv-benchmarks.git | testcomp21 |
| Benchmark Definitions | git@gitlab.com:sosy-lab/test-comp/bench-defs.git | testcomp21 |
| Tool-Info Modules | git@github.com:sosy-lab/benchexec.git | 3.6 |
| Test-Suite Format | git@gitlab.com:sosy-lab/software/test-format | testcomp21 |
| BenchExec | git@github.com:sosy-lab/benchexec.git | 3.6 |
Reproduction Archives
| Content | Reproduction Archives |
| Test-Generation Tasks | doi:10.5281/zenodo.4459132 |
| Competition Results | doi:10.5281/zenodo.4459470 |
| Test Suites (Witnesses) | doi:10.5281/zenodo.4459466 |
| BenchExec | doi:10.5281/zenodo.4317433 |
Participating Teams
| Tool | Lang. | Jury member | Affiliation | Archive | Bechmark definition | Presentation | Publication |
| Overview | Chair | Dirk Beyer | LMU Munich, Germany |
|
|||
| CMA-ES Fuzz | C | Gidon Ernst | LMU Munich, Germany | Download | cmaesfuzz.xml |
|
|
| CoVeriTest | C | Marie-Christine Jakobs | TU Darmstadt, Germany | Download | coveritest.xml |
|
|
| FuSeBMC | C | Kaled Alshmrany | University of Manchester, UK | Download | FuSeBMC.xml |
|
|
| HybridTiger | C | Sebastian Ruland | TU Darmstadt, Germany | Download | hybridtiger.xml |
|
|
| KLEE | C | Martin Nowack | Imperial College London, UK | Download | klee.xml |
|
|
| Legion | C | Dongge Liu | University of Melbourne, Australia | Download | legion.xml |
|
|
| LibKluzzer | C | Hoang M. Le | University of Bremen, Germany | Download | libkluzzer.xml |
|
|
| PRTest | C | Thomas Lemberger | LMU Munich, Germany | Download | prtest.xml |
|
|
| Symbiotic | C | Marek Chalupa | Masaryk University, Brno, Czechia | Download | symbiotic.xml |
|
|
| TracerX | C | Joxan Jaffar | National University of Singapore, Singapore | Download | tracerx.xml |
|
|
| VeriFuzz | C | Raveendra Kumar Medicherla | Tata Consultancy Services, India | Download | verifuzz.xml |
Validators
| Validator | Contact | Affiliation | Archive | Benchmark definition | Presentation | Publication |
| TestVal Test-Suite Validator | Thomas Lemberger | LMU Munich, Germany | Download | testcov-validate-test-suites.xml |
|
|