Task filter
FuSeBMC 2020-12-12 23:21:24 CET FuSeBMC.Test-Comp21_coverage-error-call.ReachSafety-Loops
Add filter for:
CMA-ES Fuzz 2020-12-13 03:33:00 CET cmaesfuzz.Test-Comp21_coverage-error-call.ReachSafety-Loops
Add filter for:
CoVeriTest 2020-12-12 23:21:46 CET coveritest.Test-Comp21_coverage-error-call.ReachSafety-Loops
Add filter for:
CPAchecker 2020-12-13 11:24:40 CET hybridtiger.Test-Comp21_coverage-error-call.ReachSafety-Loops
Add filter for:
KLEE 2020-12-13 07:36:55 CET klee.Test-Comp21_coverage-error-call.ReachSafety-Loops
Add filter for:
Legion 2020-12-13 15:46:08 CET legion.Test-Comp21_coverage-error-call.ReachSafety-Loops
Add filter for:
LibKluzzer 2020-12-13 15:28:01 CET libkluzzer.Test-Comp21_coverage-error-call.ReachSafety-Loops
Add filter for:
PRTest 2020-12-12 23:14:55 CET prtest.Test-Comp21_coverage-error-call.ReachSafety-Loops
Add filter for:
symbiotic 2020-12-13 12:43:53 CET symbiotic.Test-Comp21_coverage-error-call.ReachSafety-Loops
Add filter for:
Tracer-X 2020-12-14 20:56:36 CET tracerx.Test-Comp21_coverage-error-call.ReachSafety-Loops
Add filter for:
VeriFuzz 2020-12-13 18:29:59 CET verifuzz.Test-Comp21_coverage-error-call.ReachSafety-Loops
Add filter for:
Benchmark Setup
Benchmark | FuSeBMC | CMA-ES Fuzz | CPA/CoVeriTest | HybridTiger | KLEE | Legion | LibKluzzer | PRtest | Symbiotic | Tracer-X | VeriFuzz | ||||||||||||||||||||||||||||||||||||||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
Tool | FuSeBMC v.3.6.6 | CMA-ES Fuzz | CoVeriTest 2.0.1-svn-eda176372c+ | CPAchecker 1.9.2-svn-35606M | KLEE 2.2-pre-test-comp | Legion | LibKluzzer 1.0 | PRTest v2.2 | symbiotic 8.0.0-pre-49dd01a3 | Tracer-X 1.2.0 | VeriFuzz 1.1.0 | ||||||||||||||||||||||||||||||||||||||||||||
Limits | timelimit: 900 s, memlimit: 15000 MB, CPU core limit: 8 | timelimit: 900 s, memlimit: 15000 MB, CPU core limit: 1 | timelimit: 900 s, memlimit: 15000 MB, CPU core limit: 8 | ||||||||||||||||||||||||||||||||||||||||||||||||||||
Host | apollon* | ||||||||||||||||||||||||||||||||||||||||||||||||||||||
OS | Linux 5.4.0-56-generic | ||||||||||||||||||||||||||||||||||||||||||||||||||||||
System | CPU: Intel Xeon E3-1230 v5 @ 3.40 GHz, cores: 8, frequency: 3800 MHz, Turbo Boost: disabled; RAM: 33493 MB | ||||||||||||||||||||||||||||||||||||||||||||||||||||||
Date of execution | 2020-12-12 23:21:24 CET | 2020-12-13 03:33:00 CET | 2020-12-12 23:21:46 CET | 2020-12-13 11:24:40 CET | 2020-12-13 07:36:55 CET | 2020-12-13 15:46:08 CET | 2020-12-13 15:28:01 CET | 2020-12-12 23:14:55 CET | 2020-12-13 12:43:53 CET | 2020-12-14 20:56:36 CET | 2020-12-13 18:29:59 CET | ||||||||||||||||||||||||||||||||||||||||||||
Run set | FuSeBMC.Test-Comp21_coverage-error-call.ReachSafety-Loops | cmaesfuzz.Test-Comp21_coverage-error-call.ReachSafety-Loops | coveritest.Test-Comp21_coverage-error-call.ReachSafety-Loops | hybridtiger.Test-Comp21_coverage-error-call.ReachSafety-Loops | klee.Test-Comp21_coverage-error-call.ReachSafety-Loops | legion.Test-Comp21_coverage-error-call.ReachSafety-Loops | libkluzzer.Test-Comp21_coverage-error-call.ReachSafety-Loops | prtest.Test-Comp21_coverage-error-call.ReachSafety-Loops | symbiotic.Test-Comp21_coverage-error-call.ReachSafety-Loops | tracerx.Test-Comp21_coverage-error-call.ReachSafety-Loops | verifuzz.Test-Comp21_coverage-error-call.ReachSafety-Loops | ||||||||||||||||||||||||||||||||||||||||||||
Options |
|
|
|
|
|
|
| ||||||||||||||||||||||||||||||||||||||||||||||||
Properties | coverage-error-call |
Statistics
FuSeBMC 2020-12-12 23:21:24 CET FuSeBMC.Test-Comp21_coverage-error-call.ReachSafety-Loops
CMA-ES Fuzz 2020-12-13 03:33:00 CET cmaesfuzz.Test-Comp21_coverage-error-call.ReachSafety-Loops
CoVeriTest 2020-12-12 23:21:46 CET coveritest.Test-Comp21_coverage-error-call.ReachSafety-Loops
CPAchecker 2020-12-13 11:24:40 CET hybridtiger.Test-Comp21_coverage-error-call.ReachSafety-Loops
KLEE 2020-12-13 07:36:55 CET klee.Test-Comp21_coverage-error-call.ReachSafety-Loops
Legion 2020-12-13 15:46:08 CET legion.Test-Comp21_coverage-error-call.ReachSafety-Loops
LibKluzzer 2020-12-13 15:28:01 CET libkluzzer.Test-Comp21_coverage-error-call.ReachSafety-Loops
PRTest 2020-12-12 23:14:55 CET prtest.Test-Comp21_coverage-error-call.ReachSafety-Loops
symbiotic 2020-12-13 12:43:53 CET symbiotic.Test-Comp21_coverage-error-call.ReachSafety-Loops
Tracer-X 2020-12-14 20:56:36 CET tracerx.Test-Comp21_coverage-error-call.ReachSafety-Loops
VeriFuzz 2020-12-13 18:29:59 CET verifuzz.Test-Comp21_coverage-error-call.ReachSafety-Loops
Click here to select columns
status
raw score
cpu
(s)
(s)
mem
(MB)
(MB)
energy
(J)
(J)
status
cpu
(s)
(s)
mem
(MB)
(MB)
status
raw score
cpu
(s)
(s)
mem
(MB)
(MB)
energy
(J)
(J)
status
raw score
cpu
(s)
(s)
mem
(MB)
(MB)
energy
(J)
(J)
status
raw score
cpu
(s)
(s)
mem
(MB)
(MB)
energy
(J)
(J)
status
raw score
cpu
(s)
(s)
mem
(MB)
(MB)
energy
(J)
(J)
status
raw score
cpu
(s)
(s)
mem
(MB)
(MB)
energy
(J)
(J)
status
raw score
cpu
(s)
(s)
mem
(MB)
(MB)
energy
(J)
(J)
status
raw score
cpu
(s)
(s)
mem
(MB)
(MB)
energy
(J)
(J)
status
raw score
cpu
(s)
(s)
mem
(MB)
(MB)
energy
(J)
(J)
status
raw score
cpu
(s)
(s)
mem
(MB)
(MB)
energy
(J)
(J)
total
158
131
41000
81000
500000
158
64000
11000
158
35
99000
740000
1000000
158
53
92000
210000
1000000
158
96
39000
46000
530000
158
4
130000
160000
900000
158
138
140000
330000
1500000
158
102
140000
8200
1800000
158
82
41000
57000
530000
158
78
46000
110000
590000
158
136
14000
22000
170000
correct results
0
-
-
-
-
0
-
-
0
-
-
-
-
19
19
3400
14000
37000
96
96
3400
3300
45000
0
-
-
-
-
0
-
-
-
-
0
-
-
-
-
0
-
-
-
-
31
31
1400
3500
15000
0
-
-
-
-
correct true
0
-
-
-
-
0
-
-
0
-
-
-
-
19
19
3400
14000
37000
0
-
-
-
-
0
-
-
-
-
0
-
-
-
-
0
-
-
-
-
0
-
-
-
-
0
-
-
-
-
0
-
-
-
-
correct false
0
-
-
-
-
0
-
-
0
-
-
-
-
0
-
-
-
-
96
96
3400
3300
45000
0
-
-
-
-
0
-
-
-
-
0
-
-
-
-
0
-
-
-
-
31
31
1400
3500
15000
0
-
-
-
-
incorrect results
0
-
-
-
-
0
-
-
0
-
-
-
-
0
-
-
-
-
0
-
-
-
-
0
-
-
-
-
0
-
-
-
-
0
-
-
-
-
0
-
-
-
-
0
-
-
-
-
0
-
-
-
-
incorrect true
0
-
-
-
-
0
-
-
0
-
-
-
-
0
-
-
-
-
0
-
-
-
-
0
-
-
-
-
0
-
-
-
-
0
-
-
-
-
0
-
-
-
-
0
-
-
-
-
0
-
-
-
-
incorrect false
0
-
-
-
-
0
-
-
0
-
-
-
-
0
-
-
-
-
0
-
-
-
-
0
-
-
-
-
0
-
-
-
-
0
-
-
-
-
0
-
-
-
-
0
-
-
-
-
0
-
-
-
-
Loading...
Generated by BenchExec 3.6