Task filter


FuSeBMC 2020-12-12 23:21:24 CET FuSeBMC.Test-Comp21_coverage-branches.ReachSafety-ECA

Add filter for:

CMA-ES Fuzz 2020-12-13 03:33:00 CET cmaesfuzz.Test-Comp21_coverage-branches.ReachSafety-ECA

Add filter for:

CoVeriTest 2020-12-12 23:21:46 CET coveritest.Test-Comp21_coverage-branches.ReachSafety-ECA

Add filter for:

CPAchecker 2020-12-13 11:24:40 CET hybridtiger.Test-Comp21_coverage-branches.ReachSafety-ECA

Add filter for:

KLEE 2020-12-13 07:36:55 CET klee.Test-Comp21_coverage-branches.ReachSafety-ECA

Add filter for:

Legion 2020-12-13 15:46:08 CET legion.Test-Comp21_coverage-branches.ReachSafety-ECA

Add filter for:

LibKluzzer 2020-12-13 15:28:01 CET libkluzzer.Test-Comp21_coverage-branches.ReachSafety-ECA

Add filter for:

PRTest 2020-12-12 23:14:55 CET prtest.Test-Comp21_coverage-branches.ReachSafety-ECA

Add filter for:

symbiotic 2020-12-13 12:43:53 CET symbiotic.Test-Comp21_coverage-branches.ReachSafety-ECA

Add filter for:

Tracer-X 2020-12-14 20:56:36 CET tracerx.Test-Comp21_coverage-branches.ReachSafety-ECA

Add filter for:

VeriFuzz 2020-12-13 18:29:59 CET verifuzz.Test-Comp21_coverage-branches.ReachSafety-ECA

Add filter for:

Summary Table Quantile Plot Scatter Plot Info

Benchmark Setup

BenchmarkFuSeBMC CMA-ES Fuzz CPA/CoVeriTest HybridTiger KLEE Legion LibKluzzer PRtest Symbiotic Tracer-X VeriFuzz
ToolFuSeBMC v.3.6.6 CMA-ES Fuzz CoVeriTest 2.0.1-svn-eda176372c+ CPAchecker 1.9.2-svn-35606M KLEE 2.2-pre-test-comp Legion LibKluzzer 1.0 PRTest v2.2 symbiotic 8.0.0-pre-49dd01a3 Tracer-X 1.2.0 VeriFuzz 1.1.0
Limitstimelimit: 900 s, memlimit: 15000 MB, CPU core limit: 8 timelimit: 900 s, memlimit: 15000 MB, CPU core limit: 1 timelimit: 900 s, memlimit: 15000 MB, CPU core limit: 8
Hostapollon*
OSLinux 5.4.0-56-generic
SystemCPU: Intel Xeon E3-1230 v5 @ 3.40 GHz, cores: 8, frequency: 3800 MHz, Turbo Boost: disabled; RAM: 33493 MB
Date of execution2020-12-12 23:21:24 CET 2020-12-13 03:33:00 CET 2020-12-12 23:21:46 CET 2020-12-13 11:24:40 CET 2020-12-13 07:36:55 CET 2020-12-13 15:46:08 CET 2020-12-13 15:28:01 CET 2020-12-12 23:14:55 CET 2020-12-13 12:43:53 CET 2020-12-14 20:56:36 CET 2020-12-13 18:29:59 CET
Run setFuSeBMC.Test-Comp21_coverage-branches.ReachSafety-ECA cmaesfuzz.Test-Comp21_coverage-branches.ReachSafety-ECA coveritest.Test-Comp21_coverage-branches.ReachSafety-ECA hybridtiger.Test-Comp21_coverage-branches.ReachSafety-ECA klee.Test-Comp21_coverage-branches.ReachSafety-ECA legion.Test-Comp21_coverage-branches.ReachSafety-ECA libkluzzer.Test-Comp21_coverage-branches.ReachSafety-ECA prtest.Test-Comp21_coverage-branches.ReachSafety-ECA symbiotic.Test-Comp21_coverage-branches.ReachSafety-ECA tracerx.Test-Comp21_coverage-branches.ReachSafety-ECA verifuzz.Test-Comp21_coverage-branches.ReachSafety-ECA
Options
  • -s incr
  • -st real
  • -xml
  • -benchmark
  • -heap 10000M
  • -testcomp21
  • -setprop log.consoleLevel=SEVERE
  • -benchmark
  • -heap 10000M
  • -tigertestcomp20
  • --score uct
  • --core 8
  • --tree-depth-limit 100000
  • --conex-timeout 0
  • --symex-timeout 0
  • --min-samples 1
  • --save-tests REDUCED
  • --persistent
  • --time-penalty 0
  • --random-seed 0
  • -o ./binary.instr
  • --compile svcomp
  • --coverage-only
  • --test-comp
  • --testcomp
Propertiescoverage-branches

Statistics

FuSeBMC 2020-12-12 23:21:24 CET FuSeBMC.Test-Comp21_coverage-branches.ReachSafety-ECA
CMA-ES Fuzz 2020-12-13 03:33:00 CET cmaesfuzz.Test-Comp21_coverage-branches.ReachSafety-ECA
CoVeriTest 2020-12-12 23:21:46 CET coveritest.Test-Comp21_coverage-branches.ReachSafety-ECA
CPAchecker 2020-12-13 11:24:40 CET hybridtiger.Test-Comp21_coverage-branches.ReachSafety-ECA
KLEE 2020-12-13 07:36:55 CET klee.Test-Comp21_coverage-branches.ReachSafety-ECA
Legion 2020-12-13 15:46:08 CET legion.Test-Comp21_coverage-branches.ReachSafety-ECA
LibKluzzer 2020-12-13 15:28:01 CET libkluzzer.Test-Comp21_coverage-branches.ReachSafety-ECA
PRTest 2020-12-12 23:14:55 CET prtest.Test-Comp21_coverage-branches.ReachSafety-ECA
symbiotic 2020-12-13 12:43:53 CET symbiotic.Test-Comp21_coverage-branches.ReachSafety-ECA
Tracer-X 2020-12-14 20:56:36 CET tracerx.Test-Comp21_coverage-branches.ReachSafety-ECA
VeriFuzz 2020-12-13 18:29:59 CET verifuzz.Test-Comp21_coverage-branches.ReachSafety-ECA
Click here to select columns
status
raw score
cpu
(s)
mem
(MB)
energy
(J)
status
raw score
cpu
(s)
mem
(MB)
status
raw score
cpu
(s)
mem
(MB)
energy
(J)
status
raw score
cpu
(s)
mem
(MB)
energy
(J)
status
raw score
cpu
(s)
mem
(MB)
energy
(J)
status
raw score
cpu
(s)
mem
(MB)
energy
(J)
status
raw score
cpu
(s)
mem
(MB)
energy
(J)
status
raw score
cpu
(s)
mem
(MB)
energy
(J)
status
raw score
cpu
(s)
mem
(MB)
energy
(J)
status
raw score
cpu
(s)
mem
(MB)
energy
(J)
status
raw score
cpu
(s)
mem
(MB)
energy
(J)
total
29
4.52
25000
150000
290000
29
.156
23000
16000
29
6.00
24000
250000
230000
29
2.36
23000
250000
220000
29
7.34
26000
130000
230000
29
2.75
19000
360000
190000
29
10.1
26000
62000
250000
29
2.08
26000
7800
380000
29
9.89
22000
380000
170000
29
7.43
15000
120000
120000
29
11.7
25000
44000
320000
    correct results
0
-
-
-
-
0
-
-
-
0
-
-
-
-
0
-
-
-
-
0
-
-
-
-
0
-
-
-
-
0
-
-
-
-
0
-
-
-
-
0
-
-
-
-
6
2.91
4300
25000
32000
0
-
-
-
-
        correct true
0
-
-
-
-
0
-
-
-
0
-
-
-
-
0
-
-
-
-
0
-
-
-
-
0
-
-
-
-
0
-
-
-
-
0
-
-
-
-
0
-
-
-
-
0
-
-
-
-
0
-
-
-
-
        correct false
0
-
-
-
-
0
-
-
-
0
-
-
-
-
0
-
-
-
-
0
-
-
-
-
0
-
-
-
-
0
-
-
-
-
0
-
-
-
-
0
-
-
-
-
6
2.91
4300
25000
32000
0
-
-
-
-
    incorrect results
0
-
-
-
-
0
-
-
-
0
-
-
-
-
0
-
-
-
-
0
-
-
-
-
0
-
-
-
-
0
-
-
-
-
0
-
-
-
-
0
-
-
-
-
0
-
-
-
-
0
-
-
-
-
        incorrect true
0
-
-
-
-
0
-
-
-
0
-
-
-
-
0
-
-
-
-
0
-
-
-
-
0
-
-
-
-
0
-
-
-
-
0
-
-
-
-
0
-
-
-
-
0
-
-
-
-
0
-
-
-
-
        incorrect false
0
-
-
-
-
0
-
-
-
0
-
-
-
-
0
-
-
-
-
0
-
-
-
-
0
-
-
-
-
0
-
-
-
-
0
-
-
-
-
0
-
-
-
-
0
-
-
-
-
0
-
-
-
-
Loading...

Generated by BenchExec 3.6