Task filter


FuSeBMC 2020-12-12 23:21:24 CET FuSeBMC.Test-Comp21_coverage-branches.ReachSafety-Combinations

Add filter for:

CMA-ES Fuzz 2020-12-13 03:33:00 CET cmaesfuzz.Test-Comp21_coverage-branches.ReachSafety-Combinations

Add filter for:

CoVeriTest 2020-12-12 23:21:46 CET coveritest.Test-Comp21_coverage-branches.ReachSafety-Combinations

Add filter for:

CPAchecker 2020-12-13 11:24:40 CET hybridtiger.Test-Comp21_coverage-branches.ReachSafety-Combinations

Add filter for:

KLEE 2020-12-13 07:36:55 CET klee.Test-Comp21_coverage-branches.ReachSafety-Combinations

Add filter for:

Legion 2020-12-13 15:46:08 CET legion.Test-Comp21_coverage-branches.ReachSafety-Combinations

Add filter for:

LibKluzzer 2020-12-13 15:28:01 CET libkluzzer.Test-Comp21_coverage-branches.ReachSafety-Combinations

Add filter for:

PRTest 2020-12-12 23:14:55 CET prtest.Test-Comp21_coverage-branches.ReachSafety-Combinations

Add filter for:

symbiotic 2020-12-13 12:43:53 CET symbiotic.Test-Comp21_coverage-branches.ReachSafety-Combinations

Add filter for:

Tracer-X 2020-12-14 20:56:36 CET tracerx.Test-Comp21_coverage-branches.ReachSafety-Combinations

Add filter for:

VeriFuzz 2020-12-13 18:29:59 CET verifuzz.Test-Comp21_coverage-branches.ReachSafety-Combinations

Add filter for:

Summary Table Quantile Plot Scatter Plot Info

Benchmark Setup

BenchmarkFuSeBMC CMA-ES Fuzz CPA/CoVeriTest HybridTiger KLEE Legion LibKluzzer PRtest Symbiotic Tracer-X VeriFuzz
ToolFuSeBMC v.3.6.6 CMA-ES Fuzz CoVeriTest 2.0.1-svn-eda176372c+ CPAchecker 1.9.2-svn-35606M KLEE 2.2-pre-test-comp Legion LibKluzzer 1.0 PRTest v2.2 symbiotic 8.0.0-pre-49dd01a3 Tracer-X 1.2.0 VeriFuzz 1.1.0
Limitstimelimit: 900 s, memlimit: 15000 MB, CPU core limit: 8 timelimit: 900 s, memlimit: 15000 MB, CPU core limit: 1 timelimit: 900 s, memlimit: 15000 MB, CPU core limit: 8
Hostapollon*
OSLinux 5.4.0-56-generic
SystemCPU: Intel Xeon E3-1230 v5 @ 3.40 GHz, cores: 8, frequency: 3800 MHz, Turbo Boost: disabled; RAM: 33493 MB
Date of execution2020-12-12 23:21:24 CET 2020-12-13 03:33:00 CET 2020-12-12 23:21:46 CET 2020-12-13 11:24:40 CET 2020-12-13 07:36:55 CET 2020-12-13 15:46:08 CET 2020-12-13 15:28:01 CET 2020-12-12 23:14:55 CET 2020-12-13 12:43:53 CET 2020-12-14 20:56:36 CET 2020-12-13 18:29:59 CET
Run setFuSeBMC.Test-Comp21_coverage-branches.ReachSafety-Combinations cmaesfuzz.Test-Comp21_coverage-branches.ReachSafety-Combinations coveritest.Test-Comp21_coverage-branches.ReachSafety-Combinations hybridtiger.Test-Comp21_coverage-branches.ReachSafety-Combinations klee.Test-Comp21_coverage-branches.ReachSafety-Combinations legion.Test-Comp21_coverage-branches.ReachSafety-Combinations libkluzzer.Test-Comp21_coverage-branches.ReachSafety-Combinations prtest.Test-Comp21_coverage-branches.ReachSafety-Combinations symbiotic.Test-Comp21_coverage-branches.ReachSafety-Combinations tracerx.Test-Comp21_coverage-branches.ReachSafety-Combinations verifuzz.Test-Comp21_coverage-branches.ReachSafety-Combinations
Options
  • -s incr
  • -st real
  • -xml
  • -benchmark
  • -heap 10000M
  • -testcomp21
  • -setprop log.consoleLevel=SEVERE
  • -benchmark
  • -heap 10000M
  • -tigertestcomp20
  • --score uct
  • --core 8
  • --tree-depth-limit 100000
  • --conex-timeout 0
  • --symex-timeout 0
  • --min-samples 1
  • --save-tests REDUCED
  • --persistent
  • --time-penalty 0
  • --random-seed 0
  • -o ./binary.instr
  • --compile svcomp
  • --coverage-only
  • --test-comp
  • --testcomp
Propertiescoverage-branches

Statistics

FuSeBMC 2020-12-12 23:21:24 CET FuSeBMC.Test-Comp21_coverage-branches.ReachSafety-Combinations
CMA-ES Fuzz 2020-12-13 03:33:00 CET cmaesfuzz.Test-Comp21_coverage-branches.ReachSafety-Combinations
CoVeriTest 2020-12-12 23:21:46 CET coveritest.Test-Comp21_coverage-branches.ReachSafety-Combinations
CPAchecker 2020-12-13 11:24:40 CET hybridtiger.Test-Comp21_coverage-branches.ReachSafety-Combinations
KLEE 2020-12-13 07:36:55 CET klee.Test-Comp21_coverage-branches.ReachSafety-Combinations
Legion 2020-12-13 15:46:08 CET legion.Test-Comp21_coverage-branches.ReachSafety-Combinations
LibKluzzer 2020-12-13 15:28:01 CET libkluzzer.Test-Comp21_coverage-branches.ReachSafety-Combinations
PRTest 2020-12-12 23:14:55 CET prtest.Test-Comp21_coverage-branches.ReachSafety-Combinations
symbiotic 2020-12-13 12:43:53 CET symbiotic.Test-Comp21_coverage-branches.ReachSafety-Combinations
Tracer-X 2020-12-14 20:56:36 CET tracerx.Test-Comp21_coverage-branches.ReachSafety-Combinations
VeriFuzz 2020-12-13 18:29:59 CET verifuzz.Test-Comp21_coverage-branches.ReachSafety-Combinations
Click here to select columns
status
raw score
cpu
(s)
mem
(MB)
energy
(J)
status
raw score
cpu
(s)
mem
(MB)
status
raw score
cpu
(s)
mem
(MB)
energy
(J)
status
raw score
cpu
(s)
mem
(MB)
energy
(J)
status
raw score
cpu
(s)
mem
(MB)
energy
(J)
status
raw score
cpu
(s)
mem
(MB)
energy
(J)
status
raw score
cpu
(s)
mem
(MB)
energy
(J)
status
raw score
cpu
(s)
mem
(MB)
energy
(J)
status
raw score
cpu
(s)
mem
(MB)
energy
(J)
status
raw score
cpu
(s)
mem
(MB)
energy
(J)
status
raw score
cpu
(s)
mem
(MB)
energy
(J)
total
210
14.5
180000
310000
2200000
210
.00
34000
23000
210
30.8
190000
700000
2200000
210
7.99
190000
1000000
2100000
210
81.9
190000
940000
2000000
210
17.9
110000
3200000
890000
210
139
190000
290000
1900000
210
1.59
190000
11000
2600000
210
135
170000
2200000
1800000
210
99.1
110000
1400000
1000000
210
180
190000
73000
2400000
    correct results
0
-
-
-
-
0
-
-
-
0
-
-
-
-
0
-
-
-
-
0
-
-
-
-
0
-
-
-
-
0
-
-
-
-
0
-
-
-
-
0
-
-
-
-
30
17.1
15000
130000
120000
0
-
-
-
-
        correct true
0
-
-
-
-
0
-
-
-
0
-
-
-
-
0
-
-
-
-
0
-
-
-
-
0
-
-
-
-
0
-
-
-
-
0
-
-
-
-
0
-
-
-
-
0
-
-
-
-
0
-
-
-
-
        correct false
0
-
-
-
-
0
-
-
-
0
-
-
-
-
0
-
-
-
-
0
-
-
-
-
0
-
-
-
-
0
-
-
-
-
0
-
-
-
-
0
-
-
-
-
30
17.1
15000
130000
120000
0
-
-
-
-
    incorrect results
0
-
-
-
-
0
-
-
-
0
-
-
-
-
0
-
-
-
-
0
-
-
-
-
0
-
-
-
-
0
-
-
-
-
0
-
-
-
-
0
-
-
-
-
0
-
-
-
-
0
-
-
-
-
        incorrect true
0
-
-
-
-
0
-
-
-
0
-
-
-
-
0
-
-
-
-
0
-
-
-
-
0
-
-
-
-
0
-
-
-
-
0
-
-
-
-
0
-
-
-
-
0
-
-
-
-
0
-
-
-
-
        incorrect false
0
-
-
-
-
0
-
-
-
0
-
-
-
-
0
-
-
-
-
0
-
-
-
-
0
-
-
-
-
0
-
-
-
-
0
-
-
-
-
0
-
-
-
-
0
-
-
-
-
0
-
-
-
-
Loading...

Generated by BenchExec 3.6