Task filter


CMA-ES Fuzz 2021-12-15 19:44:24 CET cmaesfuzz.Test-Comp22_coverage-branches.ReachSafety-ProductLines

Add filter for:

CPAchecker 2021-12-15 19:57:13 CET coveritest.Test-Comp22_coverage-branches.ReachSafety-ProductLines

Add filter for:

FuSeBMC 2021-12-16 03:43:34 CET FuSeBMC.Test-Comp22_coverage-branches.ReachSafety-ProductLines

Add filter for:

CPAchecker 2021-12-16 04:41:54 CET hybridtiger.Test-Comp22_coverage-branches.ReachSafety-ProductLines

Add filter for:

KLEE 2021-12-16 10:08:26 CET klee.Test-Comp22_coverage-branches.ReachSafety-ProductLines

Add filter for:

Legion 2021-12-16 15:28:39 CET legion.Test-Comp22_coverage-branches.ReachSafety-ProductLines

Add filter for:

Legion/SymCC 2021-12-16 13:27:02 CET legion-symcc.Test-Comp22_coverage-branches.ReachSafety-ProductLines

Add filter for:

LibKluzzer 2021-12-16 20:24:25 CET libkluzzer.Test-Comp22_coverage-branches.ReachSafety-ProductLines

Add filter for:

PRTest 2021-12-16 23:53:20 CET prtest.Test-Comp22_coverage-branches.ReachSafety-ProductLines

Add filter for:

symbiotic 2021-12-17 02:47:56 CET symbiotic.Test-Comp22_coverage-branches.ReachSafety-ProductLines

Add filter for:

Tracer-X 2021-12-16 00:12:46 CET tracerx.Test-Comp22_coverage-branches.ReachSafety-ProductLines

Add filter for:

VeriFuzz 2021-12-17 07:59:01 CET verifuzz.Test-Comp22_coverage-branches.ReachSafety-ProductLines

Add filter for:

Summary Table Quantile Plot Scatter Plot Info

Benchmark Setup

BenchmarkCMA-ES FuzzCPA/CoVeriTestFuSeBMCHybridTigerKLEELegionLegion/SymCCLibKluzzerPRtestSymbioticTracer-XVeriFuzz
ToolCMA-ES Fuzz CPAchecker 2.0.1-svn-38826FuSeBMC v.4.1.14CPAchecker 1.9.2-svn-35606MKLEE 2.2-pre-test-compLegion 1.0-testcomp2021Legion/SymCC testcomp2022LibKluzzer 1.0PRTest v2.2symbiotic 9.0.0-dev-71ddb4b1Tracer-X 1.2.0VeriFuzz 1.2.10
Limitstimelimit: 900 s, memlimit: 15000 MB, CPU core limit: 1timelimit: 900 s, memlimit: 15000 MB, CPU core limit: 8
HostUnknown
OSUnknown
SystemUnknown
Date of execution2021-12-15 19:44:24 CET2021-12-15 19:57:13 CET2021-12-16 03:43:34 CET2021-12-16 04:41:54 CET2021-12-16 10:08:26 CET2021-12-16 15:28:39 CET2021-12-16 13:27:02 CET2021-12-16 20:24:25 CET2021-12-16 23:53:20 CET2021-12-17 02:47:56 CET2021-12-16 00:12:46 CET2021-12-17 07:59:01 CET
Run setcmaesfuzz.Test-Comp22_coverage-branches.ReachSafety-ProductLinescoveritest.Test-Comp22_coverage-branches.ReachSafety-ProductLinesFuSeBMC.Test-Comp22_coverage-branches.ReachSafety-ProductLineshybridtiger.Test-Comp22_coverage-branches.ReachSafety-ProductLinesklee.Test-Comp22_coverage-branches.ReachSafety-ProductLineslegion.Test-Comp22_coverage-branches.ReachSafety-ProductLineslegion-symcc.Test-Comp22_coverage-branches.ReachSafety-ProductLineslibkluzzer.Test-Comp22_coverage-branches.ReachSafety-ProductLinesprtest.Test-Comp22_coverage-branches.ReachSafety-ProductLinessymbiotic.Test-Comp22_coverage-branches.ReachSafety-ProductLinestracerx.Test-Comp22_coverage-branches.ReachSafety-ProductLinesverifuzz.Test-Comp22_coverage-branches.ReachSafety-ProductLines
Options
  • -st real
  • -xml
  • -benchmark
  • -heap 10000M
  • -testcomp22
  • -setprop log.consoleLevel=SEVERE
  • -s incr
  • -benchmark
  • -heap 10000M
  • -tigertestcomp20
  • --score uct
  • --core 8
  • --tree-depth-limit 100000
  • --conex-timeout 0
  • --symex-timeout 0
  • --min-samples 1
  • --save-tests REDUCED
  • --persistent
  • --time-penalty 0
  • --random-seed 0
  • -o ./binary.instr
  • --compile svcomp
  • --coverage-only
  • -L ubuntu2004/lib
  • --test-comp
  • --testcomp
Propertiescoverage-branches

Statistics

CMA-ES Fuzz 2021-12-15 19:44:24 CET cmaesfuzz.Test-Comp22_coverage-branches.ReachSafety-ProductLines
CPAchecker 2021-12-15 19:57:13 CET coveritest.Test-Comp22_coverage-branches.ReachSafety-ProductLines
FuSeBMC 2021-12-16 03:43:34 CET FuSeBMC.Test-Comp22_coverage-branches.ReachSafety-ProductLines
CPAchecker 2021-12-16 04:41:54 CET hybridtiger.Test-Comp22_coverage-branches.ReachSafety-ProductLines
KLEE 2021-12-16 10:08:26 CET klee.Test-Comp22_coverage-branches.ReachSafety-ProductLines
Legion 2021-12-16 15:28:39 CET legion.Test-Comp22_coverage-branches.ReachSafety-ProductLines
Legion/SymCC 2021-12-16 13:27:02 CET legion-symcc.Test-Comp22_coverage-branches.ReachSafety-ProductLines
LibKluzzer 2021-12-16 20:24:25 CET libkluzzer.Test-Comp22_coverage-branches.ReachSafety-ProductLines
PRTest 2021-12-16 23:53:20 CET prtest.Test-Comp22_coverage-branches.ReachSafety-ProductLines
symbiotic 2021-12-17 02:47:56 CET symbiotic.Test-Comp22_coverage-branches.ReachSafety-ProductLines
Tracer-X 2021-12-16 00:12:46 CET tracerx.Test-Comp22_coverage-branches.ReachSafety-ProductLines
VeriFuzz 2021-12-17 07:59:01 CET verifuzz.Test-Comp22_coverage-branches.ReachSafety-ProductLines
Click here to select columns
 
status
raw score
cpu
(s)
mem
(MB)
 
status
raw score
cpu
(s)
mem
(MB)
energy
(J)
 
status
raw score
cpu
(s)
mem
(MB)
energy
(J)
 
status
raw score
cpu
(s)
mem
(MB)
energy
(J)
 
status
raw score
cpu
(s)
mem
(MB)
energy
(J)
 
status
raw score
cpu
(s)
mem
(MB)
energy
(J)
 
status
raw score
cpu
(s)
mem
(MB)
energy
(J)
 
status
raw score
cpu
(s)
mem
(MB)
energy
(J)
 
status
raw score
cpu
(s)
mem
(MB)
energy
(J)
 
status
raw score
cpu
(s)
mem
(MB)
energy
(J)
 
status
raw score
cpu
(s)
mem
(MB)
energy
(J)
 
status
raw score
cpu
(s)
mem
(MB)
energy
(J)
total
263
19.4
94000
15000
263
76.7
78000
760000
870000
263
76.7
200000
29000
2600000
263
56.0 
220000
970000
2500000
263
73.8
10000
17000
110000
263
70.0
190000
460000
2000000
263
74.3
220000
2700000
2500000
263
76.7
240000
2200000
2600000
263
48.2
240000
6400
3300000
263
68.7
6200
19000
300000
263
76.5
11000
420000
100000
263
76.7
240000
46000
3000000
    correct results
0
-
-
-
0
-
-
-
-
0
-
-
-
-
22
1.48
3100
80000
33000
160
44.6
1600
8100
18000
0
-
-
-
-
0
-
-
-
-
0
-
-
-
-
0
-
-
-
-
0
-
-
-
-
169
50.3
8200
320000
76000
0
-
-
-
-
        correct true
0
-
-
-
0
-
-
-
-
0
-
-
-
-
22
1.48
3100
80000
33000
0
-
-
-
-
0
-
-
-
-
0
-
-
-
-
0
-
-
-
-
0
-
-
-
-
0
-
-
-
-
0
-
-
-
-
0
-
-
-
-
        correct false
0
-
-
-
0
-
-
-
-
0
-
-
-
-
0
-
-
-
-
160
44.6
1600
8100
18000
0
-
-
-
-
0
-
-
-
-
0
-
-
-
-
0
-
-
-
-
0
-
-
-
-
169
50.3
8200
320000
76000
0
-
-
-
-
    incorrect results
0
-
-
-
0
-
-
-
-
0
-
-
-
-
0
-
-
-
-
0
-
-
-
-
0
-
-
-
-
0
-
-
-
-
0
-
-
-
-
0
-
-
-
-
0
-
-
-
-
0
-
-
-
-
0
-
-
-
-
        incorrect true
0
-
-
-
0
-
-
-
-
0
-
-
-
-
0
-
-
-
-
0
-
-
-
-
0
-
-
-
-
0
-
-
-
-
0
-
-
-
-
0
-
-
-
-
0
-
-
-
-
0
-
-
-
-
0
-
-
-
-
        incorrect false
0
-
-
-
0
-
-
-
-
0
-
-
-
-
0
-
-
-
-
0
-
-
-
-
0
-
-
-
-
0
-
-
-
-
0
-
-
-
-
0
-
-
-
-
0
-
-
-
-
0
-
-
-
-
0
-
-
-
-

Generated by BenchExec 3.11-dev