Task filter


cetfuzz 2023-12-13 03:30:21 CET cetfuzz.Test-Comp24_coverage-branches.ReachSafety-Recursive

Add filter for:

CPAchecker 2023-12-10 11:47:16 CET coveritest.Test-Comp24_coverage-branches.ReachSafety-Recursive

Add filter for:

FDSE 2023-12-12 04:10:34 CET fdse.Test-Comp24_coverage-branches.ReachSafety-Recursive

Add filter for:

Fizzer 2023-12-10 16:44:12 CET fizzer.Test-Comp24_coverage-branches.ReachSafety-Recursive

Add filter for:

FuSeBMC 2023-12-10 11:48:41 CET fusebmc.Test-Comp24_coverage-branches.ReachSafety-Recursive

Add filter for:

FuSeBMC 2023-12-11 00:20:48 CET fusebmc-ia.Test-Comp24_coverage-branches.ReachSafety-Recursive

Add filter for:

CPAchecker 2023-12-20 19:51:14 CET hybridtiger.Test-Comp24_coverage-branches.ReachSafety-Recursive

Add filter for:

KLEE 2023-12-21 19:27:39 CET klee.Test-Comp24_coverage-branches.ReachSafety-Recursive

Add filter for:

KLEEF 2023-12-22 17:56:25 CET kleef.Test-Comp24_coverage-branches.ReachSafety-Recursive

Add filter for:

Legion 2023-12-21 08:25:46 CET legion.Test-Comp24_coverage-branches.ReachSafety-Recursive

Add filter for:

Legion/SymCC 2023-12-21 03:15:19 CET legion-symcc.Test-Comp24_coverage-branches.ReachSafety-Recursive

Add filter for:

Owi 2023-12-13 08:41:50 CET owic.Test-Comp24_coverage-branches.ReachSafety-Recursive

Add filter for:

PRTest 2023-12-10 11:48:15 CET prtest.Test-Comp24_coverage-branches.ReachSafety-Recursive

Add filter for:

symbiotic 2023-12-12 08:27:09 CET symbiotic.Test-Comp24_coverage-branches.ReachSafety-Recursive

Add filter for:

Tracer-X 2023-12-11 08:21:26 CET tracerx.Test-Comp24_coverage-branches.ReachSafety-Recursive

Add filter for:

TracerX-WP 2023-12-12 23:17:43 CET tracerx-wp.Test-Comp24_coverage-branches.ReachSafety-Recursive

Add filter for:

ULTIMATE TestGen 2023-12-12 15:06:35 CET utestgen.Test-Comp24_coverage-branches.ReachSafety-Recursive

Add filter for:

WASP-C 2023-12-20 19:53:37 CET wasp-c.Test-Comp24_coverage-branches.ReachSafety-Recursive

Add filter for:

Summary Table Quantile Plot Scatter Plot Info

Benchmark Setup

BenchmarkcetfuzzCPA/CoVeriTestFDSEfizzerFuSeBMCFuSeBMC_AIHybridTigerKLEEKLEEFLegionLegion/SymCCOwiPRtestSymbioticTracer-XTracerX-WPUTestGenWASP-C
Toolcetfuzz 1.0.0CPAchecker 2.2.1-svn-44999FDSE testcomp24Fizzer 0.1FuSeBMC v.4.2.1FuSeBMC v.5.1.0CPAchecker 1.9.2-svn-35606MKLEE 2.2-pre-test-compKLEEF 3.0Legion 1.0-testcomp2021Legion/SymCC testcomp2023Owi %%VERSION%%PRTest v2.3symbiotic 10.0.0-rc1-647ccad1Tracer-X 1.2.0TracerX-WP 1.2.0ULTIMATE TestGen 0.2.3-ea2a3342WASP-C version 0.1
Limitstimelimit: 900 s, memlimit: 15000 MB, CPU core limit: 4
Hostapollon*
OSLinux 5.15.0-91-generic
SystemCPU: Intel Xeon E3-1230 v5 @ 3.40 GHz, cores: 8, frequency: 3800 MHz, Turbo Boost: disabled; RAM: 33467 MB
Date of execution2023-12-13 03:30:21 CET2023-12-10 11:47:16 CET2023-12-12 04:10:34 CET2023-12-10 16:44:12 CET2023-12-10 11:48:41 CET2023-12-11 00:20:48 CET2023-12-20 19:51:14 CET2023-12-21 19:27:39 CET2023-12-22 17:56:25 CET2023-12-21 08:25:46 CET2023-12-21 03:15:19 CET2023-12-13 08:41:50 CET2023-12-10 11:48:15 CET2023-12-12 08:27:09 CET2023-12-11 08:21:26 CET2023-12-12 23:17:43 CET2023-12-12 15:06:35 CET2023-12-20 19:53:37 CET
Run setcetfuzz.Test-Comp24_coverage-branches.ReachSafety-Recursivecoveritest.Test-Comp24_coverage-branches.ReachSafety-Recursivefdse.Test-Comp24_coverage-branches.ReachSafety-Recursivefizzer.Test-Comp24_coverage-branches.ReachSafety-Recursivefusebmc.Test-Comp24_coverage-branches.ReachSafety-Recursivefusebmc-ia.Test-Comp24_coverage-branches.ReachSafety-Recursivehybridtiger.Test-Comp24_coverage-branches.ReachSafety-Recursiveklee.Test-Comp24_coverage-branches.ReachSafety-Recursivekleef.Test-Comp24_coverage-branches.ReachSafety-Recursivelegion.Test-Comp24_coverage-branches.ReachSafety-Recursivelegion-symcc.Test-Comp24_coverage-branches.ReachSafety-Recursiveowic.Test-Comp24_coverage-branches.ReachSafety-Recursiveprtest.Test-Comp24_coverage-branches.ReachSafety-Recursivesymbiotic.Test-Comp24_coverage-branches.ReachSafety-Recursivetracerx.Test-Comp24_coverage-branches.ReachSafety-Recursivetracerx-wp.Test-Comp24_coverage-branches.ReachSafety-Recursiveutestgen.Test-Comp24_coverage-branches.ReachSafety-Recursivewasp-c.Test-Comp24_coverage-branches.ReachSafety-Recursive
Options
  • -benchmark
  • -heap 10000M
  • -testcomp24
  • -setprop log.consoleLevel=SEVERE
  • --testcomp
  • --max_seconds 865
  • --optimizer_max_seconds 30
  • --max_exec_milliseconds 500
  • --max_stdin_bytes 65536
  • --stdin_model stdin_replay_bytes_then_repeat_zero
  • --test_type testcomp
  • -s incr
  • -s incr
  • -benchmark
  • -heap 10000M
  • -tigertestcomp20
  • --score uct
  • --core 8
  • --tree-depth-limit 100000
  • --conex-timeout 0
  • --symex-timeout 0
  • --min-samples 1
  • --save-tests REDUCED
  • --persistent
  • --time-penalty 0
  • --random-seed 0
  • -o ./binary.instr
  • --compile svcomp
  • --coverage-only
  • -a True
  • -q
  • -b ./binary
  • --testcomp
  • --test-comp
  • --traceabstraction.test.generation.mode Standard
  • --full-output
  • --smt-assume
  • --policy breadth
  • --test-comp
Propertiescoverage-branches

Statistics

cetfuzz 2023-12-13 03:30:21 CET cetfuzz.Test-Comp24_coverage-branches.ReachSafety-Recursive
CPAchecker 2023-12-10 11:47:16 CET coveritest.Test-Comp24_coverage-branches.ReachSafety-Recursive
FDSE 2023-12-12 04:10:34 CET fdse.Test-Comp24_coverage-branches.ReachSafety-Recursive
Fizzer 2023-12-10 16:44:12 CET fizzer.Test-Comp24_coverage-branches.ReachSafety-Recursive
FuSeBMC 2023-12-10 11:48:41 CET fusebmc.Test-Comp24_coverage-branches.ReachSafety-Recursive
FuSeBMC 2023-12-11 00:20:48 CET fusebmc-ia.Test-Comp24_coverage-branches.ReachSafety-Recursive
CPAchecker 2023-12-20 19:51:14 CET hybridtiger.Test-Comp24_coverage-branches.ReachSafety-Recursive
KLEE 2023-12-21 19:27:39 CET klee.Test-Comp24_coverage-branches.ReachSafety-Recursive
KLEEF 2023-12-22 17:56:25 CET kleef.Test-Comp24_coverage-branches.ReachSafety-Recursive
Legion 2023-12-21 08:25:46 CET legion.Test-Comp24_coverage-branches.ReachSafety-Recursive
Legion/SymCC 2023-12-21 03:15:19 CET legion-symcc.Test-Comp24_coverage-branches.ReachSafety-Recursive
Owi 2023-12-13 08:41:50 CET owic.Test-Comp24_coverage-branches.ReachSafety-Recursive
PRTest 2023-12-10 11:48:15 CET prtest.Test-Comp24_coverage-branches.ReachSafety-Recursive
symbiotic 2023-12-12 08:27:09 CET symbiotic.Test-Comp24_coverage-branches.ReachSafety-Recursive
Tracer-X 2023-12-11 08:21:26 CET tracerx.Test-Comp24_coverage-branches.ReachSafety-Recursive
TracerX-WP 2023-12-12 23:17:43 CET tracerx-wp.Test-Comp24_coverage-branches.ReachSafety-Recursive
ULTIMATE TestGen 2023-12-12 15:06:35 CET utestgen.Test-Comp24_coverage-branches.ReachSafety-Recursive
WASP-C 2023-12-20 19:53:37 CET wasp-c.Test-Comp24_coverage-branches.ReachSafety-Recursive
Click here to select columns
 
status
raw score
cpu
(s)
mem
(MB)
 
status
raw score
cpu
(s)
mem
(MB)
 
status
raw score
cpu
(s)
mem
(MB)
 
status
raw score
cpu
(s)
mem
(MB)
 
status
raw score
cpu
(s)
mem
(MB)
 
status
raw score
cpu
(s)
mem
(MB)
 
status
raw score
cpu
(s)
mem
(MB)
 
status
raw score
cpu
(s)
mem
(MB)
 
status
raw score
cpu
(s)
mem
(MB)
 
status
raw score
cpu
(s)
mem
(MB)
 
status
raw score
cpu
(s)
mem
(MB)
 
status
raw score
cpu
(s)
mem
(MB)
 
status
raw score
cpu
(s)
mem
(MB)
 
status
raw score
cpu
(s)
mem
(MB)
 
status
raw score
cpu
(s)
mem
(MB)
 
status
raw score
cpu
(s)
mem
(MB)
 
status
raw score
cpu
(s)
mem
(MB)
 
status
raw score
cpu
(s)
mem
(MB)
all results
121
38.9
49000
13000
121
90.5
110000
200000
121
93.1
30000
32000
121
94.4
33000
50000
121
97.7
110000
92000
121
97.6
110000
39000
121
84.5
78000
980000
121
58.2
67000
170000
121
94.4
88000
320000
121
64.7
72000
830000
121
66.5
100000
280000
121
65.9
67000
65000
121
40.2
42000
3600
121
87.4
84000
270000
121
87.4
47000
120000
121
57.2 
71000
340000
121
90.0
73000
230000
121
70.1
77000
110000
    correct results
0
-
-
-
0
-
-
-
0
-
-
-
0
-
-
-
0
-
-
-
0
-
-
-
0
-
-
-
22
11.1
10
550
0
-
-
-
0
-
-
-
0
-
-
-
30
21.1
950
6600
0
-
-
-
0
-
-
-
21
18.5
5500
18000
6
4.83
370
6100
0
-
-
-
0
-
-
-
        correct true
0
-
-
-
0
-
-
-
0
-
-
-
0
-
-
-
0
-
-
-
0
-
-
-
0
-
-
-
0
-
-
-
0
-
-
-
0
-
-
-
0
-
-
-
0
-
-
-
0
-
-
-
0
-
-
-
0
-
-
-
0
-
-
-
0
-
-
-
0
-
-
-
        correct false
0
-
-
-
0
-
-
-
0
-
-
-
0
-
-
-
0
-
-
-
0
-
-
-
0
-
-
-
22
11.1
10
550
0
-
-
-
0
-
-
-
0
-
-
-
30
21.1
950
6600
0
-
-
-
0
-
-
-
21
18.5
5500
18000
6
4.83
370
6100
0
-
-
-
0
-
-
-
    incorrect results
0
-
-
-
0
-
-
-
0
-
-
-
0
-
-
-
0
-
-
-
0
-
-
-
0
-
-
-
0
-
-
-
0
-
-
-
0
-
-
-
0
-
-
-
0
-
-
-
0
-
-
-
0
-
-
-
0
-
-
-
0
-
-
-
0
-
-
-
0
-
-
-
        incorrect true
0
-
-
-
0
-
-
-
0
-
-
-
0
-
-
-
0
-
-
-
0
-
-
-
0
-
-
-
0
-
-
-
0
-
-
-
0
-
-
-
0
-
-
-
0
-
-
-
0
-
-
-
0
-
-
-
0
-
-
-
0
-
-
-
0
-
-
-
0
-
-
-
        incorrect false
0
-
-
-
0
-
-
-
0
-
-
-
0
-
-
-
0
-
-
-
0
-
-
-
0
-
-
-
0
-
-
-
0
-
-
-
0
-
-
-
0
-
-
-
0
-
-
-
0
-
-
-
0
-
-
-
0
-
-
-
0
-
-
-
0
-
-
-
0
-
-
-

Generated by BenchExec 3.20