Task filter


cetfuzz 2023-12-13 03:30:21 CET cetfuzz.Test-Comp24_coverage-error-call.ReachSafety-Sequentialized

Add filter for:

CPAchecker 2023-12-10 11:47:16 CET coveritest.Test-Comp24_coverage-error-call.ReachSafety-Sequentialized

Add filter for:

ESBMC 2023-12-22 03:20:00 CET esbmc-kind.Test-Comp24_coverage-error-call.ReachSafety-Sequentialized

Add filter for:

FDSE 2023-12-12 04:10:34 CET fdse.Test-Comp24_coverage-error-call.ReachSafety-Sequentialized

Add filter for:

Fizzer 2023-12-10 16:44:12 CET fizzer.Test-Comp24_coverage-error-call.ReachSafety-Sequentialized

Add filter for:

FuSeBMC 2023-12-10 11:48:41 CET fusebmc.Test-Comp24_coverage-error-call.ReachSafety-Sequentialized

Add filter for:

FuSeBMC 2023-12-11 00:20:48 CET fusebmc-ia.Test-Comp24_coverage-error-call.ReachSafety-Sequentialized

Add filter for:

CPAchecker 2023-12-20 19:51:14 CET hybridtiger.Test-Comp24_coverage-error-call.ReachSafety-Sequentialized

Add filter for:

KLEE 2023-12-21 19:27:39 CET klee.Test-Comp24_coverage-error-call.ReachSafety-Sequentialized

Add filter for:

KLEEF 2023-12-22 17:56:25 CET kleef.Test-Comp24_coverage-error-call.ReachSafety-Sequentialized

Add filter for:

Legion 2023-12-21 08:25:46 CET legion.Test-Comp24_coverage-error-call.ReachSafety-Sequentialized

Add filter for:

Legion/SymCC 2023-12-21 03:15:19 CET legion-symcc.Test-Comp24_coverage-error-call.ReachSafety-Sequentialized

Add filter for:

Owi 2023-12-13 08:41:50 CET owic.Test-Comp24_coverage-error-call.ReachSafety-Sequentialized

Add filter for:

PRTest 2023-12-10 11:48:15 CET prtest.Test-Comp24_coverage-error-call.ReachSafety-Sequentialized

Add filter for:

Rizzer 2023-12-11 16:47:26 CET rizzer.Test-Comp24_coverage-error-call.ReachSafety-Sequentialized

Add filter for:

symbiotic 2023-12-12 08:27:09 CET symbiotic.Test-Comp24_coverage-error-call.ReachSafety-Sequentialized

Add filter for:

Tracer-X 2023-12-11 08:21:26 CET tracerx.Test-Comp24_coverage-error-call.ReachSafety-Sequentialized

Add filter for:

TracerX-WP 2023-12-12 23:17:43 CET tracerx-wp.Test-Comp24_coverage-error-call.ReachSafety-Sequentialized

Add filter for:

ULTIMATE TestGen 2023-12-12 15:06:35 CET utestgen.Test-Comp24_coverage-error-call.ReachSafety-Sequentialized

Add filter for:

WASP-C 2023-12-20 19:53:37 CET wasp-c.Test-Comp24_coverage-error-call.ReachSafety-Sequentialized

Add filter for:

Summary Table Quantile Plot Scatter Plot Info

Benchmark Setup

BenchmarkcetfuzzCPA/CoVeriTestESBMC-bkindFDSEfizzerFuSeBMCFuSeBMC_AIHybridTigerKLEEKLEEFLegionLegion/SymCCOwiPRtestRizzerSymbioticTracer-XTracerX-WPUTestGenWASP-C
Toolcetfuzz 1.0.0CPAchecker 2.2.1-svn-44999ESBMC version 7.0.0 64-bit x86_64 linuxFDSE testcomp24Fizzer 0.1FuSeBMC v.4.2.1FuSeBMC v.5.1.0CPAchecker 1.9.2-svn-35606MKLEE 2.2-pre-test-compKLEEF 3.0Legion 1.0-testcomp2021Legion/SymCC testcomp2023Owi %%VERSION%%PRTest v2.3Rizzer 0.1symbiotic 10.0.0-rc1-647ccad1Tracer-X 1.2.0TracerX-WP 1.2.0ULTIMATE TestGen 0.2.3-ea2a3342WASP-C version 0.1
Limitstimelimit: 900 s, memlimit: 15000 MB, CPU core limit: 4
Hostapollon*
OSLinux 5.15.0-91-generic
SystemCPU: Intel Xeon E3-1230 v5 @ 3.40 GHz, cores: 8, frequency: 3800 MHz, Turbo Boost: disabled; RAM: 33467 MB
Date of execution2023-12-13 03:30:21 CET2023-12-10 11:47:16 CET2023-12-22 03:20:00 CET2023-12-12 04:10:34 CET2023-12-10 16:44:12 CET2023-12-10 11:48:41 CET2023-12-11 00:20:48 CET2023-12-20 19:51:14 CET2023-12-21 19:27:39 CET2023-12-22 17:56:25 CET2023-12-21 08:25:46 CET2023-12-21 03:15:19 CET2023-12-13 08:41:50 CET2023-12-10 11:48:15 CET2023-12-11 16:47:26 CET2023-12-12 08:27:09 CET2023-12-11 08:21:26 CET2023-12-12 23:17:43 CET2023-12-12 15:06:35 CET2023-12-20 19:53:37 CET
Run setcetfuzz.Test-Comp24_coverage-error-call.ReachSafety-Sequentializedcoveritest.Test-Comp24_coverage-error-call.ReachSafety-Sequentializedesbmc-kind.Test-Comp24_coverage-error-call.ReachSafety-Sequentializedfdse.Test-Comp24_coverage-error-call.ReachSafety-Sequentializedfizzer.Test-Comp24_coverage-error-call.ReachSafety-Sequentializedfusebmc.Test-Comp24_coverage-error-call.ReachSafety-Sequentializedfusebmc-ia.Test-Comp24_coverage-error-call.ReachSafety-Sequentializedhybridtiger.Test-Comp24_coverage-error-call.ReachSafety-Sequentializedklee.Test-Comp24_coverage-error-call.ReachSafety-Sequentializedkleef.Test-Comp24_coverage-error-call.ReachSafety-Sequentializedlegion.Test-Comp24_coverage-error-call.ReachSafety-Sequentializedlegion-symcc.Test-Comp24_coverage-error-call.ReachSafety-Sequentializedowic.Test-Comp24_coverage-error-call.ReachSafety-Sequentializedprtest.Test-Comp24_coverage-error-call.ReachSafety-Sequentializedrizzer.Test-Comp24_coverage-error-call.ReachSafety-Sequentializedsymbiotic.Test-Comp24_coverage-error-call.ReachSafety-Sequentializedtracerx.Test-Comp24_coverage-error-call.ReachSafety-Sequentializedtracerx-wp.Test-Comp24_coverage-error-call.ReachSafety-Sequentializedutestgen.Test-Comp24_coverage-error-call.ReachSafety-Sequentializedwasp-c.Test-Comp24_coverage-error-call.ReachSafety-Sequentialized
Options
  • -benchmark
  • -heap 10000M
  • -testcomp24
  • -setprop log.consoleLevel=SEVERE
  • -s kinduction
  • --testcomp
  • --max_seconds 865
  • --optimizer_max_seconds 30
  • --max_exec_milliseconds 500
  • --max_stdin_bytes 65536
  • --stdin_model stdin_replay_bytes_then_repeat_zero
  • --test_type testcomp
  • -s incr
  • -s incr
  • -benchmark
  • -heap 10000M
  • -tigertestcomp20
  • --score uct
  • --core 8
  • --tree-depth-limit 100000
  • --conex-timeout 0
  • --symex-timeout 0
  • --min-samples 1
  • --save-tests REDUCED
  • --persistent
  • --time-penalty 0
  • --random-seed 0
  • -o ./binary.instr
  • --compile svcomp
  • -a True
  • -q
  • -b ./binary
  • -e
  • --testcomp
  • --max_seconds 865
  • --optimizer_max_seconds 30
  • --max_exec_milliseconds 500
  • --max_stdin_bytes 65536
  • --stdin_model stdin_replay_bytes_then_repeat_zero
  • --test_type testcomp
  • --test-comp
  • --traceabstraction.test.generation.mode Standard
  • --full-output
  • --smt-assume
  • --policy breadth
  • --test-comp
Propertiescoverage-error-call

Statistics

cetfuzz 2023-12-13 03:30:21 CET cetfuzz.Test-Comp24_coverage-error-call.ReachSafety-Sequentialized
CPAchecker 2023-12-10 11:47:16 CET coveritest.Test-Comp24_coverage-error-call.ReachSafety-Sequentialized
ESBMC 2023-12-22 03:20:00 CET esbmc-kind.Test-Comp24_coverage-error-call.ReachSafety-Sequentialized
FDSE 2023-12-12 04:10:34 CET fdse.Test-Comp24_coverage-error-call.ReachSafety-Sequentialized
Fizzer 2023-12-10 16:44:12 CET fizzer.Test-Comp24_coverage-error-call.ReachSafety-Sequentialized
FuSeBMC 2023-12-10 11:48:41 CET fusebmc.Test-Comp24_coverage-error-call.ReachSafety-Sequentialized
FuSeBMC 2023-12-11 00:20:48 CET fusebmc-ia.Test-Comp24_coverage-error-call.ReachSafety-Sequentialized
CPAchecker 2023-12-20 19:51:14 CET hybridtiger.Test-Comp24_coverage-error-call.ReachSafety-Sequentialized
KLEE 2023-12-21 19:27:39 CET klee.Test-Comp24_coverage-error-call.ReachSafety-Sequentialized
KLEEF 2023-12-22 17:56:25 CET kleef.Test-Comp24_coverage-error-call.ReachSafety-Sequentialized
Legion 2023-12-21 08:25:46 CET legion.Test-Comp24_coverage-error-call.ReachSafety-Sequentialized
Legion/SymCC 2023-12-21 03:15:19 CET legion-symcc.Test-Comp24_coverage-error-call.ReachSafety-Sequentialized
Owi 2023-12-13 08:41:50 CET owic.Test-Comp24_coverage-error-call.ReachSafety-Sequentialized
PRTest 2023-12-10 11:48:15 CET prtest.Test-Comp24_coverage-error-call.ReachSafety-Sequentialized
Rizzer 2023-12-11 16:47:26 CET rizzer.Test-Comp24_coverage-error-call.ReachSafety-Sequentialized
symbiotic 2023-12-12 08:27:09 CET symbiotic.Test-Comp24_coverage-error-call.ReachSafety-Sequentialized
Tracer-X 2023-12-11 08:21:26 CET tracerx.Test-Comp24_coverage-error-call.ReachSafety-Sequentialized
TracerX-WP 2023-12-12 23:17:43 CET tracerx-wp.Test-Comp24_coverage-error-call.ReachSafety-Sequentialized
ULTIMATE TestGen 2023-12-12 15:06:35 CET utestgen.Test-Comp24_coverage-error-call.ReachSafety-Sequentialized
WASP-C 2023-12-20 19:53:37 CET wasp-c.Test-Comp24_coverage-error-call.ReachSafety-Sequentialized
Click here to select columns
 
status
cpu
(s)
mem
(MB)
 
status
raw score
cpu
(s)
mem
(MB)
 
status
raw score
cpu
(s)
mem
(MB)
 
status
raw score
cpu
(s)
mem
(MB)
 
status
raw score
cpu
(s)
mem
(MB)
 
status
raw score
cpu
(s)
mem
(MB)
 
status
raw score
cpu
(s)
mem
(MB)
 
status
raw score
cpu
(s)
mem
(MB)
 
status
raw score
cpu
(s)
mem
(MB)
 
status
raw score
cpu
(s)
mem
(MB)
 
status
cpu
(s)
mem
(MB)
 
status
cpu
(s)
mem
(MB)
 
status
cpu
(s)
mem
(MB)
 
status
cpu
(s)
mem
(MB)
 
status
raw score
cpu
(s)
mem
(MB)
 
status
raw score
cpu
(s)
mem
(MB)
 
status
raw score
cpu
(s)
mem
(MB)
 
status
raw score
cpu
(s)
mem
(MB)
 
status
raw score
cpu
(s)
mem
(MB)
 
status
raw score
cpu
(s)
mem
(MB)
all results
106
210
12000
106
62
45000
140000
106
44
17000
850000
106
79
64000
53000
106
43
77000
48000
106
102
16000
19000
106
102
16000
24000
106
87
93000
230000
106
86
7300
43000
106
92
18000
220000
106
96000
16000
106
1000
14000
106
48000
34000
106
95000
3200
106
51
77000
45000
106
79
14000
320000
106
59
25000
1000000
106
43
12000
780000
106
9
64000
1300000
106
52
45000
39000
    correct results
0
-
-
0
-
-
-
0
-
-
-
0
-
-
-
0
-
-
-
0
-
-
-
0
-
-
-
3
3
320
8800
86
86
4400
42000
10
10
100
1300
0
-
-
0
-
-
0
-
-
0
-
-
0
-
-
-
0
-
-
-
7
7
1000
28000
10
10
64
12000
0
-
-
-
0
-
-
-
        correct true
0
-
-
0
-
-
-
0
-
-
-
0
-
-
-
0
-
-
-
0
-
-
-
0
-
-
-
3
3
320
8800
0
-
-
-
0
-
-
-
0
-
-
0
-
-
0
-
-
0
-
-
0
-
-
-
0
-
-
-
0
-
-
-
0
-
-
-
0
-
-
-
0
-
-
-
        correct false
0
-
-
0
-
-
-
0
-
-
-
0
-
-
-
0
-
-
-
0
-
-
-
0
-
-
-
0
-
-
-
86
86
4400
42000
10
10
100
1300
0
-
-
0
-
-
0
-
-
0
-
-
0
-
-
-
0
-
-
-
7
7
1000
28000
10
10
64
12000
0
-
-
-
0
-
-
-
    incorrect results
0
-
-
0
-
-
-
0
-
-
-
0
-
-
-
0
-
-
-
0
-
-
-
0
-
-
-
0
-
-
-
0
-
-
-
0
-
-
-
0
-
-
0
-
-
0
-
-
0
-
-
0
-
-
-
0
-
-
-
0
-
-
-
0
-
-
-
0
-
-
-
0
-
-
-
        incorrect true
0
-
-
0
-
-
-
0
-
-
-
0
-
-
-
0
-
-
-
0
-
-
-
0
-
-
-
0
-
-
-
0
-
-
-
0
-
-
-
0
-
-
0
-
-
0
-
-
0
-
-
0
-
-
-
0
-
-
-
0
-
-
-
0
-
-
-
0
-
-
-
0
-
-
-
        incorrect false
0
-
-
0
-
-
-
0
-
-
-
0
-
-
-
0
-
-
-
0
-
-
-
0
-
-
-
0
-
-
-
0
-
-
-
0
-
-
-
0
-
-
0
-
-
0
-
-
0
-
-
0
-
-
-
0
-
-
-
0
-
-
-
0
-
-
-
0
-
-
-
0
-
-
-

Generated by BenchExec 3.20