Task filter


cetfuzz 2023-12-13 03:30:21 CET cetfuzz.Test-Comp24_coverage-branches.Termination-MainControlFlow

Add filter for:

CPAchecker 2023-12-10 11:47:16 CET coveritest.Test-Comp24_coverage-branches.Termination-MainControlFlow

Add filter for:

FDSE 2023-12-12 04:10:34 CET fdse.Test-Comp24_coverage-branches.Termination-MainControlFlow

Add filter for:

Fizzer 2023-12-10 16:44:12 CET fizzer.Test-Comp24_coverage-branches.Termination-MainControlFlow

Add filter for:

FuSeBMC 2023-12-10 11:48:41 CET fusebmc.Test-Comp24_coverage-branches.Termination-MainControlFlow

Add filter for:

FuSeBMC 2023-12-11 00:20:48 CET fusebmc-ia.Test-Comp24_coverage-branches.Termination-MainControlFlow

Add filter for:

CPAchecker 2023-12-20 19:51:14 CET hybridtiger.Test-Comp24_coverage-branches.Termination-MainControlFlow

Add filter for:

KLEE 2023-12-21 19:27:39 CET klee.Test-Comp24_coverage-branches.Termination-MainControlFlow

Add filter for:

KLEEF 2023-12-22 17:56:25 CET kleef.Test-Comp24_coverage-branches.Termination-MainControlFlow

Add filter for:

Legion 2023-12-21 08:25:46 CET legion.Test-Comp24_coverage-branches.Termination-MainControlFlow

Add filter for:

Legion/SymCC 2023-12-21 03:15:19 CET legion-symcc.Test-Comp24_coverage-branches.Termination-MainControlFlow

Add filter for:

Owi 2023-12-13 08:41:50 CET owic.Test-Comp24_coverage-branches.Termination-MainControlFlow

Add filter for:

PRTest 2023-12-10 11:48:15 CET prtest.Test-Comp24_coverage-branches.Termination-MainControlFlow

Add filter for:

symbiotic 2023-12-12 08:27:09 CET symbiotic.Test-Comp24_coverage-branches.Termination-MainControlFlow

Add filter for:

Tracer-X 2023-12-11 08:21:26 CET tracerx.Test-Comp24_coverage-branches.Termination-MainControlFlow

Add filter for:

TracerX-WP 2023-12-12 23:17:43 CET tracerx-wp.Test-Comp24_coverage-branches.Termination-MainControlFlow

Add filter for:

ULTIMATE TestGen 2023-12-12 15:06:35 CET utestgen.Test-Comp24_coverage-branches.Termination-MainControlFlow

Add filter for:

WASP-C 2023-12-20 19:53:37 CET wasp-c.Test-Comp24_coverage-branches.Termination-MainControlFlow

Add filter for:

Summary Table Quantile Plot Scatter Plot Info

Benchmark Setup

BenchmarkcetfuzzCPA/CoVeriTestFDSEfizzerFuSeBMCFuSeBMC_AIHybridTigerKLEEKLEEFLegionLegion/SymCCOwiPRtestSymbioticTracer-XTracerX-WPUTestGenWASP-C
Toolcetfuzz 1.0.0CPAchecker 2.2.1-svn-44999FDSE testcomp24Fizzer 0.1FuSeBMC v.4.2.1FuSeBMC v.5.1.0CPAchecker 1.9.2-svn-35606MKLEE 2.2-pre-test-compKLEEF 3.0Legion 1.0-testcomp2021Legion/SymCC testcomp2023Owi %%VERSION%%PRTest v2.3symbiotic 10.0.0-rc1-647ccad1Tracer-X 1.2.0TracerX-WP 1.2.0ULTIMATE TestGen 0.2.3-ea2a3342WASP-C version 0.1
Limitstimelimit: 900 s, memlimit: 15000 MB, CPU core limit: 4
Hostapollon*
OSLinux 5.15.0-91-generic
SystemCPU: Intel Xeon E3-1230 v5 @ 3.40 GHz, cores: 8, frequency: 3800 MHz, Turbo Boost: disabled; RAM: 33467 MB
Date of execution2023-12-13 03:30:21 CET2023-12-10 11:47:16 CET2023-12-12 04:10:34 CET2023-12-10 16:44:12 CET2023-12-10 11:48:41 CET2023-12-11 00:20:48 CET2023-12-20 19:51:14 CET2023-12-21 19:27:39 CET2023-12-22 17:56:25 CET2023-12-21 08:25:46 CET2023-12-21 03:15:19 CET2023-12-13 08:41:50 CET2023-12-10 11:48:15 CET2023-12-12 08:27:09 CET2023-12-11 08:21:26 CET2023-12-12 23:17:43 CET2023-12-12 15:06:35 CET2023-12-20 19:53:37 CET
Run setcetfuzz.Test-Comp24_coverage-branches.Termination-MainControlFlowcoveritest.Test-Comp24_coverage-branches.Termination-MainControlFlowfdse.Test-Comp24_coverage-branches.Termination-MainControlFlowfizzer.Test-Comp24_coverage-branches.Termination-MainControlFlowfusebmc.Test-Comp24_coverage-branches.Termination-MainControlFlowfusebmc-ia.Test-Comp24_coverage-branches.Termination-MainControlFlowhybridtiger.Test-Comp24_coverage-branches.Termination-MainControlFlowklee.Test-Comp24_coverage-branches.Termination-MainControlFlowkleef.Test-Comp24_coverage-branches.Termination-MainControlFlowlegion.Test-Comp24_coverage-branches.Termination-MainControlFlowlegion-symcc.Test-Comp24_coverage-branches.Termination-MainControlFlowowic.Test-Comp24_coverage-branches.Termination-MainControlFlowprtest.Test-Comp24_coverage-branches.Termination-MainControlFlowsymbiotic.Test-Comp24_coverage-branches.Termination-MainControlFlowtracerx.Test-Comp24_coverage-branches.Termination-MainControlFlowtracerx-wp.Test-Comp24_coverage-branches.Termination-MainControlFlowutestgen.Test-Comp24_coverage-branches.Termination-MainControlFlowwasp-c.Test-Comp24_coverage-branches.Termination-MainControlFlow
Options
  • -benchmark
  • -heap 10000M
  • -testcomp24
  • -setprop log.consoleLevel=SEVERE
  • --testcomp
  • --max_seconds 865
  • --optimizer_max_seconds 30
  • --max_exec_milliseconds 500
  • --max_stdin_bytes 65536
  • --stdin_model stdin_replay_bytes_then_repeat_zero
  • --test_type testcomp
  • -s incr
  • -s incr
  • -benchmark
  • -heap 10000M
  • -tigertestcomp20
  • --score uct
  • --core 8
  • --tree-depth-limit 100000
  • --conex-timeout 0
  • --symex-timeout 0
  • --min-samples 1
  • --save-tests REDUCED
  • --persistent
  • --time-penalty 0
  • --random-seed 0
  • -o ./binary.instr
  • --compile svcomp
  • --coverage-only
  • -a True
  • -q
  • -b ./binary
  • --testcomp
  • --test-comp
  • --traceabstraction.test.generation.mode Standard
  • --full-output
  • --smt-assume
  • --policy breadth
  • --test-comp
Propertiescoverage-branches

Statistics

cetfuzz 2023-12-13 03:30:21 CET cetfuzz.Test-Comp24_coverage-branches.Termination-MainControlFlow
CPAchecker 2023-12-10 11:47:16 CET coveritest.Test-Comp24_coverage-branches.Termination-MainControlFlow
FDSE 2023-12-12 04:10:34 CET fdse.Test-Comp24_coverage-branches.Termination-MainControlFlow
Fizzer 2023-12-10 16:44:12 CET fizzer.Test-Comp24_coverage-branches.Termination-MainControlFlow
FuSeBMC 2023-12-10 11:48:41 CET fusebmc.Test-Comp24_coverage-branches.Termination-MainControlFlow
FuSeBMC 2023-12-11 00:20:48 CET fusebmc-ia.Test-Comp24_coverage-branches.Termination-MainControlFlow
CPAchecker 2023-12-20 19:51:14 CET hybridtiger.Test-Comp24_coverage-branches.Termination-MainControlFlow
KLEE 2023-12-21 19:27:39 CET klee.Test-Comp24_coverage-branches.Termination-MainControlFlow
KLEEF 2023-12-22 17:56:25 CET kleef.Test-Comp24_coverage-branches.Termination-MainControlFlow
Legion 2023-12-21 08:25:46 CET legion.Test-Comp24_coverage-branches.Termination-MainControlFlow
Legion/SymCC 2023-12-21 03:15:19 CET legion-symcc.Test-Comp24_coverage-branches.Termination-MainControlFlow
Owi 2023-12-13 08:41:50 CET owic.Test-Comp24_coverage-branches.Termination-MainControlFlow
PRTest 2023-12-10 11:48:15 CET prtest.Test-Comp24_coverage-branches.Termination-MainControlFlow
symbiotic 2023-12-12 08:27:09 CET symbiotic.Test-Comp24_coverage-branches.Termination-MainControlFlow
Tracer-X 2023-12-11 08:21:26 CET tracerx.Test-Comp24_coverage-branches.Termination-MainControlFlow
TracerX-WP 2023-12-12 23:17:43 CET tracerx-wp.Test-Comp24_coverage-branches.Termination-MainControlFlow
ULTIMATE TestGen 2023-12-12 15:06:35 CET utestgen.Test-Comp24_coverage-branches.Termination-MainControlFlow
WASP-C 2023-12-20 19:53:37 CET wasp-c.Test-Comp24_coverage-branches.Termination-MainControlFlow
Click here to select columns
 
status
raw score
cpu
(s)
mem
(MB)
 
status
raw score
cpu
(s)
mem
(MB)
 
status
raw score
cpu
(s)
mem
(MB)
 
status
raw score
cpu
(s)
mem
(MB)
 
status
raw score
cpu
(s)
mem
(MB)
 
status
raw score
cpu
(s)
mem
(MB)
 
status
raw score
cpu
(s)
mem
(MB)
 
status
raw score
cpu
(s)
mem
(MB)
 
status
raw score
cpu
(s)
mem
(MB)
 
status
raw score
cpu
(s)
mem
(MB)
 
status
raw score
cpu
(s)
mem
(MB)
 
status
raw score
cpu
(s)
mem
(MB)
 
status
raw score
cpu
(s)
mem
(MB)
 
status
raw score
cpu
(s)
mem
(MB)
 
status
raw score
cpu
(s)
mem
(MB)
 
status
raw score
cpu
(s)
mem
(MB)
 
status
raw score
cpu
(s)
mem
(MB)
 
status
raw score
cpu
(s)
mem
(MB)
all results
392
245
270000
43000
392
355
36000
130000
392
358
100000
68000
392
350
35000
170000
392
359
350000
150000
392
358
78000
100000
392
348  
31000
250000
392
209
16000
24000
392
355
300000
1300000
392
351
300000
2700000
392
354
260000
2800000
392
304
310000
290000
392
284
320000
34000
392
301
310000
580000
392
325
210000
260000
392
21.3
33000
72000
392
346
18000
140000
392
202
210000
3200000
    correct results
0
-
-
-
0
-
-
-
0
-
-
-
0
-
-
-
0
-
-
-
0
-
-
-
27
14.7
220
5100
0
-
-
-
0
-
-
-
0
-
-
-
0
-
-
-
0
-
-
-
0
-
-
-
0
-
-
-
0
-
-
-
0
-
-
-
0
-
-
-
0
-
-
-
        correct true
0
-
-
-
0
-
-
-
0
-
-
-
0
-
-
-
0
-
-
-
0
-
-
-
27
14.7
220
5100
0
-
-
-
0
-
-
-
0
-
-
-
0
-
-
-
0
-
-
-
0
-
-
-
0
-
-
-
0
-
-
-
0
-
-
-
0
-
-
-
0
-
-
-
        correct false
0
-
-
-
0
-
-
-
0
-
-
-
0
-
-
-
0
-
-
-
0
-
-
-
0
-
-
-
0
-
-
-
0
-
-
-
0
-
-
-
0
-
-
-
0
-
-
-
0
-
-
-
0
-
-
-
0
-
-
-
0
-
-
-
0
-
-
-
0
-
-
-
    incorrect results
0
-
-
-
0
-
-
-
0
-
-
-
0
-
-
-
0
-
-
-
0
-
-
-
0
-
-
-
0
-
-
-
0
-
-
-
0
-
-
-
0
-
-
-
0
-
-
-
0
-
-
-
0
-
-
-
0
-
-
-
0
-
-
-
0
-
-
-
0
-
-
-
        incorrect true
0
-
-
-
0
-
-
-
0
-
-
-
0
-
-
-
0
-
-
-
0
-
-
-
0
-
-
-
0
-
-
-
0
-
-
-
0
-
-
-
0
-
-
-
0
-
-
-
0
-
-
-
0
-
-
-
0
-
-
-
0
-
-
-
0
-
-
-
0
-
-
-
        incorrect false
0
-
-
-
0
-
-
-
0
-
-
-
0
-
-
-
0
-
-
-
0
-
-
-
0
-
-
-
0
-
-
-
0
-
-
-
0
-
-
-
0
-
-
-
0
-
-
-
0
-
-
-
0
-
-
-
0
-
-
-
0
-
-
-
0
-
-
-
0
-
-
-

Generated by BenchExec 3.20