Task filter


cetfuzz 2024-12-22 20:08:10 CET cetfuzz.Test-Comp25_coverage-branches.SoftwareSystems-uthash

Add filter for:

CPAchecker 2024-12-17 20:37:35 CET coveritest.Test-Comp25_coverage-branches.SoftwareSystems-uthash

Add filter for:

ESBMC 2024-12-17 20:38:01 CET esbmc-incr.Test-Comp25_coverage-branches.SoftwareSystems-uthash

Add filter for:

ESBMC 2024-12-18 07:10:21 CET esbmc-kind.Test-Comp25_coverage-branches.SoftwareSystems-uthash

Add filter for:

FDSE 2024-12-18 07:14:06 CET fdse.Test-Comp25_coverage-branches.SoftwareSystems-uthash

Add filter for:

Fizzer 2024-12-18 09:05:25 CET fizzer.Test-Comp25_coverage-branches.SoftwareSystems-uthash

Add filter for:

FuSeBMC 2024-12-18 11:02:57 CET fusebmc.Test-Comp25_coverage-branches.SoftwareSystems-uthash

Add filter for:

FuSeBMC 2024-12-24 15:22:56 CET fusebmc-ia.Test-Comp25_coverage-branches.SoftwareSystems-uthash

Add filter for:

CPAchecker 2024-12-22 12:58:39 CET hybridtiger.Test-Comp25_coverage-branches.SoftwareSystems-uthash

Add filter for:

KLEE 2024-12-22 14:23:31 CET klee.Test-Comp25_coverage-branches.SoftwareSystems-uthash

Add filter for:

KLEEF 2024-12-18 19:36:47 CET kleef.Test-Comp25_coverage-branches.SoftwareSystems-uthash

Add filter for:

Legion 2024-12-22 22:49:08 CET legion.Test-Comp25_coverage-branches.SoftwareSystems-uthash

Add filter for:

Legion/SymCC 2024-12-23 00:21:49 CET legion-symcc.Test-Comp25_coverage-branches.SoftwareSystems-uthash

Add filter for:

Owi 2024-12-23 05:13:28 CET owic.Test-Comp25_coverage-branches.SoftwareSystems-uthash

Add filter for:

PRTest 2024-12-19 03:25:41 CET prtest.Test-Comp25_coverage-branches.SoftwareSystems-uthash

Add filter for:

Sikraken 2024-12-19 12:19:23 CET sikraken.Test-Comp25_coverage-branches.SoftwareSystems-uthash

Add filter for:

symbiotic 2024-12-20 08:07:57 CET symbiotic.Test-Comp25_coverage-branches.SoftwareSystems-uthash

Add filter for:

Tracer-X 2024-12-20 08:13:36 CET tracerx.Test-Comp25_coverage-branches.SoftwareSystems-uthash

Add filter for:

TracerX-WP 2024-12-20 13:14:13 CET tracerx-wp.Test-Comp25_coverage-branches.SoftwareSystems-uthash

Add filter for:

ULTIMATE TestGen 2024-12-20 16:19:19 CET utestgen.Test-Comp25_coverage-branches.SoftwareSystems-uthash

Add filter for:

WASP-C 2024-12-23 15:22:20 CET wasp-c.Test-Comp25_coverage-branches.SoftwareSystems-uthash

Add filter for:

Summary Table Quantile Plot Scatter Plot Info

Benchmark Setup

BenchmarkcetfuzzCoVeriTestESBMC-incrESBMC-kindFDSEFizzerFuSeBMCFuSeBMC-AIHybridTigerKLEEKLEEFLegionLegion/SymCCOwiPRTestSikrakenSymbioticTracerXTracerX-WPUTestGenWASP-C
Toolcetfuzz CPAchecker 4.0ESBMC version 7.7.0 64-bit x86_64 linuxFDSE testcomp25Fizzer 0.1FuSeBMC v.4.2.2FuSeBMC v.5.1.0CPAchecker 1.9.2-svn-35606MKLEE 2.2-pre-test-compKLEEF 3.1Legion 1.0-testcomp2021Legion/SymCC testcomp2023Owi %%VERSION%%PRTest v2.3Sikraken 1.0.8symbiotic 10.0.0-rc1-f6104f53Tracer-X 1.2.0TracerX-WP 1.2.0ULTIMATE TestGen 0.2.3-222e02f9WASP-C
Limitstimelimit: 900 s, memlimit: 15000 MB, CPU core limit: 4
Hostapollon*
OSLinux 6.8.0-51-generic
SystemCPU: Intel Xeon E3-1230 v5 @ 3.40 GHz, cores: 8, frequency: 3800 MHz, Turbo Boost: disabled; RAM: 33471 MB
Date of execution2024-12-22 20:08:10 CET2024-12-17 20:37:35 CET2024-12-17 20:38:01 CET2024-12-18 07:10:21 CET2024-12-18 07:14:06 CET2024-12-18 09:05:25 CET2024-12-18 11:02:57 CET2024-12-24 15:22:56 CET2024-12-22 12:58:39 CET2024-12-22 14:23:31 CET2024-12-18 19:36:47 CET2024-12-22 22:49:08 CET2024-12-23 00:21:49 CET2024-12-23 05:13:28 CET2024-12-19 03:25:41 CET2024-12-19 12:19:23 CET2024-12-20 08:07:57 CET2024-12-20 08:13:36 CET2024-12-20 13:14:13 CET2024-12-20 16:19:19 CET2024-12-23 15:22:20 CET
Run setcetfuzz.Test-Comp25_coverage-branches.SoftwareSystems-uthashcoveritest.Test-Comp25_coverage-branches.SoftwareSystems-uthashesbmc-incr.Test-Comp25_coverage-branches.SoftwareSystems-uthashesbmc-kind.Test-Comp25_coverage-branches.SoftwareSystems-uthashfdse.Test-Comp25_coverage-branches.SoftwareSystems-uthashfizzer.Test-Comp25_coverage-branches.SoftwareSystems-uthashfusebmc.Test-Comp25_coverage-branches.SoftwareSystems-uthashfusebmc-ia.Test-Comp25_coverage-branches.SoftwareSystems-uthashhybridtiger.Test-Comp25_coverage-branches.SoftwareSystems-uthashklee.Test-Comp25_coverage-branches.SoftwareSystems-uthashkleef.Test-Comp25_coverage-branches.SoftwareSystems-uthashlegion.Test-Comp25_coverage-branches.SoftwareSystems-uthashlegion-symcc.Test-Comp25_coverage-branches.SoftwareSystems-uthashowic.Test-Comp25_coverage-branches.SoftwareSystems-uthashprtest.Test-Comp25_coverage-branches.SoftwareSystems-uthashsikraken.Test-Comp25_coverage-branches.SoftwareSystems-uthashsymbiotic.Test-Comp25_coverage-branches.SoftwareSystems-uthashtracerx.Test-Comp25_coverage-branches.SoftwareSystems-uthashtracerx-wp.Test-Comp25_coverage-branches.SoftwareSystems-uthashutestgen.Test-Comp25_coverage-branches.SoftwareSystems-uthashwasp-c.Test-Comp25_coverage-branches.SoftwareSystems-uthash
Options
  • --benchmark
  • --heap 10000M
  • --testcomp25
  • --option log.consoleLevel=SEVERE
  • -s incr
  • -o condition
  • -s kinduction
  • -o branch
  • --testcomp
  • --max_seconds 865
  • --optimizer_max_seconds 30
  • --max_exec_milliseconds 500
  • --max_exec_megabytes 13312
  • --max_stdin_bytes 65536
  • --max_trace_length 10000
  • --max_stack_size 25
  • --stdin_model stdin_replay_bytes_then_repeat_zero
  • --test_type testcomp
  • -s incr
  • -s incr
  • -benchmark
  • -heap 10000M
  • -tigertestcomp20
  • --score uct
  • --core 8
  • --tree-depth-limit 100000
  • --conex-timeout 0
  • --symex-timeout 0
  • --min-samples 1
  • --save-tests REDUCED
  • --persistent
  • --time-penalty 0
  • --random-seed 0
  • -o ./binary.instr
  • --compile svcomp
  • -a True
  • -q
  • -b ./binary
  • --testcomp
  • release budget[900]
  • --test-comp
  • --traceabstraction.test.generation.mode Standard
  • --rcfgbuilder.reuse.mode.for.test.generation ReuseUNSATmatchPrefix
  • --full-output
  • --smt-assume
  • --policy breadth
  • --test-comp
Propertiescoverage-branches

Statistics

cetfuzz 2024-12-22 20:08:10 CET cetfuzz.Test-Comp25_coverage-branches.SoftwareSystems-uthash
CPAchecker 2024-12-17 20:37:35 CET coveritest.Test-Comp25_coverage-branches.SoftwareSystems-uthash
ESBMC 2024-12-17 20:38:01 CET esbmc-incr.Test-Comp25_coverage-branches.SoftwareSystems-uthash
ESBMC 2024-12-18 07:10:21 CET esbmc-kind.Test-Comp25_coverage-branches.SoftwareSystems-uthash
FDSE 2024-12-18 07:14:06 CET fdse.Test-Comp25_coverage-branches.SoftwareSystems-uthash
Fizzer 2024-12-18 09:05:25 CET fizzer.Test-Comp25_coverage-branches.SoftwareSystems-uthash
FuSeBMC 2024-12-18 11:02:57 CET fusebmc.Test-Comp25_coverage-branches.SoftwareSystems-uthash
FuSeBMC 2024-12-24 15:22:56 CET fusebmc-ia.Test-Comp25_coverage-branches.SoftwareSystems-uthash
CPAchecker 2024-12-22 12:58:39 CET hybridtiger.Test-Comp25_coverage-branches.SoftwareSystems-uthash
KLEE 2024-12-22 14:23:31 CET klee.Test-Comp25_coverage-branches.SoftwareSystems-uthash
KLEEF 2024-12-18 19:36:47 CET kleef.Test-Comp25_coverage-branches.SoftwareSystems-uthash
Legion 2024-12-22 22:49:08 CET legion.Test-Comp25_coverage-branches.SoftwareSystems-uthash
Legion/SymCC 2024-12-23 00:21:49 CET legion-symcc.Test-Comp25_coverage-branches.SoftwareSystems-uthash
Owi 2024-12-23 05:13:28 CET owic.Test-Comp25_coverage-branches.SoftwareSystems-uthash
PRTest 2024-12-19 03:25:41 CET prtest.Test-Comp25_coverage-branches.SoftwareSystems-uthash
Sikraken 2024-12-19 12:19:23 CET sikraken.Test-Comp25_coverage-branches.SoftwareSystems-uthash
symbiotic 2024-12-20 08:07:57 CET symbiotic.Test-Comp25_coverage-branches.SoftwareSystems-uthash
Tracer-X 2024-12-20 08:13:36 CET tracerx.Test-Comp25_coverage-branches.SoftwareSystems-uthash
TracerX-WP 2024-12-20 13:14:13 CET tracerx-wp.Test-Comp25_coverage-branches.SoftwareSystems-uthash
ULTIMATE TestGen 2024-12-20 16:19:19 CET utestgen.Test-Comp25_coverage-branches.SoftwareSystems-uthash
WASP-C 2024-12-23 15:22:20 CET wasp-c.Test-Comp25_coverage-branches.SoftwareSystems-uthash
Click here to select columns
 
status
raw score
cpu
(s)
mem
(MB)
 
status
raw score
cpu
(s)
mem
(MB)
 
status
raw score
cpu
(s)
mem
(MB)
 
status
raw score
cpu
(s)
mem
(MB)
 
status
raw score
cpu
(s)
mem
(MB)
 
status
raw score
cpu
(s)
mem
(MB)
 
status
raw score
cpu
(s)
mem
(MB)
 
status
raw score
cpu
(s)
mem
(MB)
 
status
raw score
cpu
(s)
mem
(MB)
 
status
raw score
cpu
(s)
mem
(MB)
 
status
raw score
cpu
(s)
mem
(MB)
 
status
raw score
cpu
(s)
mem
(MB)
 
status
raw score
cpu
(s)
mem
(MB)
 
status
raw score
cpu
(s)
mem
(MB)
 
status
raw score
cpu
(s)
mem
(MB)
 
status
raw score
cpu
(s)
mem
(MB)
 
status
raw score
cpu
(s)
mem
(MB)
 
status
raw score
cpu
(s)
mem
(MB)
 
status
raw score
cpu
(s)
mem
(MB)
 
status
raw score
cpu
(s)
mem
(MB)
 
status
raw score
cpu
(s)
mem
(MB)
all results
51
34.6
41000
5900
51
27.1
45000
690000
51
24.1
240
3700
51
24.1
210
3200
51
34.6
45000
3600
51
34.6
44000
100000
51
34.6
46000
220000
51
29.3
20000
10000
51
.301
27000
450000
51
6.21
35000
17000
51
34.6
32000
45000
51
0
32000
380000
51
0
46000
180000
51
29.1
31000
15000
51
34.4
46000
44000
51
0
36000
130000
51
34.6
33000
50000
51
0
4.1
670
51
0
4.1
670
51
25.6
46000
68000
51
0
320
4100
    correct results
0
-
-
-
0
-
-
-
51
24.1
240
3700
51
24.1
210
3200
0
-
-
-
0
-
-
-
0
-
-
-
0
-
-
-
16
.140
4200
21000
0
-
-
-
0
-
-
-
0
-
-
-
0
-
-
-
0
-
-
-
0
-
-
-
0
-
-
-
0
-
-
-
0
-
-
-
0
-
-
-
0
-
-
-
0
-
-
-
        correct true
0
-
-
-
0
-
-
-
0
-
-
-
0
-
-
-
0
-
-
-
0
-
-
-
0
-
-
-
0
-
-
-
16
.140
4200
21000
0
-
-
-
0
-
-
-
0
-
-
-
0
-
-
-
0
-
-
-
0
-
-
-
0
-
-
-
0
-
-
-
0
-
-
-
0
-
-
-
0
-
-
-
0
-
-
-
        correct false
0
-
-
-
0
-
-
-
51
24.1
240
3700
51
24.1
210
3200
0
-
-
-
0
-
-
-
0
-
-
-
0
-
-
-
0
-
-
-
0
-
-
-
0
-
-
-
0
-
-
-
0
-
-
-
0
-
-
-
0
-
-
-
0
-
-
-
0
-
-
-
0
-
-
-
0
-
-
-
0
-
-
-
0
-
-
-
    incorrect results
0
-
-
-
0
-
-
-
0
-
-
-
0
-
-
-
0
-
-
-
0
-
-
-
0
-
-
-
0
-
-
-
0
-
-
-
0
-
-
-
0
-
-
-
0
-
-
-
0
-
-
-
0
-
-
-
0
-
-
-
0
-
-
-
0
-
-
-
0
-
-
-
0
-
-
-
0
-
-
-
0
-
-
-
        incorrect true
0
-
-
-
0
-
-
-
0
-
-
-
0
-
-
-
0
-
-
-
0
-
-
-
0
-
-
-
0
-
-
-
0
-
-
-
0
-
-
-
0
-
-
-
0
-
-
-
0
-
-
-
0
-
-
-
0
-
-
-
0
-
-
-
0
-
-
-
0
-
-
-
0
-
-
-
0
-
-
-
0
-
-
-
        incorrect false
0
-
-
-
0
-
-
-
0
-
-
-
0
-
-
-
0
-
-
-
0
-
-
-
0
-
-
-
0
-
-
-
0
-
-
-
0
-
-
-
0
-
-
-
0
-
-
-
0
-
-
-
0
-
-
-
0
-
-
-
0
-
-
-
0
-
-
-
0
-
-
-
0
-
-
-
0
-
-
-
0
-
-
-

Generated by BenchExec 3.29