Task filter


cetfuzz 2024-12-22 20:08:10 CET cetfuzz.Test-Comp25_coverage-branches.SoftwareSystems-SQLite-MemSafety

Add filter for:

CPAchecker 2024-12-17 20:37:35 CET coveritest.Test-Comp25_coverage-branches.SoftwareSystems-SQLite-MemSafety

Add filter for:

ESBMC 2024-12-17 20:38:01 CET esbmc-incr.Test-Comp25_coverage-branches.SoftwareSystems-SQLite-MemSafety

Add filter for:

ESBMC 2024-12-18 07:10:21 CET esbmc-kind.Test-Comp25_coverage-branches.SoftwareSystems-SQLite-MemSafety

Add filter for:

FDSE 2024-12-18 07:14:06 CET fdse.Test-Comp25_coverage-branches.SoftwareSystems-SQLite-MemSafety

Add filter for:

Fizzer 2024-12-18 09:05:25 CET fizzer.Test-Comp25_coverage-branches.SoftwareSystems-SQLite-MemSafety

Add filter for:

FuSeBMC 2024-12-18 11:02:57 CET fusebmc.Test-Comp25_coverage-branches.SoftwareSystems-SQLite-MemSafety

Add filter for:

FuSeBMC 2024-12-24 15:22:56 CET fusebmc-ia.Test-Comp25_coverage-branches.SoftwareSystems-SQLite-MemSafety

Add filter for:

CPAchecker 2024-12-22 12:58:39 CET hybridtiger.Test-Comp25_coverage-branches.SoftwareSystems-SQLite-MemSafety

Add filter for:

KLEE 2024-12-22 14:23:31 CET klee.Test-Comp25_coverage-branches.SoftwareSystems-SQLite-MemSafety

Add filter for:

KLEEF 2024-12-18 19:36:47 CET kleef.Test-Comp25_coverage-branches.SoftwareSystems-SQLite-MemSafety

Add filter for:

Legion 2024-12-22 22:49:08 CET legion.Test-Comp25_coverage-branches.SoftwareSystems-SQLite-MemSafety

Add filter for:

Legion/SymCC 2024-12-23 00:21:49 CET legion-symcc.Test-Comp25_coverage-branches.SoftwareSystems-SQLite-MemSafety

Add filter for:

Owi 2024-12-23 05:13:28 CET owic.Test-Comp25_coverage-branches.SoftwareSystems-SQLite-MemSafety

Add filter for:

PRTest 2024-12-19 03:25:41 CET prtest.Test-Comp25_coverage-branches.SoftwareSystems-SQLite-MemSafety

Add filter for:

Sikraken 2024-12-19 12:19:23 CET sikraken.Test-Comp25_coverage-branches.SoftwareSystems-SQLite-MemSafety

Add filter for:

symbiotic 2024-12-20 08:07:57 CET symbiotic.Test-Comp25_coverage-branches.SoftwareSystems-SQLite-MemSafety

Add filter for:

Tracer-X 2024-12-20 08:13:36 CET tracerx.Test-Comp25_coverage-branches.SoftwareSystems-SQLite-MemSafety

Add filter for:

TracerX-WP 2024-12-20 13:14:13 CET tracerx-wp.Test-Comp25_coverage-branches.SoftwareSystems-SQLite-MemSafety

Add filter for:

ULTIMATE TestGen 2024-12-20 16:19:19 CET utestgen.Test-Comp25_coverage-branches.SoftwareSystems-SQLite-MemSafety

Add filter for:

WASP-C 2024-12-23 15:22:20 CET wasp-c.Test-Comp25_coverage-branches.SoftwareSystems-SQLite-MemSafety

Add filter for:

Summary Table Quantile Plot Scatter Plot Info

Benchmark Setup

BenchmarkcetfuzzCoVeriTestESBMC-incrESBMC-kindFDSEFizzerFuSeBMCFuSeBMC-AIHybridTigerKLEEKLEEFLegionLegion/SymCCOwiPRTestSikrakenSymbioticTracerXTracerX-WPUTestGenWASP-C
Toolcetfuzz CPAchecker 4.0ESBMC version 7.7.0 64-bit x86_64 linuxFDSE testcomp25Fizzer 0.1FuSeBMC v.4.2.2FuSeBMC v.5.1.0CPAchecker 1.9.2-svn-35606MKLEE 2.2-pre-test-compKLEEF 3.1Legion 1.0-testcomp2021Legion/SymCC testcomp2023Owi %%VERSION%%PRTest v2.3Sikraken 1.0.8symbiotic 10.0.0-rc1-f6104f53Tracer-X 1.2.0TracerX-WP 1.2.0ULTIMATE TestGen 0.2.3-222e02f9WASP-C
Limitstimelimit: 900 s, memlimit: 15000 MB, CPU core limit: 4
Hostapollon*
OSLinux 6.8.0-51-generic
SystemCPU: Intel Xeon E3-1230 v5 @ 3.40 GHz, cores: 8, frequency: 3800 MHz, Turbo Boost: disabled; RAM: 33471 MB
Date of execution2024-12-22 20:08:10 CET2024-12-17 20:37:35 CET2024-12-17 20:38:01 CET2024-12-18 07:10:21 CET2024-12-18 07:14:06 CET2024-12-18 09:05:25 CET2024-12-18 11:02:57 CET2024-12-24 15:22:56 CET2024-12-22 12:58:39 CET2024-12-22 14:23:31 CET2024-12-18 19:36:47 CET2024-12-22 22:49:08 CET2024-12-23 00:21:49 CET2024-12-23 05:13:28 CET2024-12-19 03:25:41 CET2024-12-19 12:19:23 CET2024-12-20 08:07:57 CET2024-12-20 08:13:36 CET2024-12-20 13:14:13 CET2024-12-20 16:19:19 CET2024-12-23 15:22:20 CET
Run setcetfuzz.Test-Comp25_coverage-branches.SoftwareSystems-SQLite-MemSafetycoveritest.Test-Comp25_coverage-branches.SoftwareSystems-SQLite-MemSafetyesbmc-incr.Test-Comp25_coverage-branches.SoftwareSystems-SQLite-MemSafetyesbmc-kind.Test-Comp25_coverage-branches.SoftwareSystems-SQLite-MemSafetyfdse.Test-Comp25_coverage-branches.SoftwareSystems-SQLite-MemSafetyfizzer.Test-Comp25_coverage-branches.SoftwareSystems-SQLite-MemSafetyfusebmc.Test-Comp25_coverage-branches.SoftwareSystems-SQLite-MemSafetyfusebmc-ia.Test-Comp25_coverage-branches.SoftwareSystems-SQLite-MemSafetyhybridtiger.Test-Comp25_coverage-branches.SoftwareSystems-SQLite-MemSafetyklee.Test-Comp25_coverage-branches.SoftwareSystems-SQLite-MemSafetykleef.Test-Comp25_coverage-branches.SoftwareSystems-SQLite-MemSafetylegion.Test-Comp25_coverage-branches.SoftwareSystems-SQLite-MemSafetylegion-symcc.Test-Comp25_coverage-branches.SoftwareSystems-SQLite-MemSafetyowic.Test-Comp25_coverage-branches.SoftwareSystems-SQLite-MemSafetyprtest.Test-Comp25_coverage-branches.SoftwareSystems-SQLite-MemSafetysikraken.Test-Comp25_coverage-branches.SoftwareSystems-SQLite-MemSafetysymbiotic.Test-Comp25_coverage-branches.SoftwareSystems-SQLite-MemSafetytracerx.Test-Comp25_coverage-branches.SoftwareSystems-SQLite-MemSafetytracerx-wp.Test-Comp25_coverage-branches.SoftwareSystems-SQLite-MemSafetyutestgen.Test-Comp25_coverage-branches.SoftwareSystems-SQLite-MemSafetywasp-c.Test-Comp25_coverage-branches.SoftwareSystems-SQLite-MemSafety
Options
  • --benchmark
  • --heap 10000M
  • --testcomp25
  • --option log.consoleLevel=SEVERE
  • -s incr
  • -o condition
  • -s kinduction
  • -o branch
  • --testcomp
  • --max_seconds 865
  • --optimizer_max_seconds 30
  • --max_exec_milliseconds 500
  • --max_exec_megabytes 13312
  • --max_stdin_bytes 65536
  • --max_trace_length 10000
  • --max_stack_size 25
  • --stdin_model stdin_replay_bytes_then_repeat_zero
  • --test_type testcomp
  • -s incr
  • -s incr
  • -benchmark
  • -heap 10000M
  • -tigertestcomp20
  • --score uct
  • --core 8
  • --tree-depth-limit 100000
  • --conex-timeout 0
  • --symex-timeout 0
  • --min-samples 1
  • --save-tests REDUCED
  • --persistent
  • --time-penalty 0
  • --random-seed 0
  • -o ./binary.instr
  • --compile svcomp
  • -a True
  • -q
  • -b ./binary
  • --testcomp
  • release budget[900]
  • --test-comp
  • --traceabstraction.test.generation.mode Standard
  • --rcfgbuilder.reuse.mode.for.test.generation ReuseUNSATmatchPrefix
  • --full-output
  • --smt-assume
  • --policy breadth
  • --test-comp
Propertiescoverage-branches

Statistics

cetfuzz 2024-12-22 20:08:10 CET cetfuzz.Test-Comp25_coverage-branches.SoftwareSystems-SQLite-MemSafety
CPAchecker 2024-12-17 20:37:35 CET coveritest.Test-Comp25_coverage-branches.SoftwareSystems-SQLite-MemSafety
ESBMC 2024-12-17 20:38:01 CET esbmc-incr.Test-Comp25_coverage-branches.SoftwareSystems-SQLite-MemSafety
ESBMC 2024-12-18 07:10:21 CET esbmc-kind.Test-Comp25_coverage-branches.SoftwareSystems-SQLite-MemSafety
FDSE 2024-12-18 07:14:06 CET fdse.Test-Comp25_coverage-branches.SoftwareSystems-SQLite-MemSafety
Fizzer 2024-12-18 09:05:25 CET fizzer.Test-Comp25_coverage-branches.SoftwareSystems-SQLite-MemSafety
FuSeBMC 2024-12-18 11:02:57 CET fusebmc.Test-Comp25_coverage-branches.SoftwareSystems-SQLite-MemSafety
FuSeBMC 2024-12-24 15:22:56 CET fusebmc-ia.Test-Comp25_coverage-branches.SoftwareSystems-SQLite-MemSafety
CPAchecker 2024-12-22 12:58:39 CET hybridtiger.Test-Comp25_coverage-branches.SoftwareSystems-SQLite-MemSafety
KLEE 2024-12-22 14:23:31 CET klee.Test-Comp25_coverage-branches.SoftwareSystems-SQLite-MemSafety
KLEEF 2024-12-18 19:36:47 CET kleef.Test-Comp25_coverage-branches.SoftwareSystems-SQLite-MemSafety
Legion 2024-12-22 22:49:08 CET legion.Test-Comp25_coverage-branches.SoftwareSystems-SQLite-MemSafety
Legion/SymCC 2024-12-23 00:21:49 CET legion-symcc.Test-Comp25_coverage-branches.SoftwareSystems-SQLite-MemSafety
Owi 2024-12-23 05:13:28 CET owic.Test-Comp25_coverage-branches.SoftwareSystems-SQLite-MemSafety
PRTest 2024-12-19 03:25:41 CET prtest.Test-Comp25_coverage-branches.SoftwareSystems-SQLite-MemSafety
Sikraken 2024-12-19 12:19:23 CET sikraken.Test-Comp25_coverage-branches.SoftwareSystems-SQLite-MemSafety
symbiotic 2024-12-20 08:07:57 CET symbiotic.Test-Comp25_coverage-branches.SoftwareSystems-SQLite-MemSafety
Tracer-X 2024-12-20 08:13:36 CET tracerx.Test-Comp25_coverage-branches.SoftwareSystems-SQLite-MemSafety
TracerX-WP 2024-12-20 13:14:13 CET tracerx-wp.Test-Comp25_coverage-branches.SoftwareSystems-SQLite-MemSafety
ULTIMATE TestGen 2024-12-20 16:19:19 CET utestgen.Test-Comp25_coverage-branches.SoftwareSystems-SQLite-MemSafety
WASP-C 2024-12-23 15:22:20 CET wasp-c.Test-Comp25_coverage-branches.SoftwareSystems-SQLite-MemSafety
Click here to select columns
 
status
raw score
cpu
(s)
mem
(MB)
 
status
raw score
cpu
(s)
mem
(MB)
 
status
raw score
cpu
(s)
mem
(MB)
 
status
raw score
cpu
(s)
mem
(MB)
 
status
raw score
cpu
(s)
mem
(MB)
 
status
raw score
cpu
(s)
mem
(MB)
 
status
raw score
cpu
(s)
mem
(MB)
 
status
raw score
cpu
(s)
mem
(MB)
 
status
raw score
cpu
(s)
mem
(MB)
 
status
raw score
cpu
(s)
mem
(MB)
 
status
raw score
cpu
(s)
mem
(MB)
 
status
raw score
cpu
(s)
mem
(MB)
 
status
raw score
cpu
(s)
mem
(MB)
 
status
raw score
cpu
(s)
mem
(MB)
 
status
raw score
cpu
(s)
mem
(MB)
 
status
raw score
cpu
(s)
mem
(MB)
 
status
raw score
cpu
(s)
mem
(MB)
 
status
raw score
cpu
(s)
mem
(MB)
 
status
raw score
cpu
(s)
mem
(MB)
 
status
raw score
cpu
(s)
mem
(MB)
 
status
raw score
cpu
(s)
mem
(MB)
all results
1
0
110
800
1
.0003
900
1900
1
.0004
15
1400
1
0
900
1400
1
.034
900
880
1
.0328
870
1700
1
.0431
900
6400
1
.0002
.16
22
1
0
900
4400
1
.0226
30
180
1
.0062
420
9200
1
0
900
340
1
0
84
580
1
0
9.4
420
1
.0236
900
140
1
0
.032
4.9
1
.0001
22
380
1
.0208
7.3
300
1
.0208
7.3
300
1
0
62
490
1
0
180
230
    correct results
0
-
-
-
0
-
-
-
1
.0004
15
1400
0
-
-
-
0
-
-
-
0
-
-
-
0
-
-
-
0
-
-
-
0
-
-
-
0
-
-
-
1
.0062
420
9200
0
-
-
-
0
-
-
-
0
-
-
-
0
-
-
-
0
-
-
-
0
-
-
-
0
-
-
-
0
-
-
-
0
-
-
-
0
-
-
-
        correct true
0
-
-
-
0
-
-
-
0
-
-
-
0
-
-
-
0
-
-
-
0
-
-
-
0
-
-
-
0
-
-
-
0
-
-
-
0
-
-
-
0
-
-
-
0
-
-
-
0
-
-
-
0
-
-
-
0
-
-
-
0
-
-
-
0
-
-
-
0
-
-
-
0
-
-
-
0
-
-
-
0
-
-
-
        correct false
0
-
-
-
0
-
-
-
1
.0004
15
1400
0
-
-
-
0
-
-
-
0
-
-
-
0
-
-
-
0
-
-
-
0
-
-
-
0
-
-
-
1
.0062
420
9200
0
-
-
-
0
-
-
-
0
-
-
-
0
-
-
-
0
-
-
-
0
-
-
-
0
-
-
-
0
-
-
-
0
-
-
-
0
-
-
-
    incorrect results
0
-
-
-
0
-
-
-
0
-
-
-
0
-
-
-
0
-
-
-
0
-
-
-
0
-
-
-
0
-
-
-
0
-
-
-
0
-
-
-
0
-
-
-
0
-
-
-
0
-
-
-
0
-
-
-
0
-
-
-
0
-
-
-
0
-
-
-
0
-
-
-
0
-
-
-
0
-
-
-
0
-
-
-
        incorrect true
0
-
-
-
0
-
-
-
0
-
-
-
0
-
-
-
0
-
-
-
0
-
-
-
0
-
-
-
0
-
-
-
0
-
-
-
0
-
-
-
0
-
-
-
0
-
-
-
0
-
-
-
0
-
-
-
0
-
-
-
0
-
-
-
0
-
-
-
0
-
-
-
0
-
-
-
0
-
-
-
0
-
-
-
        incorrect false
0
-
-
-
0
-
-
-
0
-
-
-
0
-
-
-
0
-
-
-
0
-
-
-
0
-
-
-
0
-
-
-
0
-
-
-
0
-
-
-
0
-
-
-
0
-
-
-
0
-
-
-
0
-
-
-
0
-
-
-
0
-
-
-
0
-
-
-
0
-
-
-
0
-
-
-
0
-
-
-
0
-
-
-

Generated by BenchExec 3.29