Task filter


cetfuzz 2023-12-13 03:30:21 CET cetfuzz.Test-Comp24_coverage-error-call.SoftwareSystems-BusyBox-MemSafety

Add filter for:

CPAchecker 2023-12-10 11:47:16 CET coveritest.Test-Comp24_coverage-error-call.SoftwareSystems-BusyBox-MemSafety

Add filter for:

ESBMC 2023-12-22 03:20:00 CET esbmc-kind.Test-Comp24_coverage-error-call.SoftwareSystems-BusyBox-MemSafety

Add filter for:

FDSE 2023-12-12 04:10:34 CET fdse.Test-Comp24_coverage-error-call.SoftwareSystems-BusyBox-MemSafety

Add filter for:

Fizzer 2023-12-10 16:44:12 CET fizzer.Test-Comp24_coverage-error-call.SoftwareSystems-BusyBox-MemSafety

Add filter for:

FuSeBMC 2023-12-10 11:48:41 CET fusebmc.Test-Comp24_coverage-error-call.SoftwareSystems-BusyBox-MemSafety

Add filter for:

FuSeBMC 2023-12-11 00:20:48 CET fusebmc-ia.Test-Comp24_coverage-error-call.SoftwareSystems-BusyBox-MemSafety

Add filter for:

CPAchecker 2023-12-20 19:51:14 CET hybridtiger.Test-Comp24_coverage-error-call.SoftwareSystems-BusyBox-MemSafety

Add filter for:

KLEE 2023-12-21 19:27:39 CET klee.Test-Comp24_coverage-error-call.SoftwareSystems-BusyBox-MemSafety

Add filter for:

KLEEF 2023-12-22 17:56:25 CET kleef.Test-Comp24_coverage-error-call.SoftwareSystems-BusyBox-MemSafety

Add filter for:

Legion 2023-12-21 08:25:46 CET legion.Test-Comp24_coverage-error-call.SoftwareSystems-BusyBox-MemSafety

Add filter for:

Legion/SymCC 2023-12-21 03:15:19 CET legion-symcc.Test-Comp24_coverage-error-call.SoftwareSystems-BusyBox-MemSafety

Add filter for:

Owi 2023-12-13 08:41:50 CET owic.Test-Comp24_coverage-error-call.SoftwareSystems-BusyBox-MemSafety

Add filter for:

PRTest 2023-12-10 11:48:15 CET prtest.Test-Comp24_coverage-error-call.SoftwareSystems-BusyBox-MemSafety

Add filter for:

Rizzer 2023-12-11 16:47:26 CET rizzer.Test-Comp24_coverage-error-call.SoftwareSystems-BusyBox-MemSafety

Add filter for:

symbiotic 2023-12-12 08:27:09 CET symbiotic.Test-Comp24_coverage-error-call.SoftwareSystems-BusyBox-MemSafety

Add filter for:

Tracer-X 2023-12-11 08:21:26 CET tracerx.Test-Comp24_coverage-error-call.SoftwareSystems-BusyBox-MemSafety

Add filter for:

TracerX-WP 2023-12-12 23:17:43 CET tracerx-wp.Test-Comp24_coverage-error-call.SoftwareSystems-BusyBox-MemSafety

Add filter for:

ULTIMATE TestGen 2023-12-12 15:06:35 CET utestgen.Test-Comp24_coverage-error-call.SoftwareSystems-BusyBox-MemSafety

Add filter for:

WASP-C 2023-12-20 19:53:37 CET wasp-c.Test-Comp24_coverage-error-call.SoftwareSystems-BusyBox-MemSafety

Add filter for:

Summary Table Quantile Plot Scatter Plot Info

Benchmark Setup

BenchmarkcetfuzzCPA/CoVeriTestESBMC-bkindFDSEfizzerFuSeBMCFuSeBMC_AIHybridTigerKLEEKLEEFLegionLegion/SymCCOwiPRtestRizzerSymbioticTracer-XTracerX-WPUTestGenWASP-C
Toolcetfuzz 1.0.0CPAchecker 2.2.1-svn-44999ESBMC version 7.0.0 64-bit x86_64 linuxFDSE testcomp24Fizzer 0.1FuSeBMC v.4.2.1FuSeBMC v.5.1.0CPAchecker 1.9.2-svn-35606MKLEE 2.2-pre-test-compKLEEF 3.0Legion 1.0-testcomp2021Legion/SymCC testcomp2023Owi %%VERSION%%PRTest v2.3Rizzer 0.1symbiotic 10.0.0-rc1-647ccad1Tracer-X 1.2.0TracerX-WP 1.2.0ULTIMATE TestGen 0.2.3-ea2a3342WASP-C version 0.1
Limitstimelimit: 900 s, memlimit: 15000 MB, CPU core limit: 4
Hostapollon*
OSLinux 5.15.0-91-generic
SystemCPU: Intel Xeon E3-1230 v5 @ 3.40 GHz, cores: 8, frequency: 3800 MHz, Turbo Boost: disabled; RAM: 33467 MB
Date of execution2023-12-13 03:30:21 CET2023-12-10 11:47:16 CET2023-12-22 03:20:00 CET2023-12-12 04:10:34 CET2023-12-10 16:44:12 CET2023-12-10 11:48:41 CET2023-12-11 00:20:48 CET2023-12-20 19:51:14 CET2023-12-21 19:27:39 CET2023-12-22 17:56:25 CET2023-12-21 08:25:46 CET2023-12-21 03:15:19 CET2023-12-13 08:41:50 CET2023-12-10 11:48:15 CET2023-12-11 16:47:26 CET2023-12-12 08:27:09 CET2023-12-11 08:21:26 CET2023-12-12 23:17:43 CET2023-12-12 15:06:35 CET2023-12-20 19:53:37 CET
Run setcetfuzz.Test-Comp24_coverage-error-call.SoftwareSystems-BusyBox-MemSafetycoveritest.Test-Comp24_coverage-error-call.SoftwareSystems-BusyBox-MemSafetyesbmc-kind.Test-Comp24_coverage-error-call.SoftwareSystems-BusyBox-MemSafetyfdse.Test-Comp24_coverage-error-call.SoftwareSystems-BusyBox-MemSafetyfizzer.Test-Comp24_coverage-error-call.SoftwareSystems-BusyBox-MemSafetyfusebmc.Test-Comp24_coverage-error-call.SoftwareSystems-BusyBox-MemSafetyfusebmc-ia.Test-Comp24_coverage-error-call.SoftwareSystems-BusyBox-MemSafetyhybridtiger.Test-Comp24_coverage-error-call.SoftwareSystems-BusyBox-MemSafetyklee.Test-Comp24_coverage-error-call.SoftwareSystems-BusyBox-MemSafetykleef.Test-Comp24_coverage-error-call.SoftwareSystems-BusyBox-MemSafetylegion.Test-Comp24_coverage-error-call.SoftwareSystems-BusyBox-MemSafetylegion-symcc.Test-Comp24_coverage-error-call.SoftwareSystems-BusyBox-MemSafetyowic.Test-Comp24_coverage-error-call.SoftwareSystems-BusyBox-MemSafetyprtest.Test-Comp24_coverage-error-call.SoftwareSystems-BusyBox-MemSafetyrizzer.Test-Comp24_coverage-error-call.SoftwareSystems-BusyBox-MemSafetysymbiotic.Test-Comp24_coverage-error-call.SoftwareSystems-BusyBox-MemSafetytracerx.Test-Comp24_coverage-error-call.SoftwareSystems-BusyBox-MemSafetytracerx-wp.Test-Comp24_coverage-error-call.SoftwareSystems-BusyBox-MemSafetyutestgen.Test-Comp24_coverage-error-call.SoftwareSystems-BusyBox-MemSafetywasp-c.Test-Comp24_coverage-error-call.SoftwareSystems-BusyBox-MemSafety
Options
  • -benchmark
  • -heap 10000M
  • -testcomp24
  • -setprop log.consoleLevel=SEVERE
  • -s kinduction
  • --testcomp
  • --max_seconds 865
  • --optimizer_max_seconds 30
  • --max_exec_milliseconds 500
  • --max_stdin_bytes 65536
  • --stdin_model stdin_replay_bytes_then_repeat_zero
  • --test_type testcomp
  • -s incr
  • -s incr
  • -benchmark
  • -heap 10000M
  • -tigertestcomp20
  • --score uct
  • --core 8
  • --tree-depth-limit 100000
  • --conex-timeout 0
  • --symex-timeout 0
  • --min-samples 1
  • --save-tests REDUCED
  • --persistent
  • --time-penalty 0
  • --random-seed 0
  • -o ./binary.instr
  • --compile svcomp
  • -a True
  • -q
  • -b ./binary
  • -e
  • --testcomp
  • --max_seconds 865
  • --optimizer_max_seconds 30
  • --max_exec_milliseconds 500
  • --max_stdin_bytes 65536
  • --stdin_model stdin_replay_bytes_then_repeat_zero
  • --test_type testcomp
  • --test-comp
  • --traceabstraction.test.generation.mode Standard
  • --full-output
  • --smt-assume
  • --policy breadth
  • --test-comp
Propertiescoverage-error-call

Statistics

cetfuzz 2023-12-13 03:30:21 CET cetfuzz.Test-Comp24_coverage-error-call.SoftwareSystems-BusyBox-MemSafety
CPAchecker 2023-12-10 11:47:16 CET coveritest.Test-Comp24_coverage-error-call.SoftwareSystems-BusyBox-MemSafety
ESBMC 2023-12-22 03:20:00 CET esbmc-kind.Test-Comp24_coverage-error-call.SoftwareSystems-BusyBox-MemSafety
FDSE 2023-12-12 04:10:34 CET fdse.Test-Comp24_coverage-error-call.SoftwareSystems-BusyBox-MemSafety
Fizzer 2023-12-10 16:44:12 CET fizzer.Test-Comp24_coverage-error-call.SoftwareSystems-BusyBox-MemSafety
FuSeBMC 2023-12-10 11:48:41 CET fusebmc.Test-Comp24_coverage-error-call.SoftwareSystems-BusyBox-MemSafety
FuSeBMC 2023-12-11 00:20:48 CET fusebmc-ia.Test-Comp24_coverage-error-call.SoftwareSystems-BusyBox-MemSafety
CPAchecker 2023-12-20 19:51:14 CET hybridtiger.Test-Comp24_coverage-error-call.SoftwareSystems-BusyBox-MemSafety
KLEE 2023-12-21 19:27:39 CET klee.Test-Comp24_coverage-error-call.SoftwareSystems-BusyBox-MemSafety
KLEEF 2023-12-22 17:56:25 CET kleef.Test-Comp24_coverage-error-call.SoftwareSystems-BusyBox-MemSafety
Legion 2023-12-21 08:25:46 CET legion.Test-Comp24_coverage-error-call.SoftwareSystems-BusyBox-MemSafety
Legion/SymCC 2023-12-21 03:15:19 CET legion-symcc.Test-Comp24_coverage-error-call.SoftwareSystems-BusyBox-MemSafety
Owi 2023-12-13 08:41:50 CET owic.Test-Comp24_coverage-error-call.SoftwareSystems-BusyBox-MemSafety
PRTest 2023-12-10 11:48:15 CET prtest.Test-Comp24_coverage-error-call.SoftwareSystems-BusyBox-MemSafety
Rizzer 2023-12-11 16:47:26 CET rizzer.Test-Comp24_coverage-error-call.SoftwareSystems-BusyBox-MemSafety
symbiotic 2023-12-12 08:27:09 CET symbiotic.Test-Comp24_coverage-error-call.SoftwareSystems-BusyBox-MemSafety
Tracer-X 2023-12-11 08:21:26 CET tracerx.Test-Comp24_coverage-error-call.SoftwareSystems-BusyBox-MemSafety
TracerX-WP 2023-12-12 23:17:43 CET tracerx-wp.Test-Comp24_coverage-error-call.SoftwareSystems-BusyBox-MemSafety
ULTIMATE TestGen 2023-12-12 15:06:35 CET utestgen.Test-Comp24_coverage-error-call.SoftwareSystems-BusyBox-MemSafety
WASP-C 2023-12-20 19:53:37 CET wasp-c.Test-Comp24_coverage-error-call.SoftwareSystems-BusyBox-MemSafety
Click here to select columns
 
status
cpu
(s)
mem
(MB)
 
status
cpu
(s)
mem
(MB)
 
status
cpu
(s)
mem
(MB)
 
status
cpu
(s)
mem
(MB)
 
status
raw score
cpu
(s)
mem
(MB)
 
status
raw score
cpu
(s)
mem
(MB)
 
status
raw score
cpu
(s)
mem
(MB)
 
status
cpu
(s)
mem
(MB)
 
status
raw score
cpu
(s)
mem
(MB)
 
status
cpu
(s)
mem
(MB)
 
status
cpu
(s)
mem
(MB)
 
status
cpu
(s)
mem
(MB)
 
status
cpu
(s)
mem
(MB)
 
status
cpu
(s)
mem
(MB)
 
status
cpu
(s)
mem
(MB)
 
status
raw score
cpu
(s)
mem
(MB)
 
status
raw score
cpu
(s)
mem
(MB)
 
status
cpu
(s)
mem
(MB)
 
status
cpu
(s)
mem
(MB)
 
status
cpu
(s)
mem
(MB)
all results
4
9.0
460
4
3600
32000
4
3600
49000
4
6.0
300
4
1
3500
2000
4
1
3000
1500
4
1
3200
1500
4
2700
36000
4
1
1800  
6500
4
1800
8300
4
3600
610
4
3600
510
4
6.9
180
4
2700
9800
4
2600
1600
4
1
1100
12000
4
1
780
46000
4
.27
40
4
1900
13000
4
69
430
    correct results
0
-
-
0
-
-
0
-
-
0
-
-
0
-
-
-
0
-
-
-
0
-
-
-
0
-
-
1
1
1.8
37
0
-
-
0
-
-
0
-
-
0
-
-
0
-
-
0
-
-
0
-
-
-
0
-
-
-
0
-
-
0
-
-
0
-
-
        correct true
0
-
-
0
-
-
0
-
-
0
-
-
0
-
-
-
0
-
-
-
0
-
-
-
0
-
-
0
-
-
-
0
-
-
0
-
-
0
-
-
0
-
-
0
-
-
0
-
-
0
-
-
-
0
-
-
-
0
-
-
0
-
-
0
-
-
        correct false
0
-
-
0
-
-
0
-
-
0
-
-
0
-
-
-
0
-
-
-
0
-
-
-
0
-
-
1
1
1.8
37
0
-
-
0
-
-
0
-
-
0
-
-
0
-
-
0
-
-
0
-
-
-
0
-
-
-
0
-
-
0
-
-
0
-
-
    incorrect results
0
-
-
0
-
-
0
-
-
0
-
-
0
-
-
-
0
-
-
-
0
-
-
-
0
-
-
0
-
-
-
0
-
-
0
-
-
0
-
-
0
-
-
0
-
-
0
-
-
0
-
-
-
0
-
-
-
0
-
-
0
-
-
0
-
-
        incorrect true
0
-
-
0
-
-
0
-
-
0
-
-
0
-
-
-
0
-
-
-
0
-
-
-
0
-
-
0
-
-
-
0
-
-
0
-
-
0
-
-
0
-
-
0
-
-
0
-
-
0
-
-
-
0
-
-
-
0
-
-
0
-
-
0
-
-
        incorrect false
0
-
-
0
-
-
0
-
-
0
-
-
0
-
-
-
0
-
-
-
0
-
-
-
0
-
-
0
-
-
-
0
-
-
0
-
-
0
-
-
0
-
-
0
-
-
0
-
-
0
-
-
-
0
-
-
-
0
-
-
0
-
-
0
-
-

Generated by BenchExec 3.20