Task filter


CPAchecker 2022-12-20 15:58:25 CET coveritest.Test-Comp23_coverage-error-call.SoftwareSystems-BusyBox-MemSafety

Add filter for:

ESBMC 2022-12-20 14:32:31 CET esbmc-kind.Test-Comp23_coverage-error-call.SoftwareSystems-BusyBox-MemSafety

Add filter for:

FuSeBMC 2022-12-20 14:35:10 CET FuSeBMC.Test-Comp23_coverage-error-call.SoftwareSystems-BusyBox-MemSafety

Add filter for:

FuSeBMC 2022-12-20 14:32:46 CET FuSeBMC_IA.Test-Comp23_coverage-error-call.SoftwareSystems-BusyBox-MemSafety

Add filter for:

CPAchecker 2022-12-20 14:55:20 CET hybridtiger.Test-Comp23_coverage-error-call.SoftwareSystems-BusyBox-MemSafety

Add filter for:

KLEE 2023-02-09 12:02:40 CET klee.Test-Comp23_coverage-error-call.SoftwareSystems-BusyBox-MemSafety

Add filter for:

Legion 2023-02-10 12:16:37 CET legion.Test-Comp23_coverage-error-call.SoftwareSystems-BusyBox-MemSafety

Add filter for:

Legion/SymCC 2022-12-21 10:08:19 CET legion-symcc.Test-Comp23_coverage-error-call.SoftwareSystems-BusyBox-MemSafety

Add filter for:

PRTest 2022-12-21 19:07:52 CET prtest.Test-Comp23_coverage-error-call.SoftwareSystems-BusyBox-MemSafety

Add filter for:

symbiotic 2022-12-21 18:27:28 CET symbiotic.Test-Comp23_coverage-error-call.SoftwareSystems-BusyBox-MemSafety

Add filter for:

Tracer-X 2022-12-22 00:18:16 CET tracerx.Test-Comp23_coverage-error-call.SoftwareSystems-BusyBox-MemSafety

Add filter for:

VeriFuzz 2022-12-21 23:42:42 CET verifuzz.Test-Comp23_coverage-error-call.SoftwareSystems-BusyBox-MemSafety

Add filter for:

WASP-C 2022-12-22 03:17:26 CET wasp-c.Test-Comp23_coverage-error-call.SoftwareSystems-BusyBox-MemSafety

Add filter for:

Summary Table Quantile Plot Scatter Plot Info

Benchmark Setup

BenchmarkCPA/CoVeriTestESBMC-bkindFuSeBMCFuSeBMC_IAHybridTigerKLEELegionLegion/SymCCPRtestSymbioticTracer-XVeriFuzzWASP-C
ToolCPAchecker 2.1.2-svn-41925ESBMC version 7.0.0 64-bit x86_64 linuxFuSeBMC v.4.2.1FuSeBMC v.5.0.1CPAchecker 1.9.2-svn-35606MKLEE 2.2-pre-test-compLegion 1.0-testcomp2021Legion/SymCC testcomp2023PRTest v2.3symbiotic 9.0.0-dev-b34dd3a1Tracer-X 1.2.0VeriFuzz 1.2.11WASP-C version 0.1
Limitstimelimit: 900 s, memlimit: 15000 MB, CPU core limit: 8
Hostapollon*
OSLinux 5.15.0-56-genericLinux 5.15.0-60-genericLinux 5.15.0-56-generic
SystemCPU: Intel Xeon E3-1230 v5 @ 3.40 GHz, cores: 8, frequency: 3800 MHz, Turbo Boost: disabled; RAM: 33480 MB
Date of execution2022-12-20 15:58:25 CET2022-12-20 14:32:31 CET2022-12-20 14:35:10 CET2022-12-20 14:32:46 CET2022-12-20 14:55:20 CET2023-02-09 12:02:40 CET2023-02-10 12:16:37 CET2022-12-21 10:08:19 CET2022-12-21 19:07:52 CET2022-12-21 18:27:28 CET2022-12-22 00:18:16 CET2022-12-21 23:42:42 CET2022-12-22 03:17:26 CET
Run setcoveritest.Test-Comp23_coverage-error-call.SoftwareSystems-BusyBox-MemSafetyesbmc-kind.Test-Comp23_coverage-error-call.SoftwareSystems-BusyBox-MemSafetyFuSeBMC.Test-Comp23_coverage-error-call.SoftwareSystems-BusyBox-MemSafetyFuSeBMC_IA.Test-Comp23_coverage-error-call.SoftwareSystems-BusyBox-MemSafetyhybridtiger.Test-Comp23_coverage-error-call.SoftwareSystems-BusyBox-MemSafetyklee.Test-Comp23_coverage-error-call.SoftwareSystems-BusyBox-MemSafetylegion.Test-Comp23_coverage-error-call.SoftwareSystems-BusyBox-MemSafetylegion-symcc.Test-Comp23_coverage-error-call.SoftwareSystems-BusyBox-MemSafetyprtest.Test-Comp23_coverage-error-call.SoftwareSystems-BusyBox-MemSafetysymbiotic.Test-Comp23_coverage-error-call.SoftwareSystems-BusyBox-MemSafetytracerx.Test-Comp23_coverage-error-call.SoftwareSystems-BusyBox-MemSafetyverifuzz.Test-Comp23_coverage-error-call.SoftwareSystems-BusyBox-MemSafetywasp-c.Test-Comp23_coverage-error-call.SoftwareSystems-BusyBox-MemSafety
Options
  • -benchmark
  • -heap 10000M
  • -testcomp23
  • -setprop log.consoleLevel=SEVERE
  • -s kinduction
  • -s incr
  • -s incr
  • -benchmark
  • -heap 10000M
  • -tigertestcomp20
  • --score uct
  • --core 8
  • --tree-depth-limit 100000
  • --conex-timeout 0
  • --symex-timeout 0
  • --min-samples 1
  • --save-tests REDUCED
  • --persistent
  • --time-penalty 0
  • --random-seed 0
  • -o ./binary.instr
  • --compile svcomp
  • -a True
  • -q
  • -b ./binary
  • -e
  • --test-comp
  • --testcomp
  • --smt-assume
  • --policy breadth
  • --test-comp
Propertiescoverage-error-call

Statistics

CPAchecker 2022-12-20 15:58:25 CET coveritest.Test-Comp23_coverage-error-call.SoftwareSystems-BusyBox-MemSafety
ESBMC 2022-12-20 14:32:31 CET esbmc-kind.Test-Comp23_coverage-error-call.SoftwareSystems-BusyBox-MemSafety
FuSeBMC 2022-12-20 14:35:10 CET FuSeBMC.Test-Comp23_coverage-error-call.SoftwareSystems-BusyBox-MemSafety
FuSeBMC 2022-12-20 14:32:46 CET FuSeBMC_IA.Test-Comp23_coverage-error-call.SoftwareSystems-BusyBox-MemSafety
CPAchecker 2022-12-20 14:55:20 CET hybridtiger.Test-Comp23_coverage-error-call.SoftwareSystems-BusyBox-MemSafety
KLEE 2023-02-09 12:02:40 CET klee.Test-Comp23_coverage-error-call.SoftwareSystems-BusyBox-MemSafety
Legion 2023-02-10 12:16:37 CET legion.Test-Comp23_coverage-error-call.SoftwareSystems-BusyBox-MemSafety
Legion/SymCC 2022-12-21 10:08:19 CET legion-symcc.Test-Comp23_coverage-error-call.SoftwareSystems-BusyBox-MemSafety
PRTest 2022-12-21 19:07:52 CET prtest.Test-Comp23_coverage-error-call.SoftwareSystems-BusyBox-MemSafety
symbiotic 2022-12-21 18:27:28 CET symbiotic.Test-Comp23_coverage-error-call.SoftwareSystems-BusyBox-MemSafety
Tracer-X 2022-12-22 00:18:16 CET tracerx.Test-Comp23_coverage-error-call.SoftwareSystems-BusyBox-MemSafety
VeriFuzz 2022-12-21 23:42:42 CET verifuzz.Test-Comp23_coverage-error-call.SoftwareSystems-BusyBox-MemSafety
WASP-C 2022-12-22 03:17:26 CET wasp-c.Test-Comp23_coverage-error-call.SoftwareSystems-BusyBox-MemSafety
Click here to select columns
 
status
cpu
(s)
mem
(MB)
energy
(J)
 
status
cpu
(s)
mem
(MB)
energy
(J)
 
status
raw score
cpu
(s)
mem
(MB)
energy
(J)
 
status
cpu
(s)
mem
(MB)
energy
(J)
 
status
cpu
(s)
mem
(MB)
energy
(J)
 
status
raw score
cpu
(s)
mem
(MB)
energy
(J)
 
status
cpu
(s)
mem
(MB)
energy
(J)
 
status
cpu
(s)
mem
(MB)
energy
(J)
 
status
cpu
(s)
mem
(MB)
energy
(J)
 
status
cpu
(s)
mem
(MB)
energy
(J)
 
status
cpu
(s)
mem
(MB)
energy
(J)
 
status
cpu
(s)
mem
(MB)
energy
(J)
 
status
cpu
(s)
mem
(MB)
energy
(J)
all results
5
4500
45000
39000
5
4300
75000
60000
5
1
3000
1900
36000
5
1200
140
16000
5
3100
58000
33000
5
1
1800  
6700
23000
5
3600
740
28000
5
3600
640
44000
5
3600
15000
45000
5
910
270
11000
5
.65
75
7.0
5
2800
2800
34000
5
64
530
900
    correct results
0
-
-
-
0
-
-
-
0
-
-
-
-
0
-
-
-
0
-
-
-
1
1
1.8
36
22
0
-
-
-
0
-
-
-
0
-
-
-
0
-
-
-
0
-
-
-
0
-
-
-
0
-
-
-
        correct true
0
-
-
-
0
-
-
-
0
-
-
-
-
0
-
-
-
0
-
-
-
0
-
-
-
-
0
-
-
-
0
-
-
-
0
-
-
-
0
-
-
-
0
-
-
-
0
-
-
-
0
-
-
-
        correct false
0
-
-
-
0
-
-
-
0
-
-
-
-
0
-
-
-
0
-
-
-
1
1
1.8
36
22
0
-
-
-
0
-
-
-
0
-
-
-
0
-
-
-
0
-
-
-
0
-
-
-
0
-
-
-
    incorrect results
0
-
-
-
0
-
-
-
0
-
-
-
-
0
-
-
-
0
-
-
-
0
-
-
-
-
0
-
-
-
0
-
-
-
0
-
-
-
0
-
-
-
0
-
-
-
0
-
-
-
0
-
-
-
        incorrect true
0
-
-
-
0
-
-
-
0
-
-
-
-
0
-
-
-
0
-
-
-
0
-
-
-
-
0
-
-
-
0
-
-
-
0
-
-
-
0
-
-
-
0
-
-
-
0
-
-
-
0
-
-
-
        incorrect false
0
-
-
-
0
-
-
-
0
-
-
-
-
0
-
-
-
0
-
-
-
0
-
-
-
-
0
-
-
-
0
-
-
-
0
-
-
-
0
-
-
-
0
-
-
-
0
-
-
-
0
-
-
-

Generated by BenchExec 3.17-dev