Task filter


CPAchecker 2022-12-20 15:58:25 CET coveritest.Test-Comp23_coverage-error-call.ReachSafety-ProductLines

Add filter for:

ESBMC 2022-12-20 14:32:31 CET esbmc-kind.Test-Comp23_coverage-error-call.ReachSafety-ProductLines

Add filter for:

FuSeBMC 2022-12-20 14:35:10 CET FuSeBMC.Test-Comp23_coverage-error-call.ReachSafety-ProductLines

Add filter for:

FuSeBMC 2022-12-20 14:32:46 CET FuSeBMC_IA.Test-Comp23_coverage-error-call.ReachSafety-ProductLines

Add filter for:

CPAchecker 2022-12-20 14:55:20 CET hybridtiger.Test-Comp23_coverage-error-call.ReachSafety-ProductLines

Add filter for:

KLEE 2023-02-09 12:02:40 CET klee.Test-Comp23_coverage-error-call.ReachSafety-ProductLines

Add filter for:

Legion 2023-02-10 12:16:37 CET legion.Test-Comp23_coverage-error-call.ReachSafety-ProductLines

Add filter for:

Legion/SymCC 2022-12-21 10:08:19 CET legion-symcc.Test-Comp23_coverage-error-call.ReachSafety-ProductLines

Add filter for:

PRTest 2022-12-21 19:07:52 CET prtest.Test-Comp23_coverage-error-call.ReachSafety-ProductLines

Add filter for:

symbiotic 2022-12-21 18:27:28 CET symbiotic.Test-Comp23_coverage-error-call.ReachSafety-ProductLines

Add filter for:

Tracer-X 2022-12-22 00:18:16 CET tracerx.Test-Comp23_coverage-error-call.ReachSafety-ProductLines

Add filter for:

VeriFuzz 2022-12-21 23:42:42 CET verifuzz.Test-Comp23_coverage-error-call.ReachSafety-ProductLines

Add filter for:

WASP-C 2022-12-22 03:17:26 CET wasp-c.Test-Comp23_coverage-error-call.ReachSafety-ProductLines

Add filter for:

Summary Table Quantile Plot Scatter Plot Info

Benchmark Setup

BenchmarkCPA/CoVeriTestESBMC-bkindFuSeBMCFuSeBMC_IAHybridTigerKLEELegionLegion/SymCCPRtestSymbioticTracer-XVeriFuzzWASP-C
ToolCPAchecker 2.1.2-svn-41925ESBMC version 7.0.0 64-bit x86_64 linuxFuSeBMC v.4.2.1FuSeBMC v.5.0.1CPAchecker 1.9.2-svn-35606MKLEE 2.2-pre-test-compLegion 1.0-testcomp2021Legion/SymCC testcomp2023PRTest v2.3symbiotic 9.0.0-dev-b34dd3a1Tracer-X 1.2.0VeriFuzz 1.2.11WASP-C version 0.1
Limitstimelimit: 900 s, memlimit: 15000 MB, CPU core limit: 8
Hostapollon*
OSLinux 5.15.0-56-genericLinux 5.15.0-60-genericLinux 5.15.0-56-generic
SystemCPU: Intel Xeon E3-1230 v5 @ 3.40 GHz, cores: 8, frequency: 3800 MHz, Turbo Boost: disabled; RAM: 33480 MB
Date of execution2022-12-20 15:58:25 CET2022-12-20 14:32:31 CET2022-12-20 14:35:10 CET2022-12-20 14:32:46 CET2022-12-20 14:55:20 CET2023-02-09 12:02:40 CET2023-02-10 12:16:37 CET2022-12-21 10:08:19 CET2022-12-21 19:07:52 CET2022-12-21 18:27:28 CET2022-12-22 00:18:16 CET2022-12-21 23:42:42 CET2022-12-22 03:17:26 CET
Run setcoveritest.Test-Comp23_coverage-error-call.ReachSafety-ProductLinesesbmc-kind.Test-Comp23_coverage-error-call.ReachSafety-ProductLinesFuSeBMC.Test-Comp23_coverage-error-call.ReachSafety-ProductLinesFuSeBMC_IA.Test-Comp23_coverage-error-call.ReachSafety-ProductLineshybridtiger.Test-Comp23_coverage-error-call.ReachSafety-ProductLinesklee.Test-Comp23_coverage-error-call.ReachSafety-ProductLineslegion.Test-Comp23_coverage-error-call.ReachSafety-ProductLineslegion-symcc.Test-Comp23_coverage-error-call.ReachSafety-ProductLinesprtest.Test-Comp23_coverage-error-call.ReachSafety-ProductLinessymbiotic.Test-Comp23_coverage-error-call.ReachSafety-ProductLinestracerx.Test-Comp23_coverage-error-call.ReachSafety-ProductLinesverifuzz.Test-Comp23_coverage-error-call.ReachSafety-ProductLineswasp-c.Test-Comp23_coverage-error-call.ReachSafety-ProductLines
Options
  • -benchmark
  • -heap 10000M
  • -testcomp23
  • -setprop log.consoleLevel=SEVERE
  • -s kinduction
  • -s incr
  • -s incr
  • -benchmark
  • -heap 10000M
  • -tigertestcomp20
  • --score uct
  • --core 8
  • --tree-depth-limit 100000
  • --conex-timeout 0
  • --symex-timeout 0
  • --min-samples 1
  • --save-tests REDUCED
  • --persistent
  • --time-penalty 0
  • --random-seed 0
  • -o ./binary.instr
  • --compile svcomp
  • -a True
  • -q
  • -b ./binary
  • -e
  • --test-comp
  • --testcomp
  • --smt-assume
  • --policy breadth
  • --test-comp
Propertiescoverage-error-call

Statistics

CPAchecker 2022-12-20 15:58:25 CET coveritest.Test-Comp23_coverage-error-call.ReachSafety-ProductLines
ESBMC 2022-12-20 14:32:31 CET esbmc-kind.Test-Comp23_coverage-error-call.ReachSafety-ProductLines
FuSeBMC 2022-12-20 14:35:10 CET FuSeBMC.Test-Comp23_coverage-error-call.ReachSafety-ProductLines
FuSeBMC 2022-12-20 14:32:46 CET FuSeBMC_IA.Test-Comp23_coverage-error-call.ReachSafety-ProductLines
CPAchecker 2022-12-20 14:55:20 CET hybridtiger.Test-Comp23_coverage-error-call.ReachSafety-ProductLines
KLEE 2023-02-09 12:02:40 CET klee.Test-Comp23_coverage-error-call.ReachSafety-ProductLines
Legion 2023-02-10 12:16:37 CET legion.Test-Comp23_coverage-error-call.ReachSafety-ProductLines
Legion/SymCC 2022-12-21 10:08:19 CET legion-symcc.Test-Comp23_coverage-error-call.ReachSafety-ProductLines
PRTest 2022-12-21 19:07:52 CET prtest.Test-Comp23_coverage-error-call.ReachSafety-ProductLines
symbiotic 2022-12-21 18:27:28 CET symbiotic.Test-Comp23_coverage-error-call.ReachSafety-ProductLines
Tracer-X 2022-12-22 00:18:16 CET tracerx.Test-Comp23_coverage-error-call.ReachSafety-ProductLines
VeriFuzz 2022-12-21 23:42:42 CET verifuzz.Test-Comp23_coverage-error-call.ReachSafety-ProductLines
WASP-C 2022-12-22 03:17:26 CET wasp-c.Test-Comp23_coverage-error-call.ReachSafety-ProductLines
Click here to select columns
 
status
raw score
cpu
(s)
mem
(MB)
energy
(J)
 
status
raw score
cpu
(s)
mem
(MB)
energy
(J)
 
status
raw score
cpu
(s)
mem
(MB)
energy
(J)
 
status
raw score
cpu
(s)
mem
(MB)
energy
(J)
 
status
raw score
cpu
(s)
mem
(MB)
energy
(J)
 
status
raw score
cpu
(s)
mem
(MB)
energy
(J)
 
status
raw score
cpu
(s)
mem
(MB)
energy
(J)
 
status
raw score
cpu
(s)
mem
(MB)
energy
(J)
 
status
raw score
cpu
(s)
mem
(MB)
energy
(J)
 
status
raw score
cpu
(s)
mem
(MB)
energy
(J)
 
status
cpu
(s)
mem
(MB)
energy
(J)
 
status
raw score
cpu
(s)
mem
(MB)
energy
(J)
 
status
raw score
cpu
(s)
mem
(MB)
energy
(J)
all results
169
160
47000
390000
510000
169
169
380
15000
5000
169
169
1000
18000
15000
169
169
1000
18000
16000
169
45
140000
770000
1700000
169
169
82
4700
1000
169
34
110000
360000
1200000
169
159
8800
41000
120000
169
92
150000
5200
2000000
169
159
3400
9500
56000
169
21
2400
220
169
169
1300
26000
12000
169
5
1100
13000
16000
    correct results
0
-
-
-
-
0
-
-
-
-
0
-
-
-
-
0
-
-
-
-
0
-
-
-
-
169
169
82
4700
1000
0
-
-
-
-
125
125
330
19000
4500
0
-
-
-
-
0
-
-
-
-
0
-
-
-
0
-
-
-
-
0
-
-
-
-
        correct true
0
-
-
-
-
0
-
-
-
-
0
-
-
-
-
0
-
-
-
-
0
-
-
-
-
0
-
-
-
-
0
-
-
-
-
0
-
-
-
-
0
-
-
-
-
0
-
-
-
-
0
-
-
-
0
-
-
-
-
0
-
-
-
-
        correct false
0
-
-
-
-
0
-
-
-
-
0
-
-
-
-
0
-
-
-
-
0
-
-
-
-
169
169
82
4700
1000
0
-
-
-
-
125
125
330
19000
4500
0
-
-
-
-
0
-
-
-
-
0
-
-
-
0
-
-
-
-
0
-
-
-
-
    incorrect results
0
-
-
-
-
0
-
-
-
-
0
-
-
-
-
0
-
-
-
-
0
-
-
-
-
0
-
-
-
-
0
-
-
-
-
0
-
-
-
-
0
-
-
-
-
0
-
-
-
-
0
-
-
-
0
-
-
-
-
0
-
-
-
-
        incorrect true
0
-
-
-
-
0
-
-
-
-
0
-
-
-
-
0
-
-
-
-
0
-
-
-
-
0
-
-
-
-
0
-
-
-
-
0
-
-
-
-
0
-
-
-
-
0
-
-
-
-
0
-
-
-
0
-
-
-
-
0
-
-
-
-
        incorrect false
0
-
-
-
-
0
-
-
-
-
0
-
-
-
-
0
-
-
-
-
0
-
-
-
-
0
-
-
-
-
0
-
-
-
-
0
-
-
-
-
0
-
-
-
-
0
-
-
-
-
0
-
-
-
0
-
-
-
-
0
-
-
-
-

Generated by BenchExec 3.17-dev