Benchmark Setup

BenchmarkCPA/CoVeriTest ESBMC-falsif HybridTiger KLEE Legion LibKluzzer PRtest Symbiotic Tracer-X VeriFuzz
ToolCPAchecker 1.8-svn-32236 ESBMC version 6.0.0 64-bit x86_64 linux CPAchecker 1.8-svn-32283M KLEE 2.1-pre-test-comp Legion 0.1-testcomp2020 LibKluzzer 0.6 PRTest v2.1 symbiotic 7.0.0-dev-0b4b6e25 Tracer-X 1.2.0 VeriFuzz 1.1.0
Limitstimelimit: 900 s, memlimit: 15000 MB, CPU core limit: 8
Hostapollon*
OSLinux 4.15.0-72-generic
SystemCPU: Intel Xeon E3-1230 v5 @ 3.40 GHz, cores: 8, frequency: 3800 MHz, Turbo Boost: disabled; RAM: 33546 MB
Date of execution2019-12-08 21:02:15 CET 2019-12-08 21:07:15 CET 2019-12-08 21:12:15 CET 2019-12-08 21:17:14 CET 2019-12-08 21:22:16 CET 2019-12-08 21:27:14 CET 2019-12-09 00:39:20 CET 2019-12-09 10:23:02 CET 2019-12-09 13:19:09 CET 2019-12-09 13:14:27 CET
Run setcoveritest.[test-comp20_prop-coverage-error-call.ReachSafety-Arrays; test-comp20_prop-coverage-error-call.ReachSafety-BitVectors; test-comp20_prop-coverage-error-call.ReachSafety-ControlFlow; test-comp20_prop-coverage-error-call.ReachSafety-ECA; test-comp20_prop-coverage-error-call.ReachSafety-Floats; test-comp20_prop-coverage-error-call.ReachSafety-Heap; test-comp20_prop-coverage-error-call.ReachSafety-Loops; test-comp20_prop-coverage-error-call.ReachSafety-Recursive; test-comp20_prop-coverage-error-call.ReachSafety-Sequentialized] esbmc-falsi.[test-comp20_prop-coverage-error-call.ReachSafety-Arrays; test-comp20_prop-coverage-error-call.ReachSafety-BitVectors; test-comp20_prop-coverage-error-call.ReachSafety-ControlFlow; test-comp20_prop-coverage-error-call.ReachSafety-ECA; test-comp20_prop-coverage-error-call.ReachSafety-Floats; test-comp20_prop-coverage-error-call.ReachSafety-Heap; test-comp20_prop-coverage-error-call.ReachSafety-Loops; test-comp20_prop-coverage-error-call.ReachSafety-Recursive; test-comp20_prop-coverage-error-call.ReachSafety-Sequentialized] hybridtiger.[test-comp20_prop-coverage-error-call.ReachSafety-Arrays; test-comp20_prop-coverage-error-call.ReachSafety-BitVectors; test-comp20_prop-coverage-error-call.ReachSafety-ControlFlow; test-comp20_prop-coverage-error-call.ReachSafety-ECA; test-comp20_prop-coverage-error-call.ReachSafety-Floats; test-comp20_prop-coverage-error-call.ReachSafety-Heap; test-comp20_prop-coverage-error-call.ReachSafety-Loops; test-comp20_prop-coverage-error-call.ReachSafety-Recursive; test-comp20_prop-coverage-error-call.ReachSafety-Sequentialized] klee.[test-comp20_prop-coverage-error-call.ReachSafety-Arrays; test-comp20_prop-coverage-error-call.ReachSafety-BitVectors; test-comp20_prop-coverage-error-call.ReachSafety-ControlFlow; test-comp20_prop-coverage-error-call.ReachSafety-ECA; test-comp20_prop-coverage-error-call.ReachSafety-Floats; test-comp20_prop-coverage-error-call.ReachSafety-Heap; test-comp20_prop-coverage-error-call.ReachSafety-Loops; test-comp20_prop-coverage-error-call.ReachSafety-Recursive; test-comp20_prop-coverage-error-call.ReachSafety-Sequentialized] legion.[test-comp20_prop-coverage-error-call.ReachSafety-Arrays; test-comp20_prop-coverage-error-call.ReachSafety-BitVectors; test-comp20_prop-coverage-error-call.ReachSafety-ControlFlow; test-comp20_prop-coverage-error-call.ReachSafety-ECA; test-comp20_prop-coverage-error-call.ReachSafety-Floats; test-comp20_prop-coverage-error-call.ReachSafety-Heap; test-comp20_prop-coverage-error-call.ReachSafety-Loops; test-comp20_prop-coverage-error-call.ReachSafety-Recursive; test-comp20_prop-coverage-error-call.ReachSafety-Sequentialized] libkluzzer.[test-comp20_prop-coverage-error-call.ReachSafety-Arrays; test-comp20_prop-coverage-error-call.ReachSafety-BitVectors; test-comp20_prop-coverage-error-call.ReachSafety-ControlFlow; test-comp20_prop-coverage-error-call.ReachSafety-ECA; test-comp20_prop-coverage-error-call.ReachSafety-Floats; test-comp20_prop-coverage-error-call.ReachSafety-Heap; test-comp20_prop-coverage-error-call.ReachSafety-Loops; test-comp20_prop-coverage-error-call.ReachSafety-Recursive; test-comp20_prop-coverage-error-call.ReachSafety-Sequentialized] prtest.[test-comp20_prop-coverage-error-call.ReachSafety-Arrays; test-comp20_prop-coverage-error-call.ReachSafety-BitVectors; test-comp20_prop-coverage-error-call.ReachSafety-ControlFlow; test-comp20_prop-coverage-error-call.ReachSafety-ECA; test-comp20_prop-coverage-error-call.ReachSafety-Floats; test-comp20_prop-coverage-error-call.ReachSafety-Heap; test-comp20_prop-coverage-error-call.ReachSafety-Loops; test-comp20_prop-coverage-error-call.ReachSafety-Recursive; test-comp20_prop-coverage-error-call.ReachSafety-Sequentialized] symbiotic.[test-comp20_prop-coverage-error-call.ReachSafety-Arrays; test-comp20_prop-coverage-error-call.ReachSafety-BitVectors; test-comp20_prop-coverage-error-call.ReachSafety-ControlFlow; test-comp20_prop-coverage-error-call.ReachSafety-ECA; test-comp20_prop-coverage-error-call.ReachSafety-Floats; test-comp20_prop-coverage-error-call.ReachSafety-Heap; test-comp20_prop-coverage-error-call.ReachSafety-Loops; test-comp20_prop-coverage-error-call.ReachSafety-Recursive; test-comp20_prop-coverage-error-call.ReachSafety-Sequentialized] tracerx.[test-comp20_prop-coverage-error-call.ReachSafety-Arrays; test-comp20_prop-coverage-error-call.ReachSafety-BitVectors; test-comp20_prop-coverage-error-call.ReachSafety-ControlFlow; test-comp20_prop-coverage-error-call.ReachSafety-ECA; test-comp20_prop-coverage-error-call.ReachSafety-Floats; test-comp20_prop-coverage-error-call.ReachSafety-Heap; test-comp20_prop-coverage-error-call.ReachSafety-Loops; test-comp20_prop-coverage-error-call.ReachSafety-Recursive; test-comp20_prop-coverage-error-call.ReachSafety-Sequentialized] verifuzz.[test-comp20_prop-coverage-error-call.ReachSafety-Arrays; test-comp20_prop-coverage-error-call.ReachSafety-BitVectors; test-comp20_prop-coverage-error-call.ReachSafety-ControlFlow; test-comp20_prop-coverage-error-call.ReachSafety-ECA; test-comp20_prop-coverage-error-call.ReachSafety-Floats; test-comp20_prop-coverage-error-call.ReachSafety-Heap; test-comp20_prop-coverage-error-call.ReachSafety-Loops; test-comp20_prop-coverage-error-call.ReachSafety-Recursive; test-comp20_prop-coverage-error-call.ReachSafety-Sequentialized]
Options
  • -benchmark
  • -heap 10000M
  • -testcomp20
  • -setprop log.consoleLevel=SEVERE
  • -s falsi
  • -benchmark
  • -heap 10000M
  • -tigertestcomp20
  • --save-tests
  • --persistent
  • --time-penalty 0
  • --random-seed 0
  • --symex-timeout 10
  • --conex-timeout 10
  • -o ./binary.instr
  • --compile svcomp
  • --test-comp
  • --testcomp

Statistics

CPAchecker 2019-12-08 21:02:15 CET coveritest.[test-comp20_prop-coverage-error-call.ReachSafety-Arrays; test-comp20_prop-coverage-error-call.ReachSafety-BitVectors; test-comp20_prop-coverage-error-call.ReachSafety-ControlFlow; test-comp20_prop-coverage-error-call.ReachSafety-ECA; test-comp20_prop-coverage-error-call.ReachSafety-Floats; test-comp20_prop-coverage-error-call.ReachSafety-Heap; test-comp20_prop-coverage-error-call.ReachSafety-Loops; test-comp20_prop-coverage-error-call.ReachSafety-Recursive; test-comp20_prop-coverage-error-call.ReachSafety-Sequentialized]
ESBMC 2019-12-08 21:07:15 CET esbmc-falsi.[test-comp20_prop-coverage-error-call.ReachSafety-Arrays; test-comp20_prop-coverage-error-call.ReachSafety-BitVectors; test-comp20_prop-coverage-error-call.ReachSafety-ControlFlow; test-comp20_prop-coverage-error-call.ReachSafety-ECA; test-comp20_prop-coverage-error-call.ReachSafety-Floats; test-comp20_prop-coverage-error-call.ReachSafety-Heap; test-comp20_prop-coverage-error-call.ReachSafety-Loops; test-comp20_prop-coverage-error-call.ReachSafety-Recursive; test-comp20_prop-coverage-error-call.ReachSafety-Sequentialized]
CPAchecker 2019-12-08 21:12:15 CET hybridtiger.[test-comp20_prop-coverage-error-call.ReachSafety-Arrays; test-comp20_prop-coverage-error-call.ReachSafety-BitVectors; test-comp20_prop-coverage-error-call.ReachSafety-ControlFlow; test-comp20_prop-coverage-error-call.ReachSafety-ECA; test-comp20_prop-coverage-error-call.ReachSafety-Floats; test-comp20_prop-coverage-error-call.ReachSafety-Heap; test-comp20_prop-coverage-error-call.ReachSafety-Loops; test-comp20_prop-coverage-error-call.ReachSafety-Recursive; test-comp20_prop-coverage-error-call.ReachSafety-Sequentialized]
KLEE 2019-12-08 21:17:14 CET klee.[test-comp20_prop-coverage-error-call.ReachSafety-Arrays; test-comp20_prop-coverage-error-call.ReachSafety-BitVectors; test-comp20_prop-coverage-error-call.ReachSafety-ControlFlow; test-comp20_prop-coverage-error-call.ReachSafety-ECA; test-comp20_prop-coverage-error-call.ReachSafety-Floats; test-comp20_prop-coverage-error-call.ReachSafety-Heap; test-comp20_prop-coverage-error-call.ReachSafety-Loops; test-comp20_prop-coverage-error-call.ReachSafety-Recursive; test-comp20_prop-coverage-error-call.ReachSafety-Sequentialized]
Legion 2019-12-08 21:22:16 CET legion.[test-comp20_prop-coverage-error-call.ReachSafety-Arrays; test-comp20_prop-coverage-error-call.ReachSafety-BitVectors; test-comp20_prop-coverage-error-call.ReachSafety-ControlFlow; test-comp20_prop-coverage-error-call.ReachSafety-ECA; test-comp20_prop-coverage-error-call.ReachSafety-Floats; test-comp20_prop-coverage-error-call.ReachSafety-Heap; test-comp20_prop-coverage-error-call.ReachSafety-Loops; test-comp20_prop-coverage-error-call.ReachSafety-Recursive; test-comp20_prop-coverage-error-call.ReachSafety-Sequentialized]
LibKluzzer 2019-12-08 21:27:14 CET libkluzzer.[test-comp20_prop-coverage-error-call.ReachSafety-Arrays; test-comp20_prop-coverage-error-call.ReachSafety-BitVectors; test-comp20_prop-coverage-error-call.ReachSafety-ControlFlow; test-comp20_prop-coverage-error-call.ReachSafety-ECA; test-comp20_prop-coverage-error-call.ReachSafety-Floats; test-comp20_prop-coverage-error-call.ReachSafety-Heap; test-comp20_prop-coverage-error-call.ReachSafety-Loops; test-comp20_prop-coverage-error-call.ReachSafety-Recursive; test-comp20_prop-coverage-error-call.ReachSafety-Sequentialized]
PRTest 2019-12-09 00:39:20 CET prtest.[test-comp20_prop-coverage-error-call.ReachSafety-Arrays; test-comp20_prop-coverage-error-call.ReachSafety-BitVectors; test-comp20_prop-coverage-error-call.ReachSafety-ControlFlow; test-comp20_prop-coverage-error-call.ReachSafety-ECA; test-comp20_prop-coverage-error-call.ReachSafety-Floats; test-comp20_prop-coverage-error-call.ReachSafety-Heap; test-comp20_prop-coverage-error-call.ReachSafety-Loops; test-comp20_prop-coverage-error-call.ReachSafety-Recursive; test-comp20_prop-coverage-error-call.ReachSafety-Sequentialized]
symbiotic 2019-12-09 10:23:02 CET symbiotic.[test-comp20_prop-coverage-error-call.ReachSafety-Arrays; test-comp20_prop-coverage-error-call.ReachSafety-BitVectors; test-comp20_prop-coverage-error-call.ReachSafety-ControlFlow; test-comp20_prop-coverage-error-call.ReachSafety-ECA; test-comp20_prop-coverage-error-call.ReachSafety-Floats; test-comp20_prop-coverage-error-call.ReachSafety-Heap; test-comp20_prop-coverage-error-call.ReachSafety-Loops; test-comp20_prop-coverage-error-call.ReachSafety-Recursive; test-comp20_prop-coverage-error-call.ReachSafety-Sequentialized]
Tracer-X 2019-12-09 13:19:09 CET tracerx.[test-comp20_prop-coverage-error-call.ReachSafety-Arrays; test-comp20_prop-coverage-error-call.ReachSafety-BitVectors; test-comp20_prop-coverage-error-call.ReachSafety-ControlFlow; test-comp20_prop-coverage-error-call.ReachSafety-ECA; test-comp20_prop-coverage-error-call.ReachSafety-Floats; test-comp20_prop-coverage-error-call.ReachSafety-Heap; test-comp20_prop-coverage-error-call.ReachSafety-Loops; test-comp20_prop-coverage-error-call.ReachSafety-Recursive; test-comp20_prop-coverage-error-call.ReachSafety-Sequentialized]
VeriFuzz 2019-12-09 13:14:27 CET verifuzz.[test-comp20_prop-coverage-error-call.ReachSafety-Arrays; test-comp20_prop-coverage-error-call.ReachSafety-BitVectors; test-comp20_prop-coverage-error-call.ReachSafety-ControlFlow; test-comp20_prop-coverage-error-call.ReachSafety-ECA; test-comp20_prop-coverage-error-call.ReachSafety-Floats; test-comp20_prop-coverage-error-call.ReachSafety-Heap; test-comp20_prop-coverage-error-call.ReachSafety-Loops; test-comp20_prop-coverage-error-call.ReachSafety-Recursive; test-comp20_prop-coverage-error-call.ReachSafety-Sequentialized]
Click here to select columns
status
raw score
cpu
(s)
mem
(MB)
energy
(J)
status
raw score
cpu
(s)
mem
(MB)
energy
(J)
status
raw score
cpu
(s)
mem
(MB)
energy
(J)
status
raw score
cpu
(s)
mem
(MB)
energy
(J)
status
raw score
cpu
(s)
mem
(MB)
energy
(J)
status
raw score
cpu
(s)
mem
(MB)
energy
(J)
status
raw score
cpu
(s)
mem
(MB)
energy
(J)
status
raw score
cpu
(s)
mem
(MB)
energy
(J)
status
raw score
cpu
(s)
mem
(MB)
energy
(J)
status
raw score
cpu
(s)
mem
(MB)
energy
(J)
total
699
351
340000  
3000000
3400000
699
470
170000   
2600000
1900000  
699
369
310000  
3100000
3100000
699
473
190000   
1600000
1700000  
699
120
180000  
8500000
2100000
699
537
610000   
1500000
6100000  
699
30
590000   
280000
8400000  
699
395
180000   
3800000
1900000  
699
204
260000    
2100000
3300000   
699
638
110000  
420000
1400000
Loading...

Generated by BenchExec 2.5.1