Benchmark | CPA/CoVeriTest | TBF Test-Suite Validator | ||||||||||||||||||||||||
Tool | CPAchecker 1.8-svn 30375 | Tbf Test-suite Validator v1.2 | ||||||||||||||||||||||||
Limits | timelimit: 900 s, memlimit: 15000 MB, CPU core limit: 8 | timelimit: 10800 s, memlimit: 7000 MB, CPU core limit: 2 | ||||||||||||||||||||||||
Host | apollon* | [apollon049; apollon085; apollon116] | ||||||||||||||||||||||||
OS | Linux 4.15.0-45-generic | |||||||||||||||||||||||||
System | CPU: Intel Xeon E3-1230 v5 @ 3.40 GHz, cores: 8, frequency: 3.8 GHz, Turbo Boost: disabled; RAM: 33546 MB | |||||||||||||||||||||||||
Date of execution | 2019-02-05 21:05:56 CET | 2019-02-06 05:30:26 CET | ||||||||||||||||||||||||
Run set | coveritest.test-comp19_prop-coverage-error-call.ReachSafety-BitVectors | tbf-testsuite-validator-coveritest.test-comp19_prop-coverage-error-call.ReachSafety-BitVectors | ||||||||||||||||||||||||
Options | -benchmark -heap 10000M -testcomp19 | --verbose --sequence-file cov-seq.txt -r --stop-after-found-violation --test-suite ../../results-verified/coveritest.2019-02-05_2105.logfiles/${rundefinition_name}.${inputfile_name}.files/test-suite.zip | ||||||||||||||||||||||||
sv-benchmarks/c/ | status | score | program | test-suite | TS size | inspect test-suite | cpu (s) | wall (s) | mem (MB) | energy (J) | blkio-w (MB) | blkio-r (MB) | status | program | test-suite | TS size | inspect test-suite | cov | branch cov (%) | line cov (%) | cpu (s) | wall (s) | mem (MB) | energy | blkio-w (MB) | blkio-r (MB) |
bitvector/byte_add-1.yml | done | 1 | prog | tests | 1 | inspect | 33 | 24 | 1100 | 380 | .037 | 0 | true | prog | tests | 1 | inspect | 66.7 | 95.3 | .13 | .15 | 16 | .086 | 0 | ||
bitvector/s3_clnt_1.BV.c.cil-2.yml | done | 1 | prog | tests | 1 | inspect | 9.3 | 2.9 | 320 | 82 | .086 | 0 | true | prog | tests | 1 | inspect | 29.0 | 44.5 | .28 | .28 | 17 | .12 | 0 | ||
bitvector/s3_clnt_2.BV.c.cil-2.yml | done | 1 | prog | tests | 1 | inspect | 35 | 16 | 1000 | 330 | 1.3 | 0 | true | prog | tests | 1 | inspect | 30.6 | 41.9 | .28 | .28 | 17 | .12 | 0 | ||
bitvector/s3_clnt_3.BV.c.cil-2.yml | done | 1 | prog | tests | 1 | inspect | 7.2 | 2.4 | 350 | 58 | .061 | 0 | true | prog | tests | 1 | inspect | 22.2 | 37.7 | .27 | .28 | 17 | .12 | 0 | ||
bitvector/soft_float_1-3.c.cil.yml | done | 1 | prog | tests | 1 | inspect | 23 | 6.2 | 790 | 200 | 1.5 | 0 | true | prog | tests | 1 | inspect | 18.9 | 29.6 | .14 | .14 | 17 | .098 | 0 | ||
bitvector/soft_float_4-3.c.cil.yml | done | 1 | prog | tests | 1 | inspect | 16 | 4.4 | 600 | 130 | .89 | 0 | true | prog | tests | 1 | inspect | 23.8 | 33.1 | .13 | .13 | 16 | .090 | 0 | ||
bitvector/sum02-1.yml | timeout | prog | tests | 0 | inspect | 900 | 750 | 5500 | 9000 | 43 | 0 | error (0) | prog | tests | 0 | inspect | 0 | 0 | .064 | .065 | 8.9 | 0 | 0 | |||
bitvector-regression/recHanoi03-1.yml | out of memory | prog | tests | 0 | inspect | 140 | 78 | 15000 | 1400 | 60 | 0 | error (0) | prog | tests | 0 | inspect | 0 | 0 | .066 | .066 | 9.0 | 0 | 0 | |||
bitvector-loops/diamond_2-1.yml | done | 1 | prog | tests | 1 | inspect | 25 | 6.6 | 1100 | 230 | 2.3 | 0 | true | prog | tests | 1 | inspect | 54.2 | 96.5 | .12 | .12 | 16 | .078 | 0 | ||
bitvector-loops/verisec_sendmail_tTflag_arr_one_loop.yml | timeout | prog | tests | 0 | inspect | 900 | 880 | 1600 | 12000 | 1.3 | 0 | error (0) | prog | tests | 0 | inspect | 0 | 0 | .056 | .057 | 9.0 | 0 | 0 | |||
sv-benchmarks/c/ | status | score | program | test-suite | TS size | inspect test-suite | cpu (s) | wall (s) | mem (MB) | energy (J) | blkio-w (MB) | blkio-r (MB) | status | program | test-suite | TS size | inspect test-suite | cov | branch cov (%) | line cov (%) | cpu (s) | wall (s) | mem (MB) | energy | blkio-w (MB) | blkio-r (MB) |
total | 10 | 7 | 2100 | 1800 | 27000 | 24000 | 110 | 0 | 10 | 245 | 379 | 1.5 | 1.6 | 140 | .71 | 0 | ||||||||||
Run set | coveritest.test-comp19_prop-coverage-error-call.ReachSafety-BitVectors | tbf-testsuite-validator-coveritest.test-comp19_prop-coverage-error-call.ReachSafety-BitVectors |