Benchmark KLEE TBF Test-Suite Validator
Tool KLEE 2.0.0-pre-test-comp Tbf Test-suite Validator v1.2
Limits timelimit: 900 s, memlimit: 15000 MB, CPU core limit: 8 timelimit: 10800 s, memlimit: 7000 MB, CPU core limit: 2
Host apollon*
OS Linux 4.15.0-45-generic
System CPU: Intel Xeon E3-1230 v5 @ 3.40 GHz, cores: 8, frequency: 3.8 GHz, Turbo Boost: disabled; RAM: 33546 MB
Date of execution 2019-02-05 21:09:47 CET 2019-02-06 06:16:47 CET
Run set klee.test-comp19_prop-coverage-branches.ReachSafety-Sequentialized tbf-testsuite-validator-klee.test-comp19_prop-coverage-branches.ReachSafety-Sequentialized
Options --verbose --sequence-file cov-seq.txt -r --test-suite ../../results-verified/klee.2019-02-05_2109.logfiles/${rundefinition_name}.${inputfile_name}.files/test-suite.zip
sv-benchmarks/c/seq-mthreaded/ status score program test-suite TS size inspect test-suite cpu (s) wall (s) mem (MB) energy (J) blkio-w (MB) blkio-r (MB) status program test-suite TS size inspect test-suite cov branch cov (%) line cov (%) cpu (s) wall (s) mem (MB) energy blkio-w (MB) blkio-r (MB)
pals_STARTPALS_ActiveStandby.1.ufo.BOUNDED-10.pals.yml timeout (false(unreach-call)) .753 prog tests 53 inspect 900    900    5200 9000   .094 0   true prog tests 1 inspect
75.3 90.2 .41 .46 17 2.7 0  
pals_STARTPALS_ActiveStandby.4_1.ufo.BOUNDED-10.pals.yml timeout (false(unreach-call)) .758 prog tests 50 inspect 900    900    2600 11000   .094 0   true prog tests 1 inspect
75.8 90.6 .40 .37 17 2.6 0  
pals_STARTPALS_ActiveStandby.4_2.ufo.BOUNDED-10.pals.yml timeout (false(unreach-call)) .753 prog tests 53 inspect 900    900    3400 7800   .094 0   true prog tests 1 inspect
75.3 90.5 .41 .40 17 2.7 0  
pals_STARTPALS_ActiveStandby.5.ufo.BOUNDED-10.pals.yml timeout (false(unreach-call)) .767 prog tests 50 inspect 900    900    4900 6800   .098 0   true prog tests 1 inspect
76.7 90.7 .39 .39 17 2.6 0  
pals_STARTPALS_ActiveStandby.ufo.BOUNDED-10.pals.yml timeout .742 prog tests 46 inspect 900    900    5100 8200   .10  0   unknown prog tests 0 inspect
74.2 89.5 .36 .33 17 2.4 0  
pals_STARTPALS_Triplicated.1.ufo.BOUNDED-10.pals.yml timeout (false(unreach-call)) .838 prog tests 70 inspect 900    900    3400 10000   .098 0   true prog tests 1 inspect
83.8 92.8 .48 .54 17 3.3 0  
pals_STARTPALS_Triplicated.2.ufo.BOUNDED-10.pals.yml timeout (false(unreach-call)) .797 prog tests 64 inspect 900    900    4700 9600   .098 0   true prog tests 1 inspect
79.8 89.8 .45 .43 17 3.0 0  
pals_STARTPALS_Triplicated.ufo.BOUNDED-10.pals.yml timeout .821 prog tests 66 inspect 900    900    4900 10000   .098 0   unknown prog tests 0 inspect
82.1 90.7 .46 .49 17 3.1 0  
pals_floodmax.3.1.ufo.BOUNDED-6.pals.yml false(unreach-call) .847 prog tests 77 inspect 1.8  1.8  39 18   .033 0   true prog tests 1 inspect
84.7 93.8 .51 .60 17 3.5 0  
pals_floodmax.3.2.ufo.BOUNDED-6.pals.yml false(unreach-call) .748 prog tests 70 inspect .73 .75 34 8.9 .033 0   true prog tests 1 inspect
74.8 87.7 .50 .49 17 3.3 0  
pals_floodmax.3.3.ufo.BOUNDED-6.pals.yml false(unreach-call) .792 prog tests 71 inspect .98 1.0  34 12   .033 0   true prog tests 1 inspect
79.2 90.5 .48 .48 17 3.3 0  
pals_floodmax.3.4.ufo.BOUNDED-6.pals.yml false(unreach-call) .827 prog tests 73 inspect 1.0  1.0  33 13   .033 0   true prog tests 1 inspect
82.7 91.9 .50 .54 17 3.4 0  
pals_floodmax.3.ufo.BOUNDED-6.pals.yml done .817 prog tests 71 inspect 1.1  1.1  34 14   .033 0   unknown prog tests 0 inspect
81.7 91.2 .46 .42 17 3.3 0  
pals_floodmax.4.1.ufo.BOUNDED-8.pals.yml timeout .820 prog tests 148 inspect 900    900    3100 8600   .13  0   true prog tests 1 inspect
82.0 91.4 1.0  .96 20 11   0  
pals_floodmax.4.2.ufo.BOUNDED-8.pals.yml timeout .763 prog tests 96 inspect 900    900    3100 8400   .11  0   true prog tests 1 inspect
76.3 88.8 .74 .73 20 7.6 0  
pals_floodmax.4.3.ufo.BOUNDED-8.pals.yml timeout .761 prog tests 95 inspect 900    900    3100 11000   .11  0   true prog tests 1 inspect
76.1 88.0 .74 .70 20 7.5 0  
pals_floodmax.4.4.ufo.BOUNDED-8.pals.yml timeout .779 prog tests 97 inspect 900    900    3100 11000   .12  0   unknown prog tests 0 inspect
77.9 89.1 .70 .68 20 7.7 0  
pals_floodmax.4.ufo.BOUNDED-8.pals.yml timeout .787 prog tests 97 inspect 900    900    3100 8800   .12  0   unknown prog tests 0 inspect
78.7 89.4 .72 .70 20 7.7 0  
pals_floodmax.5.1.ufo.BOUNDED-10.pals.yml timeout .883 prog tests 342 inspect 930    930    15000 5800   .10  0   true prog tests 1 inspect
88.3 93.7 2.8  2.6  27 50   0  
pals_floodmax.5.2.ufo.BOUNDED-10.pals.yml timeout .833 prog tests 277 inspect 900    900    15000 5100   .22  0   true prog tests 1 inspect
83.3 91.2 2.3  2.8  27 43   0  
pals_floodmax.5.3.ufo.BOUNDED-10.pals.yml timeout .864 prog tests 273 inspect 900    900    15000 5200   .22  0   true prog tests 1 inspect
86.4 92.5 2.3  2.5  27 43   0  
pals_floodmax.5.4.ufo.BOUNDED-10.pals.yml timeout .855 prog tests 246 inspect 900    900    15000 7200   .21  0   unknown prog tests 0 inspect
85.5 91.8 2.0  2.1  27 39   0  
pals_floodmax.5.ufo.BOUNDED-10.pals.yml timeout .874 prog tests 276 inspect 900    900    15000 6200   .22  0   unknown prog tests 0 inspect
87.4 93.0 2.3  2.2  27 43   0  
pals_lcr-var-start-time.3.1.ufo.BOUNDED-6.pals.yml false(unreach-call) .855 prog tests 43 inspect .22 .22 22 2.7 .025 0   true prog tests 1 inspect
85.5 97.3 .31 .29 17 1.5 0  
pals_lcr-var-start-time.3.2.ufo.BOUNDED-6.pals.yml false(unreach-call) .817 prog tests 35 inspect .23 .25 21 2.8 .025 0   true prog tests 1 inspect
81.7 93.9 .28 .26 17 1.2 0  
pals_lcr-var-start-time.3.ufo.BOUNDED-6.pals.yml done .836 prog tests 39 inspect .22 .24 21 2.7 .025 0   unknown prog tests 0 inspect
83.6 96.0 .30 .34 17 1.4 0  
pals_lcr-var-start-time.4.1.ufo.BOUNDED-8.pals.yml false(unreach-call) .747 prog tests 40 inspect .75 .77 24 9.7 .029 0   true prog tests 1 inspect
74.7 89.1 .31 .42 17 1.6 0  
pals_lcr-var-start-time.4.2.ufo.BOUNDED-8.pals.yml false(unreach-call) .786 prog tests 50 inspect .60 .62 24 6.8 .029 0   true prog tests 1 inspect
78.6 91.6 .37 .35 17 1.9 0  
pals_lcr-var-start-time.4.ufo.BOUNDED-8.pals.yml done .733 prog tests 40 inspect .70 .72 24 8.1 .029 0   unknown prog tests 0 inspect
73.3 88.0 .31 .34 17 1.6 0  
pals_lcr-var-start-time.5.1.ufo.BOUNDED-10.pals.yml false(unreach-call) .788 prog tests 61 inspect 3.5  3.6  34 44   .033 0   true prog tests 1 inspect
78.8 92.0 .44 .41 17 2.8 0  
pals_lcr-var-start-time.5.2.ufo.BOUNDED-10.pals.yml false(unreach-call) .753 prog tests 47 inspect 3.0  3.1  34 34   .033 0   true prog tests 1 inspect
75.3 88.9 .37 .34 17 2.2 0  
pals_lcr-var-start-time.5.ufo.BOUNDED-10.pals.yml done .793 prog tests 62 inspect 2.9  3.0  35 44   .033 0   unknown prog tests 0 inspect
79.3 92.4 .43 .44 17 2.9 0  
pals_lcr-var-start-time.6.1.ufo.BOUNDED-12.pals.yml false(unreach-call) .759 prog tests 66 inspect 27    27    80 360   .037 0   true prog tests 1 inspect
75.9 89.5 .49 .52 18 3.4 0  
pals_lcr-var-start-time.6.2.ufo.BOUNDED-12.pals.yml false(unreach-call) .766 prog tests 67 inspect 21    21    76 250   .037 0   true prog tests 1 inspect
76.6 89.7 .51 .51 17 3.4 0  
pals_lcr-var-start-time.6.ufo.BOUNDED-12.pals.yml done .728 prog tests 60 inspect 23    23    77 280   .037 0   unknown prog tests 0 inspect
72.8 87.0 .44 .50 17 3.1 0  
pals_lcr.3.1.ufo.BOUNDED-6.pals.yml false(unreach-call) .854 prog tests 37 inspect .20 .21 21 2.8 .025 0   true prog tests 1 inspect
85.4 97.8 .28 .26 17 1.1 0  
pals_lcr.3.ufo.BOUNDED-6.pals.yml done .829 prog tests 33 inspect .21 .23 20 2.0 .025 0   unknown prog tests 0 inspect
82.9 96.3 .26 .30 17 1.0 0  
pals_lcr.4.1.ufo.BOUNDED-8.pals.yml false(unreach-call) .852 prog tests 50 inspect .59 .60 22 7.2 .029 0   true prog tests 1 inspect
85.2 97.7 .36 .38 17 1.9 0  
pals_lcr.4.ufo.BOUNDED-8.pals.yml done .824 prog tests 44 inspect .46 .48 21 5.5 .029 0   unknown prog tests 0 inspect
82.4 95.9 .32 .35 17 1.6 0  
pals_lcr.5.1.ufo.BOUNDED-10.pals.yml false(unreach-call) .823 prog tests 56 inspect 2.5  2.5  26 30   .029 0   true prog tests 1 inspect
82.3 95.7 .40 .39 17 2.4 0  
pals_lcr.5.ufo.BOUNDED-10.pals.yml done .809 prog tests 56 inspect 1.8  1.8  24 27   .029 0   unknown prog tests 0 inspect
80.9 94.8 .38 .38 17 2.4 0  
pals_lcr.6.1.ufo.BOUNDED-12.pals.yml false(unreach-call) .837 prog tests 68 inspect 14    14    48 170   .033 0   true prog tests 1 inspect
83.7 96.4 .48 .56 17 3.2 0  
pals_lcr.6.ufo.BOUNDED-12.pals.yml done .819 prog tests 67 inspect 9.2  9.2  40 130   .033 0   unknown prog tests 0 inspect
81.9 95.2 .45 .46 17 3.1 0  
pals_lcr.7.1.ufo.BOUNDED-14.pals.yml false(unreach-call) .813 prog tests 74 inspect 79    80    160 970   .037 0   true prog tests 1 inspect
81.3 94.6 .55 .62 18 4.0 0  
pals_lcr.7.ufo.BOUNDED-14.pals.yml done .803 prog tests 76 inspect 49    49    110 680   .037 0   unknown prog tests 0 inspect
80.3 93.9 .52 .49 26 4.1 0  
pals_lcr.8.1.ufo.BOUNDED-16.pals.yml false(unreach-call) .802 prog tests 85 inspect 500    500    690 6000   .041 0   true prog tests 1 inspect
80.2 93.5 .67 .73 18 5.3 0  
pals_lcr.8.ufo.BOUNDED-16.pals.yml done .806 prog tests 89 inspect 280    280    430 3500   .041 0   unknown prog tests 0 inspect
80.6 93.8 .63 .74 18 5.6 0  
pals_opt-floodmax.3.1.ufo.BOUNDED-6.pals.yml false(unreach-call) .788 prog tests 63 inspect 23    23    160 260   .037 0   true prog tests 1 inspect
78.8 89.3 .48 .51 17 2.9 0  
pals_opt-floodmax.3.2.ufo.BOUNDED-6.pals.yml false(unreach-call) .691 prog tests 62 inspect 7.8  7.8  120 110   .037 0   true prog tests 1 inspect
69.1 85.2 .46 .48 17 3.1 0  
pals_opt-floodmax.3.3.ufo.BOUNDED-6.pals.yml false(unreach-call) .759 prog tests 66 inspect 8.0  8.1  130 95   .037 0   true prog tests 1 inspect
75.9 87.8 .48 .51 17 3.3 0  
pals_opt-floodmax.3.4.ufo.BOUNDED-6.pals.yml false(unreach-call) .782 prog tests 66 inspect 9.5  9.5  130 110   .037 0   true prog tests 1 inspect
78.2 89.1 .47 .54 17 3.3 0  
pals_opt-floodmax.3.ufo.BOUNDED-6.pals.yml done .782 prog tests 65 inspect 9.7  9.7  130 110   .037 0   unknown prog tests 0 inspect
78.2 88.4 .44 .49 17 3.3 0  
pals_opt-floodmax.4.1.ufo.BOUNDED-8.pals.yml timeout (false(unreach-call)) .903 prog tests 176 inspect 900    900    13000 5700   .14  0   true prog tests 1 inspect
90.3 96.3 1.2  1.3  20 14   0  
pals_opt-floodmax.4.2.ufo.BOUNDED-8.pals.yml timeout (false(unreach-call)) .820 prog tests 143 inspect 900    900    13000 5500   .13  0   true prog tests 1 inspect
82.0 90.4 1.0  1.2  20 12   0  
pals_opt-floodmax.4.3.ufo.BOUNDED-8.pals.yml timeout (false(unreach-call)) .833 prog tests 140 inspect 900    900    13000 6400   .13  0   true prog tests 1 inspect
83.3 91.4 1.2  1.2  21 12   0  
pals_opt-floodmax.4.4.ufo.BOUNDED-8.pals.yml timeout (false(unreach-call)) .849 prog tests 152 inspect 900    900    13000 5700   .13  0   true prog tests 1 inspect
84.9 92.1 1.1  1.2  20 13   0  
pals_opt-floodmax.4.ufo.BOUNDED-8.pals.yml timeout .855 prog tests 141 inspect 900    900    13000 5500   .12  0   unknown prog tests 0 inspect
85.5 92.5 .99 .95 21 12   0  
pals_opt-floodmax.5.1.ufo.BOUNDED-10.pals.yml timeout .789 prog tests 316 inspect 900    900    15000 5800   .22  0   unknown prog tests 0 inspect
78.9 88.2 2.6  2.7  27 48   0  
pals_opt-floodmax.5.2.ufo.BOUNDED-10.pals.yml timeout .664 prog tests 245 inspect 900    900    14000 6000   .20  0   unknown prog tests 0 inspect
66.4 70.5 2.1  2.0  28 40   0  
pals_opt-floodmax.5.3.ufo.BOUNDED-10.pals.yml timeout .661 prog tests 243 inspect 900    900    14000 5400   .19  0   unknown prog tests 0 inspect
66.1 70.3 2.1  2.0  27 40   0  
pals_opt-floodmax.5.4.ufo.BOUNDED-10.pals.yml timeout .668 prog tests 240 inspect 900    900    14000 5000   .21  0   unknown prog tests 0 inspect
66.8 70.8 2.0  2.4  28 40   0  
pals_opt-floodmax.5.ufo.BOUNDED-10.pals.yml timeout .666 prog tests 241 inspect 900    900    14000 7500   .20  0   unknown prog tests 0 inspect
66.6 70.7 2.0  2.1  28 40   0  
sv-benchmarks/c/seq-mthreaded/ status score program test-suite TS size inspect test-suite cpu (s) wall (s) mem (MB) energy (J) blkio-w (MB) blkio-r (MB) status program test-suite TS size inspect test-suite cov branch cov (%) line cov (%) cpu (s) wall (s) mem (MB) energy blkio-w (MB) blkio-r (MB)
total 62 49.2 26000 26000 260000 220000 5.1  0   62 4920 5600 50 52 1200 650 0  
Run set klee.test-comp19_prop-coverage-branches.ReachSafety-Sequentialized tbf-testsuite-validator-klee.test-comp19_prop-coverage-branches.ReachSafety-Sequentialized