Benchmark Symbiotic TBF Test-Suite Validator
Tool symbiotic 6.0.3-dev-fd3f777b Tbf Test-suite Validator v1.2
Limits timelimit: 900 s, memlimit: 15000 MB, CPU core limit: 8 timelimit: 10800 s, memlimit: 7000 MB, CPU core limit: 2
Host apollon*
OS Linux 4.15.0-45-generic
System CPU: Intel Xeon E3-1230 v5 @ 3.40 GHz, cores: 8, frequency: 3.8 GHz, Turbo Boost: disabled; RAM: 33546 MB
Date of execution 2019-02-06 07:07:32 CET 2019-02-06 14:52:39 CET
Run set symbiotic.test-comp19_prop-coverage-branches.ReachSafety-Sequentialized tbf-testsuite-validator-symbiotic.test-comp19_prop-coverage-branches.ReachSafety-Sequentialized
Options --test-comp --verifier klee-testcomp --verbose --sequence-file cov-seq.txt -r --test-suite ../../results-verified/symbiotic.2019-02-06_0707.logfiles/${rundefinition_name}.${inputfile_name}.files/test-suite.zip
sv-benchmarks/c/seq-mthreaded/ status score program test-suite TS size inspect test-suite cpu (s) wall (s) mem (MB) energy (J) blkio-w (MB) blkio-r (MB) status program test-suite TS size inspect test-suite cov branch cov (%) line cov (%) cpu (s) wall (s) mem (MB) energy blkio-w (MB) blkio-r (MB)
pals_STARTPALS_ActiveStandby.1.ufo.BOUNDED-10.pals.yml out of memory (killed (signal 9, verification)) .0   prog tests 0 inspect 710    710    15000 7300 .025 .0082 error (0) prog tests 0 inspect
0   0   .057 .058 8.9 0    0  
pals_STARTPALS_ActiveStandby.4_1.ufo.BOUNDED-10.pals.yml timeout (verification) .0   prog tests 0 inspect 900    900    15000 7900 .041 7500      error (0) prog tests 0 inspect
0   0   .060 .061 8.9 0    0  
pals_STARTPALS_ActiveStandby.4_2.ufo.BOUNDED-10.pals.yml timeout (verification) .0   prog tests 0 inspect 900    900    11000 8800 .033 0      error (0) prog tests 0 inspect
0   0   .056 .057 8.9 0    0  
pals_STARTPALS_ActiveStandby.5.ufo.BOUNDED-10.pals.yml out of memory (killed (signal 9, verification)) .0   prog tests 0 inspect 780    780    15000 5400 .020 .0082 error (0) prog tests 0 inspect
0   0   .056 .056 8.9 0    0  
pals_STARTPALS_ActiveStandby.ufo.BOUNDED-10.pals.yml out of memory (killed (signal 9, verification)) .0   prog tests 0 inspect 760    760    15000 6100 .020 .0082 error (0) prog tests 0 inspect
0   0   .061 .062 8.9 0    0  
pals_STARTPALS_Triplicated.1.ufo.BOUNDED-10.pals.yml out of memory (killed (signal 9, verification)) .0   prog tests 0 inspect 760    760    15000 6900 .020 .0082 error (0) prog tests 0 inspect
0   0   .087 .088 8.9 0    0  
pals_STARTPALS_Triplicated.2.ufo.BOUNDED-10.pals.yml out of memory (killed (signal 9, verification)) .0   prog tests 0 inspect 660    660    15000 8400 .029 .0041 error (0) prog tests 0 inspect
0   0   .059 .060 8.9 0    0  
pals_STARTPALS_Triplicated.ufo.BOUNDED-10.pals.yml out of memory (killed (signal 9, verification)) .0   prog tests 0 inspect 740    740    15000 6000 .020 .0082 error (0) prog tests 0 inspect
0   0   .084 .085 8.9 0    0  
pals_floodmax.3.1.ufo.BOUNDED-6.pals.yml done .758 prog tests 10 inspect 68    68    39 990 0     0      true prog tests 1 inspect
75.8 87.5 .19  .21  17   .45 0  
pals_floodmax.3.2.ufo.BOUNDED-6.pals.yml done .490 prog tests 2 inspect 7.8  7.8  22 120 0     0      true prog tests 1 inspect
49.0 73.6 .17  .17  17   .19 0  
pals_floodmax.3.3.ufo.BOUNDED-6.pals.yml done .520 prog tests 2 inspect 36    36    27 500 0     0      true prog tests 1 inspect
52.0 75.0 .17  .18  17   .16 0  
pals_floodmax.3.4.ufo.BOUNDED-6.pals.yml done .748 prog tests 14 inspect 43    43    29 680 0     0      true prog tests 1 inspect
74.8 86.3 .24  .23  17   .73 0  
pals_floodmax.3.ufo.BOUNDED-6.pals.yml done .738 prog tests 15 inspect 43    43    29 570 0     0      unknown prog tests 0 inspect
73.8 85.6 .22  .22  17   .77 0  
pals_floodmax.4.1.ufo.BOUNDED-8.pals.yml timeout (verification) .0   prog tests 0 inspect 900    900    3000 13000 .053 0      error (0) prog tests 0 inspect
0   0   .059 .059 8.9 0    0  
pals_floodmax.4.2.ufo.BOUNDED-8.pals.yml done .342 prog tests 2 inspect 850    850    1100 10000 0     0      true prog tests 1 inspect
34.2 55.2 .23  .24  20   .29 0  
pals_floodmax.4.3.ufo.BOUNDED-8.pals.yml done .539 prog tests 5 inspect 850    850    1200 14000 0     0      true prog tests 1 inspect
54.0 69.1 .25  .25  20   .53 0  
pals_floodmax.4.4.ufo.BOUNDED-8.pals.yml done .648 prog tests 34 inspect 850    850    1200 10000 0     0      true prog tests 1 inspect
64.8 76.5 .38  .36  20   2.8  0  
pals_floodmax.4.ufo.BOUNDED-8.pals.yml done .640 prog tests 34 inspect 850    850    1200 11000 0     0      unknown prog tests 0 inspect
64.0 76.2 .36  .41  20   2.8  0  
pals_floodmax.5.1.ufo.BOUNDED-10.pals.yml out of memory (killed (signal 9, verification)) .0   prog tests 0 inspect 270    270    15000 4000 .016 .0041 error (0) prog tests 0 inspect
0   0   .059 .059 8.9 0    0  
pals_floodmax.5.2.ufo.BOUNDED-10.pals.yml out of memory (killed (signal 9, verification)) .0   prog tests 0 inspect 66    65    15000 770 .016 0      error (0) prog tests 0 inspect
0   0   .086 .087 8.9 0    0  
pals_floodmax.5.3.ufo.BOUNDED-10.pals.yml out of memory (killed (signal 9, verification)) .0   prog tests 0 inspect 65    65    15000 780 .016 0      error (0) prog tests 0 inspect
0   0   .056 .057 8.9 0    0  
pals_floodmax.5.4.ufo.BOUNDED-10.pals.yml out of memory (killed (signal 9, verification)) .0   prog tests 0 inspect 65    65    15000 790 .016 0      error (0) prog tests 0 inspect
0   0   .060 .060 8.9 0    0  
pals_floodmax.5.ufo.BOUNDED-10.pals.yml out of memory (killed (signal 9, verification)) .0   prog tests 0 inspect 65    65    15000 910 .016 0      error (0) prog tests 0 inspect
0   0   .056 .057 8.9 0    0  
pals_lcr-var-start-time.3.1.ufo.BOUNDED-6.pals.yml done .736 prog tests 12 inspect 1.1  1.1  20 18 0     0      true prog tests 1 inspect
73.6 88.7 .18  .18  17   .48 0  
pals_lcr-var-start-time.3.2.ufo.BOUNDED-6.pals.yml done .740 prog tests 7 inspect 1.1  1.1  20 16 0     0      true prog tests 1 inspect
74.0 88.4 .16  .16  17   .31 0  
pals_lcr-var-start-time.3.ufo.BOUNDED-6.pals.yml done .718 prog tests 10 inspect 1.1  1.1  20 14 0     0      unknown prog tests 0 inspect
71.8 87.2 .17  .16  17   .41 0  
pals_lcr-var-start-time.4.1.ufo.BOUNDED-8.pals.yml done .726 prog tests 15 inspect 2.0  2.0  23 30 0     0      true prog tests 1 inspect
72.6 87.6 .21  .22  17   .65 0  
pals_lcr-var-start-time.4.2.ufo.BOUNDED-8.pals.yml done .729 prog tests 10 inspect 2.0  2.0  22 30 0     0      true prog tests 1 inspect
72.9 87.4 .19  .19  17   .46 0  
pals_lcr-var-start-time.4.ufo.BOUNDED-8.pals.yml done .712 prog tests 16 inspect 2.0  2.0  23 25 0     0      unknown prog tests 0 inspect
71.2 86.5 .20  .19  17   .68 0  
pals_lcr-var-start-time.5.1.ufo.BOUNDED-10.pals.yml done .717 prog tests 18 inspect 6.0  6.0  35 77 0     0      true prog tests 1 inspect
71.7 86.5 .23  .23  17   .91 0  
pals_lcr-var-start-time.5.2.ufo.BOUNDED-10.pals.yml done .713 prog tests 12 inspect 5.7  5.7  33 85 0     0      true prog tests 1 inspect
71.3 86.0 .22  .25  17   .64 0  
pals_lcr-var-start-time.5.ufo.BOUNDED-10.pals.yml done .707 prog tests 21 inspect 5.8  5.8  33 85 0     0      unknown prog tests 0 inspect
70.7 85.7 .24  .23  17   1.0  0  
pals_lcr-var-start-time.6.1.ufo.BOUNDED-12.pals.yml done .710 prog tests 25 inspect 31    31    100 420 0     0      true prog tests 1 inspect
71.0 85.6 .30  .29  17   1.3  0  
pals_lcr-var-start-time.6.2.ufo.BOUNDED-12.pals.yml done .711 prog tests 15 inspect 27    27    87 350 0     0      true prog tests 1 inspect
71.1 85.5 .25  .25  17   .84 0  
pals_lcr-var-start-time.6.ufo.BOUNDED-12.pals.yml done .701 prog tests 28 inspect 27    27    93 410 0     0      unknown prog tests 0 inspect
70.1 84.9 .27  .26  17   1.5  0  
pals_lcr.3.1.ufo.BOUNDED-6.pals.yml done .683 prog tests 5 inspect 1.0  1.0  20 13 0     0      true prog tests 1 inspect
68.3 87.4 .17  .16  17   .23 0  
pals_lcr.3.ufo.BOUNDED-6.pals.yml done .658 prog tests 7 inspect .98 .98 20 15 0     0      unknown prog tests 0 inspect
65.8 85.8 .16  .16  17   .28 0  
pals_lcr.4.1.ufo.BOUNDED-8.pals.yml done .667 prog tests 8 inspect 2.4  2.4  21 26 0     0      true prog tests 1 inspect
66.7 86.0 .17  .16  17   .38 0  
pals_lcr.4.ufo.BOUNDED-8.pals.yml done .648 prog tests 9 inspect 1.9  1.9  20 27 0     0      unknown prog tests 0 inspect
64.8 84.8 .17  .21  17   .38 0  
pals_lcr.5.1.ufo.BOUNDED-10.pals.yml done .654 prog tests 13 inspect 8.3  8.3  25 110 0     0      true prog tests 1 inspect
65.4 84.8 .21  .21  17   .62 0  
pals_lcr.5.ufo.BOUNDED-10.pals.yml done .640 prog tests 15 inspect 6.1  6.1  24 91 0     0      unknown prog tests 0 inspect
64.0 83.8 .20  .19  17   .70 0  
pals_lcr.6.1.ufo.BOUNDED-12.pals.yml done .645 prog tests 17 inspect 37    37    43 410 0     0      true prog tests 1 inspect
64.5 83.7 .25  .24  17   .86 0  
pals_lcr.6.ufo.BOUNDED-12.pals.yml done .632 prog tests 12 inspect 24    24    38 340 0     0      unknown prog tests 0 inspect
63.2 82.9 .21  .27  17   .59 0  
pals_lcr.7.1.ufo.BOUNDED-14.pals.yml done .636 prog tests 20 inspect 170    170    140 2600 0     0      true prog tests 1 inspect
63.6 82.7 .29  .30  17   1.2  0  
pals_lcr.7.ufo.BOUNDED-14.pals.yml done .626 prog tests 20 inspect 110    110    100 1400 0     0      unknown prog tests 0 inspect
62.6 82.0 .26  .32  17   1.2  0  
pals_lcr.8.1.ufo.BOUNDED-16.pals.yml done .629 prog tests 24 inspect 840    840    560 9200 0     0      true prog tests 1 inspect
62.9 81.8 .35  .33  18   1.6  0  
pals_lcr.8.ufo.BOUNDED-16.pals.yml done .621 prog tests 22 inspect 510    510    370 5900 0     0      unknown prog tests 0 inspect
62.1 81.1 .33  .33  18   1.5  0  
pals_opt-floodmax.3.1.ufo.BOUNDED-6.pals.yml done .774 prog tests 9 inspect 60    60    38 820 0     0      true prog tests 1 inspect
77.4 88.6 .19  .19  17   .51 0  
pals_opt-floodmax.3.2.ufo.BOUNDED-6.pals.yml done .482 prog tests 2 inspect 7.1  7.1  23 92 0     0      true prog tests 1 inspect
48.2 73.0 .17  .17  17   .20 0  
pals_opt-floodmax.3.3.ufo.BOUNDED-6.pals.yml done .523 prog tests 2 inspect 37    37    27 490 0     0      true prog tests 1 inspect
52.3 74.3 .17  .22  17   .20 0  
pals_opt-floodmax.3.4.ufo.BOUNDED-6.pals.yml done .745 prog tests 14 inspect 45    45    31 590 0     0      true prog tests 1 inspect
74.5 86.5 .23  .31  17   .79 0  
pals_opt-floodmax.3.ufo.BOUNDED-6.pals.yml done .745 prog tests 17 inspect 45    45    30 580 0     0      unknown prog tests 0 inspect
74.5 85.8 .22  .21  17   .93 0  
pals_opt-floodmax.4.1.ufo.BOUNDED-8.pals.yml done .720 prog tests 10 inspect 890    890    3200 11000 0     0      true prog tests 1 inspect
72.0 82.6 .25  .26  20   .92 0  
pals_opt-floodmax.4.2.ufo.BOUNDED-8.pals.yml done .422 prog tests 2 inspect 850    850    1200 12000 0     0      true prog tests 1 inspect
42.2 63.5 .24  .24  20   .32 0  
pals_opt-floodmax.4.3.ufo.BOUNDED-8.pals.yml done .446 prog tests 2 inspect 850    850    1200 11000 0     0      true prog tests 1 inspect
44.6 64.4 .24  .24  20   .32 0  
pals_opt-floodmax.4.4.ufo.BOUNDED-8.pals.yml done .718 prog tests 38 inspect 850    850    1300 11000 0     0      true prog tests 1 inspect
71.8 82.4 .45  .43  20   3.4  0  
pals_opt-floodmax.4.ufo.BOUNDED-8.pals.yml done .660 prog tests 43 inspect 850    850    1300 12000 0     0      unknown prog tests 0 inspect
66.0 77.5 .42  .48  20   3.8  0  
pals_opt-floodmax.5.1.ufo.BOUNDED-10.pals.yml out of memory (killed (signal 9, verification)) .0   prog tests 0 inspect 270    270    15000 3900 .016 .0041 error (0) prog tests 0 inspect
0   0   .060 .061 9.0 0    0  
pals_opt-floodmax.5.2.ufo.BOUNDED-10.pals.yml out of memory (killed (signal 9, verification)) .0   prog tests 0 inspect 63    62    15000 730 .041 .0041 error (0) prog tests 0 inspect
0   0   .056 .056 8.9 0    0  
pals_opt-floodmax.5.3.ufo.BOUNDED-10.pals.yml out of memory (killed (signal 9, verification)) .0   prog tests 0 inspect 63    62    15000 900 .041 0      error (0) prog tests 0 inspect
0   0   .063 .064 8.9 0    0  
pals_opt-floodmax.5.4.ufo.BOUNDED-10.pals.yml out of memory (killed (signal 9, verification)) .0   prog tests 0 inspect 63    63    15000 780 .041 .0041 error (0) prog tests 0 inspect
0   0   .057 .058 8.9 0    0  
pals_opt-floodmax.5.ufo.BOUNDED-10.pals.yml out of memory (killed (signal 9, verification)) .0   prog tests 0 inspect 62    62    15000 790 .041 .0041 error (0) prog tests 0 inspect
0   0   .081 .082 8.9 0    0  
sv-benchmarks/c/seq-mthreaded/ status score program test-suite TS size inspect test-suite cpu (s) wall (s) mem (MB) energy (J) blkio-w (MB) blkio-r (MB) status program test-suite TS size inspect test-suite cov branch cov (%) line cov (%) cpu (s) wall (s) mem (MB) energy blkio-w (MB) blkio-r (MB)
total 62 28.0 18000 18000 280000 210000 .52 7500 62 2800 3510 11 12 930 39 0  
Run set symbiotic.test-comp19_prop-coverage-branches.ReachSafety-Sequentialized tbf-testsuite-validator-symbiotic.test-comp19_prop-coverage-branches.ReachSafety-Sequentialized