Benchmark Symbiotic TBF Test-Suite Validator
Tool symbiotic 6.0.3-dev-fd3f777b Tbf Test-suite Validator v1.2
Limits timelimit: 900 s, memlimit: 15000 MB, CPU core limit: 8 timelimit: 10800 s, memlimit: 7000 MB, CPU core limit: 2
Host apollon*
OS Linux 4.15.0-45-generic
System CPU: Intel Xeon E3-1230 v5 @ 3.40 GHz, cores: 8, frequency: 3.8 GHz, Turbo Boost: disabled; RAM: 33546 MB
Date of execution 2019-02-06 07:07:32 CET 2019-02-06 14:52:39 CET
Run set symbiotic.test-comp19_prop-coverage-branches.ReachSafety-Loops tbf-testsuite-validator-symbiotic.test-comp19_prop-coverage-branches.ReachSafety-Loops
Options --test-comp --verifier klee-testcomp --verbose --sequence-file cov-seq.txt -r --test-suite ../../results-verified/symbiotic.2019-02-06_0707.logfiles/${rundefinition_name}.${inputfile_name}.files/test-suite.zip
sv-benchmarks/c/ status score program test-suite TS size inspect test-suite cpu (s) wall (s) mem (MB) energy (J) blkio-w (MB) blkio-r (MB) status program test-suite TS size inspect test-suite cov branch cov (%) line cov (%) cpu (s) wall (s) mem (MB) energy blkio-w (MB) blkio-r (MB)
loops/array-1.yml .667  .32 .32 19 4.2 0     0      66.7  93.3 .11  .11  16   .078 0  
loops/array-2.yml 1.0    .34 .34 19 4.3 0     0      100    100   .12  .12  16   .094 0  
loops/bubble_sort-1.yml .0417 .43 .43 16 5.6 0     0      4.17 17.9 .11  .11  16   .086 0  
loops/compact.yml .833  840    840    360 11000   0     0      83.3  87.5 5.0   5.0   400   .086 0  
loops/count_up_down-1.yml .5    .31 .31 15 3.6 0     0      50.0  72.7 .12  .11  16   .078 0  
loops/count_up_down-2.yml .5    .30 .30 16 3.4 0     0      50.0  72.7 .11  .11  16   .078 0  
loops/eureka_01-1.yml .885  .83 .83 20 11   0     0      88.5  100   .14  .14  16   .21  0  
loops/for_bounded_loop1.yml 1.0    840    840    810 11000   0     0      100    100   .13  .13  16   .13  0  
loops/insertion_sort-1.yml .583  840    840    1400 9800   0     0      58.3  77.8 .13  .15  16   .11  0  
loops/insertion_sort-2.yml .7    840    840    1400 13000   0     0      70.0  82.3 .12  .14  16   .11  0  
loops/invert_string-1.yml 1.0    840    840    310 12000   0     0      100    100   .13  .13  16   .11  0  
loops/invert_string-3.yml .875  .34 .33 16 4.0 0     0      87.5  94.4 .12  .15  16   .078 0  
loops/linear_sea.ch.yml .667  840    840    45 11000   0     0      66.7  87.5 .12  .12  17   .11  0  
loops/linear_search.yml .929  4.3  4.3  21 56   0     0      92.9  100   .12  .12  17   .13  0  
loops/matrix-1.yml .75   .34 .34 18 4.4 0     0      75.0  94.1 .11  .11  16   .078 0  
loops/matrix-2.yml .0    900    900    47 12000   .012 0      0    0   .073 .082 8.9 0     0  
loops/n.c40.yml .375  .30 .30 18 3.7 0     0      37.5  63.6 .11  .14  16   .078 0  
loops/nec20.yml .833  .37 .37 19 4.9 0     0      83.3  95.8 .12  .12  16   .11  0  
loops/nec40.yml .3    .32 .32 18 4.3 0     0      30.0  46.7 .11  .11  16   .078 0  
loops/string-1.yml .846  3.9  3.9  29 52   0     0      84.6  89.7 .15  .15  16   .16  0  
loops/string-2.yml .923  8.6  8.6  33 120   0     0      92.3  93.1 .14  .15  16   .16  0  
loops/sum01-1.yml .9    840    840    410 13000   0     0      90.0  100   .15  .15  16   .11  0  
loops/sum01-2.yml .5    .32 .32 19 4.0 0     0      50.0  80.0 .12  .12  16   .094 0  
loops/sum01_bug02.yml .9    840    840    410 11000   0     0      90.0  100   .12  .13  16   .11  0  
loops/sum01_bug02_sum01_bug02_base.case.yml .9    840    840    440 9300   0     0      90.0  100   .12  .12  16   .11  0  
loops/sum03-1.yml .875  .32 .32 16 4.0 0     0      87.5  100   .11  .12  16   .078 0  
loops/sum_array-1.yml 1.0    840    840    650 11000   0     0      100    100   .12  .12  16   .11  0  
loops/sum_array-2.yml .857  850    850    2000 10000   0     0      85.7  94.4 .14  .14  16   .11  0  
loops/terminator_01.yml .75   .28 .28 16 3.3 0     0      75.0  81.8 .13  .14  16   .078 0  
loops/terminator_02-1.yml .0    740    740    15000 7900   .025 .0082 0    0   .078 .079 8.9 0     0  
loops/terminator_02-2.yml .0    790    790    15000 9800   .020 .0082 0    0   .060 .062 9.0 0     0  
loops/terminator_03-1.yml .75   850    850    91 13000   0     0      75.0  100   1.1   1.1   16   .11  0  
loops/terminator_03-2.yml .786  850    850    89 8900   0     0      78.6  92.3 2.5   2.5   16   .14  0  
loops/trex01-1.yml 1.0    840    840    360 7300   0     0      100    100   .20  .20  16   .21  0  
loops/trex01-2.yml .808  840    840    370 7100   0     0      80.8  96.5 .21  .23  17   .21  0  
loops/trex02-1.yml .667  840    840    460 8200   0     0      66.7  92.9 .14  .18  16   .094 0  
loops/trex02-2.yml .833  840    840    400 9000   0     0      83.3  100   .14  .22  16   .11  0  
loops/trex03-1.yml .0    600    600    15000 6200   .020 .0082 0    0   .057 .058 9.0 0     0  
loops/trex03-2.yml .0    530    530    15000 6500   .033 .0041 0    0   .062 .063 9.0 0     0  
loops/veris.c_sendmail_tTflag_arr_one_loop.yml .75   .40 .40 18 5.3 0     0      75.0  94.7 .12  .12  16   .094 0  
loops/vogal-1.yml .85   .43 .43 19 5.5 0     0      85.0  88.5 .12  .14  16   .11  0  
loops/vogal-2.yml .944  .67 .67 20 8.7 0     0      94.4  96.2 .12  .12  16   .094 0  
loop-acceleration/array_3-1.yml .75   15    15    22 240   0     0      75.0  81.8 .17  .17  17   .094 0  
loop-acceleration/array_3-2.yml .875  15    15    23 230   0     0      87.5  100   .20  .20  17   .11  0  
loop-acceleration/array_4.yml .833  15    15    22 220   0     0      83.3  90.0 .18  .20  17   .094 0  
loop-acceleration/diamond_1-1.yml .667  .31 .31 18 3.8 0     0      66.7  83.3 .11  .11  16   .078 0  
loop-acceleration/diamond_1-2.yml .667  .32 .32 18 4.1 0     0      66.7  83.3 .14  .14  16   .078 0  
loop-acceleration/diamond_2-2.yml .958  .39 .39 18 4.3 0     0      95.8  96.5 .12  .12  16   .094 0  
loop-acceleration/multivar_1-1.yml .75   .29 .29 16 3.4 0     0      75.0  90.9 .14  .14  16   .078 0  
loop-acceleration/multivar_1-2.yml .75   .29 .29 16 4.3 0     0      75.0  90.9 .11  .11  16   .078 0  
loop-acceleration/simple_3-2.yml .5    .32 .31 16 3.9 0     0      50.0  80.0 .11  .12  16   .078 0  
loop-crafted/simple_array_index_value_2.yml .833  420    420    64 5300   0     0      83.3  90.9 .37  .37  45   .078 0  
loop-crafted/simple_array_index_value_4.i.v+lhb-reducer.yml .0    900    900    2700 3000   .012 0      0    0   .059 .059 8.9 0     0  
loop-crafted/simple_array_index_value_4.i.v+nlh-reducer.yml .0    900    900    2700 2900   .012 0      0    0   .056 .056 8.9 0     0  
loop-crafted/simple_array_index_value_4.yml .0    900    900    2700 2800   .012 0      0    0   .080 .081 8.9 0     0  
loop-invgen/MADWiFi-encode_ie_ok.yml .682  840    840    1200 10000   0     0      68.2  92.6 .13  .13  17   .15  0  
loop-invgen/NetBSD_loop.yml .75   840    840    330 11000   0     0      75.0  93.8 .12  .12  16   .094 0  
loop-invgen/SpamAssassin-loop.i.v+cfa-reducer.yml .614  870    870    4300 11000   0     0      61.4  72.5 20     20     16   .21  0  
loop-invgen/apache-escape-absolute.i.v+cfa-reducer.yml .75   860    860    4400 10000   0     0      75.0  76.0 .19  .18  16   .61  0  
loop-invgen/apache-escape-absolute.yml .762  860    860    3200 9900   0     0      76.2  98.1 .16  .20  17   .29  0  
loop-invgen/apache-get-tag.i.p+lhb-reducer.yml .667  890    890    13000 11000   0     0      66.7  81.9 .24  .23  18   1.5   0  
loop-invgen/apache-get-tag.i.p+nlh-reducer.yml .0    900    900    15000 11000   .045 7800      0    0   .063 .064 8.9 0     0  
loop-invgen/apache-get-tag.i.p+sep-reducer.yml .0    670    670    15000 9200   .025 .012  0    0   .069 .071 9.0 0     0  
loop-invgen/apache-get-tag.i.v+lhb-reducer.yml .667  880    880    9900 9100   0     0      66.7  74.5 .16  .15  16   .38  0  
loop-invgen/apache-get-tag.i.v+nlh-reducer.yml .679  880    880    11000 11000   0     0      67.9  75.0 .16  .15  17   .38  0  
loop-invgen/apache-get-tag.yml .944  880    880    9300 10000   0     0      94.4  97.7 .15  .20  17   .22  0  
loop-invgen/down.yml .833  840    840    450 6500   0     0      83.3  94.1 .25  .25  16   .094 0  
loop-invgen/fragtest_simple.yml .786  860    860    2500 10000   0     0      78.6  96.2 .17  .17  17   .13  0  
loop-invgen/half_2.yml .875  840    840    880 11000   0     0      87.5  94.4 .12  .12  16   .11  0  
loop-invgen/heapsort.yml .9    850    850    1800 13000   0     0      90.0  97.6 .14  .13  16   .20  0  
loop-invgen/id_build.i.p+nlh-reducer.yml .297  840    840    1200 9300   0     0      29.7  41.7 .67  .66  22   .86  0  
loop-invgen/id_build.i.p+sep-reducer.yml .297  840    840    1200 11000   0     0      29.7  40.2 .63  .63  22   .86  0  
loop-invgen/id_build.i.v+lhb-reducer.yml .714  840    840    1200 9300   0     0      71.4  77.8 .56  .56  16   .11  0  
loop-invgen/id_build.yml .833  840    840    1300 9600   0     0      83.3  90.9 1.5   1.6   16   .094 0  
loop-invgen/id_trans.yml .875  840    840    1300 9000   0     0      87.5  100   .12  .17  17   .11  0  
loop-invgen/large_const.yml .722  .39 .39 19 4.5 0     0      72.2  85.2 .13  .12  17   .14  0  
loop-invgen/nest-if3.yml .714  860    860    45 11000   0     0      71.4  86.7 .13  .13  17   .13  0  
loop-invgen/nested6.yml .857  900    900    9800 9900   0     0      85.7  94.7 .13  .13  16   .16  0  
loop-invgen/nested9.yml .455  840    840    2500 10000   0     0      45.5  61.1 20     20     17   .11  0  
loop-invgen/sendmail-close-angle.yml .667  840    840    1100 10000   0     0      66.7  93.3 .12  .12  17   .11  0  
loop-invgen/seq-3.yml .75   840    840    840 10000   0     0      75.0  95.7 .13  .13  16   .11  0  
loop-invgen/up.yml .833  840    840    1600 10000   0     0      83.3  93.8 .26  .27  16   .094 0  
loop-lit/afnp2014.yml .833  14    14    22 180   0     0      83.3  91.7 .15  .15  17   .11  0  
loop-lit/bhmr2007.yml .0    900    900    56 12000   .012 0      0    0   .056 .057 8.9 0     0  
loop-lit/cggmp2005.yml .75   .26 .25 15 2.9 0     0      75.0  91.7 .11  .13  16   .078 0  
loop-lit/cggmp2005_variant.yml .75   .32 .32 19 4.1 0     0      75.0  93.3 .14  .20  16   .094 0  
loop-lit/cggmp2005b.yml .75   .27 .26 16 3.6 0     0      75.0  88.2 .11  .11  16   .078 0  
loop-lit/css2003.yml .643  10    10    870 140   0     0      64.3  93.3 .13  .14  17   .094 0  
loop-lit/ddlm2013.yml .8    840    840    52 10000   0     0      80.0  90.9 .12  .12  16   .11  0  
loop-lit/gj2007.c.i.p+lhb-reducer.yml .548  .29 .29 16 3.6 0     0      54.8  58.2 .14  .14  16   .082 0  
loop-lit/gj2007.c.i.p+nlh-reducer.yml .504  .63 .62 16 8.0 0     0      50.4  57.3 .14  .14  18   .12  0  
loop-lit/gj2007.yml .833  .25 .25 15 3.6 0     0      83.3  92.9 .11  .11  16   .078 0  
loop-lit/gj2007b.yml .5    840    840    130 10000   0     0      50.0  86.7 .14  .14  17   .094 0  
loop-lit/gr2006.yml .833  .29 .29 16 3.2 0     0      83.3  92.9 .12  .12  16   .078 0  
loop-lit/gsv2008.c.i.p+cfa-reducer.yml .875  850    850    53 11000   0     0      87.5  90.0 .14  .14  16   .13  0  
loop-lit/gsv2008.c.i.v+cfa-reducer.yml .875  850    850    52 11000   0     0      87.5  90.0 .12  .12  16   .13  0  
loop-lit/gsv2008.c.i.v+lhb-reducer.yml .8    860    860    54 12000   0     0      80.0  89.3 .12  .12  16   .13  0  
loop-lit/gsv2008.yml .75   840    840    270 8600   0     0      75.0  92.3 .12  .13  16   .11  0  
loop-lit/hhk2008.yml .6    .34 .34 19 5.5 0     0      60.0  81.2 .12  .12  16   .094 0  
loop-lit/jm2006.c.i.v+cfa-reducer.yml .9    .33 .33 18 3.7 0     0      90.0  92.0 .72  .73  16   .12  0  
loop-lit/jm2006.yml .6    .33 .32 18 4.6 0     0      60.0  81.2 .11  .15  16   .094 0  
loop-lit/jm2006_variant.yml .583  .35 .35 19 4.3 0     0      58.3  79.0 .12  .12  16   .094 0  
loop-new/count_by_1.yml .75   .27 .27 16 2.9 0     0      75.0  87.5 .12  .12  16   .078 0  
loop-new/count_by_1_variant.yml .75   5.0  5.0  950 68   0     0      75.0  87.5 .12  .16  16   .078 0  
loop-new/count_by_2.yml .75   .24 .24 16 2.9 0     0      75.0  87.5 .11  .11  16   .078 0  
loop-new/count_by_k.yml .75   850    850    20 12000   0     0      75.0  90.0 .12  .14  16   .094 0  
loop-new/count_by_nondet.yml .0    900    900    97 9500   .012 0      0    0   .057 .058 9.0 0     0  
loop-new/gauss_sum.i.p+cfa-reducer.yml .875  .66 .65 23 9.4 0     0      87.5  90.0 .14  .14  16   .11  0  
loop-new/gauss_sum.i.p+lhb-reducer.yml .889  1.2  1.2  25 16   0     0      88.9  92.5 .13  .14  16   .18  0  
loop-new/gauss_sum.i.v+cfa-reducer.yml .875  .66 .66 23 9.1 0     0      87.5  90.0 .12  .12  16   .11  0  
loop-new/gauss_sum.yml .75   .79 .79 23 9.6 0     0      75.0  91.7 .15  .15  16   .094 0  
loop-new/half.yml .714  840    840    1700 13000   0     0      71.4  92.9 .12  .12  16   .11  0  
loop-new/nested-1.yml .714  .52 .52 21 6.3 0     0      71.4  93.3 .18  .18  16   .094 0  
loop-industry-pattern/mod3.c.v+cfa-reducer.yml .75   840    840    52 12000   0     0      75.0  77.4 .12  .12  16   .11  0  
loop-industry-pattern/mod3.c.v+lhb-reducer.yml .8    890    890    7800 8200   0     0      80.0  86.7 .14  .14  17   .20  0  
loop-industry-pattern/mod3.c.v+sep-reducer.yml .7    880    880    7900 8100   0     0      70.0  70.8 .15  .18  16   .20  0  
loop-industry-pattern/mod3.yml .833  860    860    51 12000   0     0      83.3  88.2 .13  .13  16   .11  0  
sv-benchmarks/c/ status score program test-suite TS size inspect test-suite cpu (s) wall (s) mem (MB) energy (J) blkio-w (MB) blkio-r (MB) status program test-suite TS size inspect test-suite cov branch cov (%) line cov (%) cpu (s) wall (s) mem (MB) energy blkio-w (MB) blkio-r (MB)
total 117 78.4 55000 55000 210000 650000 .24 7800 117 7840 9140 67 68 2300 16 0  
Run set symbiotic.test-comp19_prop-coverage-branches.ReachSafety-Loops tbf-testsuite-validator-symbiotic.test-comp19_prop-coverage-branches.ReachSafety-Loops