Benchmark CPA/CoVeriTest TBF Test-Suite Validator
Tool CPAchecker 1.8-svn 30375 Tbf Test-suite Validator v1.2
Limits timelimit: 900 s, memlimit: 15000 MB, CPU core limit: 8 timelimit: 10800 s, memlimit: 7000 MB, CPU core limit: 2
Host apollon* [apollon049; apollon085; apollon116]
OS Linux 4.15.0-45-generic
System CPU: Intel Xeon E3-1230 v5 @ 3.40 GHz, cores: 8, frequency: 3.8 GHz, Turbo Boost: disabled; RAM: 33546 MB
Date of execution 2019-02-05 21:05:56 CET 2019-02-06 05:30:26 CET
Run set coveritest.test-comp19_prop-coverage-branches.ReachSafety-Loops tbf-testsuite-validator-coveritest.test-comp19_prop-coverage-branches.ReachSafety-Loops
Options -benchmark -heap 10000M -testcomp19 --verbose --sequence-file cov-seq.txt -r --test-suite ../../results-verified/coveritest.2019-02-05_2105.logfiles/${rundefinition_name}.${inputfile_name}.files/test-suite.zip
sv-benchmarks/c/ status score program test-suite TS size inspect test-suite cpu (s) wall (s) mem (MB) energy (J) blkio-w (MB) blkio-r (MB) status program test-suite TS size inspect test-suite cov branch cov (%) line cov (%) cpu (s) wall (s) mem (MB) energy blkio-w (MB) blkio-r (MB)
loops/array-1.yml .833  23   16   1600 240 .041  0      83.3  93.3 .14  .14  16   .14  0  
loops/array-2.yml 1.0    23   16   1300 290 .041  0      100    100   .14  .13  16   .16  0  
loops/bubble_sort-1.yml .0417 900   780   8600 9700 51      0      4.17 17.9 .15  .15  16   .25  0  
loops/compact.yml .0    900   900   2800 5200 .0041 0      0    0   .060 .060 8.9 0     0  
loops/count_up_down-1.yml .75   900   820   4900 11000 44      0      75.0  90.9 .12  .12  17   .11  0  
loops/count_up_down-2.yml .75   900   800   5000 10000 43      0      75.0  90.9 .13  .13  16   .11  0  
loops/eureka_01-1.yml .962  900   880   500 11000 .16   0      96.2  96.8 .19  .18  17   .41  0  
loops/for_bounded_loop1.yml 1.0    23   6.6 960 200 2.0    0      100    100   .14  .14  16   .19  0  
loops/insertion_sort-1.yml 1.0    900   790   4600 11000 50      0      100    100   .15  .15  16   .19  0  
loops/insertion_sort-2.yml .9    900   790   5400 11000 49      0      90.0  94.1 .14  .14  17   .18  0  
loops/invert_string-1.yml .875  900   810   5000 11000 36      0      87.5  95.0 .14  .14  16   .16  0  
loops/invert_string-3.yml .125  900   880   1900 11000 .26   0      12.5  27.8 .13  .13  16   .14  0  
loops/linear_sea.ch.yml .75   900   770   6000 11000 41      0      75.0  87.5 .15  .15  16   .18  0  
loops/linear_search.yml .714  900   790   5300 11000 44      0      71.4  88.2 .15  .15  16   .19  0  
loops/matrix-1.yml .875  23   16   1800 260 .061  0      87.5  94.1 .14  .14  16   .18  0  
loops/matrix-2.yml 1.0    23   17   1200 230 .066  0      100    100   .17  .16  16   .26  0  
loops/n.c40.yml .5    24   19   1700 290 .0041 0      50.0  63.6 .12  .12  17   .11  0  
loops/nec20.yml .833  8.2 3.4 430 73 .070  0      83.3  95.8 .17  .16  17   .21  0  
loops/nec40.yml .4    24   19   1800 310 .0041 0      40.0  46.7 .12  .12  16   .11  0  
loops/string-1.yml .808  11   3.6 480 99 .32   0      80.8  96.5 .18  .19  17   .34  0  
loops/string-2.yml .846  12   4.0 490 120 .37   0      84.6  100   .21  .20  16   .38  0  
loops/sum01-1.yml 1.0    25   6.6 870 220 1.6    0      100    100   .15  .15  17   .19  0  
loops/sum01-2.yml .833  900   720   6900 10000 46      0      83.3  90.0 .15  .15  16   .19  0  
loops/sum01_bug02.yml 1.0    22   5.9 840 180 1.6    0      100    100   .14  .14  16   .19  0  
loops/sum01_bug02_sum01_bug02_base.case.yml 1.0    10   3.2 460 90 .50   0      100    100   .14  .14  16   .21  0  
loops/sum03-1.yml .875  7.0 2.5 400 61 .14   0      87.5  100   .15  .15  16   .16  0  
loops/sum_array-1.yml 1.0    23   18   710 260 .016  0      100    100   .15  .16  16   .23  0  
loops/sum_array-2.yml .929  900   890   770 8900 .098  0      92.9  94.4 .15  .14  16   .26  0  
loops/terminator_01.yml .75   3.6 1.6 270 33 .016  0      75.0  81.8 .12  .12  16   .11  0  
loops/terminator_02-1.yml .875  4.1 1.7 270 42 .037  0      87.5  87.5 .14  .13  16   .16  0  
loops/terminator_02-2.yml .9    24   6.5 700 210 2.8    0      90.0  94.7 .16  .17  16   .27  0  
loops/terminator_03-1.yml .75   23   6.6 700 170 3.2    0      75.0  100   .14  .14  16   .19  0  
loops/terminator_03-2.yml .786  24   6.8 680 190 3.1    0      78.6  92.3 .15  .14  16   .19  0  
loops/trex01-1.yml 1.0    5.5 2.1 300 50 .11   0      100    96.2 .16  .19  17   .28  0  
loops/trex01-2.yml .962  35   12   1100 280 2.0    0      96.2  93.1 .19  .21  16   .38  0  
loops/trex02-1.yml .833  23   18   1400 260 .033  0      83.3  92.9 .14  .14  16   .14  0  
loops/trex02-2.yml 1.0    4.0 1.6 260 35 .041  0      100    100   .13  .13  16   .16  0  
loops/trex03-1.yml 1.0    23   15   1600 240 .14   0      100    93.8 .17  .16  16   .26  0  
loops/trex03-2.yml .889  22   16   1600 260 .082  0      88.9  87.5 .15  .14  16   .23  0  
loops/veris.c_sendmail_tTflag_arr_one_loop.yml .875  29   7.3 1100 240 1.8    0      87.5  94.7 .13  .13  16   .16  0  
loops/vogal-1.yml .95   900   760   5300 11000 52      0      95.0  96.2 .17  .16  16   .32  0  
loops/vogal-2.yml 1.0    31   12   840 270 1.1    0      100    100   .18  .17  16   .32  0  
loop-acceleration/array_3-1.yml .125  900   820   5900 11000 9.9    0      12.5  27.3 .12  .12  17   .078 0  
loop-acceleration/array_3-2.yml .125  900   820   5600 11000 9.8    0      12.5  33.3 .12  .12  16   .078 0  
loop-acceleration/array_4.yml .167  900   820   5900 10000 9.8    0      16.7  30.0 .11  .11  16   .078 0  
loop-acceleration/diamond_1-1.yml .833  740   660   5100 9300 39      0      83.3  91.7 .13  .13  16   .13  0  
loop-acceleration/diamond_1-2.yml .833  540   470   4900 7300 27      0      83.3  91.7 .13  .13  16   .13  0  
loop-acceleration/diamond_2-2.yml .958  26   7.0 940 200 2.1    0      95.8  96.5 .20  .19  16   .42  0  
loop-acceleration/multivar_1-1.yml .75   23   18   1400 280 .016  0      75.0  90.9 .12  .12  16   .11  0  
loop-acceleration/multivar_1-2.yml .75   22   17   1400 240 .025  0      75.0  90.9 .13  .13  16   .11  0  
loop-acceleration/simple_3-2.yml .75   900   880   1900 11000 .025  0      75.0  90.0 .12  .12  17   .11  0  
loop-crafted/simple_array_index_value_2.yml .167  910   730   7100 9400 35      0      16.7  45.5 .12  .12  17   .094 0  
loop-crafted/simple_array_index_value_4.i.v+lhb-reducer.yml .588  25   18   1300 310 .16   0      58.8  68.0 .18  .17  16   .39  0  
loop-crafted/simple_array_index_value_4.i.v+nlh-reducer.yml .714  25   18   1100 310 .15   0      71.4  77.3 .19  .18  17   .43  0  
loop-crafted/simple_array_index_value_4.yml .75   25   19   1300 290 .17   0      75.0  95.2 .17  .17  16   .23  0  
loop-invgen/MADWiFi-encode_ie_ok.yml .955  900   870   3000 11000 2.9    0      95.5  96.3 .19  .18  16   .34  0  
loop-invgen/NetBSD_loop.yml .875  900   870   5400 10000 2.0    0      87.5  93.8 .13  .13  17   .16  0  
loop-invgen/SpamAssassin-loop.i.v+cfa-reducer.yml .636  32   10   940 270 2.0    0      63.6  73.3 40     40     17   .70  0  
loop-invgen/apache-escape-absolute.i.v+cfa-reducer.yml .763  25   17   1700 310 .11   0      76.3  76.6 .32  .29  17   1.4   0  
loop-invgen/apache-escape-absolute.yml .976  25   16   1300 300 .11   0      97.6  98.1 .24  .29  16   .75  0  
loop-invgen/apache-get-tag.i.p+lhb-reducer.yml .667  900   790   5900 9700 53      0      66.7  81.9 .54  .49  18   5.6   0  
loop-invgen/apache-get-tag.i.p+nlh-reducer.yml .744  900   810   5200 12000 42      0      74.4  86.1 2.6   2.3   24   50     0  
loop-invgen/apache-get-tag.i.p+sep-reducer.yml .667  900   810   5400 12000 34      0      66.7  84.5 4.3   4.0   28   120     0  
loop-invgen/apache-get-tag.i.v+lhb-reducer.yml .667  900   790   5900 10000 58      .0041 66.7  74.5 .27  .25  16   1.2   0  
loop-invgen/apache-get-tag.i.v+nlh-reducer.yml .679  900   790   6000 11000 57      0      67.9  75.0 .25  .25  17   1.1   0  
loop-invgen/apache-get-tag.yml .944  900   790   5600 11000 65      0      94.4  97.7 .18  .18  16   .35  0  
loop-invgen/down.yml .833  900   770   7000 10000 46      0      83.3  94.1 .13  .13  17   .14  0  
loop-invgen/fragtest_simple.yml .786  900   870   1500 11000 .082  0      78.6  96.2 .15  .15  16   .21  0  
loop-invgen/half_2.yml .875  900   800   5800 12000 44      0      87.5  94.4 .14  .14  17   .18  0  
loop-invgen/heapsort.yml .95   660   590   4500 8000 57      0      95.0  97.6 .18  .17  16   .34  0  
loop-invgen/id_build.i.p+nlh-reducer.yml .297  38   26   1300 410 .025  0      29.7  41.7 .79  .76  22   16     0  
loop-invgen/id_build.i.p+sep-reducer.yml .297  37   26   1400 420 .025  0      29.7  40.2 .90  .87  22   16     0  
loop-invgen/id_build.i.v+lhb-reducer.yml .714  25   18   1600 300 .025  0      71.4  77.8 .15  .14  17   .23  0  
loop-invgen/id_build.yml .833  900   870   2100 11000 .090  0      83.3  90.9 .13  .13  16   .14  0  
loop-invgen/id_trans.yml 1.0    23   17   1300 280 .049  0      100    100   .14  .14  16   .18  0  
loop-invgen/large_const.yml .944  23   15   900 240 .18   0      94.4  96.3 .18  .17  17   .27  0  
loop-invgen/nest-if3.yml .929  900   820   4800 13000 49      0      92.9  93.3 .15  .15  16   .24  0  
loop-invgen/nested6.yml .929  900   780   8700 12000 62      .0041 92.9  94.7 .15  .15  17   .26  0  
loop-invgen/nested9.yml .955  900   870   5000 13000 .21   0      95.5  94.4 .17  .17  17   .31  0  
loop-invgen/sendmail-close-angle.yml .917  900   780   8100 11000 85      0      91.7  96.7 .16  .16  16   .23  0  
loop-invgen/seq-3.yml .938  900   720   13000 11000 47      0      93.8  95.7 .16  .16  16   .26  0  
loop-invgen/up.yml .833  900   720   13000 11000 40      0      83.3  93.8 .14  .14  16   .14  0  
loop-lit/afnp2014.yml .667  900   830   5300 13000 12      0      66.7  91.7 .13  .13  16   .13  0  
loop-lit/bhmr2007.yml .9    900   810   4500 14000 36      0      90.0  93.8 .16  .19  17   .19  0  
loop-lit/cggmp2005.yml .75   4.1 1.7 260 38 .033  0      75.0  91.7 .12  .12  17   .11  0  
loop-lit/cggmp2005_variant.yml .875  900   810   5300 11000 59      0      87.5  93.3 .13  .13  16   .16  0  
loop-lit/cggmp2005b.yml .75   110   81   3600 1200 1.3    0      75.0  88.2 .14  .13  17   .16  0  
loop-lit/css2003.yml .714  900   750   6800 9200 48      0      71.4  93.3 .17  .17  17   .16  0  
loop-lit/ddlm2013.yml .9    900   880   1000 10000 .066  0      90.0  95.5 .14  .14  17   .19  0  
loop-lit/gj2007.c.i.p+lhb-reducer.yml .548  16   4.7 610 140 .56   0      54.8  58.2 .17  .16  17   .33  0  
loop-lit/gj2007.c.i.p+nlh-reducer.yml .504  60   36   1100 700 1.4    0      50.4  57.3 .40  .37  18   4.1   0  
loop-lit/gj2007.yml .833  14   4.4 560 110 .43   0      83.3  92.9 .14  .14  17   .13  0  
loop-lit/gj2007b.yml .643  900   790   4700 11000 61      .0041 64.3  93.3 .14  .14  16   .19  0  
loop-lit/gr2006.yml .833  22   9.1 1000 220 .49   0      83.3  92.9 .13  .13  17   .14  0  
loop-lit/gsv2008.c.i.p+cfa-reducer.yml .875  900   880   1800 11000 .11   0      87.5  90.0 .14  .14  16   .16  0  
loop-lit/gsv2008.c.i.v+cfa-reducer.yml .875  900   880   1700 12000 .11   0      87.5  90.0 .14  .14  17   .16  0  
loop-lit/gsv2008.c.i.v+lhb-reducer.yml .8    900   880   1600 11000 .11   0      80.0  89.3 .14  .14  16   .16  0  
loop-lit/gsv2008.yml .875  900   880   1600 12000 .11   0      87.5  92.3 .14  .14  16   .16  0  
loop-lit/hhk2008.yml .9    900   880   2800 11000 .11   0      90.0  93.8 .14  .14  16   .18  0  
loop-lit/jm2006.c.i.v+cfa-reducer.yml .9    23   17   1500 270 .045  0      90.0  92.0 .14  .13  16   .18  0  
loop-lit/jm2006.yml .9    23   17   1200 250 .045  0      90.0  93.8 .15  .15  16   .18  0  
loop-lit/jm2006_variant.yml .917  900   810   5000 11000 43      0      91.7  94.7 .14  .15  16   .19  0  
loop-new/count_by_1.yml .75   900   880   1800 12000 .0041 0      75.0  87.5 .12  .12  16   .078 0  
loop-new/count_by_1_variant.yml .75   900   880   2100 11000 .012  0      75.0  87.5 .13  .13  16   .094 0  
loop-new/count_by_2.yml .75   900   880   1700 11000 .0041 0      75.0  87.5 .12  .12  16   .078 0  
loop-new/count_by_k.yml .875  900   810   4700 11000 72      0      87.5  90.0 .13  .16  17   .16  0  
loop-new/count_by_nondet.yml .875  900   870   1700 13000 .066  0      87.5  92.9 .13  .13  17   .16  0  
loop-new/gauss_sum.i.p+cfa-reducer.yml .875  900   770   6400 9100 40      0      87.5  90.0 .13  .13  16   .16  0  
loop-new/gauss_sum.i.p+lhb-reducer.yml .889  900   770   6200 9900 44      0      88.9  92.5 .16  .26  16   .31  0  
loop-new/gauss_sum.i.v+cfa-reducer.yml .875  900   780   5600 11000 41      .0041 87.5  90.0 .13  .13  17   .16  0  
loop-new/gauss_sum.yml .875  900   680   8300 11000 35      0      87.5  91.7 .13  .13  17   .16  0  
loop-new/half.yml .857  900   760   5800 9700 47      0      85.7  92.9 .15  .14  16   .21  0  
loop-new/nested-1.yml .929  900   770   6300 10000 55      .0041 92.9  93.3 .16  .15  16   .23  0  
loop-industry-pattern/mod3.c.v+cfa-reducer.yml .75   34   28   1500 400 .066  0      75.0  71.0 .14  .14  17   .19  0  
loop-industry-pattern/mod3.c.v+lhb-reducer.yml .75   52   46   450 700 .23   0      75.0  80.0 .16  .15  16   .29  0  
loop-industry-pattern/mod3.c.v+sep-reducer.yml .75   51   46   440 720 .23   0      75.0  79.2 .16  .15  17   .29  0  
loop-industry-pattern/mod3.yml .75   34   27   1400 410 .066  0      75.0  82.3 .16  .15  16   .19  0  
sv-benchmarks/c/ status score program test-suite TS size inspect test-suite cpu (s) wall (s) mem (MB) energy (J) blkio-w (MB) blkio-r (MB) status program test-suite TS size inspect test-suite cov branch cov (%) line cov (%) cpu (s) wall (s) mem (MB) energy blkio-w (MB) blkio-r (MB)
total 117 91.5 57000 51000 360000 690000 2000 .020 117 9150 10000 66 66 1900 240 0  
Run set coveritest.test-comp19_prop-coverage-branches.ReachSafety-Loops tbf-testsuite-validator-coveritest.test-comp19_prop-coverage-branches.ReachSafety-Loops