Benchmark CPA/Tiger-MGP TBF Test-Suite Validator
Tool CPAchecker 1.8-svn 30541M Tbf Test-suite Validator v1.2
Limits timelimit: 900 s, memlimit: 15000 MB, CPU core limit: 8 timelimit: 10800 s, memlimit: 7000 MB, CPU core limit: 2
Host apollon*
OS Linux 4.15.0-45-generic
System CPU: Intel Xeon E3-1230 v5 @ 3.40 GHz, cores: 8, frequency: 3.8 GHz, Turbo Boost: disabled; RAM: 33546 MB
Date of execution 2019-02-05 21:13:54 CET 2019-02-06 06:15:02 CET
Run set cpa-tiger.test-comp19_prop-coverage-branches.ReachSafety-Loops tbf-testsuite-validator-cpa-tiger.test-comp19_prop-coverage-branches.ReachSafety-Loops
Options -benchmark -heap 10000M -tigertestcomp19 --verbose --sequence-file cov-seq.txt -r --test-suite ../../results-verified/cpa-tiger.2019-02-05_2113.logfiles/${rundefinition_name}.${inputfile_name}.files/test-suite.zip
sv-benchmarks/c/ status score program test-suite TS size inspect test-suite cpu (s) wall (s) mem (MB) energy (J) blkio-w (MB) blkio-r (MB) status program test-suite TS size inspect test-suite cov branch cov (%) line cov (%) cpu (s) wall (s) mem (MB) energy blkio-w (MB) blkio-r (MB)
loops/array-1.yml .833  3.2 1.4 270 30 .0041 0   83.3  93.3  .14  .14  16   .13  0  
loops/array-2.yml 1.0    3.2 1.4 280 32 .0041 0   100    100    .13  .13  17   .14  0  
loops/bubble_sort-1.yml .0417 240   230   1200 3100 .0041 0   4.17 17.9  .14  .14  17   .17  0  
loops/compact.yml .0    900   900   2600 7600 .0041 0   0    0    .062 .062 9.0 0     0  
loops/count_up_down-1.yml .0    900   870   5900 12000 .0041 0   0    0    .063 .063 9.0 0     0  
loops/count_up_down-2.yml .75   240   230   2000 3100 .0041 0   75.0  90.9  .12  .12  16   .094 0  
loops/eureka_01-1.yml .962  8.1 5.1 300 97 .0041 0   96.2  96.8  .17  .17  17   .33  0  
loops/for_bounded_loop1.yml 1.0    3.3 1.4 270 28 .0041 0   100    100    .13  .12  16   .13  0  
loops/insertion_sort-1.yml 1.0    470   450   3400 6700 .0041 0   100    100    .13  .13  17   .11  0  
loops/insertion_sort-2.yml .7    330   310   4400 4200 .0041 0   70.0  82.3  .13  .13  16   .14  0  
loops/invert_string-1.yml .875  3.5 1.6 270 32 .0041 0   87.5  95.0  .12  .12  17   .078 0  
loops/invert_string-3.yml .0    5.2 2.4 300 58 .0041 0   0    0    .068 .076 9.0 0     0  
loops/linear_sea.ch.yml .75   3.5 1.6 280 34 .0041 0   75.0  87.5  .13  .14  17   .13  0  
loops/linear_search.yml 1.0    6.4 2.9 310 63 .0041 0   100    100    .14  .14  17   .18  0  
loops/matrix-1.yml .875  3.6 1.6 280 36 .0041 0   87.5  94.1  .15  .15  16   .14  0  
loops/matrix-2.yml 1.0    3.7 1.7 280 33 .0041 0   100    100    .15  .14  16   .20  0  
loops/n.c40.yml .5    5.8 3.9 270 63 .0041 0   50.0  63.6  .12  .12  16   .11  0  
loops/nec20.yml .833  3.5 1.5 270 35 .0041 0   83.3  95.8  .13  .13  16   .14  0  
loops/nec40.yml .4    6.2 4.0 290 72 .0041 0   40.0  46.7  .13  .12  16   .094 0  
loops/string-1.yml .385  700   660   15000 8000 .0041 0   38.5  48.3  .12  .12  16   .082 0  
loops/string-2.yml .731  560   540   15000 6700 .0041 0   73.1  86.2  .13  .13  17   .098 0  
loops/sum01-1.yml 1.0    5.6 2.3 290 47 .0041 0   100    100    .12  .12  16   .13  0  
loops/sum01-2.yml .75   470   450   1500 5300 .0041 0   75.0  90.0  .14  .14  17   .14  0  
loops/sum01_bug02.yml 1.0    4.3 1.8 290 37 .0041 0   100    100    .13  .12  16   .13  0  
loops/sum01_bug02_sum01_bug02_base.case.yml 1.0    3.3 1.5 280 27 .0041 0   100    100    .14  .13  17   .16  0  
loops/sum03-1.yml .875  5.4 2.4 290 63 .0041 0   87.5  100    .12  .12  16   .094 0  
loops/sum_array-1.yml 1.0    3.6 1.7 280 34 .0041 0   100    100    .15  .15  16   .14  0  
loops/sum_array-2.yml .929  4.0 1.9 280 37 .0041 0   92.9  94.4  .14  .14  17   .20  0  
loops/terminator_01.yml .75   3.1 1.3 270 27 .0041 0   75.0  81.8  .13  .13  16   .094 0  
loops/terminator_02-1.yml .875  3.2 1.4 260 32 .0041 0   87.5  87.5  .15  .15  16   .13  0  
loops/terminator_02-2.yml .9    3.3 1.5 270 32 .0041 0   90.0  94.7  .15  .14  16   .21  0  
loops/terminator_03-1.yml .75   3.3 1.4 270 29 .0041 0   75.0  100    .14  .13  17   .18  0  
loops/terminator_03-2.yml .786  3.3 1.4 270 27 .0041 0   78.6  92.3  .13  .13  16   .16  0  
loops/trex01-1.yml 1.0    3.3 1.5 270 31 .0041 0   100    96.2  .19  .20  16   .21  0  
loops/trex01-2.yml .962  15   7.4 550 140 .0041 0   96.2  93.1  .17  .16  16   .33  0  
loops/trex02-1.yml .833  3.0 1.3 270 27 .0041 0   83.3  92.9  .13  .13  17   .13  0  
loops/trex02-2.yml 1.0    3.2 1.3 270 31 .0041 0   100    100    .13  .20  16   .11  0  
loops/trex03-1.yml 1.0    3.4 1.4 270 30 .0041 0   100    93.8  .15  .18  16   .23  0  
loops/trex03-2.yml .889  3.3 1.4 270 30 .0041 0   88.9  87.5  .14  .14  16   .19  0  
loops/veris.c_sendmail_tTflag_arr_one_loop.yml .875  5.5 2.2 290 72 .0041 0   87.5  94.7  .13  .12  17   .13  0  
loops/vogal-1.yml .95   240   230   650 3100 .0041 0   95.0  96.2  .17  .20  16   .21  0  
loops/vogal-2.yml 1.0    41   34   500 470 .0041 0   100    100    .14  .15  16   .19  0  
loop-acceleration/array_3-1.yml .0    900   880   5100 11000 .0041 0   0    0    .060 .068 9.0 0     0  
loop-acceleration/array_3-2.yml .125  710   680   1600 8400 .0041 0   12.5  33.3  .11  .11  17   .078 0  
loop-acceleration/array_4.yml .0    900   880   5100 11000 .0041 0   0    0    .070 .071 9.0 0     0  
loop-acceleration/diamond_1-1.yml .833  170   150   1100 2100 .0041 0   83.3  91.7  .15  .15  16   .11  0  
loop-acceleration/diamond_1-2.yml .833  170   160   1400 2300 .0041 0   83.3  91.7  .12  .12  16   .11  0  
loop-acceleration/diamond_2-2.yml .958  4.7 2.2 290 47 .0041 0   95.8  96.5  .15  .18  17   .24  0  
loop-acceleration/multivar_1-1.yml .75   3.1 1.3 260 27 .0041 0   75.0  90.9  .12  .12  16   .11  0  
loop-acceleration/multivar_1-2.yml .75   3.1 1.3 260 27 .0041 0   75.0  90.9  .13  .13  17   .11  0  
loop-acceleration/simple_3-2.yml .75   240   230   2100 2900 .0041 0   75.0  90.0  .13  .13  16   .11  0  
loop-crafted/simple_array_index_value_2.yml .0    900   880   5000 11000 .0041 0   0    0    .058 .059 9.0 0     0  
loop-crafted/simple_array_index_value_4.i.v+lhb-reducer.yml .588  5.9 3.5 440 63 .0041 0   58.8  68.0  .18  .20  17   .24  0  
loop-crafted/simple_array_index_value_4.i.v+nlh-reducer.yml .714  5.7 3.2 440 66 .0041 0   71.4  77.3  .15  .16  16   .27  0  
loop-crafted/simple_array_index_value_4.yml .75   6.3 4.0 430 68 .0041 0   75.0  95.2  .14  .14  16   .16  0  
loop-invgen/MADWiFi-encode_ie_ok.yml .955  230   230   540 3400 .0041 0   95.5  96.3  .16  .15  16   .29  0  
loop-invgen/NetBSD_loop.yml .875  240   230   2200 3400 .0041 0   87.5  93.8  .13  .13  16   .13  0  
loop-invgen/SpamAssassin-loop.i.v+cfa-reducer.yml .614  15   9.3 460 180 .0041 0   61.4  72.5  20     20     16   .38  0  
loop-invgen/apache-escape-absolute.i.v+cfa-reducer.yml .763  6.0 2.7 300 63 .0041 0   76.3  76.6  .24  .35  17   .96  0  
loop-invgen/apache-escape-absolute.yml .976  5.2 2.5 290 52 .0041 0   97.6  98.1  .20  .19  17   .56  0  
loop-invgen/apache-get-tag.i.p+lhb-reducer.yml .635  700   680   1500 7900 .0041 0   63.5  80.5  .29  .29  18   1.9   0  
loop-invgen/apache-get-tag.i.p+nlh-reducer.yml .561  900   880   2700 12000 .0041 0   56.1  71.5  .76  .92  23   13     0  
loop-invgen/apache-get-tag.i.p+sep-reducer.yml .506  900   880   2700 10000 .0041 0   50.6  64.8  1.2   1.1   28   31     0  
loop-invgen/apache-get-tag.i.v+lhb-reducer.yml .433  900   870   6600 14000 .0041 0   43.3  52.4  .16  .16  17   .29  0  
loop-invgen/apache-get-tag.i.v+nlh-reducer.yml .643  470   450   1500 5200 .0041 0   64.3  73.6  .17  .16  16   .47  0  
loop-invgen/apache-get-tag.yml .667  700   680   1500 10000 .0041 0   66.7  74.4  .14  .20  17   .17  0  
loop-invgen/down.yml .833  620   600   3400 7300 .0041 0   83.3  94.1  .12  .12  16   .11  0  
loop-invgen/fragtest_simple.yml .786  700   680   4800 8000 .0041 0   78.6  96.2  .13  .15  16   .13  0  
loop-invgen/half_2.yml .875  230   230   700 3100 .0041 0   87.5  94.4  .12  .12  17   .13  0  
loop-invgen/heapsort.yml .05   900   890   830 12000 .0041 0   5.00 7.32 .13  .25  16   .13  0  
loop-invgen/id_build.i.p+nlh-reducer.yml .297  250   230   3300 2900 .0041 0   29.7  41.7  .33  .36  22   3.8   0  
loop-invgen/id_build.i.p+sep-reducer.yml .297  14   6.1 390 120 .0041 0   29.7  40.2  .37  .36  22   4.3   0  
loop-invgen/id_build.i.v+lhb-reducer.yml .714  250   230   2900 2600 .0041 0   71.4  77.8  .13  .13  17   .13  0  
loop-invgen/id_build.yml .833  310   300   1100 3800 .0041 0   83.3  90.9  .13  .16  16   .10  0  
loop-invgen/id_trans.yml 1.0    3.2 1.4 270 25 .0041 0   100    100    .16  .15  17   .16  0  
loop-invgen/large_const.yml .222  3.9 1.7 280 33 .0041 0   22.2  44.4  .16  .15  16   .26  0  
loop-invgen/nest-if3.yml .5    900   900   370 13000 .0041 0   50.0  66.7  .12  .12  17   .094 0  
loop-invgen/nested6.yml .929  240   230   4000 3000 .0041 0   92.9  94.7  .14  .13  16   .16  0  
loop-invgen/nested9.yml .773  460   450   1200 6400 .0041 0   77.3  94.4  .14  .14  17   .18  0  
loop-invgen/sendmail-close-angle.yml .833  470   450   2500 6200 .0041 0   83.3  93.3  .13  .13  16   .15  0  
loop-invgen/seq-3.yml .812  700   680   6900 6900 .0041 0   81.2  95.7  .14  .13  16   .18  0  
loop-invgen/up.yml .833  630   600   11000 7200 .0041 0   83.3  93.8  .12  .12  16   .094 0  
loop-lit/afnp2014.yml .333  900   880   1400 10000 .0041 0   33.3  50.0  .12  .12  17   .078 0  
loop-lit/bhmr2007.yml .7    900   890   460 11000 .0041 0   70.0  81.2  .13  .16  17   .14  0  
loop-lit/cggmp2005.yml .75   3.7 1.6 280 36 .0041 0   75.0  91.7  .12  .12  16   .094 0  
loop-lit/cggmp2005_variant.yml .75   460   450   390 6400 .0041 0   75.0  93.3  .12  .12  16   .11  0  
loop-lit/cggmp2005b.yml .75   900   880   1400 13000 .0041 0   75.0  88.2  .11  .11  16   .078 0  
loop-lit/css2003.yml .143  900   880   4500 13000 .0041 0   14.3  33.3  .12  .12  17   .078 0  
loop-lit/ddlm2013.yml .8    610   600   420 7900 .0041 0   80.0  95.5  .12  .12  16   .11  0  
loop-lit/gj2007.c.i.p+lhb-reducer.yml .548  690   680   700 8100 .0041 0   54.8  58.2  .14  .14  16   .14  0  
loop-lit/gj2007.c.i.p+nlh-reducer.yml .504  900   890   390 11000 .0041 0   50.4  57.3  .24  .23  18   1.3   0  
loop-lit/gj2007.yml .833  530   510   700 6200 .0041 0   83.3  92.9  .12  .12  16   .094 0  
loop-lit/gj2007b.yml .643  700   680   980 8100 .0041 0   64.3  93.3  .15  .15  16   .13  0  
loop-lit/gr2006.yml .833  420   400   3700 5200 .0041 0   83.3  92.9  .11  .11  16   .094 0  
loop-lit/gsv2008.c.i.p+cfa-reducer.yml .625  690   680   720 7800 .0041 0   62.5  80.0  .15  .17  16   .11  0  
loop-lit/gsv2008.c.i.v+cfa-reducer.yml .625  690   680   490 7800 .0041 0   62.5  80.0  .12  .12  16   .11  0  
loop-lit/gsv2008.c.i.v+lhb-reducer.yml .6    900   890   730 11000 .0041 0   60.0  82.1  .12  .12  16   .078 0  
loop-lit/gsv2008.yml .75   460   450   670 5600 .0041 0   75.0  92.3  .12  .12  17   .11  0  
loop-lit/hhk2008.yml .7    470   460   1000 5800 .0041 0   70.0  93.8  .14  .14  16   .11  0  
loop-lit/jm2006.c.i.v+cfa-reducer.yml .9    3.3 1.4 270 32 .0041 0   90.0  92.0  .13  .13  16   .18  0  
loop-lit/jm2006.yml .9    3.2 1.4 270 29 .0041 0   90.0  93.8  .13  .12  16   .14  0  
loop-lit/jm2006_variant.yml .583  900   870   5400 11000 .0041 0   58.3  79.0  .12  .12  16   .094 0  
loop-new/count_by_1.yml .75   710   690   1700 8100 .0041 0   75.0  87.5  .12  .12  17   .078 0  
loop-new/count_by_1_variant.yml .0    900   880   5000 12000 .0041 0   0    0    .060 .060 9.0 0     0  
loop-new/count_by_2.yml .0    900   880   4900 11000 .0041 0   0    0    .057 .057 9.1 0     0  
loop-new/count_by_k.yml .625  460   450   740 5200 .0041 0   62.5  90.0  .14  .14  16   .11  0  
loop-new/count_by_nondet.yml .875  240   230   1900 2700 .0041 0   87.5  92.9  .12  .12  16   .13  0  
loop-new/gauss_sum.i.p+cfa-reducer.yml .75   470   450   1400 4900 .0041 0   75.0  85.0  .12  .12  16   .094 0  
loop-new/gauss_sum.i.p+lhb-reducer.yml .722  690   680   1600 8600 .0041 0   72.2  85.0  .13  .21  16   .14  0  
loop-new/gauss_sum.i.v+cfa-reducer.yml .75   470   450   1200 5400 .0041 0   75.0  85.0  .12  .12  16   .094 0  
loop-new/gauss_sum.yml .625  900   870   3200 8800 .0041 0   62.5  91.7  .12  .12  16   .078 0  
loop-new/half.yml .429  900   890   1700 11000 .0041 0   42.9  78.6  .12  .12  17   .094 0  
loop-new/nested-1.yml .929  250   230   680 2900 .0041 0   92.9  93.3  .14  .14  16   .19  0  
loop-industry-pattern/mod3.c.v+cfa-reducer.yml .75   14   12   290 180 .0041 0   75.0  71.0  .14  .13  16   .18  0  
loop-industry-pattern/mod3.c.v+lhb-reducer.yml .7    29   27   300 390 .0041 0   70.0  74.7  .15  .16  17   .23  0  
loop-industry-pattern/mod3.c.v+sep-reducer.yml .65   45   43   300 600 .0041 0   65.0  76.4  .15  .15  16   .27  0  
loop-industry-pattern/mod3.yml .75   15   13   300 180 .0041 0   75.0  82.3  .14  .22  16   .18  0  
sv-benchmarks/c/ status score program test-suite TS size inspect test-suite cpu (s) wall (s) mem (MB) energy (J) blkio-w (MB) blkio-r (MB) status program test-suite TS size inspect test-suite cov branch cov (%) line cov (%) cpu (s) wall (s) mem (MB) energy blkio-w (MB) blkio-r (MB)
total 117 80.6 40000 39000 200000 490000 .48  0   117 8060 9110 38 39 1900 71 0  
Run set cpa-tiger.test-comp19_prop-coverage-branches.ReachSafety-Loops tbf-testsuite-validator-cpa-tiger.test-comp19_prop-coverage-branches.ReachSafety-Loops