Benchmark PRtest TBF Test-Suite Validator
Tool tbf v0.3.0-testcomp19 Tbf Test-suite Validator v1.2
Limits timelimit: 900 s, memlimit: 15000 MB, CPU core limit: 8 timelimit: 10800 s, memlimit: 7000 MB, CPU core limit: 2
Host apollon* [apollon010; apollon052; apollon080; apollon107; apollon135; apollon167]
OS Linux 4.15.0-45-generic
System CPU: Intel Xeon E3-1230 v5 @ 3.40 GHz, cores: 8, frequency: 3.8 GHz, Turbo Boost: disabled; RAM: 33546 MB
Date of execution 2019-02-06 06:02:44 CET 2019-02-06 13:26:19 CET
Run set prtest.test-comp19_prop-coverage-branches.ReachSafety-Loops tbf-testsuite-validator-prtest.test-comp19_prop-coverage-branches.ReachSafety-Loops
Options --stats -i random --write-xml --svcomp-nondets --verbose --sequence-file cov-seq.txt -r --test-suite ../../results-verified/prtest.2019-02-06_0602.logfiles/${rundefinition_name}.${inputfile_name}.files/test-suite.zip
sv-benchmarks/c/ status score program test-suite TS size inspect test-suite cpu (s) wall (s) mem (MB) energy (J) blkio-w (MB) blkio-r (MB) status program test-suite TS size inspect test-suite cov branch cov (%) line cov (%) cpu (s) wall (s) mem (MB) energy blkio-w (MB) blkio-r (MB)
loops/array-1.yml .833  900    840    31 14000   .012  0   83.3  93.3  .13  .13  16   .094 0  
loops/array-2.yml .667  .27 .45 30 3.1 0      0   66.7  86.7  .12  .12  16   .078 0  
loops/bubble_sort-1.yml .0    .25 .44 31 3.2 0      0   0    0    .064 .065 9.0 0     0  
loops/compact.yml .167  900    840    35 12000   .012  0   16.7  50.0  .38  .38  45   .070 0  
loops/count_up_down-1.yml .75   900    850    31 12000   .012  0   75.0  90.9  5.1   5.1   17   .078 0  
loops/count_up_down-2.yml .0    7.7  7.4  31 110   0      0   0    0    .066 .067 9.0 0     0  
loops/eureka_01-1.yml .192  900    840    31 12000   .012  0   19.2  16.1  .13  .13  17   .11  0  
loops/for_bounded_loop1.yml .625  140    130    31 1900   0      0   62.5  87.5  .37  .37  45   .094 0  
loops/insertion_sort-1.yml .0    .26 .45 31 3.0 0      0   0    0    .073 .074 9.0 0     0  
loops/insertion_sort-2.yml .0    .25 .45 31 3.0 0      0   0    0    .069 .071 9.1 0     0  
loops/invert_string-1.yml .0    .26 .35 31 3.0 0      0   .00 15.0  .13  .13  17   .078 0  
loops/invert_string-3.yml .875  900    850    31 12000   .012  0   87.5  94.4  .11  .11  16   .078 0  
loops/linear_sea.ch.yml .0    .22 .36 31 2.2 0      0   0    0    .061 .061 9.0 0     0  
loops/linear_search.yml .0    .20 .33 31 2.6 0      0   0    0    .069 .070 9.0 0     0  
loops/matrix-1.yml .875  900    840    30 13000   .012  0   87.5  94.1  .15  .15  16   .094 0  
loops/matrix-2.yml .0    .27 .46 31 2.6 0      0   0    0    .059 .060 9.0 0     0  
loops/n.c40.yml .5    900    840    31 13000   .012  0   50.0  63.6  .13  .14  17   .094 0  
loops/nec20.yml .167  .52 .63 30 6.6 0      0   16.7  33.3  .12  .12  16   .094 0  
loops/nec40.yml .4    900    840    31 12000   .012  0   40.0  46.7  .13  .13  16   .094 0  
loops/string-1.yml .808  900    840    31 12000   .012  0   80.8  96.5  .13  .13  16   .15  0  
loops/string-2.yml .808  900    840    31 14000   .0041 0   80.8  96.5  .13  .15  16   .14  0  
loops/sum01-1.yml .4    11    11    31 160   0      0   40.0  70.0  .12  .12  16   .078 0  
loops/sum01-2.yml .833  900    840    31 14000   .012  0   83.3  90.0  .13  .12  16   .13  0  
loops/sum01_bug02.yml .4    9.6  9.2  31 130   0      0   40.0  63.6  .11  .11  16   .078 0  
loops/sum01_bug02_sum01_bug02_base.case.yml .4    11    11    31 140   0      0   40.0  70.0  .11  .11  17   .078 0  
loops/sum03-1.yml .0    .26 .45 31 2.7 0      0   0    0    .060 .061 9.0 0     0  
loops/sum_array-1.yml .0    .28 .37 31 2.8 0      0   0    0    .058 .059 9.0 0     0  
loops/sum_array-2.yml .0    .27 .46 31 3.3 0      0   0    0    .063 .064 9.0 0     0  
loops/terminator_01.yml .0    3.8  3.8  30 48   0      0   0    0    .062 .062 9.0 0     0  
loops/terminator_02-1.yml .5    .32 .47 31 3.6 0      0   50.0  62.5  .39  .39  45   .078 0  
loops/terminator_02-2.yml .35   900    840    31 13000   .012  0   35.0  36.8  .13  .12  16   .13  0  
loops/terminator_03-1.yml .25   .25 .44 30 3.1 0      0   25.0  75.0  .12  .12  17   .078 0  
loops/terminator_03-2.yml .786  900    840    31 11000   .012  0   78.6  92.3  .13  .14  16   .13  0  
loops/trex01-1.yml .0    .26 .45 31 2.9 0      0   0    0    .060 .061 9.0 0     0  
loops/trex01-2.yml .654  900    830    31 12000   .012  0   65.4  65.5  .15  .15  17   .21  0  
loops/trex02-1.yml .833  900    840    34 13000   .012  0   83.3  92.9  .36  .36  45   .094 0  
loops/trex02-2.yml .0    .26 .45 31 3.2 0      0   0    0    .061 .062 9.0 0     0  
loops/trex03-1.yml .333  900    840    35 12000   .012  0   33.3  56.2  .36  .36  45   .078 0  
loops/trex03-2.yml .333  900    840    35 11000   .012  0   33.3  56.2  .38  .38  45   .078 0  
loops/veris.c_sendmail_tTflag_arr_one_loop.yml .875  900    840    31 13000   .0041 0   87.5  94.7  .13  .13  16   .11  0  
loops/vogal-1.yml .95   900    840    31 15000   .012  0   95.0  96.2  .14  .14  16   .11  0  
loops/vogal-2.yml .944  .24 .33 31 2.9 0      0   94.4  96.2  .11  .12  16   .078 0  
loop-acceleration/array_3-1.yml .875  900    850    31 12000   .012  0   87.5  90.9  .18  .18  17   .094 0  
loop-acceleration/array_3-2.yml .0    .26 .45 31 2.9 0      0   0    0    .074 .076 9.0 0     0  
loop-acceleration/array_4.yml .833  900    850    30 13000   .012  0   83.3  90.0  .14  .14  17   .078 0  
loop-acceleration/diamond_1-1.yml .833  900    840    31 12000   .012  0   83.3  91.7  .12  .12  16   .094 0  
loop-acceleration/diamond_1-2.yml .0    .24 .43 31 3.6 0      0   0    0    .085 .091 8.9 0     0  
loop-acceleration/diamond_2-2.yml .958  900    840    30 13000   .012  0   95.8  96.5  .14  .14  16   .094 0  
loop-acceleration/multivar_1-1.yml .75   900    840    30 13000   .012  0   75.0  90.9  .12  .12  17   .094 0  
loop-acceleration/multivar_1-2.yml .0    .26 .45 30 3.1 0      0   0    0    .063 .063 9.0 0     0  
loop-acceleration/simple_3-2.yml .75   900    850    31 12000   .012  0   75.0  90.0  .11  .11  16   .078 0  
loop-crafted/simple_array_index_value_2.yml .833  900    840    31 14000   .012  0   83.3  90.9  .37  .37  45   .078 0  
loop-crafted/simple_array_index_value_4.i.v+lhb-reducer.yml .176  900    840    31 12000   .012  0   17.6  14.7  .13  .13  17   .12  0  
loop-crafted/simple_array_index_value_4.i.v+nlh-reducer.yml .214  900    840    31 12000   .012  0   21.4  19.7  .13  .13  16   .12  0  
loop-crafted/simple_array_index_value_4.yml .0    900    840    31 13000   .012  0   .00 28.6  .12  .12  17   .094 0  
loop-invgen/MADWiFi-encode_ie_ok.yml .5    900    840    31 13000   .012  0   50.0  37.0  .13  .13  16   .16  0  
loop-invgen/NetBSD_loop.yml .125  900    840    31 13000   .012  0   12.5  25.0  20     20     16   .078 0  
loop-invgen/SpamAssassin-loop.i.v+cfa-reducer.yml .636  900    830    34 11000   .012  0   63.6  73.3  1.2   1.2   48   .21  0  
loop-invgen/apache-escape-absolute.i.v+cfa-reducer.yml .145  900    830    31 13000   .033  0   14.5  10.4  .16  .15  17   .24  0  
loop-invgen/apache-escape-absolute.yml .262  900    830    31 12000   .012  0   26.2  13.5  .13  .13  16   .19  0  
loop-invgen/apache-get-tag.i.p+lhb-reducer.yml .135  900    840    31 13000   .012  0   13.5  17.9  .61  .62  46   .28  0  
loop-invgen/apache-get-tag.i.p+nlh-reducer.yml .0    .22 .34 31 2.3 0      0   0    0    .063 .064 9.0 0     0  
loop-invgen/apache-get-tag.i.p+sep-reducer.yml .0    .24 .37 31 2.3 0      0   0    0    .060 .060 9.0 0     0  
loop-invgen/apache-get-tag.i.v+lhb-reducer.yml .35   900    840    31 13000   .012  0   35.0  43.0  .59  .59  46   .18  0  
loop-invgen/apache-get-tag.i.v+nlh-reducer.yml .375  900    840    31 13000   .012  0   37.5  45.7  .58  .58  46   .18  0  
loop-invgen/apache-get-tag.yml .667  900    840    30 12000   .012  0   66.7  76.7  .58  .59  46   .14  0  
loop-invgen/down.yml .833  900    840    30 12000   .012  0   83.3  94.1  16     16     17   .094 0  
loop-invgen/fragtest_simple.yml .429  900    840    33 13000   .012  0   42.9  46.1  .61  .62  46   .094 0  
loop-invgen/half_2.yml .875  900    840    31 14000   .029  0   87.5  94.4  .13  .13  17   .11  0  
loop-invgen/heapsort.yml .75   900    830    33 13000   .012  0   75.0  80.5  1.1   1.1   48   .16  0  
loop-invgen/id_build.i.p+nlh-reducer.yml .0    .21 .35 31 2.6 0      0   0    0    .060 .060 9.0 0     0  
loop-invgen/id_build.i.p+sep-reducer.yml .0021 900    840    33 14000   .012  0   .21 .55 20     20     22   .20  0  
loop-invgen/id_build.i.v+lhb-reducer.yml .0714 900    840    31 11000   .012  0   7.14 16.7  20     20     16   .078 0  
loop-invgen/id_build.yml .167  900    840    31 13000   .012  0   16.7  36.4  20     20     16   .078 0  
loop-invgen/id_trans.yml .125  900    850    31 13000   .012  0   12.5  33.3  .12  .12  16   .078 0  
loop-invgen/large_const.yml .556  900    840    31 12000   .029  0   55.6  55.6  .13  .13  16   .14  0  
loop-invgen/nest-if3.yml .5    900    840    31 12000   .012  0   50.0  46.7  20     20     28   .14  0  
loop-invgen/nested6.yml .214  900    840    30 13000   .012  0   21.4  36.8  .12  .12  16   .094 0  
loop-invgen/nested9.yml .5    900    830    31 14000   .012  0   50.0  38.9  .14  .14  16   .16  0  
loop-invgen/sendmail-close-angle.yml .917  900    830    31 13000   .012  0   91.7  96.7  .59  .58  46   .16  0  
loop-invgen/seq-3.yml .938  900    830    31 11000   .012  0   93.8  95.7  .16  .15  16   .16  0  
loop-invgen/up.yml .833  900    840    31 12000   .012  0   83.3  93.8  16     16     16   .094 0  
loop-lit/afnp2014.yml .667  900    850    31 12000   .012  0   66.7  91.7  .15  .15  17   .078 0  
loop-lit/bhmr2007.yml .8    900    840    31 12000   .012  0   80.0  81.2  .50  .50  45   .13  0  
loop-lit/cggmp2005.yml .75   900    850    31 12000   .012  0   75.0  91.7  .11  .11  16   .078 0  
loop-lit/cggmp2005_variant.yml .875  900    840    31 13000   .012  0   87.5  93.3  .12  .12  16   .11  0  
loop-lit/cggmp2005b.yml .75   900    850    31 12000   .012  0   75.0  88.2  .12  .12  16   .078 0  
loop-lit/css2003.yml .714  900    840    31 14000   .012  0   71.4  93.3  .14  .14  16   .11  0  
loop-lit/ddlm2013.yml .6    900    840    31 13000   0      0   60.0  86.4  .59  .61  45   .094 0  
loop-lit/gj2007.c.i.p+lhb-reducer.yml .548  900    850    30 12000   .012  0   54.8  58.2  .12  .12  16   .082 0  
loop-lit/gj2007.c.i.p+nlh-reducer.yml .504  900    850    31 12000   .012  0   50.4  57.3  .15  .15  17   .12  0  
loop-lit/gj2007.yml .833  900    850    31 12000   .012  0   83.3  92.9  .11  .11  16   .078 0  
loop-lit/gj2007b.yml .643  900    840    33 15000   .012  0   64.3  93.3  .65  .65  46   .13  0  
loop-lit/gr2006.yml .833  900    850    30 12000   .012  0   83.3  92.9  .13  .13  16   .078 0  
loop-lit/gsv2008.c.i.p+cfa-reducer.yml .875  900    840    31 14000   .012  0   87.5  90.0  .12  .12  17   .11  0  
loop-lit/gsv2008.c.i.v+cfa-reducer.yml .875  900    840    31 13000   .012  0   87.5  90.0  .12  .12  16   .11  0  
loop-lit/gsv2008.c.i.v+lhb-reducer.yml .8    900    840    31 13000   .012  0   80.0  89.3  .12  .12  16   .13  0  
loop-lit/gsv2008.yml .875  900    840    31 12000   .012  0   87.5  92.3  .12  .12  16   .11  0  
loop-lit/hhk2008.yml .9    900    840    31 12000   .012  0   90.0  93.8  .13  .16  17   .13  0  
loop-lit/jm2006.c.i.v+cfa-reducer.yml .7    900    840    31 12000   .012  0   70.0  72.0  1.3   1.3   17   .11  0  
loop-lit/jm2006.yml .7    900    840    31 12000   .012  0   70.0  68.8  .99  .99  16   .11  0  
loop-lit/jm2006_variant.yml .75   900    840    31 12000   .012  0   75.0  73.7  .13  .12  16   .13  0  
loop-new/count_by_1.yml .75   900    850    31 12000   .012  0   75.0  87.5  .12  .12  16   .078 0  
loop-new/count_by_1_variant.yml .75   900    850    31 12000   .012  0   75.0  87.5  .13  .12  16   .078 0  
loop-new/count_by_2.yml .75   900    850    31 11000   .012  0   75.0  87.5  .12  .12  16   .078 0  
loop-new/count_by_k.yml .875  900    840    31 16000   .012  0   87.5  90.0  .12  .12  16   .11  0  
loop-new/count_by_nondet.yml .875  900    840    31 15000   .012  0   87.5  92.9  .13  .13  17   .13  0  
loop-new/gauss_sum.i.p+cfa-reducer.yml .875  900    840    31 13000   .012  0   87.5  90.0  .12  .12  17   .11  0  
loop-new/gauss_sum.i.p+lhb-reducer.yml .722  900    840    31 13000   .012  0   72.2  85.0  .13  .12  17   .13  0  
loop-new/gauss_sum.i.v+cfa-reducer.yml .875  900    840    31 12000   .012  0   87.5  90.0  .12  .12  16   .11  0  
loop-new/gauss_sum.yml .875  900    840    31 14000   .012  0   87.5  91.7  .13  .13  16   .11  0  
loop-new/half.yml .857  900    840    30 9900   .012  0   85.7  92.9  .13  .13  16   .13  0  
loop-new/nested-1.yml .5    900    840    31 14000   .012  0   50.0  40.0  .12  .12  16   .13  0  
loop-industry-pattern/mod3.c.v+cfa-reducer.yml .583  900    840    33 13000   .012  0   58.3  71.0  .63  .63  46   .094 0  
loop-industry-pattern/mod3.c.v+lhb-reducer.yml .55   900    840    34 12000   .025  0   55.0  61.3  1.3   1.2   51   .16  0  
loop-industry-pattern/mod3.c.v+sep-reducer.yml .6    900    840    32 14000   .025  0   60.0  70.8  1.0   1.0   46   .14  0  
loop-industry-pattern/mod3.yml .583  900    840    33 14000   .012  0   58.3  58.8  .61  .61  45   .094 0  
sv-benchmarks/c/ status score program test-suite TS size inspect test-suite cpu (s) wall (s) mem (MB) energy (J) blkio-w (MB) blkio-r (MB) status program test-suite TS size inspect test-suite cov branch cov (%) line cov (%) cpu (s) wall (s) mem (MB) energy blkio-w (MB) blkio-r (MB)
total 117 59.6 79000 74000 3600 1100000 1.1 0   117 5960 6880 160 160 2500 11 0  
Run set prtest.test-comp19_prop-coverage-branches.ReachSafety-Loops tbf-testsuite-validator-prtest.test-comp19_prop-coverage-branches.ReachSafety-Loops