Benchmark VeriFuzz TBF Test-Suite Validator
Tool VeriFuzz 1.0.1 Tbf Test-suite Validator v1.2
Limits timelimit: 900 s, memlimit: 15000 MB, CPU core limit: 8 timelimit: 10800 s, memlimit: 7000 MB, CPU core limit: 2
Host apollon*
OS Linux 4.15.0-45-generic
System CPU: Intel Xeon E3-1230 v5 @ 3.40 GHz, cores: 8, frequency: 3.8 GHz, Turbo Boost: disabled; RAM: 33546 MB
Date of execution 2019-02-06 07:17:17 CET 2019-02-06 14:51:38 CET
Run set verifuzz.test-comp19_prop-coverage-branches.ReachSafety-Loops tbf-testsuite-validator-verifuzz.test-comp19_prop-coverage-branches.ReachSafety-Loops
Options --testcomp --verbose --sequence-file cov-seq.txt -r --test-suite ../../results-verified/verifuzz.2019-02-06_0717.logfiles/${rundefinition_name}.${inputfile_name}.files/test-suite.zip
sv-benchmarks/c/ status score program test-suite TS size inspect test-suite cpu (s) wall (s) mem (MB) energy (J) blkio-w (MB) blkio-r (MB) status program test-suite TS size inspect test-suite cov branch cov (%) line cov (%) cpu (s) wall (s) mem (MB) energy blkio-w (MB) blkio-r (MB)
loops/array-1.yml .833  900 920 150 12000 1.1  0     83.3  93.3 .12 .11 16 .094 0  
loops/array-2.yml 1.0    890 900 150 13000 6800    0     100    100   .12 .12 16 .094 0  
loops/bubble_sort-1.yml .0417 880 900 150 11000 .93 0     4.17 17.9 .12 .12 16 .086 0  
loops/compact.yml 1.0    900 900 150 12000 2500    0     100    100   5.2  5.2  460 .086 0  
loops/count_up_down-1.yml .75   900 900 150 12000 .71 0     75.0  90.9 .14 .18 16 .21  0  
loops/count_up_down-2.yml .75   900 900 150 9700 4.2  0     75.0  90.9 1.3  1.2  16 .21  0  
loops/eureka_01-1.yml .923  880 900 150 11000 210    0     92.3  100   .19 .18 16 .46  0  
loops/for_bounded_loop1.yml 1.0    880 900 150 13000 620    0     100    100   .21 .20 16 .34  0  
loops/insertion_sort-1.yml 1.0    900 900 150 12000 14    .094 100    100   .24 .22 17 .44  0  
loops/insertion_sort-2.yml .9    900 900 160 12000 .75 .27  90.0  94.1 .22 .21 16 .42  0  
loops/invert_string-1.yml 1.0    900 900 150 12000 700    0     100    100   .21 .20 16 .34  0  
loops/invert_string-3.yml .875  900 920 150 11000 1.1  0     87.5  94.4 .11 .11 16 .078 0  
loops/linear_sea.ch.yml .667  900 920 150 12000 1.2  0     66.7  87.5 .17 .17 16 .32  0  
loops/linear_search.yml .929  880 900 150 11000 1600    0     92.9  100   .16 .15 16 .27  0  
loops/matrix-1.yml .875  900 920 150 13000 1.1  0     87.5  94.1 .18 .17 16 .21  0  
loops/matrix-2.yml 1.0    900 900 160 11000 72    0     100    100   83    84    2800 .41  0  
loops/n.c40.yml .375  900 920 150 10000 .94 0     37.5  63.6 .11 .11 16 .078 0  
loops/nec20.yml .833  890 900 150 11000 11000    0     83.3  95.8 .12 .12 17 .13  0  
loops/nec40.yml .3    900 920 150 11000 .95 0     30.0  46.7 .11 .11 16 .078 0  
loops/string-1.yml .885  900 920 150 11000 1.3  0     88.5  96.5 .17 .16 16 .34  0  
loops/string-2.yml .962  880 900 150 13000 37    0     96.2  100   .17 .17 17 .34  0  
loops/sum01-1.yml .9    900 900 150 9400 14    0     90.0  100   .16 .16 16 .32  0  
loops/sum01-2.yml .583  900 920 150 10000 .87 0     58.3  90.0 .15 .14 16 .23  0  
loops/sum01_bug02.yml .9    900 900 150 12000 20    0     90.0  100   .45 .45 16 .27  0  
loops/sum01_bug02_sum01_bug02_base.case.yml .9    900 900 150 13000 18    0     90.0  100   .54 .54 16 .21  0  
loops/sum03-1.yml .875  890 900 150 11000 7400    0     87.5  100   .11 .11 16 .094 0  
loops/sum_array-1.yml 1.0    890 900 250 12000 440    .12  100    100   1.8  1.8  280 .27  0  
loops/sum_array-2.yml .929  900 910 280 11000 88    0     92.9  94.4 43    43    1000 .57  0  
loops/terminator_01.yml .75   900 900 150 9900 3.4  0     75.0  81.8 5.9  5.9  16 .14  0  
loops/terminator_02-1.yml .0    900 900 14000 11000 53    0     47    47    7000 0     0  
loops/terminator_02-2.yml .9    900 920 150 10000 1.0  0     90.0  94.7 .19 .20 17 .39  0  
loops/terminator_03-1.yml .75   900 900 150 12000 870    0     75.0  100   .15 .14 16 .24  0  
loops/terminator_03-2.yml .786  900 900 150 10000 .75 0     78.6  92.3 .17 .17 17 .31  0  
loops/trex01-1.yml .0    900 900 2200 14000 150    0     45    45    7000 0     0  
loops/trex01-2.yml .885  900 920 150 11000 3.7  0     88.5  96.5 1.8  1.8  140 .62  0  
loops/trex02-1.yml .0    900 900 9100 13000 .93 0     46    46    7000 0     0  
loops/trex02-2.yml .0    900 900 12000 11000 1.7  0     18    18    7000 0     0  
loops/trex03-1.yml .0    900 900 2200 14000 .56 0     45    45    7000 0     0  
loops/trex03-2.yml .0    900 900 4300 11000 .85 0     48    48    7000 .14  0  
loops/veris.c_sendmail_tTflag_arr_one_loop.yml .875  900 920 150 11000 1.1  0     87.5  94.7 .14 .14 16 .18  0  
loops/vogal-1.yml .95   900 920 150 11000 1.1  0     95.0  96.2 .22 .22 16 .29  0  
loops/vogal-2.yml 1.0    880 900 150 11000 3200    0     100    100   .18 .17 16 .37  0  
loop-acceleration/array_3-1.yml .875  900 920 150 11000 1.2  0     87.5  90.9 .38 .37 17 .21  0  
loop-acceleration/array_3-2.yml 1.0    880 900 150 12000 3000    0     100    100   .37 .38 17 .21  0  
loop-acceleration/array_4.yml .833  900 920 150 13000 1.4  0     83.3  90.0 .38 .37 17 .21  0  
loop-acceleration/diamond_1-1.yml .833  900 920 150 11000 .93 0     83.3  91.7 .14 .15 17 .094 0  
loop-acceleration/diamond_1-2.yml .833  890 900 150 12000 7500    0     83.3  91.7 .14 .14 16 .094 0  
loop-acceleration/diamond_2-2.yml .958  900 920 150 13000 .92 0     95.8  96.5 .14 .13 16 .19  0  
loop-acceleration/multivar_1-1.yml .75   900 920 150 11000 1.2  0     75.0  90.9 .14 .15 17 .21  0  
loop-acceleration/multivar_1-2.yml .75   890 900 150 12000 7700    0     75.0  90.9 .14 .14 16 .21  0  
loop-acceleration/simple_3-2.yml .75   900 920 150 10000 .90 0     75.0  90.0 .14 .13 16 .21  0  
loop-crafted/simple_array_index_value_2.yml .833  900 920 150 12000 1.7  0     83.3  90.9 .45 .45 46 .21  0  
loop-crafted/simple_array_index_value_4.i.v+lhb-reducer.yml .559  880 900 11000 11000 .97 .033 55.9  66.7 .18 .17 16 .29  0  
loop-crafted/simple_array_index_value_4.i.v+nlh-reducer.yml .679  890 900 11000 12000 .93 .033 67.9  75.8 .16 .15 16 .31  0  
loop-crafted/simple_array_index_value_4.yml .75   880 900 10000 11000 1.0  .033 75.0  95.2 .16 .15 16 .29  0  
loop-invgen/MADWiFi-encode_ie_ok.yml .909  900 920 150 10000 1.1  0     90.9  96.3 .19 .18 17 .41  0  
loop-invgen/NetBSD_loop.yml .75   900 900 150 12000 1.1  0     75.0  93.8 14    14    17 .26  0  
loop-invgen/SpamAssassin-loop.i.v+cfa-reducer.yml .0    420 420 15000 5800 1.1  0     41    41    7000 0     0  
loop-invgen/apache-escape-absolute.i.v+cfa-reducer.yml .75   880 900 180 13000 1.2  0     75.0  76.0 12    12    240 1.2   0  
loop-invgen/apache-escape-absolute.yml .786  900 910 150 12000 4.4  0     78.6  98.1 46    47    2100 .80  0  
loop-invgen/apache-get-tag.i.p+lhb-reducer.yml .667  880 900 210 11000 1.6  0     66.7  81.9 2.1  2.1  280 2.8   0  
loop-invgen/apache-get-tag.i.p+nlh-reducer.yml .744  880 900 1100 11000 2.6  0     74.4  86.1 2.7  2.6  280 20     0  
loop-invgen/apache-get-tag.i.p+sep-reducer.yml .667  890 900 1400 10000 2.8  0     66.7  84.5 3.1  3.0  280 38     0  
loop-invgen/apache-get-tag.i.v+lhb-reducer.yml .65   880 900 180 14000 1.4  0     65.0  73.8 3.4  3.4  280 1.0   0  
loop-invgen/apache-get-tag.i.v+nlh-reducer.yml .661  880 900 170 11000 1.4  0     66.1  74.3 2.0  2.0  280 1.1   0  
loop-invgen/apache-get-tag.yml .944  900 920 150 12000 2.2  0     94.4  97.7 .62 .60 25 .68  0  
loop-invgen/down.yml .833  900 900 150 11000 .68 0     83.3  94.1 .14 .14 17 .26  0  
loop-invgen/fragtest_simple.yml .786  880 900 150 14000 1.0  0     78.6  96.2 2.0  2.0  280 .39  0  
loop-invgen/half_2.yml .875  900 920 150 13000 1.0  0     87.5  94.4 .17 .16 18 .34  0  
loop-invgen/heapsort.yml .9    900 900 450 12000 .97 0     90.0  97.6 300    300    7000 .72  0  
loop-invgen/id_build.i.p+nlh-reducer.yml .297  900 920 400 11000 1.3  0     29.7  41.7 .21 .21 22 1.1   0  
loop-invgen/id_build.i.p+sep-reducer.yml .297  900 920 390 13000 1.1  0     29.7  40.2 .22 .21 22 1.1   0  
loop-invgen/id_build.i.v+lhb-reducer.yml .643  900 900 150 12000 .85 0     64.3  75.0 13    13    16 .18  0  
loop-invgen/id_build.yml .833  900 900 150 11000 .79 0     83.3  90.9 .15 .15 16 .26  0  
loop-invgen/id_trans.yml 1.0    890 900 150 11000 68    0     100    100   .34 .34 16 .27  0  
loop-invgen/large_const.yml .833  900 920 150 12000 .91 0     83.3  96.3 .13 .13 16 .18  0  
loop-invgen/nest-if3.yml .929  900 900 150 12000 1.3  0     92.9  93.3 31    31    700 .63  0  
loop-invgen/nested6.yml .929  900 900 150 9500 1.4  0     92.9  94.7 50    50    5700 .57  0  
loop-invgen/nested9.yml .818  900 900 150 11000 .79 0     81.8  94.4 .22 .21 16 .38  0  
loop-invgen/sendmail-close-angle.yml .917  900 920 150 12000 1.5  0     91.7  96.7 .20 .19 16 .46  0  
loop-invgen/seq-3.yml .812  900 920 160 11000 1.2  0     81.2  95.7 .17 .16 17 .39  0  
loop-invgen/up.yml .833  900 900 160 9600 .72 0     83.3  93.8 3.4  3.4  16 .24  0  
loop-lit/afnp2014.yml .833  900 920 150 11000 1.2  0     83.3  91.7 .22 .24 17 .21  0  
loop-lit/bhmr2007.yml .8    900 910 150 11000 3.2  0     80.0  93.8 1.4  1.4  120 .37  0  
loop-lit/cggmp2005.yml .75   900 920 150 11000 .89 0     75.0  91.7 .14 .22 16 .19  0  
loop-lit/cggmp2005_variant.yml .75   900 920 150 13000 1.1  0     75.0  93.3 .14 .14 16 .23  0  
loop-lit/cggmp2005b.yml .75   900 920 150 11000 1.1  0     75.0  88.2 .13 .13 16 .19  0  
loop-lit/css2003.yml .643  900 920 150 12000 1.1  0     64.3  93.3 .14 .14 16 .11  0  
loop-lit/ddlm2013.yml .9    900 920 150 12000 1.3  0     90.0  95.5 .23 .22 16 .47  0  
loop-lit/gj2007.c.i.p+lhb-reducer.yml .548  900 920 160 12000 1.2  0     54.8  58.2 .17 .16 16 .23  0  
loop-lit/gj2007.c.i.p+nlh-reducer.yml .504  900 920 260 11000 1.1  0     50.4  57.3 .20 .19 18 .52  0  
loop-lit/gj2007.yml .833  900 920 150 12000 1.1  0     83.3  92.9 .13 .13 16 .19  0  
loop-lit/gj2007b.yml .0    900 900 2200 13000 .74 0     42    42    7000 .21  0  
loop-lit/gr2006.yml .833  900 920 150 12000 1.0  0     83.3  92.9 .13 .13 16 .19  0  
loop-lit/gsv2008.c.i.p+cfa-reducer.yml .875  900 920 150 12000 1.1  0     87.5  90.0 .14 .14 16 .24  0  
loop-lit/gsv2008.c.i.v+cfa-reducer.yml .875  900 920 150 10000 1.0  0     87.5  90.0 .15 .15 16 .24  0  
loop-lit/gsv2008.c.i.v+lhb-reducer.yml .8    900 920 150 12000 1.1  0     80.0  89.3 .14 .14 16 .24  0  
loop-lit/gsv2008.yml .75   900 920 150 12000 1.2  0     75.0  92.3 .15 .15 16 .24  0  
loop-lit/hhk2008.yml .8    900 920 150 11000 1.2  0     80.0  93.8 .15 .14 16 .29  0  
loop-lit/jm2006.c.i.v+cfa-reducer.yml .9    900 900 150 12000 .82 0     90.0  92.0 .62 .62 16 .23  0  
loop-lit/jm2006.yml .8    900 900 160 10000 .73 0     80.0  93.8 .15 .15 16 .29  0  
loop-lit/jm2006_variant.yml .833  900 920 150 13000 1.2  0     83.3  94.7 .17 .16 16 .31  0  
loop-new/count_by_1.yml .75   900 900 150 10000 .66 0     75.0  87.5 .16 .16 16 .19  0  
loop-new/count_by_1_variant.yml .75   900 900 150 11000 .65 0     75.0  87.5 .18 .22 16 .19  0  
loop-new/count_by_2.yml .75   900 900 150 9600 .66 0     75.0  87.5 .14 .14 16 .19  0  
loop-new/count_by_k.yml .75   900 920 150 11000 1.0  0     75.0  90.0 .12 .12 16 .13  0  
loop-new/count_by_nondet.yml .875  900 920 150 10000 .93 0     87.5  92.9 .14 .14 16 .23  0  
loop-new/gauss_sum.i.p+cfa-reducer.yml .875  900 920 150 8900 1.0  0     87.5  90.0 .15 .15 16 .23  0  
loop-new/gauss_sum.i.p+lhb-reducer.yml .667  880 900 150 11000 .97 0     66.7  82.5 .17 .16 16 .23  0  
loop-new/gauss_sum.i.v+cfa-reducer.yml .875  900 920 150 13000 1.2  0     87.5  90.0 .15 .14 16 .23  0  
loop-new/gauss_sum.yml .875  900 920 150 11000 1.1  0     87.5  91.7 .15 .15 16 .24  0  
loop-new/half.yml .714  900 920 150 12000 1.1  0     71.4  92.9 .16 .15 16 .27  0  
loop-new/nested-1.yml .786  900 920 150 11000 1.3  0     78.6  93.3 .20 .22 16 .47  0  
loop-industry-pattern/mod3.c.v+cfa-reducer.yml .667  900 900 180 13000 .63 0     66.7  67.7 1.7  1.7  280 .094 0  
loop-industry-pattern/mod3.c.v+lhb-reducer.yml .0    900 900 2200 14000 .58 0     42    42    7000 .082 0  
loop-industry-pattern/mod3.c.v+sep-reducer.yml .0    900 900 2200 13000 .57 0     41    41    7000 0     0  
loop-industry-pattern/mod3.yml .0    900 900 2200 13000 .48 0     41    41    7000 0     0  
sv-benchmarks/c/ status score program test-suite TS size inspect test-suite cpu (s) wall (s) mem (MB) energy (J) blkio-w (MB) blkio-r (MB) status program test-suite TS size inspect test-suite cov branch cov (%) line cov (%) cpu (s) wall (s) mem (MB) energy blkio-w (MB) blkio-r (MB)
total 117 84.5 100000 110000 120000 1300000 54000 .58 117 8450 9460 1100 1100 100000 93 0  
Run set verifuzz.test-comp19_prop-coverage-branches.ReachSafety-Loops tbf-testsuite-validator-verifuzz.test-comp19_prop-coverage-branches.ReachSafety-Loops