Benchmark KLEE TBF Test-Suite Validator
Tool KLEE 2.0.0-pre-test-comp Tbf Test-suite Validator v1.2
Limits timelimit: 900 s, memlimit: 15000 MB, CPU core limit: 8 timelimit: 10800 s, memlimit: 7000 MB, CPU core limit: 2
Host apollon*
OS Linux 4.15.0-45-generic
System CPU: Intel Xeon E3-1230 v5 @ 3.40 GHz, cores: 8, frequency: 3.8 GHz, Turbo Boost: disabled; RAM: 33546 MB
Date of execution 2019-02-05 21:09:47 CET 2019-02-06 06:16:47 CET
Run set klee.test-comp19_prop-coverage-branches.ReachSafety-Loops tbf-testsuite-validator-klee.test-comp19_prop-coverage-branches.ReachSafety-Loops
Options --verbose --sequence-file cov-seq.txt -r --test-suite ../../results-verified/klee.2019-02-05_2109.logfiles/${rundefinition_name}.${inputfile_name}.files/test-suite.zip
sv-benchmarks/c/ status score program test-suite TS size inspect test-suite cpu (s) wall (s) mem (MB) energy (J) blkio-w (MB) blkio-r (MB) status program test-suite TS size inspect test-suite cov branch cov (%) line cov (%) cpu (s) wall (s) mem (MB) energy blkio-w (MB) blkio-r (MB)
loops/array-1.yml .833  .13 .15 21 1.1 .016 0   83.3  93.3 .12  .14  17   .11  0  
loops/array-2.yml 1.0    .14 .16 21 1.3 .016 0   100    100   .13  .13  16   .13  0  
loops/bubble_sort-1.yml .0417 .12 .14 21 1.2 .016 0   4.17 17.9 .12  .12  17   .086 0  
loops/compact.yml .833  900    900    2800 11000   .049 0   83.3  87.5 5.3   5.3   450   .086 0  
loops/count_up_down-1.yml .75   900    900    2600 13000   .070 0   75.0  90.9 .17  .17  17   .11  0  
loops/count_up_down-2.yml .75   900    900    2600 13000   .070 0   75.0  90.9 .16  .21  16   .13  0  
loops/eureka_01-1.yml 1.0    .17 .17 21 1.6 .016 0   100    100   .23  .22  16   .44  0  
loops/for_bounded_loop1.yml 1.0    900    900    1800 12000   .045 0   100    100   .14  .14  16   .18  0  
loops/insertion_sort-1.yml 1.0    .12 .14 21 1.6 .016 0   100    100   .14  .13  16   .18  0  
loops/insertion_sort-2.yml 1.0    .14 .16 21 1.1 .016 0   100    100   .13  .13  17   .18  0  
loops/invert_string-1.yml 1.0    .97 .99 27 12   .016 0   100    100   .16  .16  17   .19  0  
loops/invert_string-3.yml .875  .12 .13 21 1.4 .016 0   87.5  94.4 .12  .12  17   .078 0  
loops/linear_sea.ch.yml .75   .15 .17 21 1.5 .016 0   75.0  87.5 .15  .18  16   .18  0  
loops/linear_search.yml .714  .13 .13 21 1.6 .016 0   71.4  88.2 .14  .14  16   .19  0  
loops/matrix-1.yml .875  .14 .16 21 1.1 .016 0   87.5  94.1 .12  .12  16   .11  0  
loops/matrix-2.yml 1.0    900    900    1000 10000   .045 0   100    100   .16  .18  16   .20  0  
loops/n.c40.yml .5    .12 .14 21 1.2 .016 0   50.0  63.6 .12  .12  16   .11  0  
loops/nec20.yml .833  .12 .14 21 1.5 .016 0   83.3  95.8 .14  .14  16   .21  0  
loops/nec40.yml .4    .13 .15 21 1.4 .016 0   40.0  46.7 .12  .12  16   .11  0  
loops/string-1.yml .885  .18 .20 21 2.0 .016 0   88.5  96.5 .19  .29  16   .34  0  
loops/string-2.yml .962  .19 .21 21 1.9 .016 0   96.2  100   .19  .22  16   .36  0  
loops/sum01-1.yml 1.0    900    900    2700 12000   .070 0   100    100   40     40     17   .16  0  
loops/sum01-2.yml .833  57    57    140 850   .016 0   83.3  90.0 .14  .18  16   .19  0  
loops/sum01_bug02.yml 1.0    900    900    2700 12000   .070 0   100    100   40     40     16   .14  0  
loops/sum01_bug02_sum01_bug02_base.case.yml 1.0    900    900    2700 12000   .061 0   100    100   40     40     16   .16  0  
loops/sum03-1.yml .875  .14 .16 21 1.1 .016 0   87.5  100   .12  .12  17   .078 0  
loops/sum_array-1.yml 1.0    .15 .16 21 1.6 .016 0   100    100   .16  .17  16   .24  0  
loops/sum_array-2.yml .929  .19 .21 21 1.5 .016 0   92.9  94.4 .16  .17  16   .26  0  
loops/terminator_01.yml .5    900    900    610 10000   .045 0   50.0  72.7 .12  .12  16   .094 0  
loops/terminator_02-1.yml .75   900    900    1100 9800   .061 0   75.0  87.5 .14  .14  17   .18  0  
loops/terminator_02-2.yml .9    900    900    1300 11000   .066 0   90.0  94.7 .16  .15  16   .31  0  
loops/terminator_03-1.yml .75   900    900    100 11000   .041 0   75.0  100   .14  .14  17   .16  0  
loops/terminator_03-2.yml .786  900    900    84 14000   .033 0   78.6  92.3 .14  .16  17   .19  0  
loops/trex01-1.yml 1.0    900    900    620 12000   .074 0   100    100   .19  .18  16   .36  0  
loops/trex01-2.yml .962  900    900    330 11000   .070 0   96.2  96.5 .20  .19  17   .46  0  
loops/trex02-1.yml .833  900    900    1000 10000   .070 0   83.3  92.9 .13  .13  16   .18  0  
loops/trex02-2.yml 1.0    900    900    930 11000   .066 0   100    100   .15  .16  16   .21  0  
loops/trex03-1.yml .889  900    900    1500 11000   .070 0   88.9  100   .17  .16  16   .31  0  
loops/trex03-2.yml .889  900    900    1500 9300   .066 0   88.9  93.8 .17  .17  16   .31  0  
loops/veris.c_sendmail_tTflag_arr_one_loop.yml .75   .14 .16 21 1.7 .016 0   75.0  94.7 .12  .15  16   .070 0  
loops/vogal-1.yml .95   .14 .15 21 1.7 .016 0   95.0  96.2 .17  .16  16   .31  0  
loops/vogal-2.yml 1.0    900    900    3200 9100   .029 0   100    100   .17  .16  16   .27  0  
loop-acceleration/array_3-1.yml .875  3.2  3.2  23 48   .016 0   87.5  90.9 .20  .20  17   .11  0  
loop-acceleration/array_3-2.yml 1.0    3.4  3.4  23 45   .016 0   100    100   .26  .27  17   .14  0  
loop-acceleration/array_4.yml .833  3.2  3.3  23 47   .016 0   83.3  90.0 .20  .20  17   .11  0  
loop-acceleration/diamond_1-1.yml .833  .11 .11 21 1.5 .016 0   83.3  91.7 .13  .13  17   .13  0  
loop-acceleration/diamond_1-2.yml .833  .18 .20 20 1.1 .016 0   83.3  91.7 .14  .13  17   .14  0  
loop-acceleration/diamond_2-2.yml .958  .14 .16 21 1.2 .016 0   95.8  96.5 .18  .19  16   .26  0  
loop-acceleration/multivar_1-1.yml .75   25    25    87 330   .016 0   75.0  90.9 .12  .12  16   .11  0  
loop-acceleration/multivar_1-2.yml .75   29    29    150 390   .016 0   75.0  90.9 .12  .13  16   .12  0  
loop-acceleration/simple_3-2.yml .75   900    900    2400 12000   .066 0   75.0  90.0 .12  .12  16   .11  0  
loop-crafted/simple_array_index_value_2.yml .833  310    310    61 3400   .016 0   83.3  90.9 .63  .64  46   .094 0  
loop-crafted/simple_array_index_value_4.i.v+lhb-reducer.yml .471  .18 .19 95 1.8 .016 0   47.1  56.0 .13  .13  16   .16  0  
loop-crafted/simple_array_index_value_4.i.v+nlh-reducer.yml .571  .19 .21 95 1.9 .016 0   57.1  63.6 .14  .13  16   .18  0  
loop-crafted/simple_array_index_value_4.yml .75   .18 .20 95 1.8 .016 0   75.0  95.2 .13  .13  16   .13  0  
loop-invgen/MADWiFi-encode_ie_ok.yml .955  900    900    3100 11000   .061 0   95.5  96.3 .17  .16  16   .31  0  
loop-invgen/NetBSD_loop.yml .875  900    900    1800 14000   .061 0   87.5  93.8 .14  .14  17   .18  0  
loop-invgen/SpamAssassin-loop.i.v+cfa-reducer.yml .636  900    900    1900 11000   .061 0   63.6  73.3 60     60     16   .31  0  
loop-invgen/apache-escape-absolute.i.v+cfa-reducer.yml .763  900    900    1200 11000   .078 0   76.3  76.6 .23  .23  17   .84  0  
loop-invgen/apache-escape-absolute.yml .976  900    900    790 11000   .061 0   97.6  98.1 .23  .24  17   .72  0  
loop-invgen/apache-get-tag.i.p+lhb-reducer.yml .667  900    900    4500 11000   .066 0   66.7  81.9 .27  .27  17   1.8   0  
loop-invgen/apache-get-tag.i.p+nlh-reducer.yml .0    .11 .13 21 1.0 0     0   0    0   .061 .062 8.9 0     0  
loop-invgen/apache-get-tag.i.p+sep-reducer.yml .0    .11 .13 21 1.2 0     0   0    0   .060 .060 9.0 0     0  
loop-invgen/apache-get-tag.i.v+lhb-reducer.yml .667  900    900    3600 9100   .057 0   66.7  74.5 .20  .20  17   .55  0  
loop-invgen/apache-get-tag.i.v+nlh-reducer.yml .679  900    900    3500 11000   .049 0   67.9  75.0 .19  .20  16   .61  0  
loop-invgen/apache-get-tag.yml .944  900    900    2900 12000   .049 0   94.4  97.7 .18  .18  17   .33  0  
loop-invgen/down.yml .833  900    900    3600 13000   .070 0   83.3  94.1 18     18     16   .14  0  
loop-invgen/fragtest_simple.yml .786  900    900    160 9100   .016 0   78.6  96.2 .20  .20  17   .19  0  
loop-invgen/half_2.yml .875  900    900    3000 12000   .066 0   87.5  94.4 .16  .16  16   .19  0  
loop-invgen/heapsort.yml .95   900    900    2600 12000   .070 0   95.0  97.6 .17  .16  17   .31  0  
loop-invgen/id_build.i.p+nlh-reducer.yml .0    .12 .14 21 1.1 0     0   0    0   .058 .059 8.9 0     0  
loop-invgen/id_build.i.p+sep-reducer.yml .288  900    900    2400 13000   .13  0   28.8  39.1 140     140     22   1.6   0  
loop-invgen/id_build.i.v+lhb-reducer.yml .643  900    900    2400 11000   .066 0   64.3  72.2 60     60     16   .14  0  
loop-invgen/id_build.yml .167  900    900    2500 11000   .066 0   16.7  36.4 40     40     16   .11  0  
loop-invgen/id_trans.yml .875  900    900    2600 11000   .066 0   87.5  100   1.6   1.6   16   .21  0  
loop-invgen/large_const.yml .944  .14 .16 21 1.1 .016 0   94.4  96.3 .17  .16  16   .29  0  
loop-invgen/nest-if3.yml .929  900    900    1200 8400   .033 0   92.9  93.3 .16  .15  17   .26  0  
loop-invgen/nested6.yml .929  900    900    3900 11000   .070 0   92.9  94.7 .17  .19  16   .29  0  
loop-invgen/nested9.yml .682  900    900    100 13000   .016 0   68.2  61.1 60     60     16   .33  0  
loop-invgen/sendmail-close-angle.yml .917  900    900    440 14000   .016 0   91.7  96.7 .15  .14  16   .21  0  
loop-invgen/seq-3.yml .938  900    900    2900 8900   .070 0   93.8  95.7 .17  .16  16   .29  0  
loop-invgen/up.yml .833  900    900    380 13000   .016 0   83.3  93.8 18     18     16   .14  0  
loop-lit/afnp2014.yml .833  3.1  3.2  23 40   .016 0   83.3  91.7 .15  .15  16   .11  0  
loop-lit/bhmr2007.yml .9    900    900    1600 9400   .070 0   90.0  93.8 .17  .16  17   .21  0  
loop-lit/cggmp2005.yml .75   .11 .11 21 1.4 .016 0   75.0  91.7 .12  .15  16   .078 0  
loop-lit/cggmp2005_variant.yml .875  900    900    920 12000   .061 0   87.5  93.3 .13  .13  17   .18  0  
loop-lit/cggmp2005b.yml .75   .11 .13 21 1.2 .016 0   75.0  88.2 .11  .11  17   .078 0  
loop-lit/css2003.yml .714  900    900    45 10000   .016 0   71.4  93.3 .17  .17  17   .16  0  
loop-lit/ddlm2013.yml .8    900    900    74 8400   .037 0   80.0  90.9 .14  .14  17   .18  0  
loop-lit/gj2007.c.i.p+lhb-reducer.yml .548  .13 .14 22 1.9 .016 0   54.8  58.2 .13  .13  16   .082 0  
loop-lit/gj2007.c.i.p+nlh-reducer.yml .504  .14 .16 21 1.4 .033 0   50.4  57.3 .16  .16  18   .12  0  
loop-lit/gj2007.yml .833  .13 .15 21 1.5 .016 0   83.3  92.9 .11  .11  17   .078 0  
loop-lit/gj2007b.yml .643  900    900    2000 9700   .070 0   64.3  93.3 .14  .18  16   .18  0  
loop-lit/gr2006.yml .833  .14 .16 20 1.2 .016 0   83.3  92.9 .12  .12  16   .078 0  
loop-lit/gsv2008.c.i.p+cfa-reducer.yml .875  900    900    86 11000   .037 0   87.5  90.0 .15  .16  17   .14  0  
loop-lit/gsv2008.c.i.v+cfa-reducer.yml .875  900    900    89 13000   .033 0   87.5  90.0 .13  .13  17   .14  0  
loop-lit/gsv2008.c.i.v+lhb-reducer.yml .8    900    900    90 10000   .037 0   80.0  89.3 .13  .13  16   .14  0  
loop-lit/gsv2008.yml .875  900    900    89 11000   .037 0   87.5  92.3 .13  .13  17   .16  0  
loop-lit/hhk2008.yml .9    900    900    910 12000   .061 0   90.0  93.8 .14  .14  16   .21  0  
loop-lit/jm2006.c.i.v+cfa-reducer.yml .9    900    900    4500 13000   .070 0   90.0  92.0 .17  .19  16   .18  0  
loop-lit/jm2006.yml .9    900    900    4500 11000   .070 0   90.0  93.8 .19  .21  16   .19  0  
loop-lit/jm2006_variant.yml .917  900    900    4500 14000   .070 0   91.7  94.7 .15  .14  17   .23  0  
loop-new/count_by_1.yml .75   1.9  2.0  20 25   .016 0   75.0  87.5 .12  .12  16   .078 0  
loop-new/count_by_1_variant.yml .75   5.1  5.1  21 75   .016 0   75.0  87.5 .12  .12  16   .078 0  
loop-new/count_by_2.yml .75   1.1  1.1  21 17   .016 0   75.0  87.5 .12  .12  16   .078 0  
loop-new/count_by_k.yml .75   900    900    22 12000   .033 0   75.0  90.0 .13  .13  16   .16  0  
loop-new/count_by_nondet.yml .875  900    900    140 13000   .016 0   87.5  92.9 .14  .14  16   .16  0  
loop-new/gauss_sum.i.p+cfa-reducer.yml .875  58    58    140 820   .016 0   87.5  90.0 .13  .17  16   .16  0  
loop-new/gauss_sum.i.p+lhb-reducer.yml .889  57    57    140 740   .016 0   88.9  92.5 .17  .16  16   .23  0  
loop-new/gauss_sum.i.v+cfa-reducer.yml .875  57    57    140 720   .016 0   87.5  90.0 .13  .14  16   .16  0  
loop-new/gauss_sum.yml .875  57    57    140 790   .016 0   87.5  91.7 .14  .13  16   .18  0  
loop-new/half.yml .857  900    900    2900 15000   .061 0   85.7  92.9 .15  .14  16   .21  0  
loop-new/nested-1.yml .929  900    900    1900 12000   .061 0   92.9  93.3 .15  .15  16   .26  0  
loop-industry-pattern/mod3.c.v+cfa-reducer.yml .917  900    900    930 8000   .049 0   91.7  93.5 .14  .14  17   .18  0  
loop-industry-pattern/mod3.c.v+lhb-reducer.yml .75   900    900    1400 11000   .041 0   75.0  80.0 .18  .17  17   .31  0  
loop-industry-pattern/mod3.c.v+sep-reducer.yml .8    900    900    1400 9700   .049 0   80.0  84.7 .16  .17  17   .31  0  
loop-industry-pattern/mod3.yml .917  900    900    590 10000   .070 0   91.7  94.1 .14  .16  17   .18  0  
sv-benchmarks/c/ status score program test-suite TS size inspect test-suite cpu (s) wall (s) mem (MB) energy (J) blkio-w (MB) blkio-r (MB) status program test-suite TS size inspect test-suite cov branch cov (%) line cov (%) cpu (s) wall (s) mem (MB) energy blkio-w (MB) blkio-r (MB)
total 117 93.1 59000 59000 120000 740000 4.4  0   117 9310 10100 540 540 2400 26 0  
Run set klee.test-comp19_prop-coverage-branches.ReachSafety-Loops tbf-testsuite-validator-klee.test-comp19_prop-coverage-branches.ReachSafety-Loops