Benchmark CPA/CoVeriTest CPA/Tiger-MGP ESBMC-bkind ESBMC-falsif FairFuzz KLEE PRtest Symbiotic VeriFuzz
Tool CPAchecker 1.8-svn 30375 CPAchecker 1.8-svn 30541M ESBMC version 6.0.0 64-bit x86_64 linux FairFuzz TC-0.0.2 KLEE 2.0.0-pre-test-comp tbf v0.3.0-testcomp19 symbiotic 6.0.3-dev-fd3f777b VeriFuzz 1.0.1
Limits timelimit: 900 s, memlimit: 15000 MB, CPU core limit: 8
Host apollon*
OS Linux 4.15.0-45-generic
System CPU: Intel Xeon E3-1230 v5 @ 3.40 GHz, cores: 8, frequency: 3.8 GHz, Turbo Boost: disabled; RAM: 33546 MB
Date of execution 2019-02-05 21:05:56 CET 2019-02-05 21:13:54 CET 2019-02-05 21:13:25 CET 2019-02-05 21:11:17 CET 2019-02-08 09:39:09 CET 2019-02-05 21:09:47 CET 2019-02-06 06:02:44 CET 2019-02-06 07:07:32 CET 2019-02-06 07:17:17 CET
Run set coveritest.test-comp19_prop-coverage-branches.ReachSafety-Loops cpa-tiger.test-comp19_prop-coverage-branches.ReachSafety-Loops esbmc-kind.test-comp19_prop-coverage-branches.ReachSafety-Loops esbmc-falsi.test-comp19_prop-coverage-branches.ReachSafety-Loops fairfuzz.test-comp19_prop-coverage-branches.ReachSafety-Loops klee.test-comp19_prop-coverage-branches.ReachSafety-Loops prtest.test-comp19_prop-coverage-branches.ReachSafety-Loops symbiotic.test-comp19_prop-coverage-branches.ReachSafety-Loops verifuzz.test-comp19_prop-coverage-branches.ReachSafety-Loops
Options -benchmark -heap 10000M -testcomp19 -benchmark -heap 10000M -tigertestcomp19 -s kinduction -s falsi --stats -i random --write-xml --svcomp-nondets --test-comp --verifier klee-testcomp --testcomp
../sv-benchmarks/c/ status score cpu (s) mem (MB) energy (J) status score cpu (s) mem (MB) energy (J) status score cpu (s) mem (MB) energy (J) status score cpu (s) mem (MB) energy (J) status score cpu (s) mem (MB) energy (J) status score cpu (s) mem (MB) energy (J) status score cpu (s) mem (MB) energy (J) status score cpu (s) mem (MB) energy (J) status score cpu (s) mem (MB) energy (J)
loops/array-1.yml done .833  23   1600 240 true .833  3.2 270 30 unknown .0  .045 5.7 .081 unknown .0  .030 5.8 .20  done .667 19    16 91   done .833  .13 21 1.1 timeout .833  900    31 14000   done .667  .32 19 4.2 timeout .833  900 150 12000
loops/array-2.yml done 1.0    23   1300 290 unknown 1.0    3.2 280 32 unknown .0  .030 5.7 .16  unknown .0  .024 5.6 .074 false(unreach-call) 1.0   17    17 51   false(unreach-call) 1.0    .14 21 1.3 done .667  .27 30 3.1 done 1.0    .34 19 4.3 error 1.0    890 150 13000
loops/bubble_sort-1.yml timeout .0417 900   8600 9700 unknown .0417 240   1200 3100 unknown .0  .027 5.7 .22  unknown .0  .031 5.7 .11  done .0   14    20 55   done .0417 .12 21 1.2 done .0    .25 31 3.2 done .0417 .43 16 5.6 error .0417 880 150 11000
loops/compact.yml timeout .0    900   2800 5200 timeout .0    900   2600 7600 unknown .0  .031 5.7 .13  unknown .0  .022 5.7 .16  timeout 1.0   900    54 12000   timeout .833  900    2800 11000   timeout .167  900    35 12000   done .833  840    360 11000   timeout 1.0    900 150 12000
loops/count_up_down-1.yml timeout .75   900   4900 11000 timeout .0    900   5900 12000 unknown .0  .031 5.7 .16  unknown .0  .027 5.7 .13  done .5   18    16 110   timeout .75   900    2600 13000   timeout .75   900    31 12000   done .5    .31 15 3.6 timeout .75   900 150 12000
loops/count_up_down-2.yml timeout .75   900   5000 10000 unknown .75   240   2000 3100 unknown .0  .025 5.7 .11  unknown .0  .026 5.6 .18  couldn't run: all seeds time out or crash .0   .15 11 1.9 timeout (false(unreach-call)) .75   900    2600 13000   done .0    7.7  31 110   done .5    .30 16 3.4 timeout .75   900 150 9700
loops/eureka_01-1.yml timeout .962  900   500 11000 unknown .962  8.1 300 97 unknown .0  .024 5.6 .12  unknown .0  .026 5.7 .16  false(unreach-call) .923 75    18 240   false(unreach-call) 1.0    .17 21 1.6 timeout .192  900    31 12000   done .885  .83 20 11   error .923  880 150 11000
loops/for_bounded_loop1.yml done 1.0    23   960 200 unknown 1.0    3.3 270 28 unknown .0  .019 5.7 .072 unknown .0  .031 5.7 .15  false(unreach-call) 1.0   43    17 170   timeout (false(unreach-call)) 1.0    900    1800 12000   done .625  140    31 1900   done 1.0    840    810 11000   error 1.0    880 150 13000
loops/insertion_sort-1.yml timeout 1.0    900   4600 11000 unknown 1.0    470   3400 6700 unknown .0  .025 5.7 .081 unknown .0  .050 5.6 .083 false(unreach-call) 1.0   34    17 390   false(valid-deref) 1.0    .12 21 1.6 done .0    .26 31 3.0 done .583  840    1400 9800   timeout 1.0    900 150 12000
loops/insertion_sort-2.yml timeout .9    900   5400 11000 unknown .7    330   4400 4200 unknown .0  .024 5.7 .11  unknown .0  .025 5.6 .078 false(unreach-call) .9   440    18 5800   false(valid-deref) 1.0    .14 21 1.1 done .0    .25 31 3.0 done .7    840    1400 13000   timeout .9    900 160 12000
loops/invert_string-1.yml timeout .875  900   5000 11000 unknown .875  3.5 270 32 unknown .0  .025 5.7 .12  unknown .0  .036 5.6 .12  couldn't run: all seeds time out or crash .0   2.1  10 29   false(unreach-call) 1.0    .97 27 12   done .0    .26 31 3.0 done 1.0    840    310 12000   timeout 1.0    900 150 12000
loops/invert_string-3.yml timeout .125  900   1900 11000 unknown .0    5.2 300 58 unknown .0  .026 5.7 .23  unknown .0  .024 5.7 .12  done .875 31    16 79   done .875  .12 21 1.4 timeout .875  900    31 12000   done .875  .34 16 4.0 timeout .875  900 150 11000
loops/linear_sea.ch.yml timeout .75   900   6000 11000 unknown .75   3.5 280 34 unknown .0  .023 5.7 .13  unknown .0  .029 5.7 .13  done .667 40    18 110   done .75   .15 21 1.5 done .0    .22 31 2.2 done .667  840    45 11000   timeout .667  900 150 12000
loops/linear_search.yml timeout .714  900   5300 11000 unknown 1.0    6.4 310 63 unknown .0  .026 5.7 .25  unknown .0  .044 5.7 .10  false(unreach-call) .929 21    18 69   done .714  .13 21 1.6 done .0    .20 31 2.6 done .929  4.3  21 56   error .929  880 150 11000
loops/matrix-1.yml done .875  23   1800 260 true .875  3.6 280 36 unknown .0  .030 5.6 .18  unknown .0  .024 5.7 .071 done .75  20    16 87   done .875  .14 21 1.1 timeout .875  900    30 13000   done .75   .34 18 4.4 timeout .875  900 150 13000
loops/matrix-2.yml done 1.0    23   1200 230 unknown 1.0    3.7 280 33 unknown .0  .024 5.7 .083 unknown .0  .022 5.7 .084 timeout 1.0   900    210 12000   timeout (false(valid-deref)) 1.0    900    1000 10000   done .0    .27 31 2.6 timeout (verification) .0    900    47 12000   timeout 1.0    900 160 11000
loops/n.c40.yml done .5    24   1700 290 true .5    5.8 270 63 unknown .0  .025 5.7 .11  unknown .0  .024 5.7 .15  done .5   9.2  15 38   done .5    .12 21 1.2 timeout .5    900    31 13000   done .375  .30 18 3.7 timeout .375  900 150 10000
loops/nec20.yml done .833  8.2 430 73 true .833  3.5 270 35 unknown .0  .023 5.7 .13  unknown .0  .026 5.7 .10  false(unreach-call) .833 10    15 45   false(unreach-call) .833  .12 21 1.5 done .167  .52 30 6.6 done .833  .37 19 4.9 error .833  890 150 11000
loops/nec40.yml done .4    24   1800 310 true .4    6.2 290 72 unknown .0  .036 5.6 .14  unknown .0  .036 5.5 .14  done .4   9.5  15 36   done .4    .13 21 1.4 timeout .4    900    31 12000   done .3    .32 18 4.3 timeout .3    900 150 11000
loops/string-1.yml done .808  11   480 99 out of memory .385  700   15000 8000 unknown .0  .053 5.6 .14  unknown .0  .022 5.6 .13  done .885 92    19 250   done .885  .18 21 2.0 timeout .808  900    31 12000   done .846  3.9  29 52   timeout .885  900 150 11000
loops/string-2.yml done .846  12   490 120 out of memory .731  560   15000 6700 unknown .0  .021 5.7 .12  unknown .0  .025 5.6 .12  false(unreach-call) .962 120    20 290   false(unreach-call) .962  .19 21 1.9 timeout .808  900    31 14000   done .923  8.6  33 120   error .962  880 150 13000
loops/sum01-1.yml done 1.0    25   870 220 unknown 1.0    5.6 290 47 unknown .0  .047 5.6 .094 unknown .0  .024 5.7 .14  false(unreach-call) .9   21    19 62   timeout (false(unreach-call)) 1.0    900    2700 12000   done .4    11    31 160   done .9    840    410 13000   timeout .9    900 150 9400
loops/sum01-2.yml timeout .833  900   6900 10000 unknown .75   470   1500 5300 unknown .0  .024 5.7 .12  unknown .0  .025 5.6 .18  done .417 13    16 100   done .833  57    140 850   timeout .833  900    31 14000   done .5    .32 19 4.0 timeout .583  900 150 10000
loops/sum01_bug02.yml done 1.0    22   840 180 unknown 1.0    4.3 290 37 unknown .0  .029 5.9 .16  unknown .0  .026 5.7 .11  false(unreach-call) .9   540    11 6700   timeout (false(unreach-call)) 1.0    900    2700 12000   done .4    9.6  31 130   done .9    840    410 11000   timeout .9    900 150 12000
loops/sum01_bug02_sum01_bug02_base.case.yml done 1.0    10   460 90 unknown 1.0    3.3 280 27 unknown .0  .022 5.8 .11  unknown .0  .023 5.7 .16  false(unreach-call) 1.0   780    10 9100   timeout (false(unreach-call)) 1.0    900    2700 12000   done .4    11    31 140   done .9    840    440 9300   timeout .9    900 150 13000
loops/sum03-1.yml done .875  7.0 400 61 unknown .875  5.4 290 63 unknown .0  .025 5.7 .12  unknown .0  .024 5.7 .090 couldn't run: all seeds time out or crash .0   .15 10 1.6 false(unreach-call) .875  .14 21 1.1 done .0    .26 31 2.7 done .875  .32 16 4.0 error .875  890 150 11000
loops/sum_array-1.yml done 1.0    23   710 260 unknown 1.0    3.6 280 34 unknown .0  .028 5.7 .18  unknown .0  .028 5.7 .19  false(unreach-call) 1.0   700    150 8500   false(valid-deref) 1.0    .15 21 1.6 done .0    .28 31 2.8 done 1.0    840    650 11000   error 1.0    890 250 12000
loops/sum_array-2.yml timeout .929  900   770 8900 unknown .929  4.0 280 37 unknown .0  .026 5.7 .16  unknown .0  .022 5.6 .12  timeout .929 900    150 12000   false(valid-deref) .929  .19 21 1.5 done .0    .27 31 3.3 done .857  850    2000 10000   timeout .929  900 280 11000
loops/terminator_01.yml done .75   3.6 270 33 true .75   3.1 270 27 unknown .0  .023 5.7 .32  unknown .0  .038 5.6 .12  couldn't run: all seeds time out or crash .0   .13 11 1.5 timeout (false(unreach-call)) .5    900    610 10000   done .0    3.8  30 48   done .75   .28 16 3.3 timeout .75   900 150 9900
loops/terminator_02-1.yml done .875  4.1 270 42 true .875  3.2 260 32 unknown .0  .027 5.7 .16  unknown .0  .048 5.8 .16  couldn't run: all seeds time out or crash .0   .14 10 1.6 timeout (false(unreach-call)) .75   900    1100 9800   done .5    .32 31 3.6 out of memory (killed (signal 9, verification)) .0    740    15000 7900   timeout .0    900 14000 11000
loops/terminator_02-2.yml done .9    24   700 210 true .9    3.3 270 32 unknown .0  .028 5.7 .17  unknown .0  .024 5.7 .11  done .8   190    20 610   timeout .9    900    1300 11000   timeout .35   900    31 13000   out of memory (killed (signal 9, verification)) .0    790    15000 9800   timeout .9    900 150 10000
loops/terminator_03-1.yml done .75   23   700 170 unknown .75   3.3 270 29 unknown .0  .022 5.6 .17  unknown .0  .027 5.7 .25  false(unreach-call) .75  10    16 110   timeout (false(unreach-call)) .75   900    100 11000   done .25   .25 30 3.1 done .75   850    91 13000   timeout .75   900 150 12000
loops/terminator_03-2.yml done .786  24   680 190 true .786  3.3 270 27 unknown .0  .022 5.7 .077 unknown .0  .033 5.7 .12  done .714 690    17 10000   timeout .786  900    84 14000   timeout .786  900    31 11000   done .786  850    89 8900   timeout .786  900 150 10000
loops/trex01-1.yml done 1.0    5.5 300 50 unknown 1.0    3.3 270 31 unknown .0  .025 5.7 .12  unknown .0  .027 5.6 .13  couldn't run: all seeds time out or crash .0   1.1  14 18   timeout (false(unreach-call)) 1.0    900    620 12000   done .0    .26 31 2.9 done 1.0    840    360 7300   timeout .0    900 2200 14000
loops/trex01-2.yml done .962  35   1100 280 unknown .962  15   550 140 unknown .0  .025 5.7 .051 unknown .0  .021 5.6 .13  done .846 350    56 4000   timeout .962  900    330 11000   timeout .654  900    31 12000   done .808  840    370 7100   timeout .885  900 150 11000
loops/trex02-1.yml done .833  23   1400 260 unknown .833  3.0 270 27 unknown .0  .048 5.7 .077 unknown .0  .029 5.7 .079 timeout .833 900    40 13000   timeout .833  900    1000 10000   timeout .833  900    34 13000   done .667  840    460 8200   timeout .0    900 9100 13000
loops/trex02-2.yml done 1.0    4.0 260 35 unknown 1.0    3.2 270 31 unknown .0  .019 5.7 .047 unknown .0  .032 5.7 .15  timeout .833 900    42 12000   timeout (false(unreach-call)) 1.0    900    930 11000   done .0    .26 31 3.2 done .833  840    400 9000   timeout .0    900 12000 11000
loops/trex03-1.yml done 1.0    23   1600 240 unknown 1.0    3.4 270 30 unknown .0  .024 5.7 .16  unknown .0  .024 5.7 .087 timeout .944 900    70 11000   timeout (false(unreach-call)) .889  900    1500 11000   timeout .333  900    35 12000   out of memory (killed (signal 9, verification)) .0    600    15000 6200   timeout .0    900 2200 14000
loops/trex03-2.yml done .889  22   1600 260 true .889  3.3 270 30 unknown .0  .024 5.7 .085 unknown .0  .035 5.5 .091 timeout .778 900    83 12000   timeout .889  900    1500 9300   timeout .333  900    35 11000   out of memory (killed (signal 9, verification)) .0    530    15000 6500   timeout .0    900 4300 11000
loops/veris.c_sendmail_tTflag_arr_one_loop.yml done .875  29   1100 240 unknown .875  5.5 290 72 unknown .0  .025 5.7 .11  unknown .0  .028 5.6 .21  done .5   33    16 87   done .75   .14 21 1.7 timeout .875  900    31 13000   done .75   .40 18 5.3 timeout .875  900 150 11000
loops/vogal-1.yml timeout .95   900   5300 11000 unknown .95   240   650 3100 unknown .0  .024 5.6 .075 unknown .0  .047 5.7 .078 done .95  39    17 74   done .95   .14 21 1.7 timeout .95   900    31 15000   done .85   .43 19 5.5 timeout .95   900 150 11000
loops/vogal-2.yml done 1.0    31   840 270 unknown 1.0    41   500 470 unknown .0  .025 5.7 .11  unknown .0  .048 5.7 .096 false(unreach-call) 1.0   150    19 540   timeout (false(unreach-call)) 1.0    900    3200 9100   done .944  .24 31 2.9 done .944  .67 20 8.7 error 1.0    880 150 11000
loop-acceleration/array_3-1.yml timeout .125  900   5900 11000 timeout .0    900   5100 11000 unknown .0  .025 5.7 .059 unknown .0  .036 5.7 .15  done .5   92    11 610   done .875  3.2  23 48   timeout .875  900    31 12000   done .75   15    22 240   timeout .875  900 150 11000
loop-acceleration/array_3-2.yml timeout .125  900   5600 11000 unknown .125  710   1600 8400 unknown .0  .025 5.9 .062 unknown .0  .037 5.7 .13  done .75  320    11 2000   false(unreach-call) 1.0    3.4  23 45   done .0    .26 31 2.9 done .875  15    23 230   error 1.0    880 150 12000
loop-acceleration/array_4.yml timeout .167  900   5900 10000 timeout .0    900   5100 11000 unknown .0  .024 5.7 .075 unknown .0  .040 5.7 .084 done .667 160    10 1200   done .833  3.2  23 47   timeout .833  900    30 13000   done .833  15    22 220   timeout .833  900 150 13000
loop-acceleration/diamond_1-1.yml done .833  740   5100 9300 true .833  170   1100 2100 unknown .0  .024 5.7 .12  unknown .0  .026 5.7 .085 done .667 29    17 81   done .833  .11 21 1.5 timeout .833  900    31 12000   done .667  .31 18 3.8 timeout .833  900 150 11000
loop-acceleration/diamond_1-2.yml done .833  540   4900 7300 unknown .833  170   1400 2300 unknown .0  .029 5.7 .091 unknown .0  .017 5.6 .079 couldn't run: all seeds time out or crash .0   .15 11 1.7 false(unreach-call) .833  .18 20 1.1 done .0    .24 31 3.6 done .667  .32 18 4.1 error .833  890 150 12000
loop-acceleration/diamond_2-2.yml done .958  26   940 200 true .958  4.7 290 47 unknown .0  .024 5.7 .10  unknown .0  .022 5.7 .078 done .542 29    17 94   done .958  .14 21 1.2 timeout .958  900    30 13000   done .958  .39 18 4.3 timeout .958  900 150 13000
loop-acceleration/multivar_1-1.yml done .75   23   1400 280 unknown .75   3.1 260 27 unknown .0  .028 5.7 .18  unknown .0  .024 5.7 .15  done .75  20    16 80   done .75   25    87 330   timeout .75   900    30 13000   done .75   .29 16 3.4 timeout .75   900 150 11000
loop-acceleration/multivar_1-2.yml done .75   22   1400 240 unknown .75   3.1 260 27 unknown .0  .027 5.7 .13  unknown .0  .025 5.7 .090 couldn't run: all seeds time out or crash .0   .13 10 1.4 false(unreach-call) .75   29    150 390   done .0    .26 30 3.1 done .75   .29 16 4.3 error .75   890 150 12000
loop-acceleration/simple_3-2.yml timeout .75   900   1900 11000 unknown .75   240   2100 2900 unknown .0  .022 5.7 .093 unknown .0  .025 5.7 .082 done .75  15    15 76   timeout .75   900    2400 12000   timeout .75   900    31 12000   done .5    .32 16 3.9 timeout .75   900 150 10000
loop-crafted/simple_array_index_value_2.yml timeout .167  910   7100 9400 timeout .0    900   5000 11000 unknown .0  .021 5.7 .10  unknown .0  .028 5.7 .14  couldn't run: all seeds time out or crash .0   3.1  10 46   done .833  310    61 3400   timeout .833  900    31 14000   done .833  420    64 5300   timeout .833  900 150 12000
loop-crafted/simple_array_index_value_4.i.v+lhb-reducer.yml done .588  25   1300 310 true .588  5.9 440 63 unknown .0  .019 5.7 .14  unknown .0  .022 5.8 .054 done .588 78    18 430   unknown .471  .18 95 1.8 timeout .176  900    31 12000   timeout (verification) .0    900    2700 3000   error .559  880 11000 11000
loop-crafted/simple_array_index_value_4.i.v+nlh-reducer.yml done .714  25   1100 310 true .714  5.7 440 66 unknown .0  .026 5.7 .24  unknown .0  .027 5.7 .098 done .714 80    17 210   unknown .571  .19 95 1.9 timeout .214  900    31 12000   timeout (verification) .0    900    2700 2900   error .679  890 11000 12000
loop-crafted/simple_array_index_value_4.yml done .75   25   1300 290 true .75   6.3 430 68 unknown .0  .026 5.6 .24  unknown .0  .029 5.8 .12  timeout .75  900    11 12000   unknown .75   .18 95 1.8 timeout .0    900    31 13000   timeout (verification) .0    900    2700 2800   error .75   880 10000 11000
loop-invgen/MADWiFi-encode_ie_ok.yml timeout .955  900   3000 11000 unknown .955  230   540 3400 unknown .0  .023 5.7 .12  unknown .0  .037 5.7 .12  done .682 37    17 170   timeout .955  900    3100 11000   timeout .5    900    31 13000   done .682  840    1200 10000   timeout .909  900 150 10000
loop-invgen/NetBSD_loop.yml timeout .875  900   5400 10000 unknown .875  240   2200 3400 unknown .0  .023 5.8 .17  unknown .0  .025 5.7 .096 done .125 29    17 75   timeout .875  900    1800 14000   timeout .125  900    31 13000   done .75   840    330 11000   timeout .75   900 150 12000
loop-invgen/SpamAssassin-loop.i.v+cfa-reducer.yml done .636  32   940 270 true .614  15   460 180 unknown .0  .023 5.7 .15  unknown .0  .022 5.7 .083 timeout .636 900    85 13000   timeout .636  900    1900 11000   timeout .636  900    34 11000   done .614  870    4300 11000   probably out of memory .0    420 15000 5800
loop-invgen/apache-escape-absolute.i.v+cfa-reducer.yml done .763  25   1700 310 true .763  6.0 300 63 unknown .0  .029 5.7 .083 unknown .0  .027 5.6 .12  timeout .684 900    81 12000   timeout .763  900    1200 11000   timeout .145  900    31 13000   done .75   860    4400 10000   error .75   880 180 13000
loop-invgen/apache-escape-absolute.yml done .976  25   1300 300 unknown .976  5.2 290 52 unknown .0  .022 5.6 .13  unknown .0  .025 5.7 .12  timeout .833 900    85 12000   timeout .976  900    790 11000   timeout .262  900    31 12000   done .762  860    3200 9900   timeout .786  900 150 12000
loop-invgen/apache-get-tag.i.p+lhb-reducer.yml timeout .667  900   5900 9700 unknown .635  700   1500 7900 unknown .0  .025 5.8 .067 unknown .0  .036 5.7 .11  timeout .513 900    68 12000   timeout .667  900    4500 11000   timeout .135  900    31 13000   done .667  890    13000 11000   error .667  880 210 11000
loop-invgen/apache-get-tag.i.p+nlh-reducer.yml timeout .744  900   5200 12000 timeout .561  900   2700 12000 unknown .0  .042 5.7 .069 unknown .0  .041 5.7 .10  timeout .618 900    81 12000   error (0) .0    .11 21 1.0 done .0    .22 31 2.3 timeout (verification) .0    900    15000 11000   error .744  880 1100 11000
loop-invgen/apache-get-tag.i.p+sep-reducer.yml timeout .667  900   5400 12000 timeout .506  900   2700 10000 unknown .0  .025 5.7 .10  unknown .0  .023 5.6 .12  timeout .430 900    100 12000   error (0) .0    .11 21 1.2 done .0    .24 31 2.3 out of memory (killed (signal 9, verification)) .0    670    15000 9200   error .667  890 1400 10000
loop-invgen/apache-get-tag.i.v+lhb-reducer.yml timeout .667  900   5900 10000 timeout .433  900   6600 14000 unknown .0  .024 5.7 .11  unknown .0  .024 5.6 .12  timeout .617 900    150 13000   timeout .667  900    3600 9100   timeout .35   900    31 13000   done .667  880    9900 9100   error .65   880 180 14000
loop-invgen/apache-get-tag.i.v+nlh-reducer.yml timeout .679  900   6000 11000 unknown .643  470   1500 5200 unknown .0  .025 5.7 .12  unknown .0  .024 5.7 .16  timeout .625 900    140 11000   timeout .679  900    3500 11000   timeout .375  900    31 13000   done .679  880    11000 11000   error .661  880 170 11000
loop-invgen/apache-get-tag.yml timeout .944  900   5600 11000 unknown .667  700   1500 10000 unknown .0  .028 5.7 .15  unknown .0  .026 5.7 .099 timeout .889 900    140 12000   timeout .944  900    2900 12000   timeout .667  900    30 12000   done .944  880    9300 10000   timeout .944  900 150 12000
loop-invgen/down.yml timeout .833  900   7000 10000 unknown .833  620   3400 7300 unknown .0  .023 5.7 .082 unknown .0  .052 5.7 .16  done .833 210    11 3200   timeout .833  900    3600 13000   timeout .833  900    30 12000   done .833  840    450 6500   timeout .833  900 150 11000
loop-invgen/fragtest_simple.yml timeout .786  900   1500 11000 unknown .786  700   4800 8000 unknown .0  .025 5.7 .14  unknown .0  .031 5.7 .14  done .786 170    18 460   timeout .786  900    160 9100   timeout .429  900    33 13000   done .786  860    2500 10000   error .786  880 150 14000
loop-invgen/half_2.yml timeout .875  900   5800 12000 unknown .875  230   700 3100 unknown .0  .029 5.7 .12  unknown .0  .046 5.5 .097 done .875 42    16 100   timeout .875  900    3000 12000   timeout .875  900    31 14000   done .875  840    880 11000   timeout .875  900 150 13000
loop-invgen/heapsort.yml done .95   660   4500 8000 timeout .05   900   830 12000 unknown .0  .027 5.7 .11  unknown .0  .054 5.6 .14  timeout .9   900    110 13000   timeout .95   900    2600 12000   timeout .75   900    33 13000   done .9    850    1800 13000   timeout .9    900 450 12000
loop-invgen/id_build.i.p+nlh-reducer.yml done .297  38   1300 410 unknown .297  250   3300 2900 unknown .0  .019 5.6 .13  unknown .0  .025 5.7 .092 timeout .288 900    11 13000   error (0) .0    .12 21 1.1 done .0    .21 31 2.6 done .297  840    1200 9300   timeout .297  900 400 11000
loop-invgen/id_build.i.p+sep-reducer.yml done .297  37   1400 420 true .297  14   390 120 unknown .0  .038 5.5 .090 unknown .0  .037 5.6 .089 timeout .288 900    11 12000   timeout .288  900    2400 13000   timeout .0021 900    33 14000   done .297  840    1200 11000   timeout .297  900 390 13000
loop-invgen/id_build.i.v+lhb-reducer.yml done .714  25   1600 300 unknown .714  250   2900 2600 unknown .0  .028 5.7 .20  unknown .0  .021 5.7 .17  timeout .714 900    10 12000   timeout .643  900    2400 11000   timeout .0714 900    31 11000   done .714  840    1200 9300   timeout .643  900 150 12000
loop-invgen/id_build.yml timeout .833  900   2100 11000 unknown .833  310   1100 3800 unknown .0  .048 5.7 .092 unknown .0  .022 5.7 .089 timeout .833 900    11 13000   timeout .167  900    2500 11000   timeout .167  900    31 13000   done .833  840    1300 9600   timeout .833  900 150 11000
loop-invgen/id_trans.yml done 1.0    23   1300 280 unknown 1.0    3.2 270 25 unknown .0  .022 5.6 .078 unknown .0  .024 5.7 .13  false(unreach-call) 1.0   68    19 230   timeout (false(unreach-call)) .875  900    2600 11000   timeout .125  900    31 13000   done .875  840    1300 9000   error 1.0    890 150 11000
loop-invgen/large_const.yml done .944  23   900 240 unknown .222  3.9 280 33 unknown .0  .024 5.6 .074 unknown .0  .025 5.7 .084 done .667 10    16 57   done .944  .14 21 1.1 timeout .556  900    31 12000   done .722  .39 19 4.5 timeout .833  900 150 12000
loop-invgen/nest-if3.yml timeout .929  900   4800 13000 timeout .5    900   370 13000 unknown .0  .018 5.6 .23  unknown .0  .021 5.7 .17  done .857 720    100 8600   timeout .929  900    1200 8400   timeout .5    900    31 12000   done .714  860    45 11000   timeout .929  900 150 12000
loop-invgen/nested6.yml timeout .929  900   8700 12000 unknown .929  240   4000 3000 unknown .0  .027 5.7 .093 unknown .0  .022 5.7 .14  done .857 150    36 1000   timeout .929  900    3900 11000   timeout .214  900    30 13000   done .857  900    9800 9900   timeout .929  900 150 9500
loop-invgen/nested9.yml timeout .955  900   5000 13000 unknown .773  460   1200 6400 unknown .0  .030 5.7 .16  unknown .0  .023 5.7 .12  timeout .636 900    15 10000   timeout .682  900    100 13000   timeout .5    900    31 14000   done .455  840    2500 10000   timeout .818  900 150 11000
loop-invgen/sendmail-close-angle.yml timeout .917  900   8100 11000 unknown .833  470   2500 6200 unknown .0  .023 5.7 .13  unknown .0  .034 5.6 .16  done .75  34    19 180   timeout .917  900    440 14000   timeout .917  900    31 13000   done .667  840    1100 10000   timeout .917  900 150 12000
loop-invgen/seq-3.yml timeout .938  900   13000 11000 unknown .812  700   6900 6900 unknown .0  .024 5.7 .11  unknown .0  .025 5.6 .086 done .812 21    15 74   timeout .938  900    2900 8900   timeout .938  900    31 11000   done .75   840    840 10000   timeout .812  900 160 11000
loop-invgen/up.yml timeout .833  900   13000 11000 unknown .833  630   11000 7200 unknown .0  .025 5.7 .077 unknown .0  .057 5.8 .085 done .333 28    17 78   timeout .833  900    380 13000   timeout .833  900    31 12000   done .833  840    1600 10000   timeout .833  900 160 9600
loop-lit/afnp2014.yml timeout .667  900   5300 13000 timeout .333  900   1400 10000 unknown .0  .028 5.6 .16  unknown .0  .021 5.7 .15  done .667 45    15 240   done .833  3.1  23 40   timeout .667  900    31 12000   done .833  14    22 180   timeout .833  900 150 11000
loop-lit/bhmr2007.yml timeout .9    900   4500 14000 timeout .7    900   460 11000 unknown .0  .023 5.7 .14  unknown .0  .042 5.6 .15  timeout .8   900    120 13000   timeout .9    900    1600 9400   timeout .8    900    31 12000   timeout (verification) .0    900    56 12000   timeout .8    900 150 11000
loop-lit/cggmp2005.yml done .75   4.1 260 38 true .75   3.7 280 36 unknown .0  .025 5.6 .14  unknown .0  .038 5.6 .087 done .0   15    22 60   done .75   .11 21 1.4 timeout .75   900    31 12000   done .75   .26 15 2.9 timeout .75   900 150 11000
loop-lit/cggmp2005_variant.yml timeout .875  900   5300 11000 unknown .75   460   390 6400 unknown .0  .037 5.5 .11  unknown .0  .024 5.7 .11  done .125 26    18 91   timeout .875  900    920 12000   timeout .875  900    31 13000   done .75   .32 19 4.1 timeout .75   900 150 13000
loop-lit/cggmp2005b.yml done .75   110   3600 1200 timeout .75   900   1400 13000 unknown .0  .040 5.5 .11  unknown .0  .025 5.7 .085 done .0   12    21 54   done .75   .11 21 1.2 timeout .75   900    31 12000   done .75   .27 16 3.6 timeout .75   900 150 11000
loop-lit/css2003.yml timeout .714  900   6800 9200 timeout .143  900   4500 13000 unknown .0  .023 5.7 .12  unknown .0  .022 5.7 .081 done .643 18    14 81   timeout .714  900    45 10000   timeout .714  900    31 14000   done .643  10    870 140   timeout .643  900 150 12000
loop-lit/ddlm2013.yml timeout .9    900   1000 10000 unknown .8    610   420 7900 unknown .0  .047 5.6 .13  unknown .0  .025 5.7 .074 done .9   22    15 56   timeout .8    900    74 8400   timeout .6    900    31 13000   done .8    840    52 10000   timeout .9    900 150 12000
loop-lit/gj2007.c.i.p+lhb-reducer.yml done .548  16   610 140 unknown .548  690   700 8100 unknown .0  .046 5.7 .11  unknown .0  .031 5.7 .18  done .0   18    22 46   done .548  .13 22 1.9 timeout .548  900    30 12000   done .548  .29 16 3.6 timeout .548  900 160 12000
loop-lit/gj2007.c.i.p+nlh-reducer.yml done .504  60   1100 700 timeout .504  900   390 11000 unknown .0  .041 5.7 .16  unknown .0  .023 5.7 .14  done .0   19    23 58   done .504  .14 21 1.4 timeout .504  900    31 12000   done .504  .63 16 8.0 timeout .504  900 260 11000
loop-lit/gj2007.yml done .833  14   560 110 unknown .833  530   700 6200 unknown .0  .027 5.7 .22  unknown .0  .036 5.7 .11  done .0   16    24 57   done .833  .13 21 1.5 timeout .833  900    31 12000   done .833  .25 15 3.6 timeout .833  900 150 12000
loop-lit/gj2007b.yml timeout .643  900   4700 11000 unknown .643  700   980 8100 unknown .0  .029 5.7 .084 unknown .0  .021 5.7 .099 timeout .643 900    100 12000   timeout .643  900    2000 9700   timeout .643  900    33 15000   done .5    840    130 10000   timeout .0    900 2200 13000
loop-lit/gr2006.yml done .833  22   1000 220 unknown .833  420   3700 5200 unknown .0  .024 5.7 .11  unknown .0  .020 5.7 .17  done .0   18    20 58   done .833  .14 20 1.2 timeout .833  900    30 12000   done .833  .29 16 3.2 timeout .833  900 150 12000
loop-lit/gsv2008.c.i.p+cfa-reducer.yml timeout .875  900   1800 11000 unknown .625  690   720 7800 unknown .0  .028 5.9 .20  unknown .0  .023 5.7 .090 done .75  62    20 160   timeout .875  900    86 11000   timeout .875  900    31 14000   done .875  850    53 11000   timeout .875  900 150 12000
loop-lit/gsv2008.c.i.v+cfa-reducer.yml timeout .875  900   1700 12000 unknown .625  690   490 7800 unknown .0  .024 5.7 .11  unknown .0  .029 5.7 .19  done .75  45    18 260   timeout .875  900    89 13000   timeout .875  900    31 13000   done .875  850    52 11000   timeout .875  900 150 10000
loop-lit/gsv2008.c.i.v+lhb-reducer.yml timeout .8    900   1600 11000 timeout .6    900   730 11000 unknown .0  .022 5.7 .052 unknown .0  .031 5.7 .15  done .7   37    17 110   timeout .8    900    90 10000   timeout .8    900    31 13000   done .8    860    54 12000   timeout .8    900 150 12000
loop-lit/gsv2008.yml timeout .875  900   1600 12000 unknown .75   460   670 5600 unknown .0  .045 5.7 .11  unknown .0  .025 5.7 .18  done .75  22    17 89   timeout .875  900    89 11000   timeout .875  900    31 12000   done .75   840    270 8600   timeout .75   900 150 12000
loop-lit/hhk2008.yml timeout .9    900   2800 11000 unknown .7    470   1000 5800 unknown .0  .023 5.7 .17  unknown .0  .031 5.6 .17  done .7   11    16 38   timeout .9    900    910 12000   timeout .9    900    31 12000   done .6    .34 19 5.5 timeout .8    900 150 11000
loop-lit/jm2006.c.i.v+cfa-reducer.yml done .9    23   1500 270 unknown .9    3.3 270 32 unknown .0  .025 5.7 .12  unknown .0  .027 5.7 .044 done .9   19    15 62   timeout .9    900    4500 13000   timeout .7    900    31 12000   done .9    .33 18 3.7 timeout .9    900 150 12000
loop-lit/jm2006.yml done .9    23   1200 250 unknown .9    3.2 270 29 unknown .0  .022 5.7 .14  unknown .0  .045 5.5 .091 done .9   17    16 60   timeout .9    900    4500 11000   timeout .7    900    31 12000   done .6    .33 18 4.6 timeout .8    900 160 10000
loop-lit/jm2006_variant.yml timeout .917  900   5000 11000 timeout .583  900   5400 11000 unknown .0  .024 5.6 .099 unknown .0  .024 5.7 .070 done .75  8.4  16 32   timeout .917  900    4500 14000   timeout .75   900    31 12000   done .583  .35 19 4.3 timeout .833  900 150 13000
loop-new/count_by_1.yml timeout .75   900   1800 12000 unknown .75   710   1700 8100 unknown .0  .030 5.7 .14  unknown .0  .034 5.6 .12  done .0   17    24 60   done .75   1.9  20 25   timeout .75   900    31 12000   done .75   .27 16 2.9 timeout .75   900 150 10000
loop-new/count_by_1_variant.yml timeout .75   900   2100 11000 timeout .0    900   5000 12000 unknown .0  .025 5.7 .13  unknown .0  .047 5.7 .058 done .0   13    24 50   done .75   5.1  21 75   timeout .75   900    31 12000   done .75   5.0  950 68   timeout .75   900 150 11000
loop-new/count_by_2.yml timeout .75   900   1700 11000 timeout .0    900   4900 11000 unknown .0  .025 5.7 .11  unknown .0  .030 5.7 .10  done .0   16    23 47   done .75   1.1  21 17   timeout .75   900    31 11000   done .75   .24 16 2.9 timeout .75   900 150 9600
loop-new/count_by_k.yml timeout .875  900   4700 11000 unknown .625  460   740 5200 unknown .0  .026 5.7 .11  unknown .0  .047 5.6 .062 done .75  15    15 67   timeout .75   900    22 12000   timeout .875  900    31 16000   done .75   850    20 12000   timeout .75   900 150 11000
loop-new/count_by_nondet.yml timeout .875  900   1700 13000 unknown .875  240   1900 2700 unknown .0  .021 5.6 .084 unknown .0  .030 5.7 .13  done .75  20    18 83   timeout .875  900    140 13000   timeout .875  900    31 15000   timeout (verification) .0    900    97 9500   timeout .875  900 150 10000
loop-new/gauss_sum.i.p+cfa-reducer.yml timeout .875  900   6400 9100 unknown .75   470   1400 4900 unknown .0  .022 5.7 .13  unknown .0  .026 5.6 .086 done .75  16    14 76   done .875  58    140 820   timeout .875  900    31 13000   done .875  .66 23 9.4 timeout .875  900 150 8900
loop-new/gauss_sum.i.p+lhb-reducer.yml timeout .889  900   6200 9900 unknown .722  690   1600 8600 unknown .0  .023 5.6 .089 unknown .0  .025 5.7 .14  done .611 45    17 130   done .889  57    140 740   timeout .722  900    31 13000   done .889  1.2  25 16   error .667  880 150 11000
loop-new/gauss_sum.i.v+cfa-reducer.yml timeout .875  900   5600 11000 unknown .75   470   1200 5400 unknown .0  .022 5.6 .055 unknown .0  .028 5.5 .12  done .75  15    16 87   done .875  57    140 720   timeout .875  900    31 12000   done .875  .66 23 9.1 timeout .875  900 150 13000
loop-new/gauss_sum.yml timeout .875  900   8300 11000 timeout .625  900   3200 8800 unknown .0  .049 5.7 .15  unknown .0  .027 5.7 .21  done .75  16    17 82   done .875  57    140 790   timeout .875  900    31 14000   done .75   .79 23 9.6 timeout .875  900 150 11000
loop-new/half.yml timeout .857  900   5800 9700 timeout .429  900   1700 11000 unknown .0  .029 5.6 .18  unknown .0  .028 5.8 .19  done .786 27    18 78   timeout .857  900    2900 15000   timeout .857  900    30 9900   done .714  840    1700 13000   timeout .714  900 150 12000
loop-new/nested-1.yml timeout .929  900   6300 10000 unknown .929  250   680 2900 unknown .0  .024 5.7 .14  unknown .0  .025 5.7 .11  done .786 97    19 300   timeout .929  900    1900 12000   timeout .5    900    31 14000   done .714  .52 21 6.3 timeout .786  900 150 11000
loop-industry-pattern/mod3.c.v+cfa-reducer.yml done .75   34   1500 400 unknown .75   14   290 180 unknown .0  .035 5.7 .11  unknown .0  .029 5.7 .12  done .917 18    16 99   timeout .917  900    930 8000   timeout .583  900    33 13000   done .75   840    52 12000   timeout .667  900 180 13000
loop-industry-pattern/mod3.c.v+lhb-reducer.yml done .75   52   450 700 true .7    29   300 390 unknown .0  .027 5.6 .091 unknown .0  .024 5.7 .11  done .8   450    20 2500   timeout .75   900    1400 11000   timeout .55   900    34 12000   done .8    890    7800 8200   timeout .0    900 2200 14000
loop-industry-pattern/mod3.c.v+sep-reducer.yml done .75   51   440 720 true .65   45   300 600 unknown .0  .038 5.7 .16  unknown .0  .022 5.7 .29  done .8   140    18 440   timeout .8    900    1400 9700   timeout .6    900    32 14000   done .7    880    7900 8100   timeout .0    900 2200 13000
loop-industry-pattern/mod3.yml done .75   34   1400 410 unknown .75   15   300 180 unknown .0  .024 5.7 .064 unknown .0  .023 5.7 .12  done .917 50    16 140   timeout .917  900    590 10000   timeout .583  900    33 14000   done .833  860    51 12000   timeout .0    900 2200 13000
../sv-benchmarks/c/ status score cpu (s) mem (MB) energy (J) status score cpu (s) mem (MB) energy (J) status score cpu (s) mem (MB) energy (J) status score cpu (s) mem (MB) energy (J) status score cpu (s) mem (MB) energy (J) status score cpu (s) mem (MB) energy (J) status score cpu (s) mem (MB) energy (J) status score cpu (s) mem (MB) energy (J) status score cpu (s) mem (MB) energy (J)
total 117 91.5 57000 360000 690000 117 80.6 40000 200000 490000 117 .0  3.2 670 15 117 .0  3.4 660 14 117 73.0 31000 3800 380000 117 93.1 59000 120000 740000 117 59.6 79000 3600 1100000 117 78.4 55000 210000 650000 117 84.5 100000 120000 1300000
Run set coveritest.test-comp19_prop-coverage-branches.ReachSafety-Loops cpa-tiger.test-comp19_prop-coverage-branches.ReachSafety-Loops esbmc-kind.test-comp19_prop-coverage-branches.ReachSafety-Loops esbmc-falsi.test-comp19_prop-coverage-branches.ReachSafety-Loops fairfuzz.test-comp19_prop-coverage-branches.ReachSafety-Loops klee.test-comp19_prop-coverage-branches.ReachSafety-Loops prtest.test-comp19_prop-coverage-branches.ReachSafety-Loops symbiotic.test-comp19_prop-coverage-branches.ReachSafety-Loops verifuzz.test-comp19_prop-coverage-branches.ReachSafety-Loops