Benchmark CPA/CoVeriTest CPA/Tiger-MGP ESBMC-bkind ESBMC-falsif FairFuzz KLEE PRtest Symbiotic VeriFuzz
Tool CPAchecker 1.8-svn 30375 CPAchecker 1.8-svn 30541M ESBMC version 6.0.0 64-bit x86_64 linux FairFuzz TC-0.0.2 KLEE 2.0.0-pre-test-comp tbf v0.3.0-testcomp19 symbiotic 6.0.3-dev-fd3f777b VeriFuzz 1.0.1
Limits timelimit: 900 s, memlimit: 15000 MB, CPU core limit: 8
Host apollon*
OS Linux 4.15.0-45-generic
System CPU: Intel Xeon E3-1230 v5 @ 3.40 GHz, cores: 8, frequency: 3.8 GHz, Turbo Boost: disabled; RAM: 33546 MB
Date of execution 2019-02-05 21:05:56 CET 2019-02-05 21:13:54 CET 2019-02-05 21:13:25 CET 2019-02-05 21:11:17 CET 2019-02-08 09:39:09 CET 2019-02-05 21:09:47 CET 2019-02-06 06:02:44 CET 2019-02-06 07:07:32 CET 2019-02-06 07:17:17 CET
Run set coveritest.test-comp19_prop-coverage-branches.ReachSafety-Sequentialized cpa-tiger.test-comp19_prop-coverage-branches.ReachSafety-Sequentialized esbmc-kind.test-comp19_prop-coverage-branches.ReachSafety-Sequentialized esbmc-falsi.test-comp19_prop-coverage-branches.ReachSafety-Sequentialized fairfuzz.test-comp19_prop-coverage-branches.ReachSafety-Sequentialized klee.test-comp19_prop-coverage-branches.ReachSafety-Sequentialized prtest.test-comp19_prop-coverage-branches.ReachSafety-Sequentialized symbiotic.test-comp19_prop-coverage-branches.ReachSafety-Sequentialized verifuzz.test-comp19_prop-coverage-branches.ReachSafety-Sequentialized
Options -benchmark -heap 10000M -testcomp19 -benchmark -heap 10000M -tigertestcomp19 -s kinduction -s falsi --stats -i random --write-xml --svcomp-nondets --test-comp --verifier klee-testcomp --testcomp
../sv-benchmarks/c/seq-mthreaded/ status score cpu (s) mem (MB) energy (J) status score cpu (s) mem (MB) energy (J) status score cpu (s) mem (MB) energy (J) status score cpu (s) mem (MB) energy (J) status score cpu (s) mem (MB) energy (J) status score cpu (s) mem (MB) energy (J) status score cpu (s) mem (MB) energy (J) status score cpu (s) mem (MB) energy (J) status score cpu (s) mem (MB) energy (J)
pals_STARTPALS_ActiveStandby.1.ufo.BOUNDED-10.pals.yml .819 680 6400 6900 .681 900 540 11000 .0  .028 5.7 .17  .0  .039 5.7 .10  .0  3.2 11 41 .753 900    5200 9000   .00550 900 31 12000 .0   710    15000 7300 .819 880 170 10000
pals_STARTPALS_ActiveStandby.4_1.ufo.BOUNDED-10.pals.yml .820 270 4600 3300 .792 560 900 6500 .0  .048 5.7 .075 .0  .024 5.8 .089 .0  3.2 10 39 .758 900    2600 11000   .00560 900 31 12000 .0   900    15000 7900 .815 880 170 10000
pals_STARTPALS_ActiveStandby.4_2.ufo.BOUNDED-10.pals.yml .813 570 5200 7800 .813 140 950 1800 .0  .030 5.7 .12  .0  .027 5.7 .15  .0  3.2 10 39 .753 900    3400 7800   .00550 900 31 10000 .0   900    11000 8800 .813 880 170 9700
pals_STARTPALS_ActiveStandby.5.ufo.BOUNDED-10.pals.yml .822 900 6800 11000 .744 800 630 9400 .0  .031 5.6 .17  .0  .031 5.6 .060 .0  3.2 10 44 .767 900    4900 6800   .00560 900 31 13000 .0   780    15000 5400 .828 880 170 12000
pals_STARTPALS_ActiveStandby.ufo.BOUNDED-10.pals.yml .808 280 3900 3400 .802 420 580 5600 .0  .029 5.7 .15  .0  .025 5.7 .13  .0  3.2 11 42 .742 900    5100 8200   .00550 900 31 12000 .0   760    15000 6100 .808 880 170 13000
pals_STARTPALS_Triplicated.1.ufo.BOUNDED-10.pals.yml .838 490 5200 4900 .156 900 550 12000 .0  .024 5.7 .054 .0  .024 5.7 .13  .0  3.2 11 42 .838 900    3400 10000   .0375  900 31 12000 .0   760    15000 6900 .838 890 170 10000
pals_STARTPALS_Triplicated.2.ufo.BOUNDED-10.pals.yml .823 380 4800 4400 .760 900 600 12000 .0  .023 5.7 .11  .0  .025 5.7 .078 .0  3.2 11 50 .797 900    4700 9600   .038   900 30 14000 .0   660    15000 8400 .823 890 170 12000
pals_STARTPALS_Triplicated.ufo.BOUNDED-10.pals.yml .821 510 5600 5500 .747 780 650 9200 .0  .028 5.7 .10  .0  .023 5.7 .21  .0  3.2 10 46 .821 900    4900 10000   .0370  900 31 12000 .0   740    15000 6000 .821 890 170 13000
pals_floodmax.3.1.ufo.BOUNDED-6.pals.yml .9   900 1600 7000 .853 900 960 8500 .0  .030 5.7 .089 .0  .022 5.7 .075 .0  3.2 10 44 .847 1.8  39 18   .132   900 31 12000 .758 68    39 990 .816 880 170 11000
pals_floodmax.3.2.ufo.BOUNDED-6.pals.yml .817 36 940 320 .817 19 480 200 .0  .053 5.7 .15  .0  .029 5.7 .16  .0  3.2 10 44 .748 .73 34 8.9 .153   900 31 13000 .490 7.8  22 120 .738 880 170 9700
pals_floodmax.3.3.ufo.BOUNDED-6.pals.yml .861 900 1200 7200 .772 900 830 6300 .0  .047 5.7 .070 .0  .023 5.7 .13  .0  3.2 10 46 .792 .98 34 12   .153   900 31 13000 .520 36    27 500 .782 880 180 13000
pals_floodmax.3.4.ufo.BOUNDED-6.pals.yml .896 900 1600 11000 .807 900 1400 14000 .0  .027 5.8 .15  .0  .029 5.7 .21  .0  3.2 10 37 .827 1.0  33 13   .153   900 31 13000 .748 43    29 680 .822 880 170 10000
pals_floodmax.3.ufo.BOUNDED-6.pals.yml .886 900 1500 10000 .797 900 680 12000 .0  .030 5.6 .12  .0  .021 5.7 .16  .0  3.2 10 40 .817 1.1  34 14   .153   900 30 12000 .738 43    29 570 .802 880 180 12000
pals_floodmax.4.1.ufo.BOUNDED-8.pals.yml .934 900 1200 10000 .89  790 840 9400 .0  .030 5.7 .15  .0  .027 5.7 .20  .0  3.2 10 41 .820 900    3100 8600   .131   900 31 12000 .0   900    3000 13000 .896 880 180 12000
pals_floodmax.4.2.ufo.BOUNDED-8.pals.yml .887 320 1300 3200 .887 88 660 1000 .0  .023 5.8 .12  .0  .041 5.6 .093 .0  3.2 12 50 .763 900    3100 8400   .124   900 31 12000 .342 850    1100 10000 .848 880 190 13000
pals_floodmax.4.3.ufo.BOUNDED-8.pals.yml .909 900 1500 9000 .828 900 1300 7000 .0  .035 5.7 .091 .0  .040 5.8 .076 .0  3.2 12 41 .761 900    3100 11000   .124   900 31 11000 .539 850    1200 14000 .875 890 190 12000
pals_floodmax.4.4.ufo.BOUNDED-8.pals.yml .925 900 1600 10000 .870 900 1200 12000 .0  .029 5.7 .12  .0  .026 5.7 .11  .0  3.3 12 38 .779 900    3100 11000   .124   900 31 13000 .648 850    1200 10000 .887 890 190 12000
pals_floodmax.4.ufo.BOUNDED-8.pals.yml .921 900 1300 8500 .862 900 1300 7800 .0  .051 5.7 .092 .0  .029 5.7 .077 .0  3.2 12 47 .787 900    3100 8800   .124   900 31 12000 .640 850    1200 11000 .883 890 190 9600
pals_floodmax.5.1.ufo.BOUNDED-10.pals.yml .957 900 1200 12000 .900 900 1100 14000 .0  .023 5.6 .13  .0  .048 5.6 .10  .0  3.2 12 43 .883 930    15000 5800   .110   900 34 12000 .0   270    15000 4000 .942 890 220 12000
pals_floodmax.5.2.ufo.BOUNDED-10.pals.yml .930 900 1300 11000 .907 780 960 9700 .0  .027 5.7 .18  .0  .023 5.8 .070 .0  3.4 21 48 .833 900    15000 5100   .106   900 34 12000 .0   66    15000 770 .913 890 230 13000
pals_floodmax.5.3.ufo.BOUNDED-10.pals.yml .939 900 1400 12000 .911 900 1500 6700 .0  .021 5.7 .13  .0  .052 5.5 .13  .0  3.4 20 43 .864 900    15000 5200   .106   900 34 11000 .0   65    15000 780 .921 890 220 12000
pals_floodmax.5.4.ufo.BOUNDED-10.pals.yml .949 900 1300 12000 .917 900 1300 10000 .0  .022 5.7 .10  .0  .024 5.7 .068 .0  3.4 20 45 .855 900    15000 7200   .106   900 34 12000 .0   65    15000 790 .933 890 230 11000
pals_floodmax.5.ufo.BOUNDED-10.pals.yml .947 900 1600 11000 .916 900 1100 10000 .0  .023 5.7 .13  .0  .052 5.6 .15  .0  3.4 20 42 .874 900    15000 6200   .106   900 34 11000 .0   65    15000 910 .931 890 220 13000
pals_lcr-var-start-time.3.1.ufo.BOUNDED-6.pals.yml .882 900 1800 10000 .836 900 1200 11000 .0  .031 5.7 .12  .0  .023 5.7 .13  .0  3.2 11 48 .855 .22 22 2.7 .0909  900 31 12000 .736 1.1  20 18 .736 880 170 13000
pals_lcr-var-start-time.3.2.ufo.BOUNDED-6.pals.yml .894 900 1800 11000 .894 140 750 1800 .0  .023 5.7 .13  .0  .037 5.7 .094 .0  3.2 10 41 .817 .23 21 2.8 .0962  900 31 11000 .740 1.1  20 16 .740 880 160 11000
pals_lcr-var-start-time.3.ufo.BOUNDED-6.pals.yml .864 900 3100 12000 .8   900 1500 11000 .0  .024 5.8 .28  .0  .023 5.5 .087 .0  3.2 10 39 .836 .22 21 2.7 .0909  900 30 12000 .718 1.1  20 14 .718 880 160 11000
pals_lcr-var-start-time.4.1.ufo.BOUNDED-8.pals.yml .884 900 4200 11000 .774 900 900 11000 .0  .025 5.7 .18  .0  .035 5.7 .11  .0  3.2 10 46 .747 .75 24 9.7 .0548  900 31 11000 .726 2.0  23 30 .726 880 170 11000
pals_lcr-var-start-time.4.2.ufo.BOUNDED-8.pals.yml .886 900 2600 10000 .779 900 800 12000 .0  .024 5.7 .10  .0  .027 5.7 .24  .0  3.2 10 45 .786 .60 24 6.8 .0571  900 30 11000 .729 2.0  22 30 .729 880 170 10000
pals_lcr-var-start-time.4.ufo.BOUNDED-8.pals.yml .870 900 4000 11000 .760 900 470 12000 .0  .023 5.7 .093 .0  .037 5.7 .11  .0  3.2 10 42 .733 .70 24 8.1 .0548  900 31 12000 .712 2.0  23 25 .719 880 170 11000
pals_lcr-var-start-time.5.1.ufo.BOUNDED-10.pals.yml .886 900 4500 13000 .755 900 780 12000 .0  .025 5.7 .10  .0  .023 5.7 .094 .0  3.2 10 47 .788 3.5  34 44   .0543  900 31 13000 .717 6.0  35 77 .717 880 170 10000
pals_lcr-var-start-time.5.2.ufo.BOUNDED-10.pals.yml .882 900 4400 11000 .770 900 1000 12000 .0  .023 5.7 .084 .0  .033 5.7 .18  .0  3.2 11 38 .753 3.0  34 34   .0562  900 31 13000 .713 5.7  33 85 .719 880 170 11000
pals_lcr-var-start-time.5.ufo.BOUNDED-10.pals.yml .875 900 2900 12000 .75  900 510 13000 .0  .023 5.7 .13  .0  .025 5.8 .13  .0  3.2 10 43 .793 2.9  35 44   .0543  900 31 14000 .707 5.8  33 85 .707 880 170 11000
pals_lcr-var-start-time.6.1.ufo.BOUNDED-12.pals.yml .888 900 4600 12000 .737 900 760 13000 .0  .026 5.7 .21  .0  .024 5.7 .11  .0  3.2 10 37 .759 27    80 360   .0446  900 31 11000 .710 31    100 420 .710 880 180 13000
pals_lcr-var-start-time.6.2.ufo.BOUNDED-12.pals.yml .881 900 4500 11000 .734 900 510 12000 .0  .031 5.7 .18  .0  .046 5.6 .083 .0  3.2 11 37 .766 21    76 250   .0459  900 31 12000 .711 27    87 350 .711 880 170 12000
pals_lcr-var-start-time.6.ufo.BOUNDED-12.pals.yml .880 900 4800 12000 .755 900 780 12000 .0  .023 5.8 .13  .0  .026 5.7 .14  .0  3.2 11 39 .728 23    77 280   .0446  900 31 12000 .701 27    93 410 .701 880 170 11000
pals_lcr.3.1.ufo.BOUNDED-6.pals.yml .866 41 1100 380 .866 20 480 200 .0  .023 5.7 .14  .0  .053 5.7 .14  .0  3.2 10 44 .854 .20 21 2.8 .110   900 31 13000 .683 1.0  20 13 .683 880 160 12000
pals_lcr.3.ufo.BOUNDED-6.pals.yml .842 310 3300 3700 .842 34 600 450 .0  .030 5.7 .11  .0  .022 5.7 .15  .0  3.2 10 41 .829 .21 20 2.0 .110   900 30 13000 .658 .98 20 15 .658 880 160 13000
pals_lcr.4.1.ufo.BOUNDED-8.pals.yml .870 900 4700 8300 .732 900 1100 11000 .0  .022 5.7 .14  .0  .024 5.7 .050 .0  3.2 10 51 .852 .59 22 7.2 .0833  900 31 10000 .667 2.4  21 26 .667 880 170 11000
pals_lcr.4.ufo.BOUNDED-8.pals.yml .852 900 4600 12000 .787 900 1100 13000 .0  .023 5.6 .15  .0  .026 5.7 .059 .0  3.2 11 44 .824 .46 21 5.5 .0833  900 31 11000 .648 1.9  20 27 .648 880 170 10000
pals_lcr.5.1.ufo.BOUNDED-10.pals.yml .875 900 4700 11000 .662 900 840 9500 .0  .026 5.7 .14  .0  .027 5.6 .088 .0  3.2 11 43 .823 2.5  26 30   .0515  900 31 15000 .654 8.3  25 110 .654 880 170 13000
pals_lcr.5.ufo.BOUNDED-10.pals.yml .860 900 4700 13000 .698 900 2200 11000 .0  .026 5.6 .14  .0  .023 5.6 .16  .0  3.2 11 41 .809 1.8  24 27   .0515  900 30 11000 .640 6.1  24 91 .640 880 170 12000
pals_lcr.6.1.ufo.BOUNDED-12.pals.yml .880 900 4600 12000 .711 900 520 11000 .0  .037 5.5 .096 .0  .024 5.7 .12  .0  3.2 12 38 .837 14    48 170   .0542  900 31 13000 .645 37    43 410 .645 880 180 10000
pals_lcr.6.ufo.BOUNDED-12.pals.yml .868 900 4700 12000 .705 900 820 12000 .0  .045 5.7 .088 .0  .023 5.7 .078 .0  3.2 12 41 .819 9.2  40 130   .0542  900 31 10000 .632 24    38 340 .632 880 170 11000
pals_lcr.7.1.ufo.BOUNDED-14.pals.yml .864 900 4300 11000 .712 900 620 13000 .0  .029 5.7 .14  .0  .024 5.7 .23  .0  3.2 13 43 .813 79    160 970   .0455  900 31 12000 .636 170    140 2600 .636 880 180 11000
pals_lcr.7.ufo.BOUNDED-14.pals.yml .874 900 4400 13000 .702 900 680 7000 .0  .049 5.7 .094 .0  .025 5.7 .11  .0  3.3 13 49 .803 49    110 680   .0455  900 31 12000 .626 110    100 1400 .626 880 180 11000
pals_lcr.8.1.ufo.BOUNDED-16.pals.yml .871 900 4700 14000 .681 900 690 11000 .0  .027 5.6 .19  .0  .047 5.7 .083 .0  3.3 14 43 .802 500    690 6000   .0388  900 31 12000 .629 840    560 9200 .629 890 190 9600
pals_lcr.8.ufo.BOUNDED-16.pals.yml .871 900 4200 12000 .694 900 510 11000 .0  .048 5.7 .11  .0  .024 5.7 .16  .0  3.3 14 50 .806 280    430 3500   .0388  900 31 14000 .621 510    370 5900 .625 880 190 12000
pals_opt-floodmax.3.1.ufo.BOUNDED-6.pals.yml .909 900 1600 11000 .880 900 1000 10000 .0  .021 5.7 .17  .0  .041 5.6 .20  .0  3.2 10 43 .788 23    160 260   .120   900 31 11000 .774 60    38 820 .808 880 170 11000
pals_opt-floodmax.3.2.ufo.BOUNDED-6.pals.yml .804 37 1000 320 .804 18 500 190 .0  .023 5.7 .14  .0  .029 5.6 .12  .0  3.2 10 46 .691 7.8  120 110   .141   900 31 12000 .482 7.1  23 92 .714 880 180 12000
pals_opt-floodmax.3.3.ufo.BOUNDED-6.pals.yml .873 900 1500 11000 .773 900 920 11000 .0  .021 5.7 .077 .0  .025 5.7 .099 .0  3.2 11 39 .759 8.0  130 95   .141   900 31 15000 .523 37    27 490 .791 880 170 11000
pals_opt-floodmax.3.4.ufo.BOUNDED-6.pals.yml .895 900 2100 11000 .795 900 1000 9700 .0  .024 5.7 .066 .0  .022 5.7 .074 .0  3.2 10 40 .782 9.5  130 110   .141   900 31 12000 .745 45    31 590 .804 880 180 13000
pals_opt-floodmax.3.ufo.BOUNDED-6.pals.yml .895 900 1600 7700 .795 900 1100 9600 .0  .030 5.7 .10  .0  .043 5.7 .22  .0  3.2 10 42 .782 9.7  130 110   .141   900 30 11000 .745 45    30 580 .809 880 180 12000
pals_opt-floodmax.4.1.ufo.BOUNDED-8.pals.yml .938 900 1300 12000 .872 900 660 11000 .0  .024 5.7 .068 .0  .023 5.7 .063 .0  3.2 11 37 .903 900    13000 5700   .107   900 31 13000 .720 890    3200 11000 .895 890 190 10000
pals_opt-floodmax.4.2.ufo.BOUNDED-8.pals.yml .894 780 1300 11000 .894 380 570 4800 .0  .023 5.7 .084 .0  .041 5.7 .068 .0  3.2 12 42 .820 900    13000 5500   .102   900 31 14000 .422 850    1200 12000 .849 890 190 11000
pals_opt-floodmax.4.3.ufo.BOUNDED-8.pals.yml .914 900 1700 12000 .840 900 800 12000 .0  .049 5.8 .18  .0  .023 5.7 .10  .0  3.2 13 43 .833 900    13000 6400   .102   900 31 13000 .446 850    1200 11000 .870 890 200 13000
pals_opt-floodmax.4.4.ufo.BOUNDED-8.pals.yml .929 900 1300 12000 .857 900 1300 11000 .0  .030 5.7 .089 .0  .027 5.7 .11  .0  3.2 13 44 .849 900    13000 5700   .102   900 31 11000 .718 850    1300 11000 .887 890 190 11000
pals_opt-floodmax.4.ufo.BOUNDED-8.pals.yml .926 900 1600 13000 .855 900 610 9900 .0  .025 5.7 .13  .0  .022 5.8 .058 .0  3.2 13 38 .855 900    13000 5500   .102   900 31 12000 .660 850    1300 12000 .881 890 200 8700
pals_opt-floodmax.5.1.ufo.BOUNDED-10.pals.yml .958 900 1300 12000 .903 900 1100 9100 .0  .025 5.8 .11  .0  .027 5.7 .12  .0  3.3 13 49 .789 900    15000 5800   .0958  900 34 12000 .0   270    15000 3900 .941 890 220 13000
pals_opt-floodmax.5.2.ufo.BOUNDED-10.pals.yml .933 900 1300 12000 .891 900 1000 10000 .0  .023 5.7 .094 .0  .023 5.7 .069 .0  3.4 21 47 .664 900    14000 6000   .0928  900 35 13000 .0   63    15000 730 .914 890 230 11000
pals_opt-floodmax.5.3.ufo.BOUNDED-10.pals.yml .942 900 1400 12000 .902 900 1300 7000 .0  .035 5.7 .17  .0  .027 5.7 .24  .0  3.4 21 43 .661 900    14000 5400   .0928  900 35 11000 .0   63    15000 900 .917 890 230 11000
pals_opt-floodmax.5.4.ufo.BOUNDED-10.pals.yml .951 900 1600 12000 .909 900 1000 11000 .0  .022 5.6 .16  .0  .026 5.7 .12  .0  3.4 21 43 .668 900    14000 5000   .0928  900 35 11000 .0   63    15000 780 .927 890 250 11000
pals_opt-floodmax.5.ufo.BOUNDED-10.pals.yml .949 900 1400 11000 .907 900 1300 8200 .0  .025 5.7 .12  .0  .019 5.6 .022 .0  3.4 21 43 .666 900    14000 7500   .0928  900 35 11000 .0   62    15000 790 .927 890 230 11000
../sv-benchmarks/c/seq-mthreaded/ status score cpu (s) mem (MB) energy (J) status score cpu (s) mem (MB) energy (J) status score cpu (s) mem (MB) energy (J) status score cpu (s) mem (MB) energy (J) status score cpu (s) mem (MB) energy (J) status score cpu (s) mem (MB) energy (J) status score cpu (s) mem (MB) energy (J) status score cpu (s) mem (MB) energy (J) status score cpu (s) mem (MB) energy (J)
total 62 54.9 49000 180000 600000 62 49.4  48000 55000 570000 62 .0  1.8 350 7.8 62 .0  1.8 350 7.4 62 .0  200 770 2700 62 49.2 26000 260000 220000 62 5.23 56000 1900 750000 62 28.0 18000 280000 210000 62 48.9 55000 11000 710000
Run set coveritest.test-comp19_prop-coverage-branches.ReachSafety-Sequentialized cpa-tiger.test-comp19_prop-coverage-branches.ReachSafety-Sequentialized esbmc-kind.test-comp19_prop-coverage-branches.ReachSafety-Sequentialized esbmc-falsi.test-comp19_prop-coverage-branches.ReachSafety-Sequentialized fairfuzz.test-comp19_prop-coverage-branches.ReachSafety-Sequentialized klee.test-comp19_prop-coverage-branches.ReachSafety-Sequentialized prtest.test-comp19_prop-coverage-branches.ReachSafety-Sequentialized symbiotic.test-comp19_prop-coverage-branches.ReachSafety-Sequentialized verifuzz.test-comp19_prop-coverage-branches.ReachSafety-Sequentialized