Benchmark FairFuzz TBF Test-Suite Validator
Tool FairFuzz TC-0.0.2 Tbf Test-suite Validator v1.2
Limits timelimit: 900 s, memlimit: 15000 MB, CPU core limit: 8 timelimit: 10800 s, memlimit: 7000 MB, CPU core limit: 2
Host apollon*
OS Linux 4.15.0-45-generic
System CPU: Intel Xeon E3-1230 v5 @ 3.40 GHz, cores: 8, frequency: 3.8 GHz, Turbo Boost: disabled; RAM: 33546 MB
Date of execution 2019-02-08 09:39:09 CET 2019-02-08 13:22:04 CET
Run set fairfuzz.test-comp19_prop-coverage-branches.ReachSafety-Arrays tbf-testsuite-validator-fairfuzz.test-comp19_prop-coverage-branches.ReachSafety-Arrays
Options --verbose --sequence-file cov-seq.txt -r --test-suite ../../results-verified/fairfuzz.2019-02-08_0939.logfiles/${rundefinition_name}.${inputfile_name}.files/test-suite.zip
sv-benchmarks/c/ status score program test-suite TS size inspect test-suite cpu (s) wall (s) mem (MB) energy (J) blkio-w (MB) blkio-r (MB) status program test-suite TS size inspect test-suite cov branch cov (%) line cov (%) cpu (s) wall (s) mem (MB) energy blkio-w (MB) blkio-r (MB)
array-examples/sanfoundry_10_ground.yml timeout .944 prog tests 12 inspect 900    910    53 11000   .78 0     unknown prog tests 0 inspect
94.4 100   29     29     440   .26  0  
array-examples/sanfoundry_24-1.yml timeout .75  prog tests 31 inspect 900    910    98 13000   .38 0     unknown prog tests 0 inspect
75.0 77.8 63     64     580   .57  0  
array-examples/standard_copy1_ground-1.yml timeout .875 prog tests 1 inspect 900    910    28 10000   .30 0     unknown prog tests 0 inspect
87.5 100   2.6   2.6   410   .078 0  
array-examples/standard_copy1_ground-2.yml timeout 1.0   prog tests 8 inspect 900    910    70 11000   .32 0     true prog tests 1 inspect
100   100   40     40     930   .19  0  
array-examples/standard_copy2_ground-1.yml timeout 1.0   prog tests 7 inspect 900    910    60 14000   .31 0     true prog tests 1 inspect
100   100   36     36     920   .18  0  
array-examples/standard_copy2_ground-2.yml timeout .9   prog tests 1 inspect 900    910    23 9900   .30 0     unknown prog tests 0 inspect
90.0 100   2.6   2.6   410   .078 0  
array-examples/standard_copy3_ground-1.yml timeout .917 prog tests 1 inspect 900    910    23 14000   .31 0     unknown prog tests 0 inspect
91.7 100   2.6   2.6   410   .078 0  
array-examples/standard_copy3_ground-2.yml timeout 1.0   prog tests 10 inspect 900    910    76 12000   .31 0     true prog tests 1 inspect
100   100   50     50     950   .23  0  
array-examples/standard_copy4_ground-1.yml timeout .929 prog tests 1 inspect 900    910    23 12000   .31 0     unknown prog tests 0 inspect
92.9 100   2.6   2.6   410   .078 0  
array-examples/standard_copy4_ground-2.yml timeout 1.0   prog tests 8 inspect 900    910    69 11000   .31 0     true prog tests 1 inspect
100   100   39     39     930   .19  0  
array-examples/standard_copy5_ground-1.yml timeout .938 prog tests 1 inspect 900    910    26 12000   .30 0     unknown prog tests 0 inspect
93.8 100   2.5   2.5   410   .078 0  
array-examples/standard_copy5_ground-2.yml timeout 1.0   prog tests 8 inspect 900    910    65 11000   .31 0     true prog tests 1 inspect
100   100   42     42     930   .19  0  
array-examples/standard_copy6_ground-1.yml timeout 1.0   prog tests 8 inspect 900    910    68 11000   .31 .061 true prog tests 1 inspect
100   100   40     40     930   .19  0  
array-examples/standard_copy6_ground-2.yml timeout .944 prog tests 1 inspect 900    910    23 10000   .31 0     unknown prog tests 0 inspect
94.4 100   2.6   2.6   410   .078 0  
array-examples/standard_copy7_ground-1.yml timeout 1.0   prog tests 11 inspect 900    910    83 11000   .31 0     true prog tests 1 inspect
100   100   55     56     970   .25  0  
array-examples/standard_copy7_ground-2.yml timeout .95  prog tests 1 inspect 900    910    24 13000   .31 0     unknown prog tests 0 inspect
95.0 100   2.6   2.6   410   .082 0  
array-examples/standard_copy8_ground-1.yml timeout .955 prog tests 1 inspect 900    910    26 12000   .31 0     unknown prog tests 0 inspect
95.5 100   2.6   2.6   410   .082 0  
array-examples/standard_copy8_ground-2.yml timeout 1.0   prog tests 7 inspect 900    910    63 14000   .31 0     true prog tests 1 inspect
100   100   35     35     920   .18  0  
array-examples/standard_copy9_ground-1.yml timeout 1.0   prog tests 6 inspect 900    910    57 9900   .31 0     true prog tests 1 inspect
100   100   30     30     910   .16  0  
array-examples/standard_copy9_ground-2.yml timeout .958 prog tests 1 inspect 900    910    29 12000   .31 0     unknown prog tests 0 inspect
95.8 100   2.6   2.6   410   .082 0  
array-examples/standard_copyInitSum_ground.yml done .9   prog tests 1 inspect 49    47    11 290   .22 0     unknown prog tests 0 inspect
90.0 100   .12  .12  16   .078 0  
array-examples/standard_find_ground-1.yml timeout .9   prog tests 17 inspect 900    910    70 11000   .77 0     unknown prog tests 0 inspect
90.0 100   41     41     450   .34  0  
array-examples/standard_partition_ground-1.yml couldn't run: all seeds time out or crash .0   prog tests 0 inspect 1.4  1.4  10 15   .23 0     unknown prog tests 0 inspect
0   0   .057 .059 9.0 0     0  
array-examples/standard_sentinel-2.yml done .8   prog tests 3 inspect 710    720    29 10000   .32 0     unknown prog tests 0 inspect
80.0 100   7.3   7.3   410   .11  0  
array-examples/standard_sentinel.i.v+cfa-reducer.yml done .8   prog tests 3 inspect 720    730    28 8600   .32 0     unknown prog tests 0 inspect
80.0 86.4 7.4   7.4   410   .11  0  
array-examples/standard_strcpy_ground-1.yml timeout .8   prog tests 16 inspect 900    910    63 12000   .57 0     unknown prog tests 0 inspect
80.0 100   39     39     440   .32  0  
array-examples/standard_strcpy_original-2.yml timeout .75  prog tests 15 inspect 900    910    63 12000   .63 0     unknown prog tests 0 inspect
75.0 100   36     36     440   .31  0  
array-examples/standard_two_index_01.yml done .875 prog tests 1 inspect 420    420    15 6100   .23 0     unknown prog tests 0 inspect
87.5 100   .36  .36  44   .078 0  
array-examples/standard_two_index_02.yml timeout .875 prog tests 1 inspect 900    910    24 11000   .30 0     unknown prog tests 0 inspect
87.5 100   2.5   2.5   410   .078 0  
array-examples/standard_two_index_03.yml done .875 prog tests 1 inspect 410    410    14 5200   .24 0     unknown prog tests 0 inspect
87.5 100   .37  .37  44   .078 0  
array-examples/standard_two_index_04.yml timeout .875 prog tests 1 inspect 900    910    24 11000   .30 0     unknown prog tests 0 inspect
87.5 100   2.6   2.6   410   .078 0  
array-examples/standard_two_index_05.yml timeout .875 prog tests 1 inspect 900    910    23 12000   .32 0     unknown prog tests 0 inspect
87.5 100   2.6   2.6   410   .078 0  
array-examples/standard_two_index_06.yml done .875 prog tests 1 inspect 420    420    15 5300   .25 0     unknown prog tests 0 inspect
87.5 100   .36  .38  44   .078 0  
array-examples/standard_two_index_07.yml timeout .875 prog tests 1 inspect 900    910    25 11000   .31 0     unknown prog tests 0 inspect
87.5 100   2.5   2.5   410   .078 0  
array-examples/standard_two_index_08.yml timeout .875 prog tests 1 inspect 900    910    25 11000   .31 0     unknown prog tests 0 inspect
87.5 100   2.7   2.7   410   .078 0  
array-examples/standard_two_index_09.yml timeout .875 prog tests 1 inspect 900    910    26 11000   .32 0     unknown prog tests 0 inspect
87.5 100   2.5   2.5   410   .078 0  
array-industry-pattern/array_monotonic.yml done .9   prog tests 10 inspect 860    860    35 13000   .47 0     unknown prog tests 0 inspect
90.0 100   12     12     210   .23  0  
array-industry-pattern/array_mul_init.yml timeout .857 prog tests 48 inspect 900    910    160 11000   1.1  .045 unknown prog tests 0 inspect
85.7 100   120     120     520   .85  0  
array-industry-pattern/array_of_struct_break.yml done .9   prog tests 2 inspect 34    32    12 110   .21 0     unknown prog tests 0 inspect
90.0 100   .12  .14  16   .094 0  
array-industry-pattern/array_of_struct_ptr_mul_init.yml timeout .857 prog tests 21 inspect 900    910    83 12000   .84 .020 unknown prog tests 0 inspect
85.7 100   50     51     450   .41  0  
array-industry-pattern/array_of_struct_single_elem_init.yml timeout .917 prog tests 8 inspect 900    910    67 13000   .44 0     unknown prog tests 0 inspect
91.7 100   39     39     840   .19  0  
array-industry-pattern/array_ptr_single_elem_init-2.yml couldn't run: all seeds time out or crash .0   prog tests 0 inspect 1.3  1.4  12 16   .20 0     unknown prog tests 0 inspect
0   0   .089 .091 9.0 0     0  
array-industry-pattern/array_shadowinit.yml false(unreach-call) .875 prog tests 28 inspect 42    39    22 150   .30 .46  unknown prog tests 0 inspect
87.5 100   .21  .23  17   .52  0  
array-industry-pattern/array_single_elem_init.yml timeout .929 prog tests 18 inspect 900    910    69 13000   .70 0     unknown prog tests 0 inspect
92.9 100   46     46     450   .36  0  
reducercommutativity/avg.yml false(unreach-call) .8   prog tests 18 inspect 20    18    20 43   .33 0     unknown prog tests 0 inspect
80.0 95.2 .18  .23  17   .36  0  
reducercommutativity/avg05-1.yml done .8   prog tests 1 inspect 30    28    18 90   .21 .061 unknown prog tests 0 inspect
80.0 94.7 .12  .12  17   .078 0  
reducercommutativity/avg10-2.yml done .8   prog tests 1 inspect 33    30    17 84   .22 0     unknown prog tests 0 inspect
80.0 94.7 .12  .12  17   .078 0  
reducercommutativity/avg20-2.yml done .8   prog tests 1 inspect 35    33    14 110   .23 .13  unknown prog tests 0 inspect
80.0 94.7 .12  .12  16   .078 0  
reducercommutativity/avg40-2.yml done .8   prog tests 1 inspect 40    38    13 110   .23 .18  unknown prog tests 0 inspect
80.0 94.7 .12  .12  17   .078 0  
reducercommutativity/avg60-1.yml done .8   prog tests 1 inspect 45    42    13 130   .23 0     unknown prog tests 0 inspect
80.0 94.7 .12  .12  17   .078 0  
reducercommutativity/max.yml false(unreach-call) .8   prog tests 14 inspect 22    20    19 62   .30 0     unknown prog tests 0 inspect
80.0 95.2 .18  .18  16   .27  0  
reducercommutativity/max05-1.yml done .8   prog tests 1 inspect 30    27    16 78   .21 .061 unknown prog tests 0 inspect
80.0 94.7 .12  .12  16   .078 0  
reducercommutativity/max10-1.yml done .8   prog tests 1 inspect 22    20    15 110   .22 0     unknown prog tests 0 inspect
80.0 94.7 .13  .13  16   .078 0  
reducercommutativity/max20-1.yml done .8   prog tests 1 inspect 35    33    14 120   .23 0     unknown prog tests 0 inspect
80.0 94.7 .12  .12  16   .078 0  
reducercommutativity/max40-1.yml done .8   prog tests 1 inspect 37    35    13 110   .23 .045 unknown prog tests 0 inspect
80.0 94.7 .12  .12  16   .078 0  
reducercommutativity/max60-2.yml done .8   prog tests 1 inspect 44    42    12 140   .23 0     unknown prog tests 0 inspect
80.0 94.7 .11  .11  16   .078 0  
reducercommutativity/rangesum.yml false(unreach-call) .938 prog tests 50 inspect 730    740    60 11000   .46 0     true prog tests 1 inspect
93.8 100   30     30     870   .88  0  
reducercommutativity/rangesum05.yml false(unreach-call) .857 prog tests 2 inspect 10    9.3  16 30   .21 .11  true prog tests 1 inspect
85.7 96.3 .16  .16  16   .098 0  
reducercommutativity/rangesum10.yml false(unreach-call) .857 prog tests 2 inspect 11    10    15 33   .21 0     true prog tests 1 inspect
85.7 96.3 .12  .12  16   .098 0  
reducercommutativity/rangesum20.yml false(unreach-call) .857 prog tests 2 inspect 13    12    15 46   .24 0     true prog tests 1 inspect
85.7 96.3 .13  .13  16   .098 0  
reducercommutativity/rangesum40.yml false(unreach-call) .857 prog tests 2 inspect 14    13    12 46   .24 0     true prog tests 1 inspect
85.7 96.3 .12  .12  16   .098 0  
reducercommutativity/rangesum60.yml false(unreach-call) .857 prog tests 2 inspect 12    11    12 73   .24 .061 true prog tests 1 inspect
85.7 96.3 .13  .13  17   .098 0  
reducercommutativity/sep.yml false(unreach-call) .833 prog tests 13 inspect 31    29    20 110   .30 0     unknown prog tests 0 inspect
83.3 95.7 .20  .22  18   .27  0  
reducercommutativity/sep05-1.yml done .75  prog tests 1 inspect 31    28    15 95   .22 0     unknown prog tests 0 inspect
75.0 90.5 .12  .12  16   .078 0  
reducercommutativity/sep10-2.yml done .75  prog tests 1 inspect 28    26    15 90   .22 0     unknown prog tests 0 inspect
75.0 90.5 .12  .12  17   .078 0  
reducercommutativity/sep20-1.yml done .75  prog tests 1 inspect 25    23    13 110   .23 0     unknown prog tests 0 inspect
75.0 90.5 .11  .11  16   .078 0  
reducercommutativity/sep40-1.yml done .75  prog tests 1 inspect 42    40    14 120   .23 0     unknown prog tests 0 inspect
75.0 90.5 .12  .12  16   .078 0  
reducercommutativity/sep60-2.yml done .75  prog tests 1 inspect 38    36    12 140   .23 .061 unknown prog tests 0 inspect
75.0 90.5 .14  .14  16   .078 0  
reducercommutativity/sum.yml false(unreach-call) .8   prog tests 19 inspect 27    25    20 120   .33 0     unknown prog tests 0 inspect
80.0 95.2 .21  .20  16   .37  0  
reducercommutativity/sum05-2.yml done .8   prog tests 1 inspect 31    28    16 100   .21 .21  unknown prog tests 0 inspect
80.0 94.7 .12  .12  16   .078 0  
reducercommutativity/sum10-1.yml done .8   prog tests 1 inspect 30    28    16 96   .22 0     unknown prog tests 0 inspect
80.0 94.7 .13  .13  16   .078 0  
reducercommutativity/sum20-2.yml done .8   prog tests 1 inspect 25    23    14 130   .22 0     unknown prog tests 0 inspect
80.0 94.7 .12  .12  17   .078 0  
reducercommutativity/sum40-2.yml done .8   prog tests 1 inspect 39    37    13 110   .23 .30  unknown prog tests 0 inspect
80.0 94.7 .12  .12  16   .078 0  
reducercommutativity/sum60-2.yml done .8   prog tests 1 inspect 33    31    12 140   .23 .34  unknown prog tests 0 inspect
80.0 94.7 .12  .12  16   .078 0  
array-tiling/mbpr2.yml false(unreach-call) .8   prog tests 19 inspect 33    30    21 160   .29 0     unknown prog tests 0 inspect
80.0 91.7 .17  .17  16   .38  0  
array-tiling/mbpr3.yml false(unreach-call) .792 prog tests 22 inspect 42    39    22 190   .30 0     unknown prog tests 0 inspect
79.2 90.9 .19  .19  16   .43  0  
array-tiling/mbpr4.yml false(unreach-call) .786 prog tests 19 inspect 25    23    21 120   .30 0     unknown prog tests 0 inspect
78.6 90.9 .21  .20  19   .45  0  
array-tiling/mbpr5.yml false(unreach-call) .781 prog tests 24 inspect 67    62    24 380   .31 .061 unknown prog tests 0 inspect
78.1 91.2 .20  .19  17   .56  0  
array-tiling/mlceu.yml done .75  prog tests 16 inspect 29    28    13 130   .28 0     unknown prog tests 0 inspect
75.0 91.7 .17  .17  17   .32  0  
array-tiling/nr2.yml false(unreach-call) .889 prog tests 23 inspect 49    46    21 320   .29 0     unknown prog tests 0 inspect
88.9 100   .21  .20  21   .44  0  
array-tiling/nr3.yml false(unreach-call) .889 prog tests 27 inspect 63    58    23 230   .29 0     unknown prog tests 0 inspect
88.9 100   .26  .25  17   .51  0  
array-tiling/nr4.yml false(unreach-call) .889 prog tests 19 inspect 34    32    20 160   .29 .045 unknown prog tests 0 inspect
88.9 100   .20  .23  16   .38  0  
array-tiling/nr5.yml false(unreach-call) .889 prog tests 32 inspect 38    35    26 180   .29 0     unknown prog tests 0 inspect
88.9 100   .28  .28  16   .59  0  
array-tiling/pnr2.yml timeout .875 prog tests 46 inspect 900    900    45 12000   .78 0     unknown prog tests 0 inspect
87.5 100   20     20     350   .82  0  
array-tiling/pnr3.yml timeout .889 prog tests 51 inspect 900    900    38 10000   .79 .061 unknown prog tests 0 inspect
88.9 100   13     14     250   .90  0  
array-tiling/pnr4.yml timeout .9   prog tests 38 inspect 900    900    29 12000   .84 0     unknown prog tests 0 inspect
90.0 100   6.6   6.6   220   .69  0  
array-tiling/pnr5.yml timeout .909 prog tests 57 inspect 900    900    46 9900   1.1  0     unknown prog tests 0 inspect
90.9 100   20     20     310   1.2   0  
array-tiling/poly1.yml done .8   prog tests 23 inspect 20    19    17 69   .29 0     unknown prog tests 0 inspect
80.0 100   .20  .20  16   .44  0  
array-tiling/poly2.yml done .833 prog tests 22 inspect 16    15    16 50   .30 .22  unknown prog tests 0 inspect
83.3 100   .20  .19  16   .42  0  
array-tiling/pr2.yml false(unreach-call) .667 prog tests 21 inspect 63    59    22 420   .29 0     unknown prog tests 0 inspect
66.7 90.0 .20  .23  19   .41  0  
array-tiling/pr3.yml false(unreach-call) .65  prog tests 18 inspect 22    21    22 160   .29 0     unknown prog tests 0 inspect
65.0 87.5 .19  .18  18   .36  0  
array-tiling/pr4.yml false(unreach-call) .636 prog tests 16 inspect 11    11    23 120   .29 0     unknown prog tests 0 inspect
63.6 85.7 .19  .21  19   .33  0  
array-tiling/pr5.yml false(unreach-call) .625 prog tests 20 inspect 15    14    19 84   .29 0     unknown prog tests 0 inspect
62.5 84.4 .18  .17  16   .39  0  
array-tiling/revcpyswp2.yml done .875 prog tests 15 inspect 680    690    27 8700   .34 0     unknown prog tests 0 inspect
87.5 100   4.2   4.2   500   .31  0  
array-tiling/rew.yml done .833 prog tests 19 inspect 58    54    15 200   .28 0     unknown prog tests 0 inspect
83.3 100   .18  .17  16   .37  0  
array-tiling/rewnif.yml done .833 prog tests 19 inspect 55    51    17 180   .28 0     unknown prog tests 0 inspect
83.3 100   .18  .24  17   .37  0  
array-tiling/rewnifrev.yml done .833 prog tests 15 inspect 24    22    15 140   .29 0     unknown prog tests 0 inspect
83.3 100   .16  .16  16   .31  0  
array-tiling/rewnifrev2.yml done .8   prog tests 22 inspect 14    13    17 70   .29 .28  unknown prog tests 0 inspect
80.0 100   .22  .21  16   .42  0  
array-tiling/rewrev.yml done .833 prog tests 14 inspect 49    46    15 140   .29 0     unknown prog tests 0 inspect
83.3 100   .16  .15  17   .29  0  
array-tiling/skipped.yml timeout .889 prog tests 36 inspect 900    900    38 11000   .86 0     unknown prog tests 0 inspect
88.9 100   16     16     250   .66  0  
array-tiling/skippedu.yml timeout .944 prog tests 51 inspect 900    900    35 12000   .86 0     true prog tests 1 inspect
94.4 100   10     10     280   .90  0  
array-tiling/tcpy.yml done .857 prog tests 18 inspect 700    710    26 11000   .33 0     unknown prog tests 0 inspect
85.7 100   2.8   2.8   270   .36  0  
array-programs/copysome1-1.yml couldn't run: all seeds time out or crash .0   prog tests 0 inspect 3.1  3.2  19 44   .20 0     unknown prog tests 0 inspect
0   0   .060 .061 8.9 0     0  
array-programs/copysome1-2.yml couldn't run: all seeds time out or crash .0   prog tests 0 inspect 3.1  3.2  17 37   .20 0     unknown prog tests 0 inspect
0   0   .061 .063 9.0 0     0  
array-programs/copysome2-1.yml couldn't run: all seeds time out or crash .0   prog tests 0 inspect 3.1  3.2  18 42   .20 0     unknown prog tests 0 inspect
0   0   .062 .063 9.0 0     0  
array-programs/copysome2-2.yml couldn't run: all seeds time out or crash .0   prog tests 0 inspect 3.1  3.2  19 35   .20 0     unknown prog tests 0 inspect
0   0   .060 .061 9.0 0     0  
array-crafted/bAnd1.yml done .8   prog tests 1 inspect 44    42    12 170   .24 0     unknown prog tests 0 inspect
80.0 94.7 .12  .12  17   .078 0  
array-crafted/bAnd2.yml done .8   prog tests 1 inspect 170    160    11 1100   .25 .020 unknown prog tests 0 inspect
80.0 94.7 .14  .14  17   .078 0  
array-crafted/bAnd3.yml done .8   prog tests 1 inspect 420    420    15 5600   .27 .34  unknown prog tests 0 inspect
80.0 94.7 .36  .36  44   .078 0  
array-crafted/bAnd4.yml timeout .8   prog tests 1 inspect 900    910    24 12000   .34 .17  unknown prog tests 0 inspect
80.0 94.7 2.7   2.7   410   .078 0  
array-crafted/bAnd5.yml false(unreach-call) .833 prog tests 26 inspect 710    710    46 9100   .38 .36  unknown prog tests 0 inspect
83.3 95.7 18     18     880   .49  0  
array-crafted/bor1.yml done .8   prog tests 1 inspect 45    43    12 160   .24 .34  unknown prog tests 0 inspect
80.0 94.7 .12  .12  17   .078 0  
array-crafted/bor2.yml done .8   prog tests 1 inspect 170    170    10 1000   .25 .15  unknown prog tests 0 inspect
80.0 94.7 .15  .15  17   .078 0  
array-crafted/bor3.yml done .8   prog tests 1 inspect 420    420    15 6100   .27 .39  unknown prog tests 0 inspect
80.0 94.7 .36  .36  45   .078 0  
array-crafted/bor4.yml timeout .8   prog tests 1 inspect 900    910    25 12000   .34 .13  unknown prog tests 0 inspect
80.0 94.7 2.6   2.6   410   .078 0  
array-crafted/bor5.yml false(unreach-call) .833 prog tests 26 inspect 740    750    44 11000   .45 .29  unknown prog tests 0 inspect
83.3 95.7 17     17     1000   .49  0  
array-crafted/mapavg1.yml done .8   prog tests 1 inspect 41    39    12 190   .22 0     unknown prog tests 0 inspect
80.0 94.7 .12  .12  17   .078 0  
array-crafted/mapavg2.yml done .8   prog tests 1 inspect 170    170    11 1200   .23 0     unknown prog tests 0 inspect
80.0 94.7 .14  .14  17   .078 0  
array-crafted/mapavg3.yml done .8   prog tests 1 inspect 410    420    15 5000   .25 0     unknown prog tests 0 inspect
80.0 94.7 .42  .42  44   .078 0  
array-crafted/mapavg4.yml timeout .8   prog tests 1 inspect 900    910    25 13000   .32 0     unknown prog tests 0 inspect
80.0 94.7 2.7   2.7   410   .078 0  
array-crafted/mapavg5.yml false(unreach-call) .833 prog tests 20 inspect 720    730    42 8700   .35 0     unknown prog tests 0 inspect
83.3 95.7 16     16     1000   .38  0  
array-crafted/mapsum1.yml done .8   prog tests 1 inspect 50    48    11 180   .22 0     unknown prog tests 0 inspect
80.0 94.7 .12  .12  17   .078 0  
array-crafted/mapsum2.yml done .8   prog tests 1 inspect 170    170    11 1100   .23 0     unknown prog tests 0 inspect
80.0 94.7 .15  .15  17   .078 0  
array-crafted/mapsum3.yml done .8   prog tests 1 inspect 420    420    14 5900   .24 0     unknown prog tests 0 inspect
80.0 94.7 .37  .37  44   .078 0  
array-crafted/mapsum4.yml timeout .8   prog tests 1 inspect 900    910    24 12000   .32 0     unknown prog tests 0 inspect
80.0 94.7 2.7   2.7   410   .078 0  
array-crafted/mapsum5.yml false(unreach-call) .833 prog tests 22 inspect 710    720    50 8600   .34 0     unknown prog tests 0 inspect
83.3 95.7 23     23     870   .43  0  
array-crafted/xor1.yml done .8   prog tests 1 inspect 41    39    11 200   .24 .23  unknown prog tests 0 inspect
80.0 94.7 .12  .12  16   .078 0  
array-crafted/xor2.yml done .8   prog tests 1 inspect 170    170    10 1200   .25 0     unknown prog tests 0 inspect
80.0 94.7 .14  .14  17   .078 0  
array-crafted/xor3.yml done .8   prog tests 1 inspect 410    420    15 6100   .27 .14  unknown prog tests 0 inspect
80.0 94.7 .36  .36  44   .078 0  
array-crafted/xor4.yml timeout .8   prog tests 1 inspect 900    910    28 13000   .34 .020 unknown prog tests 0 inspect
80.0 94.7 2.6   2.6   410   .078 0  
array-crafted/xor5.yml false(unreach-call) .833 prog tests 28 inspect 690    700    40 9800   .37 0     unknown prog tests 0 inspect
83.3 95.7 13     13     1000   .52  0  
array-crafted/zero_sum1.yml done .9   prog tests 27 inspect 300    300    22 4500   .33 0     unknown prog tests 0 inspect
90.0 100   3.5   3.5   100   .50  0  
array-crafted/zero_sum2.yml done .929 prog tests 23 inspect 210    210    20 2400   .36 0     unknown prog tests 0 inspect
92.9 100   2.1   2.1   120   .44  0  
array-crafted/zero_sum3.yml done .944 prog tests 23 inspect 320    320    25 3800   .40 0     unknown prog tests 0 inspect
94.4 100   4.0   4.1   130   .44  0  
array-crafted/zero_sum4.yml done .955 prog tests 21 inspect 240    240    22 3100   .41 0     unknown prog tests 0 inspect
95.5 100   2.7   2.7   120   .41  0  
array-crafted/zero_sum5.yml done .962 prog tests 23 inspect 280    280    24 3500   .46 .18  unknown prog tests 0 inspect
96.2 100   3.1   3.1   100   .44  0  
array-crafted/zero_sum_const1.yml done .833 prog tests 19 inspect 36    33    17 89   .29 0     unknown prog tests 0 inspect
83.3 100   .18  .17  16   .37  0  
array-crafted/zero_sum_const2.yml done .875 prog tests 19 inspect 30    27    18 78   .31 0     unknown prog tests 0 inspect
87.5 100   .18  .22  16   .37  0  
array-crafted/zero_sum_const3.yml done .9   prog tests 21 inspect 19    17    17 130   .32 .061 unknown prog tests 0 inspect
90.0 100   .21  .25  16   .41  0  
array-crafted/zero_sum_const4.yml done .917 prog tests 20 inspect 19    17    15 64   .33 0     unknown prog tests 0 inspect
91.7 100   .18  .17  17   .39  0  
array-crafted/zero_sum_const5.yml done .929 prog tests 21 inspect 18    17    15 66   .34 0     unknown prog tests 0 inspect
92.9 100   .20  .19  16   .41  0  
array-crafted/zero_sum_const_m2.yml done .875 prog tests 20 inspect 35    33    17 93   .31 0     unknown prog tests 0 inspect
87.5 100   .19  .18  17   .39  0  
array-crafted/zero_sum_const_m3.yml done .9   prog tests 19 inspect 36    33    16 110   .32 .074 unknown prog tests 0 inspect
90.0 100   .19  .23  17   .38  0  
array-crafted/zero_sum_const_m4.yml done .917 prog tests 18 inspect 27    25    16 140   .33 0     unknown prog tests 0 inspect
91.7 100   .18  .18  16   .36  0  
array-crafted/zero_sum_const_m5.yml done .929 prog tests 19 inspect 18    16    16 80   .34 .18  unknown prog tests 0 inspect
92.9 100   .19  .18  16   .38  0  
array-crafted/zero_sum_m2.yml done .929 prog tests 25 inspect 310    310    20 3900   .36 0     unknown prog tests 0 inspect
92.9 100   2.5   2.5   120   .47  0  
array-crafted/zero_sum_m3.yml done .944 prog tests 24 inspect 250    250    22 3500   .39 .13  unknown prog tests 0 inspect
94.4 100   2.5   2.5   110   .46  0  
array-crafted/zero_sum_m4.yml done .955 prog tests 22 inspect 320    320    21 3900   .42 .11  unknown prog tests 0 inspect
95.5 100   2.6   2.6   100   .43  0  
array-crafted/zero_sum_m5.yml done .962 prog tests 21 inspect 260    260    23 3500   .46 0     unknown prog tests 0 inspect
96.2 100   3.2   3.2   140   .41  0  
array-multidimensional/add-2-n-u.yml couldn't run: all seeds time out or crash .0   prog tests 0 inspect .17 .19 11 1.8 .22 0     unknown prog tests 0 inspect
0   0   .059 .060 8.9 0     0  
array-multidimensional/add-3-n-u.yml couldn't run: all seeds time out or crash .0   prog tests 0 inspect .17 .18 10 1.9 .22 0     unknown prog tests 0 inspect
0   0   .065 .066 8.9 0     0  
array-multidimensional/copy-2-u.yml couldn't run: all seeds time out or crash .0   prog tests 0 inspect 3.1  3.2  18 44   .20 0     unknown prog tests 0 inspect
0   0   .074 .075 9.0 0     0  
array-multidimensional/copy-3-n-u.yml couldn't run: all seeds time out or crash .0   prog tests 0 inspect .16 .18 11 1.6 .21 0     unknown prog tests 0 inspect
0   0   .065 .066 9.0 0     0  
array-multidimensional/copy-partial-2-n-u.yml couldn't run: all seeds time out or crash .0   prog tests 0 inspect .15 .17 10 1.6 .20 0     unknown prog tests 0 inspect
0   0   .066 .067 9.0 0     0  
array-multidimensional/copy-partial-3-u.yml couldn't run: all seeds time out or crash .0   prog tests 0 inspect .18 .20 10 1.4 .20 0     unknown prog tests 0 inspect
0   0   .061 .062 9.0 0     0  
array-multidimensional/diff-2-n-u.yml couldn't run: all seeds time out or crash .0   prog tests 0 inspect .17 .18 11 1.8 .22 0     unknown prog tests 0 inspect
0   0   .059 .059 9.0 0     0  
array-multidimensional/diff-3-n-u.yml couldn't run: all seeds time out or crash .0   prog tests 0 inspect .18 .20 10 1.9 .22 .061 unknown prog tests 0 inspect
0   0   .060 .061 9.0 0     0  
array-multidimensional/init-2-n-u.yml done .9   prog tests 1 inspect 95    95    13 1200   .22 .26  unknown prog tests 0 inspect
90.0 96.2 .13  .13  16   .078 0  
array-multidimensional/init-3-u.yml couldn't run: all seeds time out or crash .0   prog tests 0 inspect .16 .17 11 1.7 .21 0     unknown prog tests 0 inspect
0   0   .063 .066 9.0 0     0  
array-multidimensional/init-4-n-u.yml couldn't run: all seeds time out or crash .0   prog tests 0 inspect .16 .17 10 2.1 .21 .35  unknown prog tests 0 inspect
0   0   .069 .070 9.1 0     0  
array-multidimensional/init-non-constant-2-n-u.yml timeout .9   prog tests 15 inspect 900    900    10 13000   .71 .26  unknown prog tests 0 inspect
90.0 96.7 .17  .16  16   .31  0  
array-multidimensional/init-non-constant-3-u.yml couldn't run: all seeds time out or crash .0   prog tests 0 inspect .16 .18 11 2.0 .22 .44  unknown prog tests 0 inspect
0   0   .059 .060 9.0 0     0  
array-multidimensional/max-2-u.yml couldn't run: all seeds time out or crash .0   prog tests 0 inspect .18 .20 10 1.8 .21 .52  unknown prog tests 0 inspect
0   0   .065 .067 9.1 0     0  
array-multidimensional/max-3-n-u.yml couldn't run: all seeds time out or crash .0   prog tests 0 inspect .16 .18 11 1.8 .22 .41  unknown prog tests 0 inspect
0   0   .057 .059 9.0 0     0  
array-multidimensional/min-2-u.yml couldn't run: all seeds time out or crash .0   prog tests 0 inspect .15 .17 11 2.2 .21 .26  unknown prog tests 0 inspect
0   0   .060 .061 9.0 0     0  
array-multidimensional/min-3-n-u.yml couldn't run: all seeds time out or crash .0   prog tests 0 inspect .17 .18 11 2.1 .22 .34  unknown prog tests 0 inspect
0   0   .065 .066 9.1 0     0  
array-multidimensional/rev-2-n-u.yml couldn't run: all seeds time out or crash .0   prog tests 0 inspect .16 .18 10 1.8 .21 .14  unknown prog tests 0 inspect
0   0   .065 .065 9.0 0     0  
array-multidimensional/rev-3-u.yml couldn't run: all seeds time out or crash .0   prog tests 0 inspect .19 .21 11 1.9 .22 .020 unknown prog tests 0 inspect
0   0   .065 .065 9.0 0     0  
array-multidimensional/transpose-u.yml couldn't run: all seeds time out or crash .0   prog tests 0 inspect 3.1  3.2  18 43   .20 0     unknown prog tests 0 inspect
0   0   .059 .060 9.0 0     0  
sv-benchmarks/c/ status score program test-suite TS size inspect test-suite cpu (s) wall (s) mem (MB) energy (J) blkio-w (MB) blkio-r (MB) status program test-suite TS size inspect test-suite cov branch cov (%) line cov (%) cpu (s) wall (s) mem (MB) energy blkio-w (MB) blkio-r (MB)
total 169 124   57000 58000 4300 730000 55   9.2 169 12400 14100 1200 1200 33000 38 0  
Run set fairfuzz.test-comp19_prop-coverage-branches.ReachSafety-Arrays tbf-testsuite-validator-fairfuzz.test-comp19_prop-coverage-branches.ReachSafety-Arrays