FASE 2026
8th Competition on Software Testing (Test-Comp 2026)

Results of the Competition

This web page presents the results of Test-Comp 2026 - 8th International Competition on Software Testing.

• Competition Report for Test-Comp 2025 (previous)
• The benchmarks and rules are also available from the competition web site.

The background color is gold for the winner, silver for the second, and bronze for the third.

Here some brief directions for reading the score-based quantile plots:

• A data point (x, y) tells you that the quantile of cumulative score x can be achieved with a minimum of y tasks.
• The right end of the graph displays the achieved score. The score is the competition's definition of quality. You can read the ranking on the top from right to left.
• More information on what to learn from and how to interpret score-based quantile plots can be found in the Test-Comp 2019 report on page 844.

Here some brief directions for navigating in the BenchExec-generated tables with the results:

• Please note the header with the menu "Select Columns, Filter Rows, etc.", which loads only after all data are retrieved from the server.
• The tables contain all information that I thought you could be interested in. If there are too many columns, you can get rid of some with "Select Columns". You can also investigate special classes of rows using "Filter Rows".
• If you want to see more data, including the time and memory needed for validation, please click on the score value in the table below (works only for base categories) to browse the most detailed tables, which also contain links to obtain the test-suites that the test-generator generated.
• In all results tables, the sources of the verification task is available via click on "prog". The log files are available via click on the status (result).
• If you want to investigate even more, you can download the Test-Comp 2026 results archive, which contains all data that I obtained with BenchExec and from which I compiled all the tables and aggregated data displays. For the most detailed tables, which end on ".xml.bz2.table.html", you can obtain the raw data by accessing the URL with the ".table.html" removed.
• More on table generation is described here.
• Column "score" shows the (not normalized) score assigned to the result of the task.
• Column "test-suite" contains a link to the test-suite archive for download/display.
• Column "inspect test-suite" opens a new page that displays information about the test-suite an a list of test-suites that other test-generators produced for the same program.
• Column "cpu" shows the consumed CPU time.
• Column "wall" shows the elapsed wall time.
• Column "energy" shows the energy consumed by the CPU.
• Column "mem" shows the peak memory usage.
• Column "blkio-w" shows the accumulated amount of information written to block devices.
• Column "blkio-r" shows the accumulated amount of information read from block devices.
• More information on the results are available in the FAQ.

Note on meta-categories: The score is not the sum of scores of the sub-categories (normalization). The run time is the sum of run times of the sub-categories, rounded to two significant digits.

Ranking by Category (with Score-Based Quantile Plots)

C.Cover-Error 1. FuSeBMC 2. Symbiotic 3. Fizzer	C.Cover-Branches 1. FuSeBMC 2. FDSE 3. Fizzer
C.Overall 1. FuSeBMC 2. FDSE 3. Fizzer

Table of All Results

In every table cell for competition results, we list the points in the first row and the CPU time (rounded to two significant digits) for successful runs in the second row.

The entry '–' means that the competition candidate opted out or was not executed in the category.
The definition of the scoring schema and the categories is given on the respective competition web pages.

Hide base categories

Participants	Plots	AFL-to-Test-Case	cetfuzz	CoVeriTest	ESBMC-incr	ESBMC-kind	FDSE	Fizzer	FuSeBMC	FuSeBMC-AI	HybridTiger	KLEE	KLEEF	Owi	PRTest	Rizzer	Sikraken	Symbiotic	TracerX	TracerX-WP	UTestGen	WASP-C
Representing Jury Member		Henrik Wachowitz	Hors Concours	Marie-Christine Jakobs	Chenfeng Wei	Chenfeng Wei	Zhenbang Chen	Marek Trtík	Kaled Alshmrany	Hors Concours	Hors Concours	Hors Concours	Hors Concours	Hors Concours	Thomas Lemberger	Hors Concours	Christophe Meudec	Martin Jonáš	Hors Concours	Hors Concours	Max Barth	Hors Concours
Affiliation		LMU Munich, Germany	--, --	LMU Munich, Germany	University of Manchester, UK	University of Manchester, UK	National University of Defense Technology, China	Masaryk University, Brno, Czechia	Institute of Public Administration, Saudi Arabia	--, --	--, --	--, --	--, --	--, --	LMU Munich, Germany	--, --	South East Technological University, Ireland	Masaryk University, Brno, Czechia	--, --	--, --	LMU Munich, Germany	--, --
C.Cover-Error 1895 valid tasks (9 void)		1082	556	853	445	472	1103	1123	1486	1338	727	1213	1413	383	331	997	—	1151	664	543	629	818
CPU time		520000  s	340000  s	1200000  s	700000  s	600000  s	1400000  s	830000  s	1100000  s	390000  s	1300000  s	220000  s	780000  s	590000  s	1200000  s	1100000  s		570000  s	400000  s	470000  s	810000  s	500000  s
C.coverage-error-call.Arrays 93 valid tasks (7 void)		89	6	66	62	62	89	73	92	75	62	82	82	38	30	75	—	67	47	42	70	81
CPU time		4100  s	5000  s	32000  s	10000  s	8700  s	83000  s	51000  s	22000  s	22000  s	62000  s	3300  s	11000  s	12000  s	26000  s	67000  s		21000  s	16000  s	18000  s	50000  s	7600  s
C.coverage-error-call.BitVectors 9 valid tasks		8	6	7	2	2	8	7	9	9	5	8	9	5	5	7	—	8	4	2	4	7
CPU time		950  s	2400  s	1700  s	930  s	950  s	8000  s	4500  s	880  s	910  s	2500  s	900  s	3  s	940  s	7200  s	4500  s		950  s	220  s	240  s	5200  s	1200  s
C.coverage-error-call.ControlFlow 5 valid tasks (2 void)		5	4	2	1	2	2	5	5	2	1	5	5	1	0	3	—	5	3	2	2	5
CPU time		60  s	3200  s	2300  s	2000  s	2200  s	3400  s	1700  s	440  s	530  s	3800  s	3  s	5  s	12  s	1  s	1100  s		120  s	2  s	2  s	4500  s	110  s
C.coverage-error-call.ECA 18 valid tasks		14	7	3	0	0	7	8	15	9	1	14	13	2	0	8	—	15	6	0	0	4
CPU time		6100  s	11000  s	10000  s	6200  s	6300  s	12000  s	15000  s	5600  s	5800  s	13000  s	4100  s	5600  s	13000  s	16000  s	14000  s		2000  s	2700  s	9900  s	12000  s	14000  s
C.coverage-error-call.Floats 33 valid tasks		15	5	25	21	21	31	33	33	33	22	8	28	3	3	18	—	0	0	0	9	23
CPU time		10000  s	5700  s	6300  s	3600  s	3600  s	29000  s	1000  s	4700  s	3900  s	6800  s	5000  s	4200  s	16000  s	30000  s	140  s		1800  s	52  s	13  s	18000  s	4300  s
C.coverage-error-call.Heap 53 valid tasks		48	20	48	23	25	45	52	51	51	42	51	51	39	13	51	—	52	37	34	44	41
CPU time		1300  s	8100  s	20000  s	2900  s	2900  s	45000  s	32000  s	3700  s	3700  s	14000  s	2000  s	2700  s	1900  s	28000  s	29000  s		1100  s	2000  s	5500  s	24000  s	1700  s
C.coverage-error-call.Loops 123 valid tasks		112	66	71	18	18	87	114	121	114	50	78	85	67	82	112	—	77	57	66	92	106
CPU time		4000  s	9900  s	85000  s	31000  s	33000  s	110000  s	32000  s	32000  s	37000  s	78000  s	37000  s	37000  s	38000  s	110000  s	34000  s		35000  s	25000  s	15000  s	41000  s	6200  s
C.coverage-error-call.ProductLines 169 valid tasks		135	169	165	51	51	167	169	169	169	49	169	169	48	92	166	—	159	121	127	104	5
CPU time		590  s	63000  s	36000  s	290  s	260  s	150000  s	120000  s	1000  s	1200  s	140000  s	84  s	100  s	1900  s	150000  s	120000  s		3300  s	200  s	2100  s	110000  s	1100  s
C.coverage-error-call.Recursive 36 valid tasks		18	12	8	18	18	21	21	35	35	5	18	33	15	8	17	—	19	12	4	14	16
CPU time		14000  s	25000  s	27000  s	14000  s	14000  s	29000  s	22000  s	3600  s	4300  s	20000  s	8100  s	3300  s	17000  s	30000  s	16000  s		4700  s	11000  s	22000  s	19000  s	17000  s
C.coverage-error-call.Sequentialized 106 valid tasks		55	0	61	0	0	76	85	103	101	91	86	97	0	0	49	—	79	45	52	57	51
CPU time		52000  s	190  s	46000  s	12000  s	12000  s	84000  s	92000  s	16000  s	13000  s	93000  s	6500  s	12000  s	48000  s	95000  s	78000  s		14000  s	15000  s	9800  s	59000  s	45000  s
C.coverage-error-call.XCSP 59 valid tasks		0	0	50	0	0	55	0	53	53	52	37	53	1	0	0	—	46	19	18	0	53
CPU time		53000  s	490  s	10000  s	6000  s	5900  s	53000  s	32000  s	7400  s	7000  s	29000  s	2900  s	8300  s	18000  s	53000  s	38000  s		19000  s	22000  s	19000  s	12000  s	7000  s
C.coverage-error-call.Hardware 497 valid tasks		65	26	91	46	46	392	43	360	290	37	322	362	0	57	41	—	49	57	51	18	35
CPU time		370000  s	4500  s	380000  s	140000  s	140000  s	440000  s	410000  s	410000  s	200000  s	410000  s	140000  s	140000  s	410000  s	420000  s	420000  s		130000  s	300000  s	360000  s	420000  s	380000  s
C.coverage-error-call.SoftwareSystems-BusyBox 4 valid tasks		0	0	0	0	0	0	0	0	0	0	1	1	0	0	0	—	1	0	0	0	0
CPU time		6  s	8  s	2500  s	3600  s	3600  s	3400  s	2700  s	3600  s	360  s	3100  s	1800  s	2700  s	4  s	2700  s	1700  s		1800  s	150  s	980  s	2700  s	48  s
C.coverage-error-call.SoftwareSystems-coreutils 29 valid tasks		25	18	16	23	23	22	20	29	29	27	19	25	2	6	26	—	27	29	25	18	6
CPU time		6100  s	20000  s	22000  s	6600  s	6600  s	26000  s	15000  s	4900  s	6200  s	15000  s	6000  s	7200  s	3600  s	10000  s	20000  s		2300  s	1700  s	3600  s	17000  s	150  s
C.coverage-error-call.SoftwareSystems-OpenBSD 1 valid tasks		1	0	0	0	0	0	1	1	1	0	1	1	0	0	1	—	1	0	0	0	0
CPU time		3  s	2  s	5  s	0  s	0  s	870  s	5  s	1  s	10  s	5  s	0  s	1  s	1  s	35  s	3  s		1  s	1  s	1  s	38  s	11  s
C.coverage-error-call.SoftwareSystems-DeviceDriversLinux64 2 valid tasks		0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	—	0	0	0	0	0
CPU time		1800  s	22  s	1500  s	1800  s	1800  s	1800  s	1800  s	1800  s	1800  s	1800  s	1800  s	1800  s	1800  s	450  s	1700  s		1800  s	1800  s	1800  s	1800  s	53  s
C.coverage-error-call.SoftwareSystems-Intel-TDX-Module 658 valid tasks		0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	—	0	0	0	0	0
CPU time		1900  s	190000  s	550000  s	450000  s	350000  s	370000  s	1900  s	540000  s	75000  s	390000  s	440  s	540000  s	1700  s	190000  s	230000  s		340000  s	200  s	170  s	20000  s	14000  s
C.Cover-Branches 14322 valid tasks (52 void)		6817	3584	7084	1378	2508	8121	7967	8237	5960	5688	4476	8282	3411	4443	—	4460	6192	4864	4679	5643	3844
CPU time		12000000  s	2100000  s	8100000  s	11000000  s	6900000  s	12000000  s	9200000  s	13000000  s	5100000  s	7100000  s	3700000  s	8600000  s	8900000  s	12000000  s		12000000  s	4000000  s	2300000  s	2700000  s	7000000  s	4500000  s
C.coverage-branches.Arrays 462 valid tasks (26 void)		441	100	313	15	69	375	377	391	316	306	154	378	147	176	—	72	303	218	209	293	292
CPU time		420000  s	150000  s	310000  s	390000  s	320000  s	330000  s	230000  s	420000  s	170000  s	310000  s	62000  s	260000  s	130000  s	190000  s		410000  s	350000  s	100000  s	95000  s	310000  s	250000  s
C.coverage-branches.BitVectors 62 valid tasks		44	37	49	19	19	49	48	50	36	16	33	47	29	33	—	20	46	36	29	43	34
CPU time		56000  s	34000  s	19000  s	25000  s	25000  s	55000  s	38000  s	56000  s	22000  s	7600  s	22000  s	21000  s	35000  s	55000  s		54000  s	35000  s	24000  s	20000  s	27000  s	30000  s
C.coverage-branches.ControlFlow 98 valid tasks		61	50	57	3	4	64	65	66	29	28	40	66	22	18	—	10	61	44	45	52	47
CPU time		88000  s	66000  s	50000  s	70000  s	68000  s	83000  s	71000  s	86000  s	32000  s	53000  s	46000  s	41000  s	35000  s	46000  s		87000  s	49000  s	34000  s	38000  s	49000  s	35000  s
C.coverage-branches.ECA 32 valid tasks		12	9	6	0	0	11	8	13	6	3	8	14	3	2	—	11	13	10	2	3	4
CPU time		29000  s	21000  s	25000  s	25000  s	25000  s	27000  s	25000  s	29000  s	15000  s	23000  s	28000  s	24000  s	25000  s	29000  s		29000  s	14000  s	15000  s	15000  s	20000  s	26000  s
C.coverage-branches.Floats 852 valid tasks		175	48	251	85	87	550	551	498	301	482	29	536	31	102	—	128	136	67	65	122	63
CPU time		510000  s	120000  s	570000  s	410000  s	550000  s	750000  s	250000  s	760000  s	330000  s	540000  s	16000  s	340000  s	480000  s	770000  s		770000  s	27000  s	12000  s	11000  s	560000  s	49000  s
C.coverage-branches.Hardness 6789 valid tasks (11 void)		3016	192	4467	46	2813	5388	5545	5172	2250	4497	1573	5635	115	1103	—	4213	693	338	625	4773	231
CPU time		6100000  s	37000  s	3700000  s	5500000  s	2900000  s	6100000  s	5800000  s	6100000  s	2600000  s	3000000  s	1600000  s	5100000  s	5600000  s	6100000  s		6100000  s	640000  s	290000  s	440000  s	3400000  s	1700000  s
C.coverage-branches.Hardware 1186 valid tasks		118	704	982	7	64	1058	936	976	374	698	317	975	1	971	—	201	111	534	786	446	19
CPU time		1100000  s	9600  s	930000  s	1000000  s	970000  s	1100000  s	1000000  s	1100000  s	450000  s	960000  s	760000  s	940000  s	1000000  s	1100000  s		1100000  s	240000  s	710000  s	820000  s	1000000  s	1000000  s
C.coverage-branches.Heap 384 valid tasks		228	93	203	37	89	220	229	228	203	168	191	233	128	113	—	79	214	147	141	181	132
CPU time		350000  s	110000  s	140000  s	230000  s	140000  s	330000  s	200000  s	340000  s	140000  s	120000  s	160000  s	190000  s	100000  s	250000  s		310000  s	160000  s	70000  s	80000  s	130000  s	84000  s
C.coverage-branches.Loops 773 valid tasks		600	279	609	166	194	623	624	628	505	495	414	620	432	376	—	551	578	409	407	600	455
CPU time		680000  s	270000  s	460000  s	390000  s	360000  s	660000  s	250000  s	700000  s	280000  s	410000  s	220000  s	230000  s	250000  s	680000  s		680000  s	300000  s	160000  s	160000  s	250000  s	210000  s
C.coverage-branches.ProductLines 263 valid tasks		57	66	77	40	40	76	77	77	60	55	74	77	59	48	—	5	69	50	57	64	1
CPU time		240000  s	140000  s	52000  s	60000  s	60000  s	230000  s	160000  s	200000  s	90000  s	220000  s	10000  s	5200  s	8700  s	240000  s		240000  s	8200  s	9800  s	31000  s	170000  s	1700  s
C.coverage-branches.Recursive 138 valid tasks		95	42	94	9	9	100	100	103	88	87	59	104	68	63	—	49	91	64	59	92	71
CPU time		120000  s	65000  s	120000  s	110000  s	110000  s	120000  s	92000  s	120000  s	53000  s	90000  s	73000  s	77000  s	81000  s	110000  s		120000  s	88000  s	49000  s	64000  s	78000  s	90000  s
C.coverage-branches.Sequentialized 108 valid tasks		72	2	81	20	20	86	87	94	41	58	37	94	3	17	—	54	54	38	41	65	64
CPU time		97000  s	3400  s	64000  s	41000  s	41000  s	89000  s	91000  s	97000  s	40000  s	75000  s	29000  s	42000  s	48000  s	97000  s		96000  s	45000  s	37000  s	27000  s	66000  s	43000  s
C.coverage-branches.XCSP 119 valid tasks		104	0	116	78	78	117	102	107	105	119	102	118	36	102	—	119	78	59	55	114	79
CPU time		110000  s	1300  s	4800  s	49000  s	49000  s	110000  s	81000  s	110000  s	53000  s	5900  s	15000  s	22000  s	30000  s	110000  s		110000  s	30000  s	34000  s	33000  s	19000  s	16000  s
C.coverage-branches.Combinations 671 valid tasks		428	98	233	73	73	417	325	406	252	170	202	447	109	73	—	376	401	254	149	130	152
CPU time		600000  s	110000  s	540000  s	530000  s	530000  s	600000  s	520000  s	600000  s	260000  s	500000  s	470000  s	450000  s	550000  s	600000  s		600000  s	360000  s	280000  s	380000  s	410000  s	530000  s
C.coverage-branches.SoftwareSystems-AWS-C-Common 352 valid tasks (4 void)		33	5	33	15	15	33	37	36	25	14	29	37	8	8	—	8	29	24	22	20	0
CPU time		320000  s	23000  s	190000  s	89000  s	85000  s	240000  s	59000  s	300000  s	130000  s	100000  s	28000  s	18000  s	2600  s	310000  s		320000  s	23000  s	6400  s	1800  s	270000  s	5100  s
C.coverage-branches.SoftwareSystems-BusyBox 75 valid tasks (2 void)		0	0	6	0	0	19	28	27	14	6	26	31	0	16	—	0	25	15	17	6	0
CPU time		84  s	120  s	67000  s	63000  s	44000  s	49000  s	66000  s	62000  s	30000  s	48000  s	53000  s	59000  s	83  s	66000  s		68000  s	50000  s	7100  s	19000  s	65000  s	720  s
C.coverage-branches.SoftwareSystems-coreutils 143 valid tasks (9 void)		24	9	23	1	9	28	20	28	19	23	20	27	0	9	—	2	22	21	20	18	6
CPU time		130000  s	120000  s	120000  s	110000  s	89000  s	38000  s	20000  s	130000  s	54000  s	23000  s	23000  s	45000  s	6300  s	14000  s		39000  s	31000  s	22000  s	20000  s	48000  s	36000  s
C.coverage-branches.SoftwareSystems-OpenBSD 5 valid tasks		0	0	0	0	0	0	0	0	0	0	0	0	0	0	—	0	0	0	0	0	0
CPU time		4500  s	10  s	24  s	0  s	1  s	4400  s	18  s	3100  s	1  s	24  s	2  s	4  s	5  s	470  s		4  s	10  s	4  s	5  s	180  s	57  s
C.coverage-branches.SoftwareSystems-DeviceDriversLinux64 279 valid tasks		59	50	61	0	27	59	60	60	55	5	26	59	0	19	—	9	43	55	56	57	0
CPU time		250000  s	180000  s	34000  s	250000  s	34000  s	250000  s	160000  s	250000  s	120000  s	23000  s	52000  s	46000  s	3000  s	240000  s		97000  s	230000  s	37000  s	43000  s	51000  s	1500  s
C.coverage-branches.SoftwareSystems-Intel-TDX-Module 836 valid tasks		0	8	9	0	2	58	53	61	23	3	4	41	1	11	—	4	2	4	4	0	0
CPU time		2500  s	230000  s	600000  s	590000  s	410000  s	500000  s	2400  s	670000  s	110000  s	480000  s	560  s	600000  s	2100  s	240000  s		740000  s	750000  s	250  s	220  s	25000  s	17000  s
C.coverage-branches.SoftwareSystems-uthash 9 valid tasks		6	6	5	0	0	6	6	6	5	0	1	6	5	6	—	5	6	0	0	4	0
CPU time		8100  s	7300  s	7300  s	4600  s	4500  s	7900  s	5700  s	8100  s	4300  s	4700  s	6300  s	5700  s	5600  s	8100  s		8100  s	5900  s	1  s	1  s	350  s	56  s
C.coverage-branches.SoftwareSystems-SQLite 1 valid tasks		0	0	0	0	0	0	0	0	0	0	0	0	0	0	—	0	0	0	0	0	0
CPU time		900  s	110  s	900  s	900  s	900  s	3  s	870  s	900  s	900  s	900  s	30  s	430  s	13  s	900  s		2  s	100  s	7  s	7  s	76  s	170  s
C.coverage-branches.Termination-BitVectors 27 valid tasks		26	21	27	9	14	27	27	27	25	27	15	26	20	22	—	22	24	25	26	16	13
CPU time		24000  s	20000  s	180  s	14000  s	17  s	24000  s	610  s	24000  s	950  s	270  s	5  s	7  s	18000  s	23000  s		22000  s	13000  s	9800  s	12000  s	630  s	8900  s
C.coverage-branches.Termination-MainControlFlow 407 valid tasks		352	256	383	26	187	388	387	388	352	375	223	376	309	269	—	345	331	342	328	221	208
CPU time		370000  s	290000  s	46000  s	330000  s	68000  s	360000  s	49000  s	370000  s	45000  s	42000  s	18000  s	48000  s	330000  s	350000  s		360000  s	320000  s	260000  s	280000  s	17000  s	220000  s
C.coverage-branches.Termination-MainHeap 251 valid tasks		228	77	238	5	120	232	235	239	218	220	136	230	152	91	—	67	210	213	207	185	207
CPU time		220000  s	91000  s	25000  s	220000  s	34000  s	120000  s	43000  s	230000  s	72000  s	23000  s	8500  s	32000  s	83000  s	120000  s		220000  s	200000  s	86000  s	84000  s	9500  s	130000  s
C.Overall 16217 valid tasks (61 void)		8488	4410	7660	2685	3438	9319	9317	11024	9099	6329	7723	10735	3569	3932	—	—	8429	5594	4974	5888	5674
CPU time		12000000  s	2400000  s	9300000  s	11000000  s	7500000  s	14000000  s	10000000  s	14000000  s	5500000  s	8400000  s	3900000  s	9300000  s	9500000  s	13000000  s			4600000  s	2700000  s	3100000  s	7800000  s	5000000  s
Participants	Plots	AFL-to-Test-Case	cetfuzz	CoVeriTest	ESBMC-incr	ESBMC-kind	FDSE	Fizzer	FuSeBMC	FuSeBMC-AI	HybridTiger	KLEE	KLEEF	Owi	PRTest	Rizzer	Sikraken	Symbiotic	TracerX	TracerX-WP	UTestGen	WASP-C

Note on meta-categories: The score is not the sum of scores of the sub-categories (normalization). The run time is the sum of run times of the sub-categories, rounded to two significant digits.

If you did not find what you are looking for, please do not hesitate to contact Dirk Beyer (competition chair).