MiniZinc - Challenge 2014 Results

MiniZinc Challenge 2014 Results

Entrants

The entrants for this year (with their descriptions, when provided):

Choco 3 (description). A Java FD solver.
Concrete (description). A CP system written in Scala.
HaifaCSP (description). A C++ FD solver using SAT solving algorithms.
iZplus (description).
JaCoP (description). A Java FD solver.
MinisatID (description). A solver combining techniques from the fields SAT, SMT, CP, and ASP.
Mistral 2.0 (description). A C++ FD solver.
Opturion CPX (description). A FD solver using Lazy clause generation.
OR-Tools (description).
Picat CP (description).
Picat SAT (description).
SICStus Prolog (description). A Prolog development environment with a FD constraint programming module.

In addition, the challenge organisers entered the following FlatZinc implementations:

Chuffed (description). A C++ FD solver using lazy clause generation.
G12/FD. A Mercury FD solver (the G12 FlatZinc interpreter's default solver).
G12/LazyFD. A Mercury FD solver using lazy clause generation.
Gecode (description). A C++ FD solver.
sunny-cp (description). A single-threaded CP portfolio solver using, e.g., Chuffed, G12 solvers, Gecode, MinisatID.
sunny-cp-presolve. sunny-cp with pre-solving.

As per the challenge rules, these entries are not eligible for prizes, but do modify the scoring results. Furthermore, entries in the FD search category (JaCoP and SICStus Prolog) were automatically included in the free search category, while entries in the free search category (Concrete, Chuffed, G12/FD, G12/LazyFD, HaifaCSP, MinisatID, Mistral, Opturion CPX, Picat CP, Picat SAT and promoted FD entries) were automatically included in the parallel search category. Lastly, all entries in the free search category and promoted entries into that category were automatically included in the open search category.

Summary of Results

The results for the MiniZinc Challenge 2014 were

Category	Gold	Silver	Bronze
Fixed	OR-Tools	Opturion CPX	SICStus Prolog
Free	iZplus	Opturion CPX	Choco
Parallel	OR-Tools	Opturion CPX	Choco
Open	OR-Tools	Opturion CPX	Choco

Description of Results

All times are given in milliseconds.

A score of 0.0 indicates a worse answer in quality (worse objective, no proof of optimality, or no answer for satisfaction problems), 1.0 a better solution in quality. When the quality is the same, the 1.0 purse is split with respect to time used.

If a promoted entry does not recognize an option (or states that it is just ignored), times and solutions from the previous category are used for scoring. The suffixes -fd, -free, -par or -open (for the parallel portfolio solver entered) at the end of the solvers names indicate which configuration the solvers were run with.

mzn2fzn was run with the same time and memory limits as the solvers.

In the Status column:

S indicates that a solution was found,
C indicates that the search was complete,
INC indicates an incorrect answer
MZN indicates that flattening aborted (time-out or out-of-memory)
UNK indicates that no answer was returned in the time limit or the solver aborted.

Download all problems

All problems are available in a zipped tar-ball here.

Selection:

Select a list of solvers and benchmarks and click on “Compute Results” to score the solvers against each other on the selected benchmarks. The entrants for each of the fd search, free search and parallel search categories can be selected with the corresponding buttons.

Solver selection:	Problem selection:

Summary:

Total per problem:

Solver	Score

Problem	Solver	Score

Individual results:

Problem	Instance	Solver	Status	Time	Objective	Score

Global constraint per model

The following table lists the global constraints used by each model in this year's challenge.

Problem	Type	Kind	RC	MiniZinc Globals
amaze3	puzzle	sat	✓	count
cyclic-rcpsp	combi	sat		cumulative
elitserien	real	min		all_different, global_cardinality_closed, inverse, member, regular
fillomino	puzzle	sat
jp-encoding	real	min		count
liner-sf-repositioning	real	min		all_different, all_different_except_0
mario	puzzle	max		subcircuit
mqueens	puzzle	min		lex_lesseq
multi-knapsack	combi	max		knapsack
openshop	combi	min		cumulative
rectangle-packing-2014	combi	sat		cumulative, diffn
road_construction	combi	min
ship-schedule	real	max
smelt	puzzle	min		cumulative
solbat	combi	sat
spot5	real	min		table
stochastic-fjsp	combi	min
stochastic-vrp	combi	min		circuit
train	real	min
traveling-tppv	combi	min		all_different, regular

The files on this page are for MiniZinc version 1.6.