As per the challenge rules, these entries are not
eligible for prizes, but do modify the scoring results.
Furthermore, entries in the FD search category (BProlog, Gecode, JaCoP,
Chuffed, CPX and G12/FD) were automatically included in the free search
category, while entries in the free search category (Bumblebee, Fzn2smt, SCIP,
CBC, CPLEX, Gurobi and promoted FD entries) were automatically included in the
parallel search category.
Summary of Results
The results for the MiniZinc Challenge 2011 were
Fixed search:
Gold medal: Gecode
Silver medal: JaCoP
Bronze medal: BProlog
Free search:
Gold medal: Gecode
Silver medal: fzn2smt
Bronze medal: JaCoP
Parallel search:
Gold medal: Gecode
Silver medal: fzn2smt
Bronze medal: JaCoP
Results Presentation
The slides for the presentation of the results at CP2011 are
here in [PDF]
Description of Results
All times are given in seconds.
Scores of 0, 1 and 2 are used in the tables rather than 0, 0.5 and 1.
If a promoted entry does not recognize an option (or states that it is just
ignored), times and solutions from the previous category are used for scoring.
The suffixes -fd, -free and -par at the end of the solvers names indicate
which configuration the solvers were run with.
mzn2fzn was run with the same time and memory limits as the solvers.
In the Status column:
S and U indicates whether a solution was found (S - Solved)
or not (U - Unsolved).
C and U indicates whether the search was complete (C) or
uncomplete (U).
E indicates an error of any kind,
or an incorrect answer from the solver.
Incorrect answers:
Bumblebee returns UC (unsatisfiable) for all instances of cyclic-rcpsp.
Bumblebee returns UC (unsatisfiable) for all instances of wwtpp-real, in
around 70 seconds for each of them. We have decided to count all 5 instances
results as an error (effectively disqualified bumblebee on that problem).
SCIP completes the search on 8Ships (ship-scheduling) with a suboptimal answer.
cpx-par finds a better-than-optimal solution for 6ShipsMixedUnconst
(ship-schedule).
Errors:
SCIP aborts on the carpet-cutting instances when parsing the generated
FlatZinc.
(We checked the FlatZinc files manually and they are valid.)
Bumblebee aborts on the vrp, four prize-collecting instances,
ship-scheduling and table layout with a compiler failed error.
Linearisation aborts (CBC, Cplex, Gurobi) on carpet-cutting,
open stacks and pentominoes.
CBC/Cplex/Gurobi aborts on pattern set mining due to a lack of
support for the FlatZinc built-in bool_le_reif/3.
SCIP prints out the complete search terminator even for SAT instances.
It did not affect the scoring.
Other errors are mostly due to memory exhaustion.
Selection:
Select a list of solvers and benchmarks and click on "Compute Results" to score
the solvers against each other on the selected benchmarks.
The entrants for each of the fd search, free search and parallel search
categories can be selected with the corresponding buttons.
Solver selection:
Problem selection:
Summary:
Total per problem:
Solver
Score
Problem
Solver
Score
Individual results:
Problem
Instance
Solver
Status
Time
Objective
Score
Global constraint per model
The following table lists the global constraints used by each model
in this year's challenge.
Model
Global constraints used
bacp
black-hole
inverse, alldifferent
carpet-cutting
cumulative, diffn
costas-array
alldifferent
cyclic-rcpsp
cumulative
depot-placement
alldifferent
fast-food
fillomino
grid-colouring
nonogram
regular
open-stacks
alldifferent
pattern-set-mining
lex_less
pentominoes
regular
prize-collecting
roster
at_most, at_least, exactly
ship-scheduling
solbat
table layout
vrp
wwtp_real
The files on this page are for MiniZinc version 1.3.