BlackBoxOptimizationBenchmarking.jl
A Julia implementation of the Black-Box-Optimization-Benchmarking (BBOB) functions.
Benchmark results
The average sucess rate (meaning the optimizer reached the minimum + 1e-6) in function of the number of objective function evaluations :
Since some global optimizers have poor final convergence, they were chained into a Nelder-Mead using 10% of the objective function evaluation budget.
The average sucess rate across the dimension of the function:
The total relative run time of each optimizer
Note that the Python algorithms are called from Julia, which might cause some overhead.
Functions
Functions can be accessed as BlackBoxOptimizationBenchmarking.F1
, which returns a BBOBFunction
with fields f
containing the function itself, f_opt
its minimal value, and x_opt
its minimizer, i.e. f(x_opt) = f_opt
.
Functions can be listed using enumerate
:
julia> enumerate(BBOBFunction)
20-element Array{BlackBoxOptimizationBenchmarking.BBOBFunction,1}:
Sphere
Ellipsoidal Function
Discus Function
Bent Cigar Function
Sharp Ridge Function
Different Powers Function
Rastrigin Function
Weierstrass Function
Schaffers F7 Function
Schaffers F7 Function, moderately ill-conditioned
Composite Griewank-Rosenbrock Function F8F2
Ellipsoidal
Schwefel Function
Rastrigin
Buche-Rastrigin
Linear Slope
Attractive Sector
Step Ellipsoidal Function
Rosenbrock Function, original
Rosenbrock Function, rotated
A benchmark for a single optimizer and function can be run with:
benchmark(optimizer::Any, f::BBOBFunction, run_lengths, Ntrials, dimensions, Δf)
Or for a collection of optimizers with:
benchmark(optimizers::Vector{T}, funcs, run_lengths, Ntrials, dimensions, Δf)
See test/runtests.jl
The optimizer must implement the methods optimize
, minimum
and minimizer
, see
scripts/optimizers_interface.jl
Generating new instance of the functions
To avoid overfiting and test if algorithms are robust with respect to rotations of the error function, rotation matrices are randomly generated the first time the package is used.
If needed new rotations can be generated by running the following:
@eval BlackBoxOptimizationBenchmarking begin
@memoize function Q(D)
r = randn(D); r = r/norm(r)
Q = [r nullspace(Matrix(r'))]
end
@memoize function R(D)
r = randn(D); r = r/norm(r)
R = [r nullspace(Matrix(r'))]
end
end
Reference:
http://coco.lri.fr/downloads/download15.01/bbobdocfunctions.pdf