Benchmark Test Problems

This notebook introduces the benchmark test evaluators available in the Standard Evaluator library. These evaluators implement well-known mathematical optimization problems with known properties, making them ideal for testing optimization algorithms, validating surrogate models, and demonstrating library functionality. You will learn how to browse the catalog of available benchmarks, instantiate and evaluate them, inspect their problem definitions, and access known optimal solutions.

Benchmark Evaluator Catalog

All benchmark test evaluators are available under se.evaluators.test. They are organized into the following categories:

Unconstrained Optimization

Class Name	Design Variables
`CosineTensorProduct`	User defined
`ExponentialTensorProduct`	User defined
`HyperbolicTangentTensorProduct`	User defined
`Sphere`	User defined
`Rosenbrock`	User defined, >=2
`ExtendedRosenbrock`	2
`Trigonometric`	2
`HelicalValley`	3
`HS38`	4
`PowellSingularFunction`	4

Constrained Optimization

Class Name	Design Variables	Constraints
`ConstrainedBetts`	2	1
`WrkBkPrb1`	2	1
`G6Problem`	2	2
`OptlibTest`	3	1
`TP37`	3	2
`CantileveredBeam`	4	3
`CantileveredBeamContinuous`	4	3
`HS47`	5	3
`TwoBarTruss`	5	2
`CantileveredBeamFixedVariable`	6	3
`HS100`	7	4
`SpeedReducer`	7	11
`G7Problem`	10	8
`HS118`	15	17

Multi-Fidelity

Class Name	Design Variables
`ForresterMultiFiHi` / `ForresterMultiFiLo`	1
`ExponentialMultiFiHi` / `ExponentialMultiFiLo`	2
`SimpleMultiFiHi` / `SimpleMultiFiMid` / `SimpleMultiFiLo`	2
`BoreholeMultiFiHi` / `BoreholeMultiFiLo`	8

Feasibility-Discovery

Class Name	Design Variables	Constraints
`SmallCircleFeasibleRegion`	2	1
`DisconnectedFeasibleRegions`	2	1
`G6Problem`	2	2
`G7Problem`	10	8
`SpeedReducer`	7	11

Instantiating and Evaluating Benchmarks

Let’s instantiate one evaluator from each of three categories (unconstrained, constrained, and feasibility-discovery) and evaluate each at two sample points.

import standard_evaluator as se
import pandas as pd

matlabengine not available.

Unconstrained: Rosenbrock (2 variables)

The Rosenbrock function is a classic non-convex optimization test problem. We evaluate it at two points.

rosenbrock = se.evaluators.test.Rosenbrock()
df_rosen = pd.DataFrame({"x0": [0.0, 1.0], "x1": [0.0, 1.0]})
rosenbrock(df_rosen)
df_rosen

	x0	x1	f
0	0.0	0.0	1.0
1	1.0	1.0	0.0

The response column f shows the Rosenbrock function value at each point. The global minimum is f=0 at (1, 1).

Constrained: ConstrainedBetts (2 variables, 1 constraint)

This problem includes both an objective f and a constraint c1.

betts = se.evaluators.test.ConstrainedBetts()
df_betts = pd.DataFrame({"x1": [2.0, 10.0], "x2": [0.0, 5.0]})
betts(df_betts)
df_betts

	x1	x2	f	c1
0	2.0	0.0	-99.96	20.0
1	10.0	5.0	-74.00	95.0

The output DataFrame includes the objective f and the constraint c1. A feasible solution requires the constraint to satisfy its bounds.

Feasibility-Discovery: SmallCircleFeasibleRegion (2 variables, 1 constraint)

This evaluator has no objective. It defines only a constraint that determines feasibility.

circle = se.evaluators.test.SmallCircleFeasibleRegion()
df_circle = pd.DataFrame({"x1": [0.5, -0.3], "x2": [-0.3, -0.3]})
circle(df_circle)
df_circle

	x1	x2	g1
0	0.5	-0.3	-0.0225
1	-0.3	-0.3	0.6175

The constraint g1 is satisfied (feasible) when g1 <= 0. Points inside the small circle centered at (0.5, -0.3) with radius 0.15 are feasible.

Feasibility-Discovery Benchmarks

The following five benchmarks are particularly useful for testing algorithms that need to discover feasible regions in the design space:

SmallCircleFeasibleRegion

Variables: 2 (x1, x2 in [-1, 1])
Constraints: 1 (g1 <= 0)
Purpose: Tests an algorithm’s ability to find a small circular feasible region (radius 0.15) in a 2D space. The feasible region occupies a tiny fraction of the design space.

DisconnectedFeasibleRegions

Variables: 2 (x1, x2 in [-1, 1])
Constraints: 1 (g1 <= 0)
Purpose: Tests an algorithm’s ability to discover multiple disconnected circular feasible islands. Algorithms must explore broadly to find all feasible regions rather than converging to a single one.

G6Problem

Variables: 2 (x1 in [13, 100], x2 in [0, 100])
Constraints: 2 (g1 <= 0, g2 <= 0)
Purpose: Features a narrow crescent-shaped feasible region defined by the intersection of two nonlinear constraints. Tests an algorithm’s ability to navigate tight, curved feasible boundaries.

G7Problem

Variables: 10 (x1 to x10 in [-10, 10])
Constraints: 8 (g1 to g8 <= 0)
Purpose: A high-dimensional problem with 8 nonlinear inequality constraints. Tests feasibility discovery in a 10D space where the feasible region is defined by many interacting constraints.

SpeedReducer

Variables: 7 (x1 to x7 with mixed bounds)
Constraints: 11 (g1 to g11 <= 0)
Purpose: A real-world engineering design problem (gear box weight minimization) with 11 constraints on stress, deflection, and geometry. Tests feasibility discovery under many engineering constraints simultaneously.

Accessing the OptProblem Definition

Each benchmark evaluator has an opt_problem property that provides the full optimization problem definition, including variables with bounds, responses, objectives, and constraints.

g6 = se.evaluators.test.G6Problem()
opt = g6.opt_problem

print("Variables:")
for v in opt.variables:
    print(f"  {v.name}: bounds={v.bounds}")

print("\nResponses:")
for r in opt.responses:
    print(f"  {r.name}")

print(f"\nObjectives: {opt.objectives}")
print(f"Constraints: {opt.constraints}")

Variables:
  x1: bounds=(13.0, 100.0)
  x2: bounds=(0.0, 100.0)

Responses:
  f
  g1
  g2

Objectives: ['f']
Constraints: ['g1', 'g2']

The OptProblem tells us the G6 problem has 2 design variables with specific bounds, 3 responses (one objective and two constraints), and that we are minimizing f subject to g1 <= 0 and g2 <= 0.

Known Solutions

Many benchmark evaluators provide a known_solution property that returns a DataFrame containing the optimal input values and corresponding response values. This is useful for validating optimization results.

print("G6Problem known solution:")
g6.known_solution

G6Problem known solution:

	x1	x2	f	g1	g2
0	14.095	0.84296	-6961.814744	-0.000007	0.000007

The known solution DataFrame shows the optimal design point and the corresponding objective and constraint values. For the G6 problem, the minimum of f is approximately -6961.81 at x1=14.095, x2=0.84296, with both constraints nearly active (close to zero).

Not all benchmarks have known solutions. Feasibility-only problems (like SmallCircleFeasibleRegion) return None since they have no objective to optimize.