{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "a1b2c3d4",
   "metadata": {},
   "source": [
    "# Benchmark Test Problems\n",
    "\n",
    "This notebook introduces the benchmark test evaluators available in the Standard Evaluator library. These evaluators implement well-known mathematical optimization problems with known properties, making them ideal for testing optimization algorithms, validating surrogate models, and demonstrating library functionality. You will learn how to browse the catalog of available benchmarks, instantiate and evaluate them, inspect their problem definitions, and access known optimal solutions."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e5f6a7b8",
   "metadata": {},
   "source": [
    "## Benchmark Evaluator Catalog\n",
    "\n",
    "All benchmark test evaluators are available under `se.evaluators.test`. They are organized into the following categories:\n",
    "\n",
    "### Unconstrained Optimization\n",
    "\n",
    "| Class Name | Design Variables |\n",
    "|---|---|\n",
    "| `CosineTensorProduct` | User defined |\n",
    "| `ExponentialTensorProduct` | User defined |\n",
    "| `HyperbolicTangentTensorProduct` | User defined |\n",
    "| `Sphere` | User defined |\n",
    "| `Rosenbrock` | User defined, >=2 |\n",
    "| `ExtendedRosenbrock` | 2 |\n",
    "| `Trigonometric` | 2 |\n",
    "| `HelicalValley` | 3 |\n",
    "| `HS38` | 4 |\n",
    "| `PowellSingularFunction` | 4 |\n",
    "\n",
    "### Constrained Optimization\n",
    "\n",
    "| Class Name | Design Variables | Constraints |\n",
    "|---|---|---|\n",
    "| `ConstrainedBetts` | 2 | 1 |\n",
    "| `WrkBkPrb1` | 2 | 1 |\n",
    "| `G6Problem` | 2 | 2 |\n",
    "| `OptlibTest` | 3 | 1 |\n",
    "| `TP37` | 3 | 2 |\n",
    "| `CantileveredBeam` | 4 | 3 |\n",
    "| `CantileveredBeamContinuous` | 4 | 3 |\n",
    "| `HS47` | 5 | 3 |\n",
    "| `TwoBarTruss` | 5 | 2 |\n",
    "| `CantileveredBeamFixedVariable` | 6 | 3 |\n",
    "| `HS100` | 7 | 4 |\n",
    "| `SpeedReducer` | 7 | 11 |\n",
    "| `G7Problem` | 10 | 8 |\n",
    "| `HS118` | 15 | 17 |\n",
    "\n",
    "### Multi-Fidelity\n",
    "\n",
    "| Class Name | Design Variables |\n",
    "|---|---|\n",
    "| `ForresterMultiFiHi` / `ForresterMultiFiLo` | 1 |\n",
    "| `ExponentialMultiFiHi` / `ExponentialMultiFiLo` | 2 |\n",
    "| `SimpleMultiFiHi` / `SimpleMultiFiMid` / `SimpleMultiFiLo` | 2 |\n",
    "| `BoreholeMultiFiHi` / `BoreholeMultiFiLo` | 8 |\n",
    "\n",
    "### Feasibility-Discovery\n",
    "\n",
    "| Class Name | Design Variables | Constraints |\n",
    "|---|---|---|\n",
    "| `SmallCircleFeasibleRegion` | 2 | 1 |\n",
    "| `DisconnectedFeasibleRegions` | 2 | 1 |\n",
    "| `G6Problem` | 2 | 2 |\n",
    "| `G7Problem` | 10 | 8 |\n",
    "| `SpeedReducer` | 7 | 11 |"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c9d0e1f2",
   "metadata": {},
   "source": [
    "## Instantiating and Evaluating Benchmarks\n",
    "\n",
    "Let's instantiate one evaluator from each of three categories (unconstrained, constrained, and feasibility-discovery) and evaluate each at two sample points."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "a3b4c5d6",
   "metadata": {},
   "outputs": [],
   "source": [
    "import standard_evaluator as se\n",
    "import pandas as pd"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e7f8a9b0",
   "metadata": {},
   "source": [
    "### Unconstrained: Rosenbrock (2 variables)\n",
    "\n",
    "The Rosenbrock function is a classic non-convex optimization test problem. We evaluate it at two points."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "c1d2e3f4",
   "metadata": {},
   "outputs": [],
   "source": [
    "rosenbrock = se.evaluators.test.Rosenbrock()\n",
    "df_rosen = pd.DataFrame({\"x0\": [0.0, 1.0], \"x1\": [0.0, 1.0]})\n",
    "rosenbrock(df_rosen)\n",
    "df_rosen"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a5b6c7d8",
   "metadata": {},
   "source": [
    "The response column `f` shows the Rosenbrock function value at each point. The global minimum is `f=0` at `(1, 1)`.\n",
    "\n",
    "### Constrained: ConstrainedBetts (2 variables, 1 constraint)\n",
    "\n",
    "This problem includes both an objective `f` and a constraint `c1`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "e9f0a1b2",
   "metadata": {},
   "outputs": [],
   "source": [
    "betts = se.evaluators.test.ConstrainedBetts()\n",
    "df_betts = pd.DataFrame({\"x1\": [2.0, 10.0], \"x2\": [0.0, 5.0]})\n",
    "betts(df_betts)\n",
    "df_betts"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c3d4e5f6",
   "metadata": {},
   "source": [
    "The output DataFrame includes the objective `f` and the constraint `c1`. A feasible solution requires the constraint to satisfy its bounds.\n",
    "\n",
    "### Feasibility-Discovery: SmallCircleFeasibleRegion (2 variables, 1 constraint)\n",
    "\n",
    "This evaluator has no objective. It defines only a constraint that determines feasibility."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "a7b8c9d0",
   "metadata": {},
   "outputs": [],
   "source": [
    "circle = se.evaluators.test.SmallCircleFeasibleRegion()\n",
    "df_circle = pd.DataFrame({\"x1\": [0.5, -0.3], \"x2\": [-0.3, -0.3]})\n",
    "circle(df_circle)\n",
    "df_circle"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e1f2a3b4",
   "metadata": {},
   "source": [
    "The constraint `g1` is satisfied (feasible) when `g1 <= 0`. Points inside the small circle centered at `(0.5, -0.3)` with radius `0.15` are feasible."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c5d6e7f8",
   "metadata": {},
   "source": [
    "## Feasibility-Discovery Benchmarks\n",
    "\n",
    "The following five benchmarks are particularly useful for testing algorithms that need to discover feasible regions in the design space:\n",
    "\n",
    "### SmallCircleFeasibleRegion\n",
    "- **Variables:** 2 (`x1`, `x2` in [-1, 1])\n",
    "- **Constraints:** 1 (`g1 <= 0`)\n",
    "- **Purpose:** Tests an algorithm's ability to find a small circular feasible region (radius 0.15) in a 2D space. The feasible region occupies a tiny fraction of the design space.\n",
    "\n",
    "### DisconnectedFeasibleRegions\n",
    "- **Variables:** 2 (`x1`, `x2` in [-1, 1])\n",
    "- **Constraints:** 1 (`g1 <= 0`)\n",
    "- **Purpose:** Tests an algorithm's ability to discover multiple disconnected circular feasible islands. Algorithms must explore broadly to find all feasible regions rather than converging to a single one.\n",
    "\n",
    "### G6Problem\n",
    "- **Variables:** 2 (`x1` in [13, 100], `x2` in [0, 100])\n",
    "- **Constraints:** 2 (`g1 <= 0`, `g2 <= 0`)\n",
    "- **Purpose:** Features a narrow crescent-shaped feasible region defined by the intersection of two nonlinear constraints. Tests an algorithm's ability to navigate tight, curved feasible boundaries.\n",
    "\n",
    "### G7Problem\n",
    "- **Variables:** 10 (`x1` to `x10` in [-10, 10])\n",
    "- **Constraints:** 8 (`g1` to `g8 <= 0`)\n",
    "- **Purpose:** A high-dimensional problem with 8 nonlinear inequality constraints. Tests feasibility discovery in a 10D space where the feasible region is defined by many interacting constraints.\n",
    "\n",
    "### SpeedReducer\n",
    "- **Variables:** 7 (`x1` to `x7` with mixed bounds)\n",
    "- **Constraints:** 11 (`g1` to `g11 <= 0`)\n",
    "- **Purpose:** A real-world engineering design problem (gear box weight minimization) with 11 constraints on stress, deflection, and geometry. Tests feasibility discovery under many engineering constraints simultaneously."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a9b0c1d2",
   "metadata": {},
   "source": [
    "## Accessing the OptProblem Definition\n",
    "\n",
    "Each benchmark evaluator has an `opt_problem` property that provides the full optimization problem definition, including variables with bounds, responses, objectives, and constraints."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "e3f4a5b6",
   "metadata": {},
   "outputs": [],
   "source": [
    "g6 = se.evaluators.test.G6Problem()\n",
    "opt = g6.opt_problem\n",
    "\n",
    "print(\"Variables:\")\n",
    "for v in opt.variables:\n",
    "    print(f\"  {v.name}: bounds={v.bounds}\")\n",
    "\n",
    "print(\"\\nResponses:\")\n",
    "for r in opt.responses:\n",
    "    print(f\"  {r.name}\")\n",
    "\n",
    "print(f\"\\nObjectives: {opt.objectives}\")\n",
    "print(f\"Constraints: {opt.constraints}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c7d8e9f0",
   "metadata": {},
   "source": [
    "The `OptProblem` tells us the G6 problem has 2 design variables with specific bounds, 3 responses (one objective and two constraints), and that we are minimizing `f` subject to `g1 <= 0` and `g2 <= 0`."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a1b2c3e4",
   "metadata": {},
   "source": [
    "## Known Solutions\n",
    "\n",
    "Many benchmark evaluators provide a `known_solution` property that returns a DataFrame containing the optimal input values and corresponding response values. This is useful for validating optimization results."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "e5f6a7c8",
   "metadata": {},
   "outputs": [],
   "source": [
    "print(\"G6Problem known solution:\")\n",
    "g6.known_solution"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c9d0e1g2",
   "metadata": {},
   "source": [
    "The known solution DataFrame shows the optimal design point and the corresponding objective and constraint values. For the G6 problem, the minimum of `f` is approximately -6961.81 at `x1=14.095`, `x2=0.84296`, with both constraints nearly active (close to zero).\n",
    "\n",
    "Not all benchmarks have known solutions. Feasibility-only problems (like `SmallCircleFeasibleRegion`) return `None` since they have no objective to optimize."
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": ".venv",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "name": "python",
   "version": "3.12.8"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}