{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "a1b2c3d4",
   "metadata": {},
   "source": [
    "# Evaluator Interface\n",
    "\n",
    "This notebook explains how to define the interface for an evaluator in the Standard Evaluator library. You will learn the two supported approaches: **OptProblem** for optimization problems with objectives and constraints, and **EvaluatorInfo** for general-purpose evaluators that only describe inputs and outputs. By the end, you will know how to create both interface types, pass them to a `NumpyEvaluator`, and access the resulting input/output metadata."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b2c3d4e5",
   "metadata": {},
   "source": [
    "## Two Interface Approaches\n",
    "\n",
    "Every evaluator in the Standard Evaluator library requires an interface definition that describes its inputs and outputs. There are two ways to provide this:\n",
    "\n",
    "1. **`OptProblem`** — Used when your evaluator represents an optimization problem. It defines typed input variables, response variables, and additionally specifies which responses serve as objectives and which serve as constraints. This is the right choice when you plan to optimize the evaluator.\n",
    "\n",
    "2. **`EvaluatorInfo`** — Used for general-purpose evaluators that only need input/output descriptions without optimization semantics. It defines named, typed inputs and outputs but does not include objectives or constraints.\n",
    "\n",
    "Both approaches use the same variable types (`FloatVariable`, `IntVariable`, `ArrayVariable`, `CategoricalVariable`) to describe individual inputs and outputs with names, bounds, defaults, and other metadata."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c3d4e5f6",
   "metadata": {},
   "source": [
    "## Setup\n",
    "\n",
    "We begin by importing the library using the standard alias convention."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "d4e5f6a7",
   "metadata": {},
   "outputs": [],
   "source": [
    "import standard_evaluator as se\n",
    "from standard_evaluator.evaluators.numpy_evaluator import NumpyEvaluator\n",
    "import numpy as np\n",
    "import pandas as pd"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e5f6a7b8",
   "metadata": {},
   "source": [
    "## Creating an OptProblem\n",
    "\n",
    "An `OptProblem` defines the full optimization context: input variables with bounds, response variables, objectives to minimize, and constraints to satisfy. Here we create a problem with two float inputs, two responses, one objective, and one constraint."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "f6a7b8c9",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Create an OptProblem with 2 inputs, 2 responses, 1 objective, 1 constraint\n",
    "opt_problem = se.OptProblem(\n",
    "    name=\"demo_problem\",\n",
    "    variables=[\n",
    "        se.FloatVariable(name=\"x\", bounds=(-5.0, 5.0)),\n",
    "        se.FloatVariable(name=\"y\", bounds=(0.0, 10.0)),\n",
    "    ],\n",
    "    responses=[\n",
    "        se.FloatVariable(name=\"f\"),\n",
    "        se.FloatVariable(name=\"g\"),\n",
    "    ],\n",
    "    objectives=[\"f\"],\n",
    "    constraints=[\"g\"],\n",
    ")\n",
    "\n",
    "print(\"Problem name:\", opt_problem.name)\n",
    "print(\"Variables:\", opt_problem.variable_names())\n",
    "print(\"Responses:\", opt_problem.response_names())\n",
    "print(\"Objectives:\", opt_problem.objectives)\n",
    "print(\"Constraints:\", opt_problem.constraints)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a8b9c0d1",
   "metadata": {},
   "source": [
    "The `OptProblem` validates that all objective and constraint names reference existing responses (or variables for objectives). The bounds on each `FloatVariable` define the feasible design space."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b9c0d1e2",
   "metadata": {},
   "source": [
    "## Creating an EvaluatorInfo\n",
    "\n",
    "An `EvaluatorInfo` describes inputs and outputs without optimization semantics. It supports the same variable types as `OptProblem` but has no concept of objectives or constraints. Here we create an interface with a `FloatVariable` and an `IntVariable` as inputs, and a `FloatVariable` as output."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "c0d1e2f3",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Create an EvaluatorInfo with 2 typed inputs and 1 output\n",
    "eval_info = se.EvaluatorInfo(\n",
    "    name=\"weighted_sum\",\n",
    "    inputs=[\n",
    "        se.FloatVariable(name=\"a\", bounds=(0.0, 100.0)),\n",
    "        se.IntVariable(name=\"n\", bounds=(1, 10)),\n",
    "    ],\n",
    "    outputs=[\n",
    "        se.FloatVariable(name=\"result\"),\n",
    "    ],\n",
    ")\n",
    "\n",
    "print(\"Interface name:\", eval_info.name)\n",
    "print(\"Inputs:\", eval_info.input_names())\n",
    "print(\"Outputs:\", [v.name for v in eval_info.outputs])"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d1e2f3a4",
   "metadata": {},
   "source": [
    "## Instantiating a NumpyEvaluator with EvaluatorInfo\n",
    "\n",
    "To create a `NumpyEvaluator`, you subclass it and implement the `eval_np` method. You can pass an `EvaluatorInfo` as the `interface` parameter to define what the evaluator expects as inputs and produces as outputs."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "e2f3a4b5",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Define a NumpyEvaluator that computes result = a * n\n",
    "class WeightedEvaluator(NumpyEvaluator):\n",
    "    \"\"\"Computes result = a * n.\"\"\"\n",
    "\n",
    "    def eval_np(self, sites, **kwargs):\n",
    "        a = sites[:, 0]\n",
    "        n = sites[:, 1]\n",
    "        return (a * n).reshape(-1, 1)\n",
    "\n",
    "\n",
    "# Instantiate using the interface parameter\n",
    "weighted_eval = WeightedEvaluator(interface=eval_info)\n",
    "\n",
    "print(\"Evaluator inputs:\", weighted_eval.inputs)\n",
    "print(\"Evaluator outputs:\", weighted_eval.outputs)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f3a4b5c6",
   "metadata": {},
   "source": [
    "The evaluator now knows it expects columns `a` and `n` as inputs and produces a column `result` as output. Internally, the `EvaluatorInfo` is converted to an `OptProblem` (without objectives or constraints) so that both interface types share the same underlying data model."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a4b5c6d7",
   "metadata": {},
   "source": [
    "## Instantiating a NumpyEvaluator with OptProblem\n",
    "\n",
    "When you have an optimization problem definition, pass it directly as the `opt_problem` parameter. This gives the evaluator access to objective and constraint metadata in addition to the variable definitions."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "b5c6d7e8",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Define a NumpyEvaluator that computes f = x^2 + y^2, g = x + y\n",
    "class OptEvaluator(NumpyEvaluator):\n",
    "    \"\"\"Computes f = x^2 + y^2 and g = x + y.\"\"\"\n",
    "\n",
    "    def eval_np(self, sites, **kwargs):\n",
    "        x = sites[:, 0]\n",
    "        y = sites[:, 1]\n",
    "        f = x**2 + y**2\n",
    "        g = x + y\n",
    "        return np.column_stack([f, g])\n",
    "\n",
    "\n",
    "# Instantiate using the opt_problem parameter\n",
    "opt_eval = OptEvaluator(opt_problem=opt_problem)\n",
    "\n",
    "# Call the evaluator with a DataFrame\n",
    "sites = pd.DataFrame({\"x\": [1.0, -2.0, 3.0], \"y\": [2.0, 5.0, 1.0]})\n",
    "opt_eval(sites)\n",
    "\n",
    "print(\"Inputs property:\", opt_eval.inputs)\n",
    "print(\"Outputs property:\", opt_eval.outputs)\n",
    "print(\"\\nEvaluation results:\")\n",
    "sites"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c6d7e8f9",
   "metadata": {},
   "source": [
    "The `inputs` property returns the list of input variable names (`['x', 'y']`) and the `outputs` property returns the list of response names (`['f', 'g']`). The DataFrame is modified in-place with the computed response columns appended."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d7e8f9a0",
   "metadata": {},
   "source": [
    "## When to Use OptProblem vs EvaluatorInfo\n",
    "\n",
    "Choose the interface type based on your use case:\n",
    "\n",
    "| Use Case | Interface Type | Reason |\n",
    "| -------- | -------------- | ------ |\n",
    "| Running optimization with an optimizer | `OptProblem` | Defines objectives and constraints needed by optimization algorithms |\n",
    "| Wrapping analysis code for general evaluation | `EvaluatorInfo` | Only input/output descriptions are needed, no optimization semantics |\n",
    "| Integration with `EvaluatorOpenMdaoComponent` | `OptProblem` | The OpenMDAO component uses objectives and constraints for driver setup |\n",
    "| Building surrogate models | Either | Surrogate training only needs input/output structure |\n",
    "\n",
    "**Summary:**\n",
    "- Use **`OptProblem`** when your evaluator will be used in an optimization context and you need to specify which responses are objectives to minimize and which are constraints to satisfy.\n",
    "- Use **`EvaluatorInfo`** when you only need to describe the evaluator's inputs and outputs without optimization semantics — for example, when wrapping a general-purpose analysis tool or building a data pipeline."
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "name": "python",
   "version": "3.11"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}