{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "f35051af",
   "metadata": {},
   "source": [
    "# Evaluator Hierarchy\n",
    "\n",
    "This notebook introduces the core abstraction in the Standard Evaluator library: the **Evaluator** class hierarchy. You will learn how the abstract `Evaluator` base class defines a common API for wrapping analysis codes, explore the available concrete subclasses, and see how decorators add caching and logging capabilities. By the end, you will be able to create evaluators, call them with DataFrames or NumPy arrays, and leverage built-in caching and logging."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "091de718",
   "metadata": {},
   "source": [
    "## The Evaluator Class Hierarchy\n",
    "\n",
    "The `Evaluator` abstract base class defines the standard interface that all evaluators share:\n",
    "\n",
    "- **`__call__(sites: DataFrame)`** — Evaluate sites provided as a pandas DataFrame. The DataFrame is modified in-place to include response columns.\n",
    "- **`eval_np(sites: ndarray)`** — Evaluate sites provided as a NumPy array with unrolled scalar columns (excludes fixed variables and unrolls ArrayVariables).\n",
    "- **`eval_list(sites: List)`** — Same as `eval_np` but accepts and returns Python lists.\n",
    "\n",
    "### Concrete Subclasses\n",
    "\n",
    "| Class | Purpose |\n",
    "| ----- | ------- |\n",
    "| `NumpyEvaluator` | Wraps a user-defined function that operates on NumPy arrays |\n",
    "| `ExecutableEvaluator` | Wraps an external executable (command-line tool) |\n",
    "| `ShiftScaleEvaluator` | Applies shift/scale transformations before delegating to another evaluator |\n",
    "| `OpenMdaoEvaluator` | Wraps an OpenMDAO component as an evaluator |\n",
    "| `ExcelEvaluator` | Wraps an Excel workbook as an evaluator |\n",
    "| `MatlabEvaluator` | Wraps a MATLAB function via the MATLAB Engine API |\n",
    "\n",
    "### Decorators\n",
    "\n",
    "The evaluator constructor supports two decorator parameters:\n",
    "\n",
    "- **`cache`** — Pass a path to a SQLite database file to enable caching of evaluated sites. Previously computed sites are looked up rather than re-evaluated.\n",
    "- **`logging`** — Set to `True` to record every evaluation call. The logged history is accessible via `get_log()`."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "520c3e2a",
   "metadata": {},
   "source": [
    "## Setup\n",
    "\n",
    "We begin by importing the library using the standard alias convention."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "912206bc",
   "metadata": {},
   "outputs": [],
   "source": [
    "import standard_evaluator as se\n",
    "import pandas as pd\n",
    "import numpy as np"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d27b6bf4",
   "metadata": {},
   "source": [
    "## Creating and Calling an Evaluator\n",
    "\n",
    "We create a `Sphere` benchmark evaluator with 3 design variables. This is a `NumpyEvaluator` subclass representing the sphere function $f(\\mathbf{x}) = \\sum x_i^2$. We then build an input DataFrame with 3 rows and call the evaluator using `__call__`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "1a97a681",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Create a Sphere benchmark evaluator with 3 design variables\n",
    "evaluator = se.evaluators.Sphere(num_independent=3)\n",
    "\n",
    "print(\"Inputs:\", evaluator.inputs)\n",
    "print(\"Outputs:\", evaluator.outputs)\n",
    "print(\"Variable bounds:\")\n",
    "for var in evaluator.opt_problem.variables:\n",
    "    print(f\"  {var.name}: {var.bounds}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "43e1dd3b",
   "metadata": {},
   "source": [
    "Now we build an input DataFrame with 3 sample sites and call the evaluator. The `__call__` method modifies the DataFrame in-place, appending the response column `f`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "e0f08963",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Build an input DataFrame with 3 rows of sample sites\n",
    "sites = pd.DataFrame({\n",
    "    \"x0\": [1.0, 2.0, -3.0],\n",
    "    \"x1\": [0.5, -1.0, 4.0],\n",
    "    \"x2\": [2.0, 3.0, -1.0],\n",
    "})\n",
    "\n",
    "# Call the evaluator — modifies sites in-place\n",
    "evaluator(sites)\n",
    "\n",
    "# Display the result with both input and response columns\n",
    "sites"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8da88896",
   "metadata": {},
   "source": [
    "The DataFrame now contains the original input columns (`x0`, `x1`, `x2`) along with the computed response column `f`, where each row's value equals $x_0^2 + x_1^2 + x_2^2$."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "535b4b2c",
   "metadata": {},
   "source": [
    "## Using `eval_np` and `eval_list`\n",
    "\n",
    "The `eval_np` and `eval_list` methods provide a convenience interface that works with raw NumPy arrays or Python lists instead of DataFrames. Key differences from `__call__`:\n",
    "\n",
    "- **Fixed variables are excluded** from the input — you only provide varying design variables.\n",
    "- **ArrayVariables are unrolled** into individual scalar columns (e.g., a shape `(3,)` array becomes 3 separate values).\n",
    "- Only the `__call__` method preserves array variables as arrays within DataFrame cells.\n",
    "\n",
    "These methods are useful when integrating with optimization libraries that operate on flat numeric arrays."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "c1922f5d",
   "metadata": {},
   "outputs": [],
   "source": [
    "# eval_np: pass a 2D NumPy array (rows=sites, cols=unrolled inputs)\n",
    "sites_np = np.array([\n",
    "    [1.0, 0.5, 2.0],\n",
    "    [2.0, -1.0, 3.0],\n",
    "    [-3.0, 4.0, -1.0],\n",
    "])\n",
    "\n",
    "results_np = evaluator.eval_np(sites_np)\n",
    "print(\"eval_np results (NumPy array):\")\n",
    "print(results_np)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6f7baada",
   "metadata": {},
   "source": [
    "The `eval_list` method works identically but accepts and returns Python lists of lists."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "9943d29d",
   "metadata": {},
   "outputs": [],
   "source": [
    "# eval_list: pass a list of lists\n",
    "sites_list = [\n",
    "    [1.0, 0.5, 2.0],\n",
    "    [2.0, -1.0, 3.0],\n",
    "]\n",
    "\n",
    "results_list = evaluator.eval_list(sites_list)\n",
    "print(\"eval_list results (Python lists):\")\n",
    "print(results_list)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a167f9ca",
   "metadata": {},
   "source": [
    "Both `eval_np` and `eval_list` return only the response values (here, the `f` column as a single-column array or nested list)."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "7d450346",
   "metadata": {},
   "source": [
    "## Caching Decorator\n",
    "\n",
    "By passing a `cache` parameter (a path to a SQLite database file) when constructing an evaluator, the caching decorator is applied. Previously evaluated sites are retrieved from the cache instead of being re-computed. After evaluation, you can inspect the cache using `get_cache()`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "35f386c6",
   "metadata": {},
   "outputs": [],
   "source": [
    "import tempfile\n",
    "import os\n",
    "\n",
    "# Create a temporary database file for caching\n",
    "cache_file = os.path.join(tempfile.gettempdir(), \"sphere_cache_demo.db\")\n",
    "\n",
    "# Create evaluator with caching enabled\n",
    "cached_evaluator = se.evaluators.Sphere(num_independent=2, cache=cache_file)\n",
    "\n",
    "# Evaluate two sites\n",
    "sites_cached = pd.DataFrame({\"x0\": [1.0, 2.0], \"x1\": [3.0, 4.0]})\n",
    "cached_evaluator(sites_cached)\n",
    "print(\"Evaluation result:\")\n",
    "print(sites_cached)\n",
    "\n",
    "# Inspect the cache\n",
    "print(\"\\nCached sites:\")\n",
    "print(cached_evaluator.get_cache())"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f9e2d37a",
   "metadata": {},
   "source": [
    "The cache stores all evaluated sites. On subsequent calls, any site matching a cached entry (within a configurable tolerance) will retrieve its response from the database rather than re-evaluating."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "632a24fd",
   "metadata": {},
   "source": [
    "## Logging Decorator\n",
    "\n",
    "Setting `logging=True` when constructing an evaluator enables the logging decorator. Every evaluation call is recorded, and the full history can be retrieved using `get_log()`. Each logged site includes a `call_num` column indicating which evaluation call it belongs to."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "7717cb21",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Create evaluator with logging enabled\n",
    "logged_evaluator = se.evaluators.Sphere(num_independent=2, logging=True)\n",
    "\n",
    "# First evaluation call\n",
    "sites_a = pd.DataFrame({\"x0\": [1.0, 2.0], \"x1\": [3.0, 4.0]})\n",
    "logged_evaluator(sites_a)\n",
    "\n",
    "# Second evaluation call\n",
    "sites_b = pd.DataFrame({\"x0\": [5.0], \"x1\": [6.0]})\n",
    "logged_evaluator(sites_b)\n",
    "\n",
    "# Retrieve the full evaluation log\n",
    "log = logged_evaluator.get_log()\n",
    "print(\"Evaluation log:\")\n",
    "log"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "11ad7249",
   "metadata": {},
   "source": [
    "The log shows all 3 evaluated sites across both calls. The `call_num` column distinguishes which invocation produced each row — sites from the first call have `call_num=1` and the site from the second call has `call_num=2`."
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "name": "python",
   "version": "3.12.0"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}