{
"cells": [
{
"cell_type": "markdown",
"id": "f17a116a",
"metadata": {},
"source": [
"# Replacing a group with a component\n",
"In this demonstration we show how we can replace a group of components with a single component by using the functionality implemented in the `standard_evaluator` library.\n",
"\n",
"\n",
"\n",
"We first load all required libraries and methods."
]
},
{
"cell_type": "code",
"execution_count": 1,
"id": "acef51ce-e90f-43a3-9716-ecb16167b371",
"metadata": {
"metadata": {}
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"matlabengine not available.\n"
]
}
],
"source": [
"import typing\n",
"import openmdao.api as om\n",
"import openmdao.core.system as oms\n",
"\n",
"from standard_evaluator import get_interface, show_structure, create_problem, get_state, set_state, load_state, save_state\n",
"from standard_evaluator import load_assembly, save_assembly, convert_om_var, get_opt_problem, set_opt_problem\n",
"from standard_evaluator import Variable, FloatVariable, IntVariable, ArrayVariable, OptProblem"
]
},
{
"cell_type": "markdown",
"id": "763bdf4f-e824-456c-898f-1513dcff5e7c",
"metadata": {},
"source": [
"## Creating the OpenMDAO assembly\n",
"\n",
"Next we generate an OpenMDAO assembly with wo sub-groups, one of which represents an aero analyses with two components. The first component is a stand-in for a geometry builder, and the second component is a stand-in for a Computational Fluid Dynamics (CFD) component. There is a link between those two components which can be thought of as a large, multi-dimensional grid. In this example we are just using a 1-D array, but this can be thought of as a large three dimensional grid.\n",
"\n",
"The idea is that the geometry builder takes as inputs a small number of geometry parameters (in this case all individual floats), and produces a large, even multi-dimensional output. The CFD component takes that output from the geometry builder and some parameters and performs calculations that together with a post-processing step (which we did not model) creates a small number of outputs (in this example we only use a single float output).\n",
"\n",
"After building the assembly we add optimization design variables, objectives, and constraints, run the assembly, and show the values of all inputs and outputs."
]
},
{
"cell_type": "code",
"execution_count": 2,
"id": "d9bdb3c0",
"metadata": {
"metadata": {}
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"14 Input(s) in 'model'\n",
"\n",
"varname val shape prom_name \n",
"---------- ------------------ ----- -------------\n",
"sub1\n",
" z1\n",
" a [1.] (1,) a \n",
" b [1.] (1,) sub1.my_alias\n",
" c [1.] (1,) sub1.z1.c \n",
" d [1.] (1,) sub1.d \n",
" z2\n",
" a [1.] (1,) sub1.z2.a \n",
" b [1.] (1,) sub1.z2.b \n",
" c [4.] (1,) sub1.c \n",
" d [1.] (1,) sub1.d \n",
"aero\n",
" geometry\n",
" l [1.] (1,) aero.l \n",
" q [1.] (1,) aero.q \n",
" z [1.] (1,) aero.z \n",
" cfd\n",
" grid |8.94427191| (20,) aero.cfd.grid\n",
" k [7.] (1,) aero.k \n",
" r [1.] (1,) aero.r \n",
"\n",
"\n",
"5 Explicit Output(s) in 'model'\n",
"\n",
"varname val shape prom_name \n",
"---------- ------------------ -------- ------------------\n",
"sub1\n",
" z1\n",
" y [4.] (1,) sub1.z1.y \n",
" z2\n",
" v [7.] (1,) sub1.v \n",
"aero\n",
" geometry\n",
" bla |20.0| (20, 20) aero.geometry.bla \n",
" grid |8.94427191| (20,) aero.geometry.grid\n",
" cfd\n",
" drag [0.047] (1,) aero.drag \n",
"\n",
"\n",
"0 Implicit Output(s) in 'model'\n",
"\n",
"\n"
]
}
],
"source": [
"grid_size = 20\n",
"\n",
"class Sub1(om.Group):\n",
" def setup(self):\n",
" self.add_subsystem('z1', om.ExecComp(\"y = a + b + c + d\",\n",
" ),\n",
" promotes_inputs=[\n",
" 'a',\n",
" ('b', 'my_alias'),\n",
" 'd',\n",
" ])\n",
"\n",
" self.add_subsystem('z2', om.ExecComp(\"v = a + b + c + d\",\n",
" ),\n",
" promotes_inputs=['c', 'd'],\n",
" promotes_outputs=['*'])\n",
" self.connect('z1.y', 'c')\n",
"\n",
"class Aero(om.Group):\n",
" def setup(self):\n",
" self.add_subsystem('geometry', om.ExecComp([f\"grid = (q + z *l)*ones({grid_size})\", f\"bla = outer(ones({grid_size}), ones({grid_size}))\"], \n",
" grid={'tags': 'internal', 'shape': (grid_size)},\n",
" bla ={'tags': 'internal', 'shape': (grid_size, grid_size)}\n",
" ),\n",
" promotes_inputs=['q', 'z', 'l'])\n",
" self.add_subsystem('cfd', om.ExecComp(f\"drag = (inner(grid, ones({grid_size})) + r * k)/1000\", \n",
" grid={'tags': 'internal', 'shape_by_conn': True},\n",
" drag={'units': 'N'}),\n",
" promotes_inputs=['r', 'k'],\n",
" promotes_outputs=['drag'])\n",
" self.connect('geometry.grid', 'cfd.grid')\n",
"\n",
"\n",
"class Intermediate3(om.Group):\n",
" def setup(self):\n",
" self.add_subsystem('sub1', Sub1(),\n",
" promotes_inputs=['a'])\n",
"\n",
" self.add_subsystem('aero', Aero())\n",
" self.connect('sub1.v', 'aero.k')\n",
"\n",
"prob = om.Problem(model=Intermediate3())\n",
"\n",
"# Define an optimization problem\n",
"prob.model.add_design_var('a', lower=-50, upper=50, scaler= .1, adder=30.)\n",
"prob.model.add_design_var('sub1.my_alias', lower=-50, upper=50)\n",
"prob.model.add_objective('aero.drag')\n",
"prob.model.add_constraint('sub1.v', lower=0, upper=10.)\n",
"prob.model.add_constraint('sub1.z1.y', lower=-32.32, upper=1829.)\n",
"\n",
"prob.setup()\n",
"prob.final_setup()\n",
"\n",
"prob.run_model()\n",
"_ = prob.model.list_inputs(shape=True)\n",
"_ = prob.model.list_outputs(shape=True)"
]
},
{
"cell_type": "markdown",
"id": "079c55d7",
"metadata": {},
"source": [
"Next we look at the N2 diagram of this assembly. Note that the grid output from the `geometry` component only goes to the `cfd` component, which means we can mark it as an internal variable.\n",
"\n",
"\n",
"Note: Right now we have to add the 'internal' tag to an input or output of a component to signal to the standard evaluator that this is internal to the group. This should be something that can be automated, although there is an argument that can be made that the developer of a group should be explicit what inputs and outputs that are only used internally within a group should be accessible outside the group. That is, the developer of the group should be defining the interface of the group explicitly.\n",
"
"
]
},
{
"cell_type": "code",
"execution_count": 3,
"id": "e14f6152",
"metadata": {
"metadata": {}
},
"outputs": [],
"source": [
"import os\n",
"# Check if running in VS Code\n",
"if 'VSCODE_PID' in os.environ:\n",
" display_in_notebook = False\n",
"else:\n",
" display_in_notebook = True\n",
"om.n2(prob, 'replace.html', display_in_notebook=display_in_notebook, )"
]
},
{
"cell_type": "markdown",
"id": "942d49a7",
"metadata": {},
"source": [
"We can now take this OpenMDAO assembly and create a Pydantic description of this assembly using the `get_interface` method. This description can be easily saved to a JSON file, and we can also load the information from a JSON file. We also get the state of the OpenMDAO problem which will be saved in a different, binary HDF5 formatted file. The `state` variable right now is just a dictionary with values of all inputs and outputs of the problem at the top level.\n",
"\n",
"In this demonstration we do not need to save either the assembly information or the state to disk since we can manipulate them in memory. See the other demonstrations for how to store and load assemblies and state."
]
},
{
"cell_type": "code",
"execution_count": 4,
"id": "d3bec7b1",
"metadata": {
"metadata": {}
},
"outputs": [],
"source": [
"info = get_interface(prob.model)\n",
"state = get_state(prob, info)"
]
},
{
"cell_type": "markdown",
"id": "ddcbc3a0",
"metadata": {},
"source": [
"# Replacing a group with a Surrogate or simplified model\n",
"In the next step we creata a new OpenMDAO component that we will want to use to replace the complex aero group in the original model with. In this example since we have two simple equations we can replace those two equations with a single equation where we substitute the first equation into the second equation. Note that because we used a simple way of creating the one dimensional grid and using the grid in the second formula we can calculate what the inner product of two vectors of length 20 with all ones in them is, which is 20.\n",
"\n",
"We then take this component and add it to an OpenMDAO `Problem` and run `setup()` to fully instantiate the problem.\n",
"\n",
"Note that this step could easily be replaced with a process where we take the original aero group, instantiate it as an OpenMDAO problem, run a Design of Experiment against it, and build a surrogate model which we can then expose as an OpenMDAO component. This is not part of the contract, but Boeing has some internal tools that demonstrate this process."
]
},
{
"cell_type": "code",
"execution_count": 5,
"id": "78b6dba2",
"metadata": {
"metadata": {}
},
"outputs": [
{
"data": {
"text/plain": [
""
]
},
"execution_count": 5,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"class Surrogate(om.Group):\n",
" def setup(self):\n",
" self.add_subsystem('aero', \n",
" om.ExecComp(\"drag = ((q + z *l)*20 + r * k)/1000\",\n",
" drag={'units': 'N'}),\n",
" promotes_inputs=['q', 'z', 'l', 'r', 'k'],\n",
" promotes_outputs=['drag'])\n",
"\n",
"prob_replace = om.Problem(model=Surrogate())\n",
"prob_replace.setup()"
]
},
{
"cell_type": "markdown",
"id": "846bc05e",
"metadata": {},
"source": [
"In the next step we get the Pydantic representation of the new surrogate using the `get_interface` method from the standard evaluator. We give it the same name as the component we are trying to replace."
]
},
{
"cell_type": "code",
"execution_count": 6,
"id": "1c87cd80",
"metadata": {
"metadata": {}
},
"outputs": [],
"source": [
"surrogate_info = get_interface(prob_replace.model)"
]
},
{
"cell_type": "markdown",
"id": "c674b27f",
"metadata": {},
"source": [
"We can now replace the original group with the surrogate component / group in the Pydantic description. \n",
"\n",
"This is the key step in this, we are replacing the interface description of the group in the overall interface description with the interface description of the simplified component.\n",
"\n",
"We show the process in the below graphic. \n",
"\n",
""
]
},
{
"cell_type": "code",
"execution_count": 7,
"id": "443904ed",
"metadata": {
"metadata": {}
},
"outputs": [],
"source": [
"info.components['aero'] = surrogate_info"
]
},
{
"cell_type": "markdown",
"id": "46199a55",
"metadata": {},
"source": [
"Create a new OpenMDAO problem from the assembly information where the original `aero` group is replaced by a simplified group / component."
]
},
{
"cell_type": "code",
"execution_count": 8,
"id": "6efc7740",
"metadata": {
"metadata": {}
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"** This is a group, special handling needed for sub1 in group \n",
"Adding z1 component\n",
"Building Equation class z1\n",
"Adding z2 component\n",
"Building Equation class z2\n",
"** This is a group, special handling needed for aero in group \n",
"Adding aero component\n",
"Building Equation class aero\n",
"Finished this step\n",
"Finished this step2\n",
"Started this step4\n"
]
},
{
"ename": "KeyboardInterrupt",
"evalue": "",
"output_type": "error",
"traceback": [
"\u001b[31m---------------------------------------------------------------------------\u001b[39m",
"\u001b[31mKeyboardInterrupt\u001b[39m Traceback (most recent call last)",
"\u001b[36mCell\u001b[39m\u001b[36m \u001b[39m\u001b[32mIn[8]\u001b[39m\u001b[32m, line 1\u001b[39m\n\u001b[32m----> \u001b[39m\u001b[32m1\u001b[39m new_prob5 = create_problem(info)\n",
"\u001b[36mFile \u001b[39m\u001b[32mC:\\dev\\standard-evaluator\\src\\standard_evaluator\\om_converter.py:476\u001b[39m, in \u001b[36mcreate_problem\u001b[39m\u001b[34m(info, state, opt_problem)\u001b[39m\n\u001b[32m 474\u001b[39m \u001b[38;5;28mprint\u001b[39m(\u001b[33m\"\u001b[39m\u001b[33mFinished this step3\u001b[39m\u001b[33m\"\u001b[39m)\n\u001b[32m 475\u001b[39m \u001b[38;5;28mprint\u001b[39m(\u001b[33m\"\u001b[39m\u001b[33mStarted this step4\u001b[39m\u001b[33m\"\u001b[39m)\n\u001b[32m--> \u001b[39m\u001b[32m476\u001b[39m \u001b[30;43mprob\u001b[39;49m\u001b[30;43m.\u001b[39;49m\u001b[30;43msetup\u001b[39;49m\u001b[30;43m(\u001b[39;49m\u001b[30;43m)\u001b[39;49m\n\u001b[32m 477\u001b[39m \u001b[38;5;28mprint\u001b[39m(\u001b[33m\"\u001b[39m\u001b[33mFinished this step4\u001b[39m\u001b[33m\"\u001b[39m)\n\u001b[32m 479\u001b[39m \u001b[38;5;66;03m# Set the state of all he elements in the model if state was defined.\u001b[39;00m\n",
"\u001b[36mFile \u001b[39m\u001b[32mc:\\dev\\standard-evaluator\\.venv\\Lib\\site-packages\\openmdao\\core\\problem.py:1142\u001b[39m, in \u001b[36mProblem.setup\u001b[39m\u001b[34m(self, check, logger, mode, force_alloc_complex, distributed_vector_class, local_vector_class, derivatives, parent)\u001b[39m\n\u001b[32m 1139\u001b[39m \u001b[38;5;28mself\u001b[39m._metadata[\u001b[33m'\u001b[39m\u001b[33mreports_dir\u001b[39m\u001b[33m'\u001b[39m] = \u001b[38;5;28mself\u001b[39m.get_reports_dir()\n\u001b[32m 1141\u001b[39m \u001b[38;5;28;01mtry\u001b[39;00m:\n\u001b[32m-> \u001b[39m\u001b[32m1142\u001b[39m \u001b[30;43mmodel\u001b[39;49m\u001b[30;43m.\u001b[39;49m\u001b[30;43m_setup\u001b[39;49m\u001b[30;43m(\u001b[39;49m\u001b[30;43mmodel_comm\u001b[39;49m\u001b[30;43m,\u001b[39;49m\u001b[30;43m \u001b[39;49m\u001b[30;43mself\u001b[39;49m\u001b[30;43m.\u001b[39;49m\u001b[30;43m_metadata\u001b[39;49m\u001b[30;43m)\u001b[39;49m\n\u001b[32m 1143\u001b[39m \u001b[38;5;28;01mfinally\u001b[39;00m:\n\u001b[32m 1144\u001b[39m \u001b[38;5;66;03m# whenever we're outside of model._setup, static mode should be True so that anything\u001b[39;00m\n\u001b[32m 1145\u001b[39m \u001b[38;5;66;03m# added outside of _setup will persist.\u001b[39;00m\n\u001b[32m 1146\u001b[39m \u001b[38;5;28mself\u001b[39m._metadata[\u001b[33m'\u001b[39m\u001b[33mstatic_mode\u001b[39m\u001b[33m'\u001b[39m] = \u001b[38;5;28;01mTrue\u001b[39;00m\n",
"\u001b[36mFile \u001b[39m\u001b[32mc:\\dev\\standard-evaluator\\.venv\\Lib\\site-packages\\openmdao\\core\\group.py:812\u001b[39m, in \u001b[36mGroup._setup\u001b[39m\u001b[34m(self, comm, prob_meta)\u001b[39m\n\u001b[32m 806\u001b[39m grp._has_bounds |= subsys._has_bounds\n\u001b[32m 808\u001b[39m \u001b[38;5;66;03m# promoted names must be known to determine implicit connections so this must be\u001b[39;00m\n\u001b[32m 809\u001b[39m \u001b[38;5;66;03m# called after _setup_var_data, and _setup_var_data will have to be partially redone\u001b[39;00m\n\u001b[32m 810\u001b[39m \u001b[38;5;66;03m# after auto_ivcs have been added, but auto_ivcs can't be added until after we know all of\u001b[39;00m\n\u001b[32m 811\u001b[39m \u001b[38;5;66;03m# the connections.\u001b[39;00m\n\u001b[32m--> \u001b[39m\u001b[32m812\u001b[39m \u001b[30;43mself\u001b[39;49m\u001b[30;43m.\u001b[39;49m\u001b[30;43mget_conn_graph\u001b[39;49m\u001b[30;43m(\u001b[39;49m\u001b[30;43m)\u001b[39;49m\u001b[30;43m.\u001b[39;49m\u001b[30;43msetup_global_connections\u001b[39;49m\u001b[30;43m(\u001b[39;49m\u001b[30;43mself\u001b[39;49m\u001b[30;43m)\u001b[39;49m\n",
"\u001b[36mFile \u001b[39m\u001b[32mc:\\dev\\standard-evaluator\\.venv\\Lib\\site-packages\\openmdao\\core\\conn_graph.py:3932\u001b[39m, in \u001b[36mAllConnGraph.setup_global_connections\u001b[39m\u001b[34m(self, model)\u001b[39m\n\u001b[32m 3927\u001b[39m errmsg = \u001b[33m'\u001b[39m\u001b[38;5;130;01m\\n\u001b[39;00m\u001b[33m'\u001b[39m.join([\u001b[33mf\u001b[39m\u001b[33m'\u001b[39m\u001b[33m \u001b[39m\u001b[38;5;132;01m{\u001b[39;00medge[\u001b[32m0\u001b[39m]\u001b[38;5;132;01m}\u001b[39;00m\u001b[33m ---> \u001b[39m\u001b[38;5;132;01m{\u001b[39;00medge[\u001b[32m1\u001b[39m]\u001b[38;5;132;01m}\u001b[39;00m\u001b[33m'\u001b[39m\n\u001b[32m 3928\u001b[39m \u001b[38;5;28;01mfor\u001b[39;00m edge \u001b[38;5;129;01min\u001b[39;00m cycle_edges])\n\u001b[32m 3929\u001b[39m \u001b[38;5;28mself\u001b[39m._collect_error(\u001b[33m'\u001b[39m\u001b[33mCycle detected in input-to-input connections. \u001b[39m\u001b[33m'\u001b[39m\n\u001b[32m 3930\u001b[39m \u001b[33mf\u001b[39m\u001b[33m'\u001b[39m\u001b[33mThis is not allowed.\u001b[39m\u001b[38;5;130;01m\\n\u001b[39;00m\u001b[38;5;132;01m{\u001b[39;00merrmsg\u001b[38;5;132;01m}\u001b[39;00m\u001b[33m'\u001b[39m)\n\u001b[32m-> \u001b[39m\u001b[32m3932\u001b[39m \u001b[30;43mself\u001b[39;49m\u001b[30;43m.\u001b[39;49m\u001b[30;43madd_auto_ivc_nodes\u001b[39;49m\u001b[30;43m(\u001b[39;49m\u001b[30;43mmodel\u001b[39;49m\u001b[30;43m)\u001b[39;49m\n\u001b[32m 3933\u001b[39m \u001b[38;5;28mself\u001b[39m.update_src_inds_lists(model)\n\u001b[32m 3935\u001b[39m model._setup_auto_ivcs()\n",
"\u001b[36mFile \u001b[39m\u001b[32mc:\\dev\\standard-evaluator\\.venv\\Lib\\site-packages\\openmdao\\core\\conn_graph.py:3268\u001b[39m, in \u001b[36mAllConnGraph.add_auto_ivc_nodes\u001b[39m\u001b[34m(self, model)\u001b[39m\n\u001b[32m 3265\u001b[39m above_anchors.update(ins)\n\u001b[32m 3266\u001b[39m \u001b[38;5;28;01mcontinue\u001b[39;00m\n\u001b[32m-> \u001b[39m\u001b[32m3268\u001b[39m \u001b[30;43mstack\u001b[39;49m\u001b[30;43m.\u001b[39;49m\u001b[30;43mextend\u001b[39;49m\u001b[30;43m(\u001b[39;49m\u001b[30;43mins\u001b[39;49m\u001b[30;43m)\u001b[39;49m\n\u001b[32m 3270\u001b[39m \u001b[38;5;28;01mif\u001b[39;00m src \u001b[38;5;129;01mis\u001b[39;00m \u001b[38;5;28;01mNone\u001b[39;00m:\n\u001b[32m 3271\u001b[39m dangling_inputs.add(node)\n",
"\u001b[31mKeyboardInterrupt\u001b[39m: "
]
}
],
"source": [
"new_prob5 = create_problem(info)"
]
},
{
"cell_type": "markdown",
"id": "2e2cd721",
"metadata": {},
"source": [
"We can now save and show the N2 diagram of the new OpenMDAO Problem."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "f72c0321",
"metadata": {
"metadata": {}
},
"outputs": [],
"source": [
"om.n2(new_prob5, 'replace_after_surrogate.html', display_in_notebook=display_in_notebook)"
]
},
{
"cell_type": "markdown",
"id": "7605e669",
"metadata": {},
"source": [
"Finally, we are setting values for all inputs to the two different assemblies, and run both of them. You will notice that the calculated drag is the same for the two calculations."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "3898d275",
"metadata": {
"metadata": {}
},
"outputs": [],
"source": [
"prob.set_val('a', 34.)\n",
"prob.set_val('sub1.my_alias', 5)\n",
"prob.set_val('sub1.z1.c', 24.)\n",
"prob.set_val('sub1.d', 3.)\n",
"prob.set_val('sub1.c', 12.)\n",
"prob.set_val('aero.l', 400.)\n",
"prob.set_val('aero.q', 5.)\n",
"prob.set_val('aero.r', 6.)\n",
"prob.set_val('aero.z', 7.)\n",
"prob.run_model()\n",
"_ = prob.model.list_outputs(shape=True, units=True)"
]
},
{
"cell_type": "markdown",
"id": "0b9e1f49",
"metadata": {},
"source": [
"To transfer the state from the original problem to the new problem with the simplified component we use the `get_state` and `set_state` methods from the `standard_evalautor` library."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "466cb59d",
"metadata": {
"metadata": {}
},
"outputs": [],
"source": [
"# We can get a Python dictionary with the current state of the problem using the `get_state` method\n",
"new_state = get_state(prob, info)\n",
"\n",
"\n",
"# Load the state from the HDF5 file using the load_state helper method. Note that we need to capture the info from the new \n",
"# instance so that variables and responses that were part of the old components are not seen anymore.\n",
"new_info = get_interface(new_prob5.model)\n",
"set_state(new_prob5, new_info, new_state)"
]
},
{
"cell_type": "markdown",
"id": "5c6e0a82",
"metadata": {},
"source": [
"Now when we run the problem with the simplified component we can see that we get the same value for drag."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "41f22fe8",
"metadata": {
"metadata": {}
},
"outputs": [],
"source": [
"new_prob5.run_model()\n",
"_ = new_prob5.model.list_inputs(shape=True, units=True)\n",
"_ = new_prob5.model.list_outputs(shape=True, units=True)"
]
}
],
"metadata": {
"kernelspec": {
"display_name": ".venv",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.12.8"
}
},
"nbformat": 4,
"nbformat_minor": 5
}