Thoughts on why we need to be explicit with input and outputs
When we are trying to implement analyses that are chained together into workflows we want to understand what the relationships are between components. Often downstreams processes will only leverage some of the information generated by another process, or even none at all.
Below is a simple example of several disciplines that are interacting with each other. In this first example we assume all disciplines just say they use the ADH without specifying which part of the ADH they use and modify. We have a short loop between disciplines 2 and 3 in this example, and the final discipline being executed is discipline 5.
Any time anything in the input ADH is changed all analyses need to be re-executed since there is no visibility if the change has any impact on these disciplines.
sequenceDiagram
autonumber
participant Discipline 1
participant Discipline 2
participant Discipline 3
participant Discipline 4
participant Discipline 5
participant Discipline 6
Note left of Discipline 1: Input ADH
rect rgb(230, 255, 255)
activate Discipline 1
Note left of Discipline 1: Execute
deactivate Discipline 1
Discipline 1->>Discipline 2: ADH
loop "Until converged"
activate Discipline 2
Note left of Discipline 2: Execute
deactivate Discipline 2
Discipline 2->>Discipline 3: ADH
activate Discipline 3
Note left of Discipline 3: Execute
deactivate Discipline 3
Discipline 3->>Discipline 2: ADH
end
Discipline 3->>Discipline 4: ADH
activate Discipline 4
Note left of Discipline 4: Execute
deactivate Discipline 4
Discipline 4->>Discipline 6: ADH
end
Discipline 1->>Discipline 5: ADH
activate Discipline 5
Note left of Discipline 5: Execute
deactivate Discipline 5
Discipline 5->>Discipline 6: ADH
activate Discipline 6
Note left of Discipline 6: Execute
deactivate Discipline 6
Note right of Discipline 6: Final ADH
On the other hand if each discipline is explicit of their inputs and outputs executing analyses can become a lot more flexible. For example, if the end user only changes entries in the ADH that effect Discipline 5 only Disciplines 1 and 5 need to be executed again. That can significantly reduce the amount of computing needed after an update of the ADH. If each discipline is explicit about what parts of the ADH is being used all the work in light blue will not need to be executed again. We show this in the next figure.
Another major issue with not being explicit about what each discipline changes in the ADH is that you will generate race conditions. In the above figure we know that when we are trying to execute Discipline 6 we will have two different versions of the ADH being passed to Discipline 6: One for Discipline 4 and one from Discipline 5. By definition those two will be different since Discipline 2, 3, and 4 all made updates to some part of the ADH. When running Discipline 6 how will we know what ADH to use? What if Discipline 5 makes changes to the same element of the ADH that was changed by Disciplines 2, 3, or 4? These kind of race conditions will be very hard to discover without explicit identification of what components are used.
sequenceDiagram
autonumber
participant Discipline 1
participant Discipline 2
participant Discipline 3
participant Discipline 4
participant Discipline 5
participant Discipline 6
Note left of Discipline 1: Input ADH
rect rgb(230, 255, 255)
Discipline 1->>Discipline 2: ADH Subset
loop "Until converged"
Discipline 2->>Discipline 3: ADH Subset
Discipline 3->>Discipline 2: ADH Subset
end
Discipline 3->>Discipline 4: ADH Subset
Discipline 4->>Discipline 6: ADH
end
Discipline 1->>Discipline 5: ADH
activate Discipline 5
Note left of Discipline 5: Execute
deactivate Discipline 5
Discipline 5->>Discipline 6: ADH
activate Discipline 6
Note left of Discipline 6: Execute
deactivate Discipline 6
Note right of Discipline 6: Final ADH
Another instance where it is really important to be explicit about what parts of the ADH a discipline touches is when there is an opportunity to execute analyses in parallel. In the below figure we want to execute three instances of Discipline 2. Each run is independent, but if we are not explicit about which part of the ADH are inputs and outputs the system will not be able to know that the three runs can be executed in parallel.
sequenceDiagram
participant Discipline 1
participant Discipline 2 (1)
participant Discipline 2 (2)
participant Discipline 2 (3)
participant Discipline 3
Note left of Discipline 1: ADH
activate Discipline 1
Note left of Discipline 1: Execute
deactivate Discipline 1
Discipline 1->>Discipline 2 (1): ADH
activate Discipline 2 (1)
Note left of Discipline 2 (1): Execute
deactivate Discipline 2 (1)
Discipline 2 (1)->>Discipline 3: ADH
Discipline 1->>Discipline 2 (2): ADH
activate Discipline 2 (2)
Note left of Discipline 2 (2): Execute
deactivate Discipline 2 (2)
Discipline 2 (2)->>Discipline 3: ADH
Discipline 1->>Discipline 2 (3): ADH
activate Discipline 2 (3)
Note left of Discipline 2 (3): Execute
deactivate Discipline 2 (3)
Discipline 2 (3)->>Discipline 3: ADH
activate Discipline 3
Note left of Discipline 3: Execute
deactivate Discipline 3
Note right of Discipline 3: Final ADH
On the other hand if each instance of Discipline 2 is explicit what data from the ADH they use the system can execute the three instances in parallel.
sequenceDiagram
participant Discipline 1
participant Discipline 2 (1)
participant Discipline 2 (2)
participant Discipline 2 (3)
participant Discipline 3
Note left of Discipline 1: Input ADH
activate Discipline 1
Note left of Discipline 1: Execute
deactivate Discipline 1
par
Discipline 1->>Discipline 2 (1): ADH Subset
activate Discipline 2 (1)
Note left of Discipline 2 (1): Execute
deactivate Discipline 2 (1)
Discipline 2 (1)->>Discipline 3: ADH Subset
and
Discipline 1->>Discipline 2 (2): ADH Subset
activate Discipline 2 (2)
Note left of Discipline 2 (2): Execute
deactivate Discipline 2 (2)
Discipline 2 (2)->>Discipline 3: ADH Subset
and
Discipline 1->>Discipline 2 (3): ADH Subset
activate Discipline 2 (3)
Note left of Discipline 2 (3): Execute
deactivate Discipline 2 (3)
Discipline 2 (3)->>Discipline 3: ADH Subset
end
activate Discipline 3
Note left of Discipline 3: Execute
deactivate Discipline 3
Note right of Discipline 3: Final ADH