Python 3.x – Python3: How to change the design of a class hierarchy to improve access to objects buried there?

I already asked this question in stackoverflow along with a part related to the serialization and in the code view only for the design part. Since the design-related part does not receive responses or comments about the stack exchange and was not addressed in the code view, I would like to ask about it here.

The problem: I have a complicated class hierarchy in which the classes are similar to each other, but each class contains a group of variables with more or less complicated status. To give you an impression, take a look at this example of minimal work:

#! / usr / bin / env python3
# - * - encoding: utf-8 - * -

import numpy as np

OptimizableVariable class:
the complicated state variable. It is intended to return mainly
floating point values. The state determines whether an optimization
later it will change the value ("V") or not ("F"). The "P" status
it is for a variable that depends on a series of other variables.
def __init __ (self, name = "", status = "F", ** kwargs): = name
self.status = state
own parameters = kwargs
self.eval_dict = {"F": self.eval_fixed,
"V": self.eval_variable,
"P": self.eval_pickup}

def eval_fixed (self):
returns self.parameters.get ("value", 0.0)

def eval_variable (self):
returns self.parameters.get ("value", 0.0)

def eval_pickup (self):
fun = self.parameters.get ("function", None)
optvar_arguments = self.parameters.get ("arguments", tuple ())

If the fun is none:
call_args = tuple ([ov.evaluate() for ov in optvar_arguments])
return fun (* call_args)

def evaluate (self):
self.eval_dict return[self.status]()

def setvalue (self, value):
if self.status == "F" or self.status == "V":
parameters of the self["value"] = value

OptimizableVariableContainer class:
Class that contains several nested OptVar and OptVarContainer objects
classes It is responsible for the management of OptVar of its sub-OptVarContainers
with their respective OptVar objects.
def __init __ (self, name = "", ** kwargs): = name
for (key, value_dict) in kwargs.items ():
setattr (self, key, OptimizableVariable (name = key, ** value_dict))

def getAllOptVars (self):

def addOptVarToDict (var,
dictOfOptVars = {},
idlist =[],
reductionkeystring = ""):
Accumulate optimizable variables in var and its linked objects.
Ignore links and double links.

@param var: object to be evaluated (object)
@param dictOfOptVars: optimizable variables found so far (dict dictation of objects)
@param idlist: ID of already evaluated objects (int list)
@param reduces the key chain: to generate the key dict (string)

if id (var) is not in idlist:
idlist.append (id (var))

If it is an instance (var, OptimizableVariableContainer):
for (k, v) in var .__ dict __. Elements ():
newredkeystring = averagekeystring + + "."
dictOfOptVars, idlist = addOptVarToDict (v,
identification list

elif isinstance (var, OptimizableVariable):
newreducedkeystring = averagekeystring +
dictOfOptVars[newreducedkeystring] = var
# other elifs are also possible here, looking for
# OptimizableVariables in lists or dictations within the
# class instance

returns dictOfOptVars, idlist

(dict_opt_vars, idlist) = addOptVarToDict (self, reducedkeystring = "")
back dict_opt_vars

def getStatusVVariables (self):
Obtain all optimizable variables with "V" status
optvars = self.getAllOptVars ()
he came back [ov for ov in optvars.values() if ov.status == "V"]

    def getVValues ​​(self):
Function to obtain all the state values ​​of the variables "V".
in a large np.array. Interface for functions scipy.optimize.
return np.array ([a.evaluate() for a in self.getStatusVVariables()])

def setVValues ​​(self, x):
Function to set all the values ​​of the active variables to the values.
in the large np.array x. Interface for functions scipy.optimize.
for i, var in enumerate (self.getStatusVVariables ()):
var.setvalue (x[i])

LocalCoordinates class (OptimizableVariableContainer):
Specific implementation of the OptVarContainer class
def __init __ (self, name = ""):
super (LocalCoordinates, self) .__ init __ (name = name,
** {"x": {"state": "F", "value": 0.},
"y": {"state": "F", "value": 0.},
"z": {"state": "F", "value": 0.}})

Surface class (OptimizableVariableContainer):
Specific implementation of the OptVarContainer class
def __init __ (self, name = ""):
super (Surface, self) .__ init __ (name = name,
** {"curvature": {"state": "F", "value": 0.0}}) = LocalCoordinates (name = name + "_lc")

Class system (OptimizableVariableContainer):
Specific implementation of the OptVarContainer class
def __init __ (self, name = ""):
super (System, auto) .__ init __ (name = name,
** {"some_property": {"status": "F", "value": 1.0},
"some_other_property": {"status": "F", "value": 0.0}})
self.surf1 = Surface (name = "surf1")
self.surf2 = Surface (name = "surf2")

def surf1_lc_on_circle (mysys):
Scalar function to verify that x, y of lc of surf1 are in a unit circle
return ( () ** 2 + () ** 2 - 1) ** 2

# optvars = mysys.getStatusVVariables ()
# numpy_vector = np.asarray ([ov.evaluate() for ov in optvars])

def main ():
# creating OptimizableVariableContainer with some nested
# OptimizableVariableContainers.
my_sys = System (name = "system")

# The behavior is intended to access the OptimizableVariable objects through
# scope within the class hierarchy.
print ( ())["value"] = 2.0
print ( = OptimizableVariable (name = "z", status = "P",
function = lambda x, y: x ** 2 + y ** 2,
arguments = (, = "V" = "V"
print ( ())

# The following construction is pretty ugly:
# a) due to the dict: the order is not fixed
# b) due to the recursion (and perhaps the lexical classification) is not fast
# c) objective: deliver these optvars in a numerical optimization
# through a matrix numpy => should be fast
optvarsdict = my_sys.getAllOptVars ()
for (key, ov) in optvarsdict.items ():
print ("% s (% s):% f"% (key, ov.status, ov.evaluate ()))

# Optimization of state variables "V" due to scalar optimization
Criterion # is an important point in the code. This is done manually in
# monitoring:

print (surf1_lc_on_circle (my_sys)) # check the value of the scalar function (> 0)
print (my_sys.getVValues ​​()) # get a numpy array of variables V
my_sys.setVValues ​​(np.array ([0., 1.])) # this pair is in a unit circle
# Note that since the values ​​come from a dict, it is not guaranteed
# that the matrix is [x, y] nor is it guaranteed that the order is preserved
print (surf1_lc_on_circle (my_sys)) # check the value of the scalar function (== 0)

# Notice that my_sys was modified (which is the expected behavior):
# Also note that my_sys.surf1.surf1_lc.z has changed since it is from
# status "P" (also expected behavior)
optvarsdict = my_sys.getAllOptVars ()
for (key, ov) in optvarsdict.items ():
print ("% s (% s):% f"% (key, ov.status, ov.evaluate ()))

yes __name__ == "__main__":

The question now is: How to better access the OptimizableVariable objects? I already thought about using some type of group object and using some proxy within the class hierarchy to have a link to the group. This grouping should not be a singleton, since it should be possible to manage more than one grouping at a time. So far, access to the variables OptimizableVariable within my own code is done through the function getAllOptVars. This is pretty ugly and only considered temporary. Is there a better alternative to this function?

To summarize and clarify my goals:

  1. The class hierarchy is fine and reflects the model context of our problem
  2. The Optimizable Variables belong to each object in the hierarchy and must also remain there due to the context of the model
  3. The access and subsequent processing of OptimizableVariables (that is, collecting them from the hierarchy and delivering them to an optimizer such as a numpy matrix) is not considered optimal. There I need some suggestions to make it better (that is, get rid of the instance and identification queries).
  4. A "good to have" would be: decouple the serialization of the variable variables and the version management of the classes in the hierarchy of objects (that is, it would be good to have an interface to take snapshots of certain objects in the hierarchy)
  5. Another "good to have" would be: How to incorporate Optimizable Variables shared and "singleton" Optimizable Variables better in this class hierarchy?

I am aware that there are no unique solutions for these design problems, but I need more information. Thank you!

PS: If the minimum work example is too long and does not show the purpose of the code, I'm sorry and I could reduce it.