On Jan 11, 4:29 pm, dg.google.gro...@thesamovar.net wrote:
> Hi all,
>
> I'm part of a small team writing a Python package for a scientific
> computing project. The idea is to make it easy to use for relatively
> inexperienced programmers. As part of that aim, we're using what we're
> calling 'magic functions', and I'm a little bit concerned that they
> are dangerous code. I'm looking for advice on what the risks are (e.g.
> possibility of introducing subtle bugs, code won't be compatible with
> future versions of Python, etc.).
>
> Quick background: Part of the way our package works is that you create
> a lot of objects, and then you create a new object which collects
> together these objects and operates on them. We originally were
> writing things like:
>
> obj1 = Obj(params1)
> obj2 = Obj(params2)
> ...
> bigobj = Bigobj(objects=[obj1,obj2])
> bigobj.run()
>
> This is fine, but we decided that for clarity of these programs, and
> to make it easier for inexperienced programmers, we would like to be
> able to write something like:
>
> obj1 = Obj(params1)
> obj2 = Obj(params2)
> ...
> run()
>
> The idea is that the run() function inspects the stack, and looks for
> object which are instances of class Obj, creates a Bigobj with those
> objects and calls its run() method.
>
> So, any comments on that approach?
>
> I'm including the code I've written to do this, and if you have time
> to look through it, I'd also be very grateful for any more specific
> comments about the way I've implemented it (in particular, can it be
> made faster, is my program creating cycles that stop the garbage
> collection from working, etc.). I hope the code will be formatted
> correctly:
>
> def
> getInstances(instancetype,level=1,includeglobals=True,containersearchdepth=1,exclude={},predicate=lambda
> x:True):
> """Find all instances of a given class at a given level in the
> stack
> """
> vars = {}
> # Note: we use level+1 because level refers to the level relative
> to the function calling this one
> if includeglobals: vars.update(stack()[level+1][0].f_globals)
> vars.update(stack()[level+1][0].f_locals)
> # Note that you can't extract the names from vars.itervalues() so
> we provide via knownnames the names vars.iterkeys(),
> # containersearchdepth+1 is used because vars.itervalues() is the
> initial container from the point of view of this
> # function, but not from the point of view of the person calling
> getInstances
> objs, names =
> extractInstances(instancetype,vars.itervalues(),containersearchdepth
> +1,knownnames=vars.iterkeys(),exclude=exclude,predicate=predicate)
> return (objs,names)
>
> def
> extractInstances(instancetype,container,depth,containingname='vars()',knownnames=None,exclude={},predicate=lambda
> x:True):
> if depth<=0: return ([],[])
> if isinstance(container,str): return ([],[]) # Assumption: no need
> to search through strings
> # Ideally, this line wouldn't be here, but it seems to cause
> programs to crash, probably because
> # some of the simulator objects are iterable but shouldn't be
> iterated over normally
> # TODO: Investigate what is causing this to crash, and possibly
> put in a global preference to turn this line off?
> if not isinstance(container,
> (list,tuple,dict,type({}.itervalues()))): return ([],[])
> # Note that knownnames is only provided by the initial call of
> extractInstances and the known
> # names are from the dictionary of variables. After the initial
> call, names can only come from
> # the __name__ attribute of a variable if it has one, and that is
> checked explicitly below
> if knownnames is None:
> knewnames = False
> knownnames = repeat(containingname)
> else:
> knewnames = True
> objs = []
> names = []
> try: # container may not be a container, if it isn't, we'll
> encounter a TypeError
> for x,name in zip(container,knownnames):
> # Note that we always have a name variable defined, but if
> knewnames=False then this is just
> # a copy of containingname, so the name we want to give it
> in this instance is redefined in this
> # case. We have to use this nasty check because we want to
> iterate over the pair (x,name) as
> # variables in the same position in the container have the
> same name, and we can't necessarily
> # use __getitem__
> if hasattr(x,'__name__'): name = x.__name__
> elif not knewnames: name = 'Unnamed object, id =
> '+str(id(x))+', contained in: '+containingname
> if isinstance(x,instancetype):
> if x not in exclude and predicate(x):
> objs.append(x)
> names.append(name)
> else: # Assumption: an object of the instancetype is not
> also a container we want to search in.
> # Note that x may not be a container, but then
> extractInstances will just return an empty list
> newobjs, newnames =
> extractInstances(instancetype,x,depth-1,containingname=name,predicate=predicate)
> objs += newobjs
> names += newnames
> return (objs,names)
> except: # if we encounter a TypeError from the for loop, we just
> return an empty pair, container wasn't a container
> return ([],[])
>
> In case that doesn't work, here it is without the comments:
>
> def
> getInstances(instancetype,level=1,includeglobals=True,containersearchdepth=1,exclude={},predicate=lambda
> x:True):
> vars = {}
> if includeglobals: vars.update(stack()[level+1][0].f_globals)
> vars.update(stack()[level+1][0].f_locals)
> objs, names =
> extractInstances(instancetype,vars.itervalues(),containersearchdepth
> +1,knownnames=vars.iterkeys(),exclude=exclude,predicate=predicate)
> return (objs,names)
>
> def
> extractInstances(instancetype,container,depth,containingname='vars()',knownnames=None,exclude={},predicate=lambda
> x:True):
> if depth<=0: return ([],[])
> if isinstance(container,str): return ([],[])
> if not isinstance(container,
> (list,tuple,dict,type({}.itervalues()))): return ([],[])
> if knownnames is None:
> knewnames = False
> knownnames = repeat(containingname)
> else:
> knewnames = True
> objs = []
> names = []
> try:
> for x,name in zip(container,knownnames):
> if hasattr(x,'__name__'): name = x.__name__
> elif not knewnames: name = 'Unnamed object, id =
> '+str(id(x))+', contained in: '+containingname
> if isinstance(x,instancetype):
> if x not in exclude and predicate(x):
> objs.append(x)
> names.append(name)
> else:
> newobjs, newnames =
> extractInstances(instancetype,x,depth-1,containingname=name,predicate=predicate)
> objs += newobjs
> names += newnames
> return (objs,names)
> except:
> return ([],[])
If you are the author of class Obj, then why not just make the class
maintain a record of any objects that have been instantiated?
That way, run could simply call a class method to obtain a list of all
the objects it needs.