Prev: Anybody use web2py?
Next: Mechanize/ClientForm - How to select IgnoreControl button and submit form
From: lordofcode on 23 Dec 2009 04:41
Hi All

Not an expert in Python, so sorry if this sounds like a silly
question.
I went through other few threads in the mailing list but they are not
helping me much.
I have run into a problem related to dynamically loading and unloading
a module.
I need to dynamically load a module and unload it and load another
module.

For example I have many files(All files in Python are modules right?)
like mobile_1.py ,mobile_2.py, mobile_3.py etc.. in my project folder
which contains classes and methods with same name but different
functionality.(am afraid I cannot change this structure as these files
are generated randomly by the user)

So initially when my program starts I have to load a default module. I
do this as follows:
##############################
>>MODULE_name = "mobile_1"
>>exec "from "+MODULE_name+" import *"
##############################
And use the methods defined in "mobile_1.py" file

Now as the application continues , I may have to use the methods
defined in "mobile_2.py" or "mobile_3.py" etc instead of the
previously loaded module,which I incorrectly try to do as below:
####################
>>MODULE_name = "mobile_2"
>>exec "from "+MODULE_name+" import *"
#####################
The above import does not have any impact and the methods called from
my application still pertain to mobile_1.py as its still in the
current namespace(?).
I tried below code with del(), reload() etc but could not figure it
out.
###Code to unload a dll####
>>del sys.modules[MODULE_name] #==> does not delete the reference in namespace


1)How do I unload a module dynamically and completely remove the
references in the module so a new module with same name references can
be loaded?
2)Are there any alternative way to do the above requirement?
Currently I am working around by restarting the whole initial setup
for each new module which is unnecessary waste.Can I avoid this
"reset"?

From: Lie Ryan on 23 Dec 2009 06:08
On 12/23/2009 8:41 PM, lordofcode wrote:
> Hi All
>
> Not an expert in Python, so sorry if this sounds like a silly
> question.
> I went through other few threads in the mailing list but they are not
> helping me much.
> I have run into a problem related to dynamically loading and unloading
> a module.
> I need to dynamically load a module and unload it and load another
> module.
>
> For example I have many files(All files in Python are modules right?)
> like mobile_1.py ,mobile_2.py, mobile_3.py etc.. in my project folder
> which contains classes and methods with same name but different
> functionality.(am afraid I cannot change this structure as these files
> are generated randomly by the user)
>
> So initially when my program starts I have to load a default module. I
> do this as follows:
> ##############################
>>> MODULE_name = "mobile_1"
>>> exec "from "+MODULE_name+" import *"
> ##############################
> And use the methods defined in "mobile_1.py" file

You probably shouldn't use "from module import *", use the __import__
function:

name = __import__('one')
name.foo() # defined in `one`
name = __import__('two')
name.bar() # defined in `two`

> Now as the application continues , I may have to use the methods
> defined in "mobile_2.py" or "mobile_3.py" etc instead of the
> previously loaded module,which I incorrectly try to do as below:
> ####################
>>> MODULE_name = "mobile_2"
>>> exec "from "+MODULE_name+" import *"
> #####################
> The above import does not have any impact and the methods called from
> my application still pertain to mobile_1.py as its still in the
> current namespace(?).

that will just imports mobile_2's function into the current namespace,
possibly overriding the previously defined names from mobile_1 but it
won't delete anything that doesn't get overriden.

> I tried below code with del(), reload() etc but could not figure it
> out.
> ###Code to unload a dll####
>>> del sys.modules[MODULE_name] #==> does not delete the reference in namespace

> 1)How do I unload a module dynamically and completely remove the
> references in the module so a new module with same name references can
> be loaded?
> 2)Are there any alternative way to do the above requirement?
> Currently I am working around by restarting the whole initial setup
> for each new module which is unnecessary waste.Can I avoid this
> "reset"?

You can start a python subprocess for each new module you're loading.
That restarts the whole thing, but python loads fairly quickly and it
ensures that there won't be any leftover side effects caused by the
execution of the previous module.
From: Jean-Michel Pichavant on 23 Dec 2009 07:37
lordofcode wrote:
> Hi All
>
> Not an expert in Python, so sorry if this sounds like a silly
> question.
> I went through other few threads in the mailing list but they are not
> helping me much.
> I have run into a problem related to dynamically loading and unloading
> a module.
> I need to dynamically load a module and unload it and load another
> module.
>
> For example I have many files(All files in Python are modules right?)
> like mobile_1.py ,mobile_2.py, mobile_3.py etc.. in my project folder
> which contains classes and methods with same name but different
> functionality.(am afraid I cannot change this structure as these files
> are generated randomly by the user)
>
> So initially when my program starts I have to load a default module. I
> do this as follows:
> ##############################
>
>>> MODULE_name = "mobile_1"
>>> exec "from "+MODULE_name+" import *"
>>>
> ##############################
> And use the methods defined in "mobile_1.py" file
>
> Now as the application continues , I may have to use the methods
> defined in "mobile_2.py" or "mobile_3.py" etc instead of the
> previously loaded module,which I incorrectly try to do as below:
> ####################
>
>>> MODULE_name = "mobile_2"
>>> exec "from "+MODULE_name+" import *"
>>>
> #####################
> The above import does not have any impact and the methods called from
> my application still pertain to mobile_1.py as its still in the
> current namespace(?).
> I tried below code with del(), reload() etc but could not figure it
> out.
> ###Code to unload a dll####
>
>>> del sys.modules[MODULE_name] #==> does not delete the reference in namespace
>>>
>
>
> 1)How do I unload a module dynamically and completely remove the
> references in the module so a new module with same name references can
> be loaded?
> 2)Are there any alternative way to do the above requirement?
> Currently I am working around by restarting the whole initial setup
> for each new module which is unnecessary waste.Can I avoid this
> "reset"?
>
>
1/ Do not use from x import *
2/ How many modules would you like to import in the end ? If this number
is bound I would suggest to use a static approach:

import mod1
import mod2
import mod3
mod4 = __import__('mod4') # equivalent to import mod4

for mod in [mod1, mod2, mod3, mod4]:
# will call func1 on each modules
print "now using module %s functions " % mod.__file__
mod.func1()
mod.func2()

With this approach it's pretty much easy to call the correct
methods/attributes, as every symbols is scopped by module namespace. You
will have less hard time debugging your code.


3/ if you really need to unload the previous module, it's a little bit
tedious.

import mod1
del mod1
sys.modules['mod1'] = None # will unload mod1 assuming mod1 was the only
reference to that module.

But believe me, you don't want to mess up with the python import mechanism.

JM
From: Jean-Michel Pichavant on 23 Dec 2009 07:54
Jean-Michel Pichavant wrote:
> lordofcode wrote:
>> Hi All
>>
>> Not an expert in Python, so sorry if this sounds like a silly
>> question.
>> I went through other few threads in the mailing list but they are not
>> helping me much.
>> I have run into a problem related to dynamically loading and unloading
>> a module.
>> I need to dynamically load a module and unload it and load another
>> module.
>>
>> For example I have many files(All files in Python are modules right?)
>> like mobile_1.py ,mobile_2.py, mobile_3.py etc.. in my project folder
>> which contains classes and methods with same name but different
>> functionality.(am afraid I cannot change this structure as these files
>> are generated randomly by the user)
>>
>> So initially when my program starts I have to load a default module. I
>> do this as follows:
>> ##############################
>>
>>>> MODULE_name = "mobile_1"
>>>> exec "from "+MODULE_name+" import *"
>>>>
>> ##############################
>> And use the methods defined in "mobile_1.py" file
>>
>> Now as the application continues , I may have to use the methods
>> defined in "mobile_2.py" or "mobile_3.py" etc instead of the
>> previously loaded module,which I incorrectly try to do as below:
>> ####################
>>
>>>> MODULE_name = "mobile_2"
>>>> exec "from "+MODULE_name+" import *"
>>>>
>> #####################
>> The above import does not have any impact and the methods called from
>> my application still pertain to mobile_1.py as its still in the
>> current namespace(?).
>> I tried below code with del(), reload() etc but could not figure it
>> out.
>> ###Code to unload a dll####
>>
>>>> del sys.modules[MODULE_name] #==> does not delete the reference
>>>> in namespace
>>>>
>>
>>
>> 1)How do I unload a module dynamically and completely remove the
>> references in the module so a new module with same name references can
>> be loaded?
>> 2)Are there any alternative way to do the above requirement?
>> Currently I am working around by restarting the whole initial setup
>> for each new module which is unnecessary waste.Can I avoid this
>> "reset"?
>>
>>
> 1/ Do not use from x import *
> 2/ How many modules would you like to import in the end ? If this
> number is bound I would suggest to use a static approach:
>
> import mod1
> import mod2
> import mod3
> mod4 = __import__('mod4') # equivalent to import mod4
>
> for mod in [mod1, mod2, mod3, mod4]:
> # will call func1 on each modules
> print "now using module %s functions " % mod.__file__
> mod.func1()
> mod.func2()
>
> With this approach it's pretty much easy to call the correct
> methods/attributes, as every symbols is scopped by module namespace.
> You will have less hard time debugging your code.
>
>
> 3/ if you really need to unload the previous module, it's a little bit
> tedious.
>
> import mod1
> del mod1
> sys.modules['mod1'] = None # will unload mod1 assuming mod1 was the
> only reference to that module.
>
> But believe me, you don't want to mess up with the python import
> mechanism.
>
> JM
You may also want to look at the imp module,
http://docs.python.org/library/imp.html
It could help you in your task.

JM
From: Steven D'Aprano on 23 Dec 2009 08:40
On Wed, 23 Dec 2009 13:37:06 +0100, Jean-Michel Pichavant wrote:

> 3/ if you really need to unload the previous module, it's a little bit
> tedious.
>
> import mod1
> del mod1
> sys.modules['mod1'] = None

Assigning sys.modules[name] to None is not the same as deleting the
entry. None has special meaning to imports from packages, and for modules
it is interpreted as meaning that the module doesn't exist.

>>> import math
>>> del math
>>> sys.modules['math'] = None
>>> import math
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
ImportError: No module named math



> # will unload mod1 assuming mod1 was the only
> reference to that module.

Which is highly unlikely. Any classes or functions from the module will
keep the module alive.


> But believe me, you don't want to mess up with the python import
> mechanism.

Unless you understand how it works.


--
Steven
 |  Next  |  Last
Pages: 1 2 3
Prev: Anybody use web2py?
Next: Mechanize/ClientForm - How to select IgnoreControl button and submit form