Prev: Loading pyd fails with "The operating system cannot run %1"
Next: Python Script Cannot Write to Directory
From: Navkirat Singh on 4 Aug 2010 00:00
Hey guys,

I am using a multiprocessing program, where the new process is supposed to change a variable in the main class that it branches out from. This is somehow not working, following is an approximate code. Would really appreciate any insight into this matter:


var = {}

class Something():

def set_var(self):
global var
var = somevalue

def get_var(self):
return var

def newprocess(self):
self.set_var()

def do_multiprocessing(self):
while true:
self.get_var()
new_process = process(target=newprocess)
new_process.start()


I am really confused here !

Any help would be awesome : )

Regards,
Nav

From: Matteo Landi on 4 Aug 2010 04:18
Usually, modify global variables in a multi-thread/multi-process
scenario is not the right to operate: you better re-implement your
solution in a way that the shared resource is either protected with
synchronized objects or accessed by a single thread/process (and in
this case, it won't be a shared resource anymore).

Think about the the needs of the shared resources: in this extremely
simple example, you can see that the increment of the session number
could be done by the server before spawning the child, but obviously
this is not a real scenario.

If you can't give up with shared resources, I recommend you to create
a synchronized object owned by the server but shared with the children
(take a look at the args keywords of the Process constructor).

Regards.

On Wed, Aug 4, 2010 at 9:47 AM, Navkirat Singh <navkirats(a)gmail.com> wrote:
>  : (
> False alarm, the earlier solution breaks multiprocessing. Whats happening
> here is the child needs to change a variable in the parent process, So I
> think I am looking at shared memory (maybe). Any suggestions?
> Regards,
> Nav
>
>
> On 04-Aug-2010, at 12:41 PM, Navkirat Singh wrote:
>
> Thanks a lot guys !!
> I solved the problem:
> In the lines:
>>>
>>> new_process = process(target=newprocess)
>>>                        new_process.start()
>
> The target=newprocess is pointing towards a variable, instead of a function.
> So, appending a () will make it goto that function, thereby changing the
> global variable : )
> Thanks,
> Nav
>
> On 04-Aug-2010, at 11:42 AM, Daniel da Silva wrote:
>
> Your problem lies somewhere in the use of the Process class, not with global
> variables.
>
> If you replace your "p = ..." and "p.start()" lines with a direct call to
> self.handle_connection(), your code works as expected. I don't know much
> about the multiprocessing module, so I can't really comment on what you're
> doing wrong, but I hope this points you in the right direction.
>
> Sorry I couldn't be of more help,
>
> Daniel
>
>
> On Tue, Aug 3, 2010 at 9:48 PM, Navkirat Singh <navkirats(a)gmail.com> wrote:
>>
>> On 04-Aug-2010, at 9:46 AM, Daniel da Silva wrote:
>>
>> Please post approximate code that actually works and displays the problem.
>>
>> On Tue, Aug 3, 2010 at 9:00 PM, Navkirat Singh <navkirats(a)gmail.com>
>> wrote:
>>>
>>> Hey guys,
>>>
>>> I am using a multiprocessing program, where the new process is supposed
>>> to change a variable in the main class that it branches out from. This is
>>> somehow not working, following is an approximate code. Would really
>>> appreciate any insight into this matter:
>>>
>>>
>>> var = {}
>>>
>>> class Something():
>>>
>>>        def set_var(self):
>>>                global var
>>>                var = somevalue
>>>
>>>        def get_var(self):
>>>                return var
>>>
>>>        def newprocess(self):
>>>                self.set_var()
>>>
>>>        def do_multiprocessing(self):
>>>                while true:
>>>                        self.get_var()
>>>                        new_process = process(target=newprocess)
>>>                        new_process.start()
>>>
>>>
>>> I am really confused here !
>>>
>>> Any help would be awesome : )
>>>
>>> Regards,
>>> Nav
>>>
>>> --
>>> http://mail.python.org/mailman/listinfo/python-list
>>
>>
>> This is a working code, streamlined, but it is where the problem is:
>> from multiprocessing import *
>> dicts = 0
>> print('global ', dicts)
>> class WebServer():
>> def set_sessionInfo(self):
>> global dicts
>> dicts = dicts + 1
>> def get_sessionInfo(self):
>> return dicts
>> def handle_connection(self):
>> self.set_sessionInfo()
>> def serve_forever(self):
>> for x in range(10):
>> p = Process(target=self.handle_connection)
>> p.start()
>> print(self.get_sessionInfo())
>> ws = WebServer()
>> ws.serve_forever()
>> print(dicts)
>>
>
>
>
>
> --
> http://mail.python.org/mailman/listinfo/python-list
>
>



--
Matteo Landi
http://www.matteolandi.net/
From: Jean-Michel Pichavant on 4 Aug 2010 05:24
Navkirat Singh wrote:
>
> On 04-Aug-2010, at 9:46 AM, Daniel da Silva wrote:
>
>> Please post approximate code that actually works and displays the
>> problem.
>>
>> On Tue, Aug 3, 2010 at 9:00 PM, Navkirat Singh <navkirats(a)gmail.com
>> <mailto:navkirats(a)gmail.com>> wrote:
>>
>> Hey guys,
>>
>> I am using a multiprocessing program, where the new process is
>> supposed to change a variable in the main class that it branches
>> out from. This is somehow not working, following is an
>> approximate code. Would really appreciate any insight into this
>> matter:
>>
>>
>> var = {}
>>
>> class Something():
>>
>> def set_var(self):
>> global var
>> var = somevalue
>>
>> def get_var(self):
>> return var
>>
>> def newprocess(self):
>> self.set_var()
>>
>> def do_multiprocessing(self):
>> while true:
>> self.get_var()
>> new_process = process(target=newprocess)
>> new_process.start()
>>
>>
>> I am really confused here !
>>
>> Any help would be awesome : )
>>
>> Regards,
>> Nav
>>
>> --
>> http://mail.python.org/mailman/listinfo/python-list
>>
>>
>
> This is a working code, streamlined, but it is where the problem is:
>
> from multiprocessing import *
>
> dicts = 0
> print('global ', dicts)
>
> class WebServer():
> def set_sessionInfo(self):
> global dicts
> dicts = dicts + 1
> def get_sessionInfo(self):
> return dicts
>
> def handle_connection(self):
> self.set_sessionInfo()
> def serve_forever(self):
> for x in range(10):
> p = Process(target=self.handle_connection)
> p.start()
> print(self.get_sessionInfo())
> ws = WebServer()
> ws.serve_forever()
> print(dicts)
>
>

"As mentioned above, when doing concurrent programming it is usually
best to avoid using shared state as far as possible. This is
particularly true when using multiple processes.

However, if you really do need to use some shared data then
multiprocessing provides a couple of ways of doing so."

source :

http://docs.python.org/library/multiprocessing.html


Read it, everything is explained with examples, including how to
properly solve your problem.
"When using multiple processes, one generally uses message passing for
communication between processes and avoids having to use any
synchronization primitives like locks."

JM
 | 
Pages: 1
Prev: Loading pyd fails with "The operating system cannot run %1"
Next: Python Script Cannot Write to Directory