Prev: Pick items from list with probability based upon property of list member ?
Next: Pick items from list with probability based upon property of list member ?
From: hywhy on 20 Jun 2010 08:43


first sorry for my poor english. Is there any problem in the follow code?
thanks!


from multiprocessing.managers import BaseManager
import Queue

class CrawlerManager(BaseManager):
pass

downloader_queue = Queue.Queue()

downloader_queue.put('hello')

CrawlerManager.register('get_downloader_queue', callable=lambda:
downloader_queue)

mgr = CrawlerManager()
mgr.start()

q = mgr.get_downloader_queue()



error:

pickle.PicklingError: Can't pickle <function <lambda> at 0x00C02F70>: it's
not found as __parents_main__.<lambda>
Traceback (most recent call last):

print q.get()
--
View this message in context: http://old.nabble.com/question-about-multiprocessing-tp28940614p28940614.html
Sent from the Python - python-list mailing list archive at Nabble.com.

From: Gabriel Genellina on 22 Jun 2010 07:49
En Sun, 20 Jun 2010 09:43:09 -0300, hywhy <hywhy(a)live.com> escribi�:

> from multiprocessing.managers import BaseManager
> import Queue
>
> class CrawlerManager(BaseManager):
> pass
>
> downloader_queue = Queue.Queue()
> downloader_queue.put('hello')
>
> CrawlerManager.register('get_downloader_queue', callable=lambda:
> downloader_queue)
>
> mgr = CrawlerManager()
> mgr.start()
>
> q = mgr.get_downloader_queue()
>
>
>
> error:
>
> pickle.PicklingError: Can't pickle <function <lambda> at 0x00C02F70>:
> it's
> not found as __parents_main__.<lambda>

Multiprocessing uses pickle to transfer data between processes. You can't
pickle a lambda expression - functions must have a name and be available
at the outermost module level. (See "Programming guidelines" in the
multiprocessing documentation, and the pickle module)
But you can't share a Queue object - use the multiprocessing one (see
"Exchanging data between processes").

--
Gabriel Genellina

From: hywhy on 24 Jun 2010 09:54

thanks，friend！
I wanna use Queue to share objects，but on windows， the multiprocessing
module can‘t do this。 Is there any way to solve this problem！
thanks
--
View this message in context: http://old.nabble.com/question-about-multiprocessing-tp28940614p28982744.html
Sent from the Python - python-list mailing list archive at Nabble.com.

From: News123 on 24 Jun 2010 16:55
hywhy wrote:
> thanks，friend！
> I wanna use Queue to share objects，but on windows， the multiprocessing
> module can't do this。 Is there any way to solve this problem！
> thanks



With multiprocessing.SyncManager you should be able to exchange Queues
between different processes.

From: hywhy on 26 Jun 2010 10:54

thanks,firend. I have a try with SyncManager, Is there any problem in my
code?


from multiprocessing.managers import SyncManager,BaseProxy

import multiprocessing
import Queue

class ResourceController(object):

def __init__(self):
self.text = 'Hello world!'
self.queue = multiprocessing.Queue()

self.queue.put('I am queue')

def say(self):
print self.text

def get_queue(self):
return self.queue

class ResourceProxy(BaseProxy):

def say_hello(self):
return self._callmethod('say')

def get_queue(self):
return self._callmethod('get_queue')


class CrawlerManager(SyncManager):

def __init__(self):
SyncManager.__init__(self)
self.register('ResourceController', ResourceController,
ResourceProxy)



if __name__ == '__main__':
cm = CrawlerManager()
cm.start()

rc = cm.ResourceController()
rc.say_hello()
q = rc.get_queue()

print q.get()

--
View this message in context: http://old.nabble.com/question-about-multiprocessing-tp28940614p29000781.html
Sent from the Python - python-list mailing list archive at Nabble.com.

 | 
Pages: 1
Prev: Pick items from list with probability based upon property of list member ?
Next: Pick items from list with probability based upon property of list member ?