Prev: Seek support for new slice syntax PEP.
Next: multiprocessing module
From: Pete Hunt on 14 Dec 2009 19:37
Hey, everyone -

I'm trying to use multiprocessing to create a process pool similar to
multiprocessing.Pool, except that it works across multiple hosts. For
various reasons, I would like the result of async_apply() to return a
Twisted Deferred object.

My first attempt at the implementation is:

class DeferredProxy(managers.BaseProxy):
_exposed_ = ("callback","errback","addCallback","addErrback")
def callback(self, v):
return self._callmethod("callback",(v,))

def errback(self, v):
return self._callmethod("errback",(v,))

def addCallback(self, cb):
return self._callmethod("addCallback",(cb,))

def addErrback(self, eb):
return self.__callmethod("adderrback",(eb,))

class DispatchManager(managers.BaseManager):
pass
DispatchManager.register("Deferred", defer.Deferred, DeferredProxy)

This, however, fails, because "cb" and "eb" cannot be pickled. I don't
actually need to pickle them, though, because I want them to run on
the machine on which they were created. Is there a way, in
multiprocessing, that I can pass some sort of "callback" around that
serves as a proxy to a remote function?

Thanks in advance,

Pete
 | 
Pages: 1
Prev: Seek support for new slice syntax PEP.
Next: multiprocessing module