Prev: which
Next: Terminating threaded programs
From: Gaetan de Menten on 5 Feb 2010 09:25
Hi all,

I am trying to write an (optional!) C extension for SQLAlchemy, and I
am struggling to make an extension type picklable *and* using the same
format as the pure Python version (so that computers with the C
extension can unpickle pickles from computers without it and
vice-versa). Also the extension type (MyBase in the example code
below) only serves as a base class for the type that get pickled
(MyPythonType). MyPythonType also has further subclasses and those
should be picklable too.

My setup is as follow:

On the Python side:
================

try:
from cextension import MyBase
except ImportError:
class MyBase(object):
__slots__ = ('a', 'b', 'c')
def __init__(self, a, b, c):
self.a, self.b, self.c = a, b, c

def __getstate__(self):
...

def __setstate__(self, state):
...
...


class MyPythonType(MyBase):
...

On the .c side:
===========
I implemented a reduce method, which returns

Py_BuildValue("(O(s)N)", Py_TYPE(self), "__state__", state);

and in my BaseRowProxy_init method, I check the number of arguments,
whether the "__state__" marker is present and unpack the state if it
is. This makes the type nicely pickleable, but this is obviously not
the same "pickle format" as the Python version.

What would you recommend in this kind of situation? I'm a bit tired of
trial and error... My last hope is to define __reduce__ on the Python
side too, and make it use a module-level "reconstructor" function as
follow:

def mypythontype_reconstructor(cls, state):
obj = object.__new__(cls, state):
....
return obj

Will this work? Any other idea?

Thanks in advance,
--
Gaëtan de Menten
From: Gabriel Genellina on 9 Feb 2010 05:11
En Fri, 05 Feb 2010 11:25:02 -0300, Gaetan de Menten <gdementen(a)gmail.com>
escribi�:

> I am trying to write an (optional!) C extension for SQLAlchemy, and I
> am struggling to make an extension type picklable *and* using the same
> format as the pure Python version (so that computers with the C
> extension can unpickle pickles from computers without it and
> vice-versa). [...]
> My last hope is to define __reduce__ on the Python
> side too,

That seems reasonable.

> and make it use a module-level "reconstructor" function as
> follow:
>
> def mypythontype_reconstructor(cls, state):
> obj = object.__new__(cls, state):
> ....
> return obj
>
> Will this work? Any other idea?

If you're going to call object.__new__(cls, state) anyway, then just
return the object's type as the first item in __reduce__; no need to
create such global function


--
Gabriel Genellina

 | 
Pages: 1
Prev: which
Next: Terminating threaded programs