Prev: Mark built-in module as deprecated
Next: Colour TIFF support (PIL or otherwise)
From: Thomas Jollans on 13 Jun 2010 13:15
Hi,

I'm writing some buffer-centric number-crunching routines in C for
Python code that uses array.array objects for storing/manipulating data.
I would like to:

1. allocate a buffer of a certain size
2. fill it
3. return it as an array.

I can't see any obvious way to do this with the array module, but I was
hoping somebody here might be able to help. My best shot would be to:

1. create a bytearray with PyByteArray_FromStringAndSize(NULL, byte_len)
2. fill its buffer
3. initialize an array from the bytearray.

The issue I have with this approach is that array will copy the data to
its own buffer. I'd much rather create an array of a certain size, get a
write buffer, and fill it directly -- is that possible?

I expect that numpy allows this, but I don't really want to depend on
numpy, especially as they haven't released a py3k version yet.

-- Thomas
From: Stephen Hansen on 13 Jun 2010 13:28
On 6/13/10 10:15 AM, Thomas Jollans wrote:
> Hi,
>
> I'm writing some buffer-centric number-crunching routines in C for
> Python code that uses array.array objects for storing/manipulating data.
> I would like to:


Take this with a grain of salt: I am *not* a C programmer, and my usage
of the Python/C API is extremely minimal.

However, I think you're looking for The Buffer Interface. Its, IIUC, the
way to read/write efficiently into existing data structures without
having to go all Pythony.

It changed in Py3k: http://www.python.org/dev/peps/pep-3118/

Again: I may be entirely wrong. Not my area of expertise. But I lurk a lot.

--

Stephen Hansen
... Also: Ixokai
... Mail: me+list/python (AT) ixokai (DOT) io
... Blog: http://meh.ixokai.io/

From: Stefan Behnel on 13 Jun 2010 14:41
Thomas Jollans, 13.06.2010 19:15:
> I'm writing some buffer-centric number-crunching routines in C for
> Python code that uses array.array objects for storing/manipulating data.
> I would like to:
>
> 1. allocate a buffer of a certain size
> 2. fill it
> 3. return it as an array.

Take a look at a) NumPy and b) Cython. You can also use Cython with the
array module, but NumPy is a much more common way to deal with "number
crunching routines", especially multi-dimentional arrays.

Cython interacts with both through the buffer interface, which Stephen
Hansen already mentioned, but it also has support for Python versions that
do not already have it, and it certainly makes the interaction between
Python code and C code really easy. It's also an optimising (mostly) Python
compiler that's used a lot for number crunching code, in case you are not
really into writing the code in C.

Stefan

From: Stephen Hansen on 13 Jun 2010 15:05
On 6/13/10 11:41 AM, Stefan Behnel wrote:
> Take a look at a) NumPy and b) Cython. You can also use Cython with the
> array module, but NumPy is a much more common way to deal with "number
> crunching routines", especially multi-dimentional arrays.

Does Cython support Py3k yet? The OP seemed to be concerned about using
numpy due to it not yet being ready for 3+.

If so, that's very awesome.

--

Stephen Hansen
... Also: Ixokai
... Mail: me+list/python (AT) ixokai (DOT) io
... Blog: http://meh.ixokai.io/

From: Stefan Behnel on 13 Jun 2010 15:17
Stephen Hansen, 13.06.2010 21:05:
> On 6/13/10 11:41 AM, Stefan Behnel wrote:
>> Take a look at a) NumPy and b) Cython. You can also use Cython with the
>> array module, but NumPy is a much more common way to deal with "number
>> crunching routines", especially multi-dimentional arrays.
>
> Does Cython support Py3k yet? The OP seemed to be concerned about using
> numpy due to it not yet being ready for 3+.

I didn't read that from the post, but you may be right.

Cython supports Py 2.3-3.1+, but NumPy still does not support Py3k (they
are actually considering to rewrite parts of it in Cython to simplify the
migration ;).

If the OP is targeting Py3, the array module might still be a viable
solution together with Cython.

Stefan

 |  Next  |  Last
Pages: 1 2 3
Prev: Mark built-in module as deprecated
Next: Colour TIFF support (PIL or otherwise)