Heavy malloc()/free() load and threads
in [Unix Programming]
|
From: Urs Thuermann on 11 Mar 2010 10:39 Casper H.S. Dik <Casper.Dik(a)Sun.COM> writes: > >The average rate will be roughly 40 to 50 allocations/deallocations in > >a strict LIFO order and there will be 30000 to 60000 items on the list. > > Per what? Oops, a typo. That should read "40 to 50 allocations/deallocations per second". > Never use your own memory management except in specific cases. OK. My plan was to start with malloc()/free() from libc and see if it copes well with the rate of calls to malloc() and free() and with the number of items allocated at a time. But I wanted to have an estimation from people here beforehand. Thanks. > >2. Are malloc() and free() thread-safe (according to POSIX and/or in > > typical libc implementations) or do I have to use a mutex? > > Yes, required by the standard. OK, thanks. Do you have a pointer? I probably overlooked something when searching opengroup.org and POSIX man pages. urs
From: Casper H.S. Dik on 11 Mar 2010 11:06 Urs Thuermann <urs(a)isnogud.escape.de> writes: >OK, thanks. Do you have a pointer? I probably overlooked something >when searching opengroup.org and POSIX man pages. It's in Section 2.9: All functions defined by this volume of IEEE Std 1003.1-2001 shall be thread-safe, except that the following functions need not be thread-safe:.. (Malloc is not listed) Casper -- Expressed in this posting are my opinions. They are in no way related to opinions held by my employer, Sun Microsystems. Statements on Sun products included here are not gospel and may be fiction rather than truth.
From: Måns Rullgård on 11 Mar 2010 11:12 Urs Thuermann <urs(a)isnogud.escape.de> writes: > Casper H.S. Dik <Casper.Dik(a)Sun.COM> writes: > >> >The average rate will be roughly 40 to 50 allocations/deallocations in >> >a strict LIFO order and there will be 30000 to 60000 items on the list. >> >> Per what? > > Oops, a typo. That should read "40 to 50 allocations/deallocations > per second". > >> Never use your own memory management except in specific cases. Almost all cases are specific. That said, one should only use custom memory management after the default malloc has been shown to be a problem. > OK. My plan was to start with malloc()/free() from libc and see if it > copes well with the rate of calls to malloc() and free() and with the > number of items allocated at a time. But I wanted to have an > estimation from people here beforehand. Thanks. There are usually better designs avoiding huge numbers of malloc and free calls. If, as you say, you have strict LIFO order, and you there is an upper bound to the total amount required at any one time, a simple stack allocator will be much faster and simpler. It will need about 10 lines of code. >> >2. Are malloc() and free() thread-safe (according to POSIX and/or in >> > typical libc implementations) or do I have to use a mutex? >> >> Yes, required by the standard. > > OK, thanks. Do you have a pointer? I probably overlooked something > when searching opengroup.org and POSIX man pages. http://www.opengroup.org/onlinepubs/000095399/functions/xsh_chap02_09.html#tag_02_09_01 -- M�ns Rullg�rd mans(a)mansr.com
From: Chris Friesen on 11 Mar 2010 11:32 On 03/11/2010 08:01 AM, Nicolas George wrote: > Urs Thuermann wrote in message > <ygfzl2fxh5k.fsf(a)janus.isnogud.escape.de>: >> 1. Will typical implementations of malloc()/free() in libc handle this >> load well? Or should I implement my own memory management? > > Code it, debug it, and test it. If you have performances problems, profile. > If the profiling shows that memory allocation is the bottleneck, then you > can think about changing it. Absolutely. There's no point messing with it if it's already fast enough. Chris
From: Rainer Weikusat on 11 Mar 2010 13:44
Chris Friesen <cbf123(a)mail.usask.ca> writes: > On 03/11/2010 08:01 AM, Nicolas George wrote: >> Urs Thuermann wrote in message >> <ygfzl2fxh5k.fsf(a)janus.isnogud.escape.de>: >>> 1. Will typical implementations of malloc()/free() in libc handle this >>> load well? Or should I implement my own memory management? >> >> Code it, debug it, and test it. If you have performances problems, profile. >> If the profiling shows that memory allocation is the bottleneck, then you >> can think about changing it. > > Absolutely. There's no point messing with it if it's already fast enough. Except that 'it' doesn't presently exist. And once 'it' has been written with ad hoc usage of the malloc-interface in mind, the amount of work which was necessary to change to another approach will likely be prohibitive. It is conceivable that runtime performance is at all irrelavant, usually because the program isn't supposed to be operating continously as one of (potentially) many tasks on some loosely monitored system. In this case, using the easiest way (which may be malloc) is appropriate, although such a program probably shouldn't usually be written in C anymore nowadays. Otherwise, it is better to design such that potential problems are avoided than to hope to be will able to find a quick(!) workaround when a problem has already occurred. |