Heavy malloc()/free() load and threads
in [Unix Programming]
|
From: Ersek, Laszlo on 15 Mar 2010 18:29 In article <87vdcxwd1l.fsf(a)fever.mssgmbh.com>, Rainer Weikusat <rweikusat(a)mssgmbh.com> writes: > lacos(a)ludens.elte.hu (Ersek, Laszlo) writes: >> (On a side note, you chose to ignore my earlier question (or so it >> seems) targeted at the portability of those 50-300 SLOC custom >> allocators and whether they are based on malloc().) > > malloc is a libray routine on UNIX(*) which uses some facility the > kernel provides for actually allocating address space (not memory, > strictly speaking). This will usually be brk/ sbrk or mmap. Because > library routines will likely call malloc behind ones back, using brk/ > sbrk in a more sophisticated custom allocator would be asking for > trouble. Which leaves mmap. While brk is no longer a 'standardized' > UNIX(*) routine, it will be available on UNIX(*) because existing > allocators (eg, dlmalloc) use it. mmap is a standardized > UNIX(*)-routine. Thank you for explaining. Would you please describe the arguments you pass to mmap() (if they are not based on MAP_ANONYMOUS)? Thank you very much, lacos
From: Scott Lurndal on 15 Mar 2010 19:56 lacos(a)ludens.elte.hu (Ersek, Laszlo) writes: >In article <87vdcxwd1l.fsf(a)fever.mssgmbh.com>, >Rainer Weikusat <rweikusat(a)mssgmbh.com> writes: > >> lacos(a)ludens.elte.hu (Ersek, Laszlo) writes: > >>> (On a side note, you chose to ignore my earlier question (or so it >>> seems) targeted at the portability of those 50-300 SLOC custom >>> allocators and whether they are based on malloc().) >> >> malloc is a libray routine on UNIX(*) which uses some facility the >> kernel provides for actually allocating address space (not memory, >> strictly speaking). This will usually be brk/ sbrk or mmap. Because >> library routines will likely call malloc behind ones back, using brk/ >> sbrk in a more sophisticated custom allocator would be asking for >> trouble. Which leaves mmap. While brk is no longer a 'standardized' >> UNIX(*) routine, it will be available on UNIX(*) because existing >> allocators (eg, dlmalloc) use it. mmap is a standardized >> UNIX(*)-routine. > >Thank you for explaining. Would you please describe the arguments you >pass to mmap() (if they are not based on MAP_ANONYMOUS)? > Typically, a file descriptor derived from opening /dev/zero. scott
From: Ersek, Laszlo on 15 Mar 2010 20:26 In article <YOznn.650022$L8.160111(a)news.usenetserver.com>, scott(a)slp53.sl.home (Scott Lurndal) writes: > lacos(a)ludens.elte.hu (Ersek, Laszlo) writes: >>In article <87vdcxwd1l.fsf(a)fever.mssgmbh.com>, >>Rainer Weikusat <rweikusat(a)mssgmbh.com> writes: >>> malloc is a libray routine on UNIX(*) which uses some facility the >>> kernel provides for actually allocating address space (not memory, >>> strictly speaking). This will usually be brk/ sbrk or mmap. Because >>> library routines will likely call malloc behind ones back, using brk/ >>> sbrk in a more sophisticated custom allocator would be asking for >>> trouble. Which leaves mmap. While brk is no longer a 'standardized' >>> UNIX(*) routine, it will be available on UNIX(*) because existing >>> allocators (eg, dlmalloc) use it. mmap is a standardized >>> UNIX(*)-routine. >> >>Thank you for explaining. Would you please describe the arguments you >>pass to mmap() (if they are not based on MAP_ANONYMOUS)? >> > > Typically, a file descriptor derived from opening /dev/zero. Thanks for the answer. However, the pathname /dev/zero is not standardized. Assuming it was and it referred to a character special with the usual behavior, mmap()'s support for character specials is still unspecified by SUSv[1-4]. (Please note that I'm not on a crusade to prove that this can't be done portably. I'd like to learn *how* it can be done portably if it is possible at all.) Thank you, lacos
From: Chris Friesen on 15 Mar 2010 21:12 On 03/15/2010 06:26 PM, Ersek, Laszlo wrote: > However, the pathname /dev/zero is not standardized. Assuming it was and > it referred to a character special with the usual behavior, mmap()'s > support for character specials is still unspecified by SUSv[1-4]. > > (Please note that I'm not on a crusade to prove that this can't be done > portably. I'd like to learn *how* it can be done portably if it is > possible at all.) Would this work portably? It's more overhead so it would likely only makes sense if MAP_ANONYMOUS isn't supported. 1) open a scratch file 2) unlink it 3) ftruncate it to the desired size 4) mmap it 5) close it The main weakness seems to be that step 3 could fail due to file size limits on the user or filesystem size limits. Chris
From: Scott Lurndal on 15 Mar 2010 21:40
lacos(a)ludens.elte.hu (Ersek, Laszlo) writes: >In article <YOznn.650022$L8.160111(a)news.usenetserver.com>, >scott(a)slp53.sl.home (Scott Lurndal) writes: > >> lacos(a)ludens.elte.hu (Ersek, Laszlo) writes: > >>>In article <87vdcxwd1l.fsf(a)fever.mssgmbh.com>, >>>Rainer Weikusat <rweikusat(a)mssgmbh.com> writes: > >>>> malloc is a libray routine on UNIX(*) which uses some facility the >>>> kernel provides for actually allocating address space (not memory, >>>> strictly speaking). This will usually be brk/ sbrk or mmap. Because >>>> library routines will likely call malloc behind ones back, using brk/ >>>> sbrk in a more sophisticated custom allocator would be asking for >>>> trouble. Which leaves mmap. While brk is no longer a 'standardized' >>>> UNIX(*) routine, it will be available on UNIX(*) because existing >>>> allocators (eg, dlmalloc) use it. mmap is a standardized >>>> UNIX(*)-routine. >>> >>>Thank you for explaining. Would you please describe the arguments you >>>pass to mmap() (if they are not based on MAP_ANONYMOUS)? >>> >> >> Typically, a file descriptor derived from opening /dev/zero. > >Thanks for the answer. > >However, the pathname /dev/zero is not standardized. Assuming it was and >it referred to a character special with the usual behavior, mmap()'s >support for character specials is still unspecified by SUSv[1-4]. > >(Please note that I'm not on a crusade to prove that this can't be done >portably. I'd like to learn *how* it can be done portably if it is >possible at all.) I haven't yet seen a system that supports mmap that doesn't also support /dev/zero. That said, it is possible that some embedded POSIX systems won't. scott |