Virtual memory control
in [Solaris]
|
Prev: Fresh install of Solaris 10u9 from running system, pfinstall?
Next: how to get access to patches at sunsolve.sun.com
From: JKB on 11 May 2010 04:16 Hello, I'm trying to port a huge program from Linux/NetBSD to Solaris 10 Sparc and I have a lot of trouble with virtual memory. This program uses a lot of threads and memory (allocated by mmap() or malloc()). For some reasons, I use a best fit allocator (ptmalloc3) that is better than mtmalloc. All other allocators (umem, malloc, mutex protected bsdmalloc) do not work. My test server is a Sun Enterprise T1000 with 8 GB of RAM. This program uses 4 GB of RAM (code + data). When I launch this process, it uses 100% of CPU. After several hours, process sends a lot of time in default pages : PID USERNAME USR SYS TRP TFL DFL LCK SLP LAT VCX ICX SCL SIG PROCESS/LWPID 22597 bertrand 95 0,0 5,3 0,0 0,0 0,0 0,0 0,0 0 9 0 0 serveur.rpl/124 22597 bertrand 90 0,1 9,3 0,0 0,9 0,0 0,0 0,0 1 9 0 0 serveur.rpl/127 22597 bertrand 83 0,3 7,0 0,0 10 0,0 0,0 0,0 5 9 0 0 serveur.rpl/127 22597 bertrand 4,1 29 0,0 0,0 9,1 0,0 58 0,0 41 3 2 0 serveur.rpl/126 8053 bertrand 0,8 17 0,0 0,0 0,0 0,0 82 0,0 61 0 485 0 prstat/1 22597 bertrand 7,4 6,3 0,0 0,0 14 0,0 72 0,0 49 3 1K 0 serveur.rpl/122 22597 bertrand 5,9 6,4 0,0 0,0 5,1 0,0 83 0,1 47 3 1K 0 serveur.rpl/124 22597 bertrand 5,8 6,0 0,0 0,0 6,1 0,0 82 0,1 49 7 1K 0 serveur.rpl/119 22597 bertrand 5,5 6,2 0,0 0,0 5,9 0,0 82 0,1 46 9 1K 0 serveur.rpl/127 22597 bertrand 4,8 6,6 0,0 0,0 12 0,0 77 0,1 58 2 1K 0 serveur.rpl/128 22597 bertrand 4,9 5,7 0,0 0,0 3,2 0,0 86 0,1 38 2 1K 0 serveur.rpl/123 22597 bertrand 5,5 5,1 0,0 0,0 5,5 0,0 84 0,0 44 3 986 0 serveur.rpl/119 22597 bertrand 5,5 5,1 0,0 0,0 8,4 0,0 81 0,1 45 3 976 0 serveur.rpl/122 22597 bertrand 5,0 4,6 0,0 0,0 3,8 0,0 87 0,0 37 2 1K 0 serveur.rpl/127 22597 bertrand 5,0 4,3 0,0 0,0 7,3 0,0 83 0,1 51 4 1K 0 serveur.rpl/125 22597 bertrand 3,8 5,2 0,0 0,0 5,3 0,7 85 0,1 34 4 528 0 serveur.rpl/118 22597 bertrand 4,6 3,9 0,0 0,0 6,4 61 24 0,0 41 2 990 0 serveur.rpl/119 22597 bertrand 4,6 3,9 0,0 0,0 12 30 49 0,0 36 3 942 0 serveur.rpl/117 22597 bertrand 4,0 4,4 0,0 0,0 5,0 21 65 0,0 31 4 530 0 serveur.rpl/117 22597 bertrand 4,5 3,8 0,0 0,0 6,0 0,0 86 0,0 36 3 938 0 serveur.rpl/121 22597 bertrand 4,0 4,0 0,0 0,0 11 0,0 81 0,1 30 2 976 0 serveur.rpl/125 22597 bertrand 4,2 3,7 0,0 0,0 2,8 0,0 89 0,0 40 3 1K 0 serveur.rpl/125 22597 bertrand 4,2 3,4 0,0 0,0 5,0 0,0 87 0,0 33 2 939 0 serveur.rpl/121 22597 bertrand 3,9 3,8 0,0 0,0 4,8 0,0 88 0,0 33 5 526 0 serveur.rpl/120 22597 bertrand 4,3 3,3 0,0 0,0 2,6 47 43 0,0 32 1 942 0 serveur.rpl/122 22597 bertrand 3,5 4,1 0,0 0,0 1,7 0,0 91 0,0 34 3 1K 0 serveur.rpl/127 22597 bertrand 4,2 3,3 0,0 0,0 36 0,0 56 0,0 25 4 938 0 serveur.rpl/120 22597 bertrand 3,7 3,8 0,0 0,0 3,4 29 60 0,0 28 4 493 0 serveur.rpl/118 22597 bertrand 4,1 3,3 0,0 0,0 6,8 0,0 86 0,1 38 2 938 0 serveur.rpl/120 22597 bertrand 4,3 3,1 0,0 0,0 21 0,0 72 0,1 31 5 938 0 serveur.rpl/126 22597 bertrand 3,1 3,5 0,0 0,0 4,0 0,0 89 0,1 27 2 526 0 serveur.rpl/124 22597 bertrand 3,1 3,1 0,0 0,0 3,3 58 32 0,0 25 3 527 0 serveur.rpl/122 22597 bertrand 1,7 1,3 0,0 0,0 2,0 0,0 95 0,0 14 2 451 0 serveur.rpl/117 I have tried to lock this process in memory (I have between 3 and 4 GB available) with mlockall(MCL_CURRENT | MCL_FUTURE) without any result. I suppose that default pages come from initialization of mapped memory. How can I force Solaris to map virtual memory in memory ? tchaikovski:[~] > prctl -i project user.bertrand project: 100: user.bertrand NAME PRIVILEGE VALUE FLAG ACTION RECIPIENT project.max-contracts privileged 10,0K - deny - system 2,15G max deny - project.max-device-locked-memory privileged 492MB - deny - system 16,0EB max deny - project.max-locked-memory privileged 6,00GB - deny - system 16,0EB max deny - project.max-port-ids privileged 8,19K - deny - system 65,5K max deny - project.max-shm-memory privileged 788GB - deny - system 16,0EB max deny - project.max-shm-ids privileged 128 - deny - system 16,8M max deny - project.max-msg-ids privileged 128 - deny - system 16,8M max deny - project.max-sem-ids privileged 128 - deny - system 16,8M max deny - project.max-crypto-memory privileged 1,92GB - deny - system 16,0EB max deny - project.max-tasks system 2,15G max deny - project.max-lwps system 2,15G max deny - project.cpu-cap system 4,29G inf deny - project.cpu-shares privileged 1 - none - system 65,5K max none - zone.max-swap system 16,0EB max deny - zone.max-locked-memory system 16,0EB max deny - zone.max-shm-memory system 16,0EB max deny - zone.max-shm-ids system 16,8M max deny - zone.max-sem-ids system 16,8M max deny - zone.max-msg-ids system 16,8M max deny - zone.max-lwps system 2,15G max deny - zone.cpu-cap system 4,29G inf deny - zone.cpu-shares privileged 1 - none - tchaikovski:[~] > Regards, JKB
From: Casper H.S. Dik on 11 May 2010 05:01 JKB <knatschke(a)koenigsberg.fr> writes: > I'm trying to port a huge program from Linux/NetBSD to Solaris 10 > Sparc and I have a lot of trouble with virtual memory. This program > uses a lot of threads and memory (allocated by mmap() or malloc()). > For some reasons, I use a best fit allocator (ptmalloc3) that is > better than mtmalloc. All other allocators (umem, malloc, mutex > protected bsdmalloc) do not work. What does "do not work" mean is this context? > My test server is a Sun Enterprise T1000 with 8 GB of RAM. This > program uses 4 GB of RAM (code + data). Is it a 64 bit process? > I have tried to lock this process in memory (I have between 3 and 4 GB > available) with mlockall(MCL_CURRENT | MCL_FUTURE) without any > result. Have you tried using large pages? Casper -- Expressed in this posting are my opinions. They are in no way related to opinions held by my employer, Sun Microsystems. Statements on Sun products included here are not gospel and may be fiction rather than truth.
From: JKB on 11 May 2010 05:24 Le 11-05-2010, ? propos de Re: Virtual memory control, Casper H.S Dik ?crivait dans comp.unix.solaris : > JKB <knatschke(a)koenigsberg.fr> writes: > >> I'm trying to port a huge program from Linux/NetBSD to Solaris 10 >> Sparc and I have a lot of trouble with virtual memory. This program >> uses a lot of threads and memory (allocated by mmap() or malloc()). > >> For some reasons, I use a best fit allocator (ptmalloc3) that is >> better than mtmalloc. All other allocators (umem, malloc, mutex >> protected bsdmalloc) do not work. > > What does "do not work" mean is this context? malloc and umem allocate both more and more pages in swap and do not try to reuse freed memory. Of course, I have verified (on linux with valgrind) there is no memory leak in my code. >> My test server is a Sun Enterprise T1000 with 8 GB of RAM. This >> program uses 4 GB of RAM (code + data). > > Is it a 64 bit process? Yes. >> I have tried to lock this process in memory (I have between 3 and 4 GB >> available) with mlockall(MCL_CURRENT | MCL_FUTURE) without any >> result. > > Have you tried using large pages? I don't know this option. I have tried to force pages size with ppgsz -o heap=4194304 -p 15561 and ppgsz -o anon=4194304 -p 15561. Regards, JKB -- Le cerveau, c'est un véritable scandale écologique. Il représente 2% de notre masse corporelle, mais disperse à lui seul 25% de l'énergie que nous consommons tous les jours.
From: Casper H.S. Dik on 11 May 2010 05:56 JKB <knatschke(a)koenigsberg.fr> writes: > malloc and umem allocate both more and more pages in swap and do not > try to reuse freed memory. Of course, I have verified (on linux with > valgrind) there is no memory leak in my code. Both will reuse memory but there might be a fragmentation; you may also want to try use libumem and run "findleaks". Note that some "memory leaks" can't be found; you still keep a reference but you didn't know. >> Have you tried using large pages? > I don't know this option. I have tried to force pages size with > ppgsz -o heap=4194304 -p 15561 and ppgsz -o anon=4194304 -p 15561. Did it work and did it help? Casper -- Expressed in this posting are my opinions. They are in no way related to opinions held by my employer, Sun Microsystems. Statements on Sun products included here are not gospel and may be fiction rather than truth.
From: JKB on 11 May 2010 05:58
Le 11-05-2010, ? propos de Re: Virtual memory control, Casper H.S Dik ?crivait dans comp.unix.solaris : > JKB <knatschke(a)koenigsberg.fr> writes: > >> malloc and umem allocate both more and more pages in swap and do not >> try to reuse freed memory. Of course, I have verified (on linux with >> valgrind) there is no memory leak in my code. > > Both will reuse memory but there might be a fragmentation; you may also > want to try use libumem and run "findleaks". > > Note that some "memory leaks" can't be found; you still keep a reference > but you didn't know. > >>> Have you tried using large pages? > >> I don't know this option. I have tried to force pages size with >> ppgsz -o heap=4194304 -p 15561 and ppgsz -o anon=4194304 -p 15561. > > Did it work and did it help? I don't know. Problem occurs after several hours... JKB -- Le cerveau, c'est un véritable scandale écologique. Il représente 2% de notre masse corporelle, mais disperse à lui seul 25% de l'énergie que nous consommons tous les jours. |