Prev: snv_134 Zpool Failure (again)
Next: How to clone Solaris 10 ZFS boot disk?
From: Boardy on 30 Mar 2010 17:13

Perhaps I'm tired but...

I have a Solaris 10 server with 32,768MB RAM.
It's running 4 non-global zones.

sar -r reports that it has about 200,000 8KB pages free (1,600MB).
vmstat -p agrees, showing 1,500 to 1,600MB free.

This would suggest that I've used 31,000MB of RAM

Looking at prstat -Z -s rss I can only account for about 6,500MB

So where has all the freemem gone?
From: Ian Collins on 30 Mar 2010 17:26
On 03/31/10 10:13 AM, Boardy wrote:
>
> Perhaps I'm tired but...
>
> I have a Solaris 10 server with 32,768MB RAM.
> It's running 4 non-global zones.
>
> sar -r reports that it has about 200,000 8KB pages free (1,600MB).
> vmstat -p agrees, showing 1,500 to 1,600MB free.
>
> This would suggest that I've used 31,000MB of RAM
>
> Looking at prstat -Z -s rss I can only account for about 6,500MB
>
> So where has all the freemem gone?

Lurking backstage until its required! Freed pages are reclaimed when
required. If you are using ZFS, it will be using free memory for its cache.

--
Ian Collins
From: Boardy on 30 Mar 2010 17:59
>
> > So where has all the freemem gone?
>
> Lurking backstage until its required!  Freed pages are reclaimed when
> required.  If you are using ZFS, it will be using free memory for its cache.
>

This makes monitoring the amount of freemem a bit pointless.
Is there a better way to keep an eye on the systems virtual and
physical memory?
Is there a way to force the pages to be reclaimed?
Is there a way to determine how much the ZFS cache is using?

From: chuckers on 30 Mar 2010 19:49
On Mar 31, 6:59 am, Boardy <jonathan.bo...(a)googlemail.com> wrote:
> > > So where has all the freemem gone?
>
> > Lurking backstage until its required!  Freed pages are reclaimed when
> > required.  If you are using ZFS, it will be using free memory for its cache.
>
> This makes monitoring the amount of freemem a bit pointless.
> Is there a better way to keep an eye on the systems virtual and
> physical memory?
> Is there a way to force the pages to be reclaimed?

Not sure about that but there is a way to limit the ARC in advance.

Add the following to /etc/system and reboot:

* setting the zfs_arc_min to 2GB (hex value of - 1024 * 2 * 1024 *
1024).
set zfs:zfs_arc_min=0x80000000

* setting the zfs_arc_max to 8GB (hex value of - 1024 * 8 * 1024 *
1024).
set zfs:zfs_arc_max=0x200000000


> Is there a way to determine how much the ZFS cache is using?

kstat -m zfs -s size

For another method check out:

http://cuddletech.com/arc_summary/

Also refer to:

http://www.solarisinternals.com/wiki/index.php/ZFS_Evil_Tuning_Guide


From: Boardy on 30 Mar 2010 20:45
On Mar 31, 12:49 pm, chuckers <chucker...(a)gmail.com> wrote:
> On Mar 31, 6:59 am, Boardy <jonathan.bo...(a)googlemail.com> wrote:
>
> > > > So where has all the freemem gone?
>
> > > Lurking backstage until its required!  Freed pages are reclaimed when
> > > required.  If you are using ZFS, it will be using free memory for its cache.
>
> > This makes monitoring the amount of freemem a bit pointless.
> > Is there a better way to keep an eye on the systems virtual and
> > physical memory?
> > Is there a way to force the pages to be reclaimed?
>
> Not sure about that but there is a way to limit the ARC in advance.
>
> Add the following to /etc/system and reboot:
>
> * setting the zfs_arc_min to 2GB (hex value of - 1024 * 2 * 1024 *
> 1024).
> set zfs:zfs_arc_min=0x80000000
>
> * setting the zfs_arc_max to 8GB (hex value of - 1024 * 8 * 1024 *
> 1024).
> set zfs:zfs_arc_max=0x200000000
>
> > Is there a way to determine how much the ZFS cache is using?
>
> kstat -m zfs -s size
>
> For another method check out:
>
> http://cuddletech.com/arc_summary/
>
> Also refer to:
>
> http://www.solarisinternals.com/wiki/index.php/ZFS_Evil_Tuning_Guide

Thanks for the info, it prompted a bit of googling on my part and I
came across this article http://southbrain.com/south/2008/04/wheres-all-my-memory-gone-sola.html
which reminded me of mdb memstat

My memstat on my system shows me the following;

# mdb -k
Loading modules: [ unix genunix specfs dtrace ufs sd mpt px md ip hook
neti sctp arp usba nca fcp fctl qlc zfs ssd cpc random crypto wrsmd
fcip logindmux ptm sppp nfs ipc lofs ]
> ::memstat
Page Summary Pages MB %Tot
------------ ---------------- ---------------- ----
Kernel 2023535 15808 49%
Anon 1374133 10735 33%
Exec and libs 49188 384 1%
Page cache 70819 553 2%
Free (cachelist) 38984 304 1%
Free (freelist) 560947 4382 14%

Total 4117606 32168
Physical 4096047 32000
>

If I'm reading it correctly: Kernel is taking up 15,808MB or which
12,000MB is ZFS cache according to

[yvwdomain-app02:/root]# kstat -m zfs -s size
module: zfs instance: 0
name: arcstats class: misc
size 12973940960 <== 12,000MB

My next question is why do we have such an enormous amount of memory
devoted to Anon (stacks and heaps?)?

I'm guessing that this is all pretty normal and I'm simply noticing it
because of the way this system is monitored. This begs another
question; if Solaris 10 and ZFS simply don't like unused memory,
what's the best way to monitor the virtual memory system and generate
alerts when I have issues?
 |  Next  |  Last
Pages: 1 2 3
Prev: snv_134 Zpool Failure (again)
Next: How to clone Solaris 10 ZFS boot disk?