Tuesday, September 05, 2006

How much memory does ZFS consume?

When using ZFS standard tools give inaccurate values for free memory as ZFS doesn't use normal page cache and rather allocates directly kernel memory. When low-memory condition occurs ZFS should free its buffer memory. So how to get how much additional memory is possibly free?

bash-3.00# mdb -k
Loading modules: [ unix krtld genunix specfs dtrace cpu.AuthenticAMD.15 ufs md ip sctp usba fcp fctl lofs zfs random nfs crypto fcip cpc logindmux ptm ipc ]
> ::memstat
Page Summary Pages MB %Tot
------------ ---------------- ---------------- ----
Kernel 859062 3355 41%
Anon 675625 2639 32%
Exec and libs 7994 31 0%
Page cache 39319 153 2%
Free (cachelist) 110881 433 5%
Free (freelist) 385592 1506 19%

Total 2078473 8119
Physical 2049122 8004

bash-3.00# echo "::kmastat"|mdb -k|grep zio_buf|awk 'BEGIN {c=0} {c=c+$5} END {print c}'

So kernel consumes about 3.2TB of memory and about 2.7GB is allocated to ZFS buffers and basically should be treated as free memory. Approximately free memory on this host is: Free (cachelist) + Free (freelist) + 2923298816.

I guess small script which do all the calculations automatically would be useful.


Anonymous said...


# ZFS I/O buffer (can be treated as free memory)
ZIO_BUF=`echo "::kmastat"|mdb -k|grep zio_buf|awk 'BEGIN {c=0} {c=c+$5} END {pri
nt c}'`
((ZFS_BUF = $ZIO_BUF / 1024 / 1024 ))

# Free memory from cachelist and freelist out of memstat
FREE=`echo "::memstat"|mdb -k|grep Free|awk 'BEGIN {c=0} {c=c+$4} END {print c}'


echo "Free memory = $FREE_MEM MB"

BK said...

A faster (and less resource intensive method) is :

kstat -p zfs:0:arcstats:size


FREE=$(($(pagesize)*$(kstat -p unix:0:system_pages:freemem|awk '{print $NF}')))

ZIO_BUF=$(kstat -p zfs:0:arcstats:size|awk '{print $NF}')