Sysstat contains various utilities to collect and historize performance and monitor server activity on the linux. Sar is part of the sysstat package. by using sar, we can collect, report, or save system resource utilization information. It will help system administrator to troubleshoot performance issues, or to optimize performance. This post will show how to setup and the usage of sar on linux CentOS 6.5.
1. Install sar (systat) :
[root@oss ~]# yum install sysstat -y
2. Verify the sar version using “sar -V”
[root@oss ~]# sar -V sysstat version 9.0.4 (C) Sebastien Godard (sysstatorange.fr)
3. Sar usage examples :
a) Check swap usage :
[root@oss ~]# sar -W Linux 2.6.32-431.el6.x86_64 (oss.ehowstuff.local) 08/18/2014 _x86_64_ (2 CPU) 11:10:01 PM pswpin/s pswpout/s 11:20:01 PM 0.00 0.00 11:30:01 PM 0.00 0.00 11:40:01 PM 0.00 0.00 Average: 0.00 0.00
b) Check system load :
[root@oss ~]# sar -q Linux 2.6.32-431.el6.x86_64 (oss.ehowstuff.local) 08/18/2014 _x86_64_ (2 CPU) 11:10:01 PM runq-sz plist-sz ldavg-1 ldavg-5 ldavg-15 11:20:01 PM 0 86 0.00 0.00 0.00 11:30:01 PM 0 85 0.00 0.00 0.00 11:40:01 PM 0 85 0.00 0.00 0.00 Average: 0 85 0.00 0.00 0.00
c) Collect CPU statistics 3 times within 1 second interval :
[root@oss ~]# sar 1 3 Linux 2.6.32-431.el6.x86_64 (oss.ehowstuff.local) 08/18/2014 _x86_64_ (2 CPU) 11:46:55 PM CPU %user %nice %system %iowait %steal %idle 11:46:56 PM all 0.00 0.00 0.50 0.00 0.00 99.50 11:46:57 PM all 0.00 0.00 0.00 0.00 0.00 100.00 11:46:58 PM all 0.00 0.00 0.00 0.00 0.00 100.00 Average: all 0.00 0.00 0.17 0.00 0.00 99.83
d) Displaying the RAM Load:
[root@oss ~]# sar -r Linux 2.6.32-431.el6.x86_64 (oss.ehowstuff.local) 08/18/2014 _x86_64_ (2 CPU) 11:10:01 PM kbmemfree kbmemused %memused kbbuffers kbcached kbcommit %commit 11:20:01 PM 3670608 254080 6.47 11872 160172 67684 1.12 11:30:01 PM 3670616 254072 6.47 11892 160172 67684 1.12 11:40:01 PM 3670508 254180 6.48 11920 160180 67684 1.12 Average: 3670577 254111 6.47 11895 160175 67684 1.12
e) Display paging statistic :
[root@oss ~]# sar -B Linux 2.6.32-431.el6.x86_64 (oss.ehowstuff.local) 08/18/2014 _x86_64_ (2 CPU) 11:10:01 PM pgpgin/s pgpgout/s fault/s majflt/s pgfree/s pgscank/s pgscand/s pgsteal/s %vmeff 11:20:01 PM 0.00 0.23 8.21 0.00 4.36 0.00 0.00 0.00 0.00 11:30:01 PM 0.00 0.19 2.70 0.00 1.29 0.00 0.00 0.00 0.00 11:40:01 PM 0.00 0.14 3.75 0.00 1.62 0.00 0.00 0.00 0.00 11:50:01 PM 0.00 0.16 6.60 0.00 2.80 0.00 0.00 0.00 0.00 Average: 0.00 0.18 5.31 0.00 2.52 0.00 0.00 0.00 0.00
f) Display Swap space utilization statistics :
[root@oss ~]# sar -S Linux 2.6.32-431.el6.x86_64 (oss.ehowstuff.local) 08/18/2014 _x86_64_ (2 CPU) 11:10:01 PM kbswpfree kbswpused %swpused kbswpcad %swpcad 11:20:01 PM 2097144 0 0.00 0 0.00 11:30:01 PM 2097144 0 0.00 0 0.00 11:40:01 PM 2097144 0 0.00 0 0.00 11:50:01 PM 2097144 0 0.00 0 0.00 Average: 2097144 0 0.00 0 0.00
4. Monitor previous day sar statictics :
Every sar statictics by default stored in /var/log/sa. Es example :
[root@oss sa]# ls sa18 sa19 sa20 sa21 sa22 sa23 sa24 sar18 sar19 sar20 sar23
a. Display cpu load on sa22 :
[root@oss sa]# sar -f /var/log/sa/sa22 Linux 2.6.32-431.el6.x86_64 (oss.ehowstuff.local) 08/22/2014 _x86_64_ (2 CPU) 12:00:01 AM CPU %user %nice %system %iowait %steal %idle 12:10:01 AM all 0.04 0.00 0.06 0.06 0.00 99.84 12:20:01 AM all 0.03 0.00 0.05 0.07 0.00 99.85 12:30:01 AM all 0.03 0.00 0.05 0.02 0.00 99.90 12:40:01 AM all 0.34 0.00 0.71 0.02 0.00 98.93 12:50:01 AM all 1.67 0.00 3.19 0.01 0.00 95.13 01:00:01 AM all 0.64 0.00 0.83 0.74 0.00 97.79 01:10:01 AM all 0.64 0.00 1.51 0.03 0.00 97.83 01:20:01 AM all 0.35 0.00 0.71 0.13 0.00 98.81 01:30:01 AM all 0.03 0.00 0.05 0.02 0.00 99.90 01:40:01 AM all 0.03 0.00 0.05 0.02 0.00 99.90 01:50:01 AM all 0.03 0.00 0.05 0.02 0.00 99.91 02:00:01 AM all 0.03 0.00 0.05 0.02 0.00 99.90 02:10:01 AM all 0.03 0.00 0.05 0.02 0.00 99.90 02:20:01 AM all 0.03 0.00 0.04 0.02 0.00 99.91 02:30:01 AM all 0.03 0.00 0.04 0.02 0.00 99.91 02:40:01 AM all 0.03 0.00 0.04 0.02 0.00 99.91 02:50:01 AM all 0.03 0.00 0.04 0.01 0.00 99.92 03:00:01 AM all 0.03 0.00 0.04 0.02 0.00 99.92 03:10:01 AM all 0.03 0.00 0.04 0.02 0.00 99.91 03:20:01 AM all 0.03 0.00 0.04 0.01 0.00 99.92 03:30:01 AM all 0.03 0.00 0.04 0.01 0.00 99.92 03:40:01 AM all 0.03 0.00 0.05 0.02 0.00 99.91 03:50:01 AM all 0.03 0.00 0.05 0.03 0.00 99.89 04:00:01 AM all 0.03 0.00 0.04 0.02 0.00 99.91 04:10:01 AM all 0.03 0.00 0.04 0.02 0.00 99.91 04:20:01 AM all 0.03 0.00 0.04 0.02 0.00 99.92 04:30:01 AM all 0.03 0.00 0.04 0.02 0.00 99.91 04:40:01 AM all 0.03 0.00 0.04 0.02 0.00 99.91 04:50:01 AM all 0.03 0.00 0.04 0.02 0.00 99.91 05:00:01 AM all 0.03 0.00 0.04 0.02 0.00 99.92 05:10:01 AM all 0.03 0.00 0.05 0.02 0.00 99.90 05:20:01 AM all 0.03 0.00 0.05 0.02 0.00 99.91 05:30:01 AM all 0.03 0.00 0.04 0.02 0.00 99.91 05:30:01 AM CPU %user %nice %system %iowait %steal %idle 05:40:01 AM all 0.03 0.00 0.05 0.01 0.00 99.91 05:50:01 AM all 0.03 0.00 0.05 0.01 0.00 99.91 06:00:01 AM all 0.03 0.00 0.05 0.02 0.00 99.91 06:10:01 AM all 0.03 0.00 0.05 0.01 0.00 99.90 06:20:01 AM all 0.03 0.00 0.05 0.01 0.00 99.91 06:30:01 AM all 0.03 0.00 0.05 0.01 0.00 99.91 06:40:02 AM all 0.03 0.00 0.05 0.01 0.00 99.92 06:50:01 AM all 0.03 0.00 0.04 0.01 0.00 99.92 07:00:01 AM all 0.03 0.00 0.05 0.01 0.00 99.91 07:10:01 AM all 0.03 0.00 0.05 0.01 0.00 99.91 07:20:01 AM all 0.03 0.00 0.05 0.01 0.00 99.92 07:30:01 AM all 0.03 0.00 0.04 0.01 0.00 99.92 07:40:01 AM all 0.03 0.00 0.05 0.01 0.00 99.91 07:50:01 AM all 0.03 0.00 0.05 0.01 0.00 99.91 08:00:01 AM all 0.03 0.00 0.05 0.01 0.00 99.91 08:10:01 AM all 0.03 0.00 0.05 0.02 0.00 99.90 08:20:01 AM all 0.03 0.00 0.05 0.01 0.00 99.91 Average: all 0.10 0.00 0.18 0.03 0.00 99.69
b. Display RAM Load on sa22 :
[root@oss sa]# sar -f /var/log/sa/sa22 -r Linux 2.6.32-431.el6.x86_64 (oss.ehowstuff.local) 08/22/2014 _x86_64_ (2 CPU) 12:00:01 AM kbmemfree kbmemused %memused kbbuffers kbcached kbcommit %commit 12:10:01 AM 3807336 117352 2.99 8252 32352 66920 1.11 12:20:01 AM 3804708 119980 3.06 8712 32812 71440 1.19 12:30:01 AM 3804584 120104 3.06 8744 32816 71592 1.19 12:40:01 AM 3799784 124904 3.18 8768 33124 72228 1.20 12:50:01 AM 3799668 125020 3.19 8800 33124 72228 1.20 01:00:01 AM 3723612 201076 5.12 10048 110608 71592 1.19 01:10:01 AM 3718804 205884 5.25 10232 111044 72228 1.20 01:20:01 AM 3711560 213128 5.43 10284 122396 71592 1.19 01:30:01 AM 3711684 213004 5.43 10324 122408 71592 1.19 01:40:01 AM 3711436 213252 5.43 10348 122408 71592 1.19 01:50:01 AM 3711436 213252 5.43 10372 122412 71592 1.19 02:00:01 AM 3711560 213128 5.43 10404 122416 71592 1.19 02:10:01 AM 3711576 213112 5.43 10444 122424 71592 1.19 02:20:01 AM 3711568 213120 5.43 10476 122428 71592 1.19 02:30:01 AM 3711444 213244 5.43 10516 122436 71592 1.19 02:40:01 AM 3711320 213368 5.44 10552 122436 71592 1.19 02:50:01 AM 3711320 213368 5.44 10580 122444 71592 1.19 03:00:01 AM 3711188 213500 5.44 10612 122448 71592 1.19 03:10:01 AM 3711180 213508 5.44 10652 122456 72168 1.20 03:20:01 AM 3711180 213508 5.44 10676 122456 72168 1.20 03:30:01 AM 3710932 213756 5.45 10708 122464 72168 1.20 03:40:01 AM 3710692 213996 5.45 10740 122464 72168 1.20 03:50:01 AM 3710700 213988 5.45 10928 122604 71592 1.19 04:00:01 AM 3710568 214120 5.46 10960 122608 71592 1.19 04:10:01 AM 3710816 213872 5.45 11000 122608 71592 1.19 04:20:01 AM 3710576 214112 5.46 11032 122616 71592 1.19 04:30:01 AM 3710576 214112 5.46 11068 122616 71592 1.19 04:40:01 AM 3710708 213980 5.45 11096 122624 71592 1.19 04:50:01 AM 3710452 214236 5.46 11128 122628 71592 1.19 05:00:01 AM 3710576 214112 5.46 11160 122636 71592 1.19 05:10:01 AM 3710452 214236 5.46 11200 122636 71592 1.19 05:20:01 AM 3710576 214112 5.46 11232 122644 71592 1.19 05:30:01 AM 3710708 213980 5.45 11268 122644 71592 1.19 05:30:01 AM kbmemfree kbmemused %memused kbbuffers kbcached kbcommit %commit 05:40:01 AM 3710320 214368 5.46 11296 122652 71592 1.19 05:50:01 AM 3710072 214616 5.47 11320 122652 71592 1.19 06:00:01 AM 3710204 214484 5.46 11364 122660 71592 1.19 06:10:01 AM 3710072 214616 5.47 11400 122664 71592 1.19 06:20:01 AM 3710204 214484 5.46 11424 122668 71592 1.19 06:30:01 AM 3710204 214484 5.46 11464 122676 71592 1.19 06:40:02 AM 3709948 214740 5.47 11488 122680 71592 1.19 06:50:01 AM 3710072 214616 5.47 11512 122680 71592 1.19 07:00:01 AM 3710204 214484 5.46 11552 122680 71592 1.19 07:10:01 AM 3709824 214864 5.47 11584 122692 71592 1.19 07:20:01 AM 3709948 214740 5.47 11612 122688 71592 1.19 07:30:01 AM 3709956 214732 5.47 11648 122700 71592 1.19 07:40:01 AM 3709956 214732 5.47 11680 122704 71592 1.19 07:50:01 AM 3709824 214864 5.47 11704 122708 71592 1.19 08:00:01 AM 3709832 214856 5.47 11744 122712 71592 1.19 08:10:01 AM 3709700 214988 5.48 11784 122724 71592 1.19 08:20:01 AM 3709700 214988 5.48 11812 122720 71592 1.19 Average: 3720306 204382 5.21 10794 113138 71580 1.19
5. Other sar options :
[root@oss ~]# sar -h Usage: sar [ options ] [[ ] ] Main options and reports: -b I/O and transfer rate statistics -B Paging statistics -d Block device statistics -I { | SUM | ALL | XALL } Interrupts statistics -m Power management statistics -n { [,...] | ALL } Network statistics Keywords are: DEV Network interfaces EDEV Network interfaces (errors) NFS NFS client NFSD NFS server SOCK Sockets (v4) IP IP traffic (v4) EIP IP traffic (v4) (errors) ICMP ICMP traffic (v4) EICMP ICMP traffic (v4) (errors) TCP TCP traffic (v4) ETCP TCP traffic (v4) (errors) UDP UDP traffic (v4) SOCK6 Sockets (v6) IP6 IP traffic (v6) EIP6 IP traffic (v6) (errors) ICMP6 ICMP traffic (v6) EICMP6 ICMP traffic (v6) (errors) UDP6 UDP traffic (v6) -q Queue length and load average statistics -r Memory utilization statistics -R Memory statistics -S Swap space utilization statistics -u [ ALL ] CPU utilization statistics -v Kernel table statistics -w Task creation and system switching statistics -W Swapping statistics -y TTY device statistics
6. Other sysstat utilities:
sar collects and displays ALL system activities statistics.
sadc stands for “system activity data collector”. This is the sar backend tool that does the data collection.
sa1 stores system activities in binary data file. sa1 depends on sadc for this purpose. sa1 runs from cron.
sa2 creates daily summary of the collected statistics. sa2 runs from cron.
sadf can generate sar report in CSV, XML, and various other formats. Use this to integrate sar data with other tools.
iostat generates CPU, I/O statistics
mpstat displays CPU statistics.
pidstat reports statistics based on the process id (PID)
nfsiostat displays NFS I/O statistics.
cifsiostat generates CIFS statistics.
