How to Configure Linux TCP keepalive Setting

TCP keepalive Setting

In line with the increase in internet users, the traffic and workload on the web server is also increased. Hence, the webmaster or system administrator needs to make sure that the web server is able to accommodate a sufficient number of TCP connections.

If your web server has begun to show an increase in the number of visitors, you may start planning to perform basic tcp tuning on the linux operating system.

On average, most people that visit the website or blog that comes from search engines only read a page just for 1-2 minutes. After they got the answer for what they really want, they simply leave the page and visit other sites. But the old opened connection still remains and unused for a long time.

For low and average number of website visitors, the default values for the keepalive parameter should be sufficient.

But for high concurrency web server or in a busy server, decrease timeouts on TCP sockets can help to clean up the tcp connections from clients that have been disconnected. This can be done by changing the default value of tcp_keepalive setting in sysctl.conf.

What is TCP Keepalive Setting?

TCP keepalive is a mechanism for TCP connections that help to determine whether the other end has stopped responding or not.

TCP will send the keepalive probe contains null data to the network peer several times after a period of idle time. If the peer does not respond, the socket will be closed automatically.

The application will then receive a notification about the socket closure, which it should handle in the correct manner.

Most of the operating systems and hosts that support TCP also support TCP Keepalive.

Basically, tuning some of the settings in sysctl.conf really help speeding things up under heavy usage.

Tunable TCP settings can be found on /proc/sys/net/ipv4

What are the default values of TCP KeepAlive setting ?

tcp_keepalive_time = 7200 (seconds)
tcp_keepalive_intvl = 75 (seconds)
tcp_keepalive_probes = 9 (number of probes)

TCP keepalive process waits for two hours (7200 secs) for socket activity before sending the first keepalive probe, and then resend it every 75 seconds. As long as there is TCP/IP socket communications going on and active, no keepalive packets are needed.

How to Configure Linux TCP keepalive Settings ?

Please note that the following tuning is for linux operating system only. This steps has been tested in CentOS 5/6/7, RHEL 5/6/7 and Oracle Linux 6/7.

Optionally you can do further tuning of the web applications level such as Apache or Nginx web server.

1. Edit your /etc/sysctl.conf

# vi /etc/sysctl.conf

2. Add the following setting :

net.ipv4.tcp_keepalive_time = 60
net.ipv4.tcp_keepalive_intvl = 10
net.ipv4.tcp_keepalive_probes = 6

Explanation for above parameter in section a), b) and c).

3. To load settings, enter the following command :

# sysctl -p

KeepAlive Parameter Details

a) Decrease the time default value for tcp_keepalive_time connection from 7200 seconds to 60 seconds. This determine the time of connection inactivity after which the first keep alive request is sent. Parameter below shows that the TCP will begin sending keepalive null packets after 1 minute.

net.ipv4.tcp_keepalive_time = 60

b) The following parameter (tcp_keepalive_intvl) determines the keepalive probe will resend every 10 seconds after first keep alive probe. This reduce from 75 seconds to 10 seconds gap or time interval between each of the keep alive probes.

net.ipv4.tcp_keepalive_intvl = 10

c) Next parameter (tcp_keepalive_probes) is expressed in the pure number. The following setting determine the number of probes before timing out. We recommend to reduce number of retransmitted from 9 to 6 before the connection is considered broken.

net.ipv4.tcp_keepalive_probes = 6

With this, your application will detect dead TCP connections after 120 seconds (60s + 10s + 10s + 10s + 10s + 10s + 10s).

How to Install and Setup Munin on CentOS 7

Munin on CentOS 7

Munin is open source and free software for monitoring computer system, network monitoring and application infrastructure monitoring software. Munin offers monitoring and alerting for servers, switches, applications, and services.

Munin can help system administrators to analyze the trend of the computer system whether it is experiencing problems or not. It can be an easier alternative to the popular open-source software zabbix monitoring.

In this article, I will explain how you can monitor your linux CentOS with Munin and the simple steps to install and setup Munin on CentOS 7.

Steps to Install and Setup Munin on CentOS 7

1. Enable or install the EPEL Repository into CentOS 7. Read more on how to Enable EPEL Repository on CentOS 7 / RHEL 7

2. Munin requires a web server to run. In this article, we will use apache. Install apache, Munin and Munin Node with yum command :

# yum install httpd munin munin-node -y

3. Start and enable apache and munin at boot.

# systemctl start httpd
# systemctl enable httpd
# systemctl start munin-node
# systemctl enable munin-node

4. We want munin to use the name centos72.ehowstuff.local instead of localhost. Please open edit the setting in /etc/munin/munin.conf

# vim /etc/munin/munin.conf

Original :

[localhost]
    address 127.0.0.1
    use_node_name yes

Change to :

[centos72.ehowstuff.local]
    address 127.0.0.1
    use_node_name yes

5. You also have optional to change the munin node hostname :

# vim /etc/munin/munin-node.conf

Original :

host_name localhost.localdomain

Change to :

host_name centos72.ehowstuff.local

6. Next go to the Apache virtual host configuration file to add the permission to access your network.

# vim /etc/httpd/conf.d/munin.conf

Add network segment that you allow to access to the CentOS server.

AuthUserFile /etc/munin/munin-htpasswd
AuthName "Munin"
AuthType Basic
require valid-user

Order Deny,Allow
Deny from all
Allow from 127.0.0.1 192.168.0.0/24
..
..

7. Munin statistics page shall be protected by a username and password. We can add the new user (admin) and password to /etc/munin/munin-htpasswd with htpasswd command line. So we have to setup basic Apache authentication before we can start access the munin statistic page.

# htpasswd /etc/munin/munin-htpasswd admin
New password:
Re-type new password:
Adding password for user admin

8. Allow port 80 in the firewalld permanently. learn more how to configure Firewalld on CentOS 7.

a) Get default zone :

# firewall-cmd --get-active-zones
public
  interfaces: ens160

b) Allow port 80 permanently in firewalld :

# firewall-cmd --permanent --zone=public --add-port=80/tcp
success

c) reload the setting to take effect immediately :

# firewall-cmd --reload
success

d) List all active firewalld configuration :

# firewall-cmd --list-all
public (default, active)
  interfaces: ens160
  sources:
  services: dhcpv6-client ssh
  ports: 80/tcp
  masquerade: no
  forward-ports:
  icmp-blocks:
  rich rules:

9. Try access munin statistic page from client.

http://192.168.0.14/munin

Munin on CentOS 7

NGINX DDos Attack Tutorial – Implement Basic Protection

DDoS attacks are usually intended to paralyze websites and web services and it is better to mitigate it at the firewall level. But for the web server that runs on Nginx, I have prepared a basic step to provide DDoS protection which proved to work for small-scale DDoS attacks and DDoS attacks that aimed at applications. This DDos Attack Tutorial protection for Nginx guidelines has been tested on CentOS 6, CentOS 7, RHEL 7 and Oracle Linux 7. This steps may work on your environment but please note that this guidelines is not an official document and official recommendation from Nginx website.

DDos Attack Tutorial – Implement Basic Protection for Nginx :

1. In /etc/nginx/nginx.conf, include the following parameters :

client_body_buffer_size 128k;
large_client_header_buffers 4 256k;
limit_conn_zone $binary_remote_addr zone=conn_limit_per_ip:10m;
limit_req_zone $binary_remote_addr zone=req_limit_per_ip:10m rate=50r/s;
server {
    limit_conn conn_limit_per_ip 10;
    limit_req zone=req_limit_per_ip burst=10 nodelay;
}

2. Then restart or reload your Nginx service to apply DDoS protection for Nginx :

# /etc/init.d/nginx restart

or

# /etc/init.d/nginx reload

Explanation :

a) Limit the number of connections per single IP :

limit_conn_zone $binary_remote_addr zone=conn_limit_per_ip:10m;

b) Limit the number of requests for a given session :

limit_req_zone $binary_remote_addr zone=req_limit_per_ip:10m rate=50r/s;

C) Zone which we want to limit by upper values, we want limit whole server :

server {
limit_conn conn_limit_per_ip 10;
limit_req zone=req_limit_per_ip burst=10 nodelay;
}

If your WordPress is under DDoS attack, you will get the following log into Nginx files domain.access.log :

1.2.3.4 - - [25/Mar/2015:16:52:38 +0800] "POST /wp-login.php HTTP/1.0" 200 6203 "-" "-"
1.2.3.4 - - [25/Mar/2015:16:52:39 +0800] "POST /wp-login.php HTTP/1.0" 200 6203 "-" "-"
1.2.3.4 - - [25/Mar/2015:16:52:39 +0800] "POST /wp-login.php HTTP/1.0" 200 6203 "-" "-"
1.2.3.4 - - [25/Mar/2015:16:52:40 +0800] "POST /wp-login.php HTTP/1.0" 200 6203 "-" "-"
1.2.3.4 - - [25/Mar/2015:16:52:40 +0800] "POST /wp-login.php HTTP/1.0" 200 6203 "-" "-"
1.2.3.4 - - [25/Mar/2015:16:52:41 +0800] "POST /wp-login.php HTTP/1.0" 200 6203 "-" "-"
1.2.3.4 - - [25/Mar/2015:16:52:41 +0800] "POST /wp-login.php HTTP/1.0" 200 6203 "-" "-"
1.2.3.4 - - [25/Mar/2015:16:52:42 +0800] "POST /wp-login.php HTTP/1.0" 200 6203 "-" "-"
1.2.3.4 - - [25/Mar/2015:16:52:42 +0800] "POST /wp-login.php HTTP/1.0" 200 6203 "-" "-"
1.2.3.4 - - [25/Mar/2015:16:52:43 +0800] "POST /wp-login.php HTTP/1.0" 200 6203 "-" "-"
1.2.3.4 - - [25/Mar/2015:16:52:43 +0800] "POST /wp-login.php HTTP/1.0" 200 6203 "-" "-"
1.2.3.4 - - [25/Mar/2015:16:52:44 +0800] "POST /wp-login.php HTTP/1.0" 200 6203 "-" "-"
1.2.3.4 - - [25/Mar/2015:16:52:44 +0800] "POST /wp-login.php HTTP/1.0" 200 6203 "-" "-"
1.2.3.4 - - [25/Mar/2015:16:52:45 +0800] "POST /wp-login.php HTTP/1.0" 200 6203 "-" "-"
1.2.3.4 - - [25/Mar/2015:16:52:45 +0800] "POST /wp-login.php HTTP/1.0" 200 6203 "-" "-"

Here is an example of the results after you perform basic DDoS protection for Nginx :

2015/03/28 11:44:33 [error] 22370#0: *71492 limiting connections by zone "conn_limit_per_ip", client: 1.2.3.4, server: www.ehowstuff.com, request: "GET /wp-login.php HTTP/1.0", host: "www.ehowstuff.com"
2015/03/28 11:44:33 [error] 22370#0: *71493 limiting connections by zone "conn_limit_per_ip", client: 1.2.3.4, server: www.ehowstuff.com, request: "GET /wp-login.php HTTP/1.0", host: "www.ehowstuff.com"
2015/03/28 11:44:33 [error] 22370#0: *71494 limiting connections by zone "conn_limit_per_ip", client: 1.2.3.4, server: www.ehowstuff.com, request: "GET /wp-login.php HTTP/1.0", host: "www.ehowstuff.com"
2015/03/28 11:44:33 [error] 22370#0: *71498 limiting connections by zone "conn_limit_per_ip", client: 1.2.3.4, server: www.ehowstuff.com, request: "GET /wp-login.php HTTP/1.0", host: "www.ehowstuff.com"
2015/03/28 11:44:33 [error] 22370#0: *71502 limiting connections by zone "conn_limit_per_ip", client: 1.2.3.4, server: www.ehowstuff.com, request: "GET /wp-login.php HTTP/1.0", host: "www.ehowstuff.com"
2015/03/28 11:44:33 [error] 22370#0: *71506 limiting connections by zone "conn_limit_per_ip", client: 1.2.3.4, server: www.ehowstuff.com, request: "GET /wp-login.php HTTP/1.0", host: "www.ehowstuff.com"

Hope this DDos Attack Tutorial to Implement Basic Protection on NGINX help!!

DDos Attack Tutorial

How to Add Linux Swap Space on CentOS 6 VPS

Linux swapI have a virtual private server (VPS) and plans to run a WordPress blog in it. A limited amount of RAM and swap size will be an issue because some software applications like MySQL, Apache, NGINX, PHP, HHVM and Varnish requires more memory to operate.

From my past experience in Ubiquity Hosting and Linode, the pre-allocated swap sizes of 2GB and 1GB VPS are 1GB and 512MB only. After switched to 2GB plan for RamNode VPS, linux swap also provided in the size of 1GB.

Why we need Linux Swap Space ?

To make your server more responsive and prevent crash when it runs out of memory, giving additional linux swap space will help. Linux swap is an area on the storage or hard disk where the operating system can temporarily store data that it no longer can hold in memory. Without the Linux swap, the VPS that runs out of memory can crash or start kill some applications to free up memory. This can cause lose for unsaved data or experience downtime.

However, swapping does have drawbacks. Reading from and writing to swap is slower than using memory. But swapping should take advantages on the SSD server. In the hard disk drive, If VPS started using linux swap space, the VPS performance can slow down significantly.

As a reference for webmasters and system administrators, here are the steps on how to increase the size of the linux swap on CentOS 6 VPS. This steps may also works and tested on CentOS 7 / RHEL 7 and oracle linux 7.

1. Check and verify the allocated disk partition for swap using “fdisl -l” command :

[root@vps ~]# fdisk -l

Disk /dev/vda: 53.7 GB, 53687091200 bytes
16 heads, 63 sectors/track, 104025 cylinders
Units = cylinders of 1008 * 512 = 516096 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x00059ca1

   Device Boot      Start         End      Blocks   Id  System
/dev/vda1   *           3        1018      512000   83  Linux
Partition 1 does not end on cylinder boundary.
/dev/vda2            1018        3099     1048576   82  Linux swap / Solaris
Partition 2 does not end on cylinder boundary.
/dev/vda3            3099      104023    50865920   83  Linux
Partition 3 does not end on cylinder boundary.

2. From the operating system, verify the swap size from command line utility “swapon -s” and “free -m“. The current swap size is 1GB :

[root@vps ~]# swapon -s
Filename                                Type            Size    Used    Priority
/dev/vda2                               partition       1048568 0       -1
[root@vps ~]# free -m
             total       used       free     shared    buffers     cached
Mem:          1877       1624        253          0        210        503
-/+ buffers/cache:        909        967
Swap:         1023          0       1023

3. Start Create the Swap file by determine the size of the new swap file and assign the swap file name. At a shell prompt as root, type the following dd command with count being equal to the desired block size. As an example i will add 2048MB as additional space size and assigned swap file name is /swapfile-ext.

[root@vps ~]# dd if=/dev/zero of=/swapfile-ext bs=1M count=2048
2048+0 records in
2048+0 records out
2147483648 bytes (2.1 GB) copied, 4.05978 s, 529 MB/s

4. Creating a linux swap area with mkswap command :

[root@vps ~]# mkswap /swapfile-ext
mkswap: /swapfile-ext: warning: don't erase bootbits sectors
        on whole disk. Use -f to force.
Setting up swapspace version 1, size = 2097148 KiB
no label, UUID=cc49d098-52e3-4fdf-8157-c40b23e00eef

5. Assign the correct permission to prevent swap space from world-readable :

[root@vps ~]# chmod 600 /swapfile-ext

6. Activatethe swap file immediately :

[root@vps ~]# swapon /swapfile-ext

7. Verify by viewing the output using “free -m” utility. 1GB plus 2GB equal to 3GB new swap size :

[root@vps ~]# free -m
             total       used       free     shared    buffers     cached
Mem:          1877       1801         76          0        137        765
-/+ buffers/cache:        897        979
Swap:         3071          0       3071

Verify by viewing the output using “swapon -s” utility

[root@vps ~]# swapon -s
Filename                                Type            Size    Used    Priority
/dev/vda2                               partition       1048568 556     -1
/swapfile-ext                           file            2097144 0       -2

8. To enable it at boot time, edit /etc/fstab to include the following entry:

/swapfile-ext          swap            swap    defaults        0 0

How to Setup Sar (Sysstat) – Monitor Linux System Performance

Sysstat contains various utilities to collect and historize performance and monitor server activity on the linux. Sar is part of the sysstat package. by using sar, we can collect, report, or save system resource utilization information. It will help system administrator to troubleshoot performance issues, or to optimize performance. This post will show how to setup and the usage of sar on linux CentOS 6.5.

1. Install sar (systat) :

[root@oss ~]# yum install sysstat -y

2. Verify the sar version using “sar -V”

[root@oss ~]# sar -V
sysstat version 9.0.4
(C) Sebastien Godard (sysstat  orange.fr)

3. Sar usage examples :

a) Check swap usage :

[root@oss ~]# sar -W
Linux 2.6.32-431.el6.x86_64 (oss.ehowstuff.local)       08/18/2014      _x86_64_        (2 CPU)

11:10:01 PM  pswpin/s pswpout/s
11:20:01 PM      0.00      0.00
11:30:01 PM      0.00      0.00
11:40:01 PM      0.00      0.00
Average:         0.00      0.00

b) Check system load :

[root@oss ~]# sar -q
Linux 2.6.32-431.el6.x86_64 (oss.ehowstuff.local)       08/18/2014      _x86_64_        (2 CPU)

11:10:01 PM   runq-sz  plist-sz   ldavg-1   ldavg-5  ldavg-15
11:20:01 PM         0        86      0.00      0.00      0.00
11:30:01 PM         0        85      0.00      0.00      0.00
11:40:01 PM         0        85      0.00      0.00      0.00
Average:            0        85      0.00      0.00      0.00

c) Collect CPU statistics 3 times within 1 second interval :

[root@oss ~]# sar 1 3
Linux 2.6.32-431.el6.x86_64 (oss.ehowstuff.local)       08/18/2014      _x86_64_        (2 CPU)

11:46:55 PM     CPU     %user     %nice   %system   %iowait    %steal     %idle
11:46:56 PM     all      0.00      0.00      0.50      0.00      0.00     99.50
11:46:57 PM     all      0.00      0.00      0.00      0.00      0.00    100.00
11:46:58 PM     all      0.00      0.00      0.00      0.00      0.00    100.00
Average:        all      0.00      0.00      0.17      0.00      0.00     99.83

d) Displaying the RAM Load:

[root@oss ~]# sar -r
Linux 2.6.32-431.el6.x86_64 (oss.ehowstuff.local)       08/18/2014      _x86_64_        (2 CPU)

11:10:01 PM kbmemfree kbmemused  %memused kbbuffers  kbcached  kbcommit   %commit
11:20:01 PM   3670608    254080      6.47     11872    160172     67684      1.12
11:30:01 PM   3670616    254072      6.47     11892    160172     67684      1.12
11:40:01 PM   3670508    254180      6.48     11920    160180     67684      1.12
Average:      3670577    254111      6.47     11895    160175     67684      1.12

e) Display paging statistic :

[root@oss ~]# sar -B
Linux 2.6.32-431.el6.x86_64 (oss.ehowstuff.local)       08/18/2014      _x86_64_        (2 CPU)

11:10:01 PM  pgpgin/s pgpgout/s   fault/s  majflt/s  pgfree/s pgscank/s pgscand/s pgsteal/s    %vmeff
11:20:01 PM      0.00      0.23      8.21      0.00      4.36      0.00      0.00      0.00      0.00
11:30:01 PM      0.00      0.19      2.70      0.00      1.29      0.00      0.00      0.00      0.00
11:40:01 PM      0.00      0.14      3.75      0.00      1.62      0.00      0.00      0.00      0.00
11:50:01 PM      0.00      0.16      6.60      0.00      2.80      0.00      0.00      0.00      0.00
Average:         0.00      0.18      5.31      0.00      2.52      0.00      0.00      0.00      0.00

f) Display Swap space utilization statistics :

[root@oss ~]# sar -S
Linux 2.6.32-431.el6.x86_64 (oss.ehowstuff.local)       08/18/2014      _x86_64_        (2 CPU)

11:10:01 PM kbswpfree kbswpused  %swpused  kbswpcad   %swpcad
11:20:01 PM   2097144         0      0.00         0      0.00
11:30:01 PM   2097144         0      0.00         0      0.00
11:40:01 PM   2097144         0      0.00         0      0.00
11:50:01 PM   2097144         0      0.00         0      0.00
Average:      2097144         0      0.00         0      0.00

4. Monitor previous day sar statictics :

Every sar statictics by default stored in /var/log/sa. Es example :

[root@oss sa]# ls
sa18  sa19  sa20  sa21  sa22  sa23  sa24  sar18  sar19  sar20  sar23

a. Display cpu load on sa22 :

[root@oss sa]# sar -f /var/log/sa/sa22
Linux 2.6.32-431.el6.x86_64 (oss.ehowstuff.local)       08/22/2014      _x86_64_        (2 CPU)

12:00:01 AM     CPU     %user     %nice   %system   %iowait    %steal     %idle
12:10:01 AM     all      0.04      0.00      0.06      0.06      0.00     99.84
12:20:01 AM     all      0.03      0.00      0.05      0.07      0.00     99.85
12:30:01 AM     all      0.03      0.00      0.05      0.02      0.00     99.90
12:40:01 AM     all      0.34      0.00      0.71      0.02      0.00     98.93
12:50:01 AM     all      1.67      0.00      3.19      0.01      0.00     95.13
01:00:01 AM     all      0.64      0.00      0.83      0.74      0.00     97.79
01:10:01 AM     all      0.64      0.00      1.51      0.03      0.00     97.83
01:20:01 AM     all      0.35      0.00      0.71      0.13      0.00     98.81
01:30:01 AM     all      0.03      0.00      0.05      0.02      0.00     99.90
01:40:01 AM     all      0.03      0.00      0.05      0.02      0.00     99.90
01:50:01 AM     all      0.03      0.00      0.05      0.02      0.00     99.91
02:00:01 AM     all      0.03      0.00      0.05      0.02      0.00     99.90
02:10:01 AM     all      0.03      0.00      0.05      0.02      0.00     99.90
02:20:01 AM     all      0.03      0.00      0.04      0.02      0.00     99.91
02:30:01 AM     all      0.03      0.00      0.04      0.02      0.00     99.91
02:40:01 AM     all      0.03      0.00      0.04      0.02      0.00     99.91
02:50:01 AM     all      0.03      0.00      0.04      0.01      0.00     99.92
03:00:01 AM     all      0.03      0.00      0.04      0.02      0.00     99.92
03:10:01 AM     all      0.03      0.00      0.04      0.02      0.00     99.91
03:20:01 AM     all      0.03      0.00      0.04      0.01      0.00     99.92
03:30:01 AM     all      0.03      0.00      0.04      0.01      0.00     99.92
03:40:01 AM     all      0.03      0.00      0.05      0.02      0.00     99.91
03:50:01 AM     all      0.03      0.00      0.05      0.03      0.00     99.89
04:00:01 AM     all      0.03      0.00      0.04      0.02      0.00     99.91
04:10:01 AM     all      0.03      0.00      0.04      0.02      0.00     99.91
04:20:01 AM     all      0.03      0.00      0.04      0.02      0.00     99.92
04:30:01 AM     all      0.03      0.00      0.04      0.02      0.00     99.91
04:40:01 AM     all      0.03      0.00      0.04      0.02      0.00     99.91
04:50:01 AM     all      0.03      0.00      0.04      0.02      0.00     99.91
05:00:01 AM     all      0.03      0.00      0.04      0.02      0.00     99.92
05:10:01 AM     all      0.03      0.00      0.05      0.02      0.00     99.90
05:20:01 AM     all      0.03      0.00      0.05      0.02      0.00     99.91
05:30:01 AM     all      0.03      0.00      0.04      0.02      0.00     99.91

05:30:01 AM     CPU     %user     %nice   %system   %iowait    %steal     %idle
05:40:01 AM     all      0.03      0.00      0.05      0.01      0.00     99.91
05:50:01 AM     all      0.03      0.00      0.05      0.01      0.00     99.91
06:00:01 AM     all      0.03      0.00      0.05      0.02      0.00     99.91
06:10:01 AM     all      0.03      0.00      0.05      0.01      0.00     99.90
06:20:01 AM     all      0.03      0.00      0.05      0.01      0.00     99.91
06:30:01 AM     all      0.03      0.00      0.05      0.01      0.00     99.91
06:40:02 AM     all      0.03      0.00      0.05      0.01      0.00     99.92
06:50:01 AM     all      0.03      0.00      0.04      0.01      0.00     99.92
07:00:01 AM     all      0.03      0.00      0.05      0.01      0.00     99.91
07:10:01 AM     all      0.03      0.00      0.05      0.01      0.00     99.91
07:20:01 AM     all      0.03      0.00      0.05      0.01      0.00     99.92
07:30:01 AM     all      0.03      0.00      0.04      0.01      0.00     99.92
07:40:01 AM     all      0.03      0.00      0.05      0.01      0.00     99.91
07:50:01 AM     all      0.03      0.00      0.05      0.01      0.00     99.91
08:00:01 AM     all      0.03      0.00      0.05      0.01      0.00     99.91
08:10:01 AM     all      0.03      0.00      0.05      0.02      0.00     99.90
08:20:01 AM     all      0.03      0.00      0.05      0.01      0.00     99.91
Average:        all      0.10      0.00      0.18      0.03      0.00     99.69

b. Display RAM Load on sa22 :

[root@oss sa]# sar -f /var/log/sa/sa22 -r
Linux 2.6.32-431.el6.x86_64 (oss.ehowstuff.local)       08/22/2014      _x86_64_        (2 CPU)

12:00:01 AM kbmemfree kbmemused  %memused kbbuffers  kbcached  kbcommit   %commit
12:10:01 AM   3807336    117352      2.99      8252     32352     66920      1.11
12:20:01 AM   3804708    119980      3.06      8712     32812     71440      1.19
12:30:01 AM   3804584    120104      3.06      8744     32816     71592      1.19
12:40:01 AM   3799784    124904      3.18      8768     33124     72228      1.20
12:50:01 AM   3799668    125020      3.19      8800     33124     72228      1.20
01:00:01 AM   3723612    201076      5.12     10048    110608     71592      1.19
01:10:01 AM   3718804    205884      5.25     10232    111044     72228      1.20
01:20:01 AM   3711560    213128      5.43     10284    122396     71592      1.19
01:30:01 AM   3711684    213004      5.43     10324    122408     71592      1.19
01:40:01 AM   3711436    213252      5.43     10348    122408     71592      1.19
01:50:01 AM   3711436    213252      5.43     10372    122412     71592      1.19
02:00:01 AM   3711560    213128      5.43     10404    122416     71592      1.19
02:10:01 AM   3711576    213112      5.43     10444    122424     71592      1.19
02:20:01 AM   3711568    213120      5.43     10476    122428     71592      1.19
02:30:01 AM   3711444    213244      5.43     10516    122436     71592      1.19
02:40:01 AM   3711320    213368      5.44     10552    122436     71592      1.19
02:50:01 AM   3711320    213368      5.44     10580    122444     71592      1.19
03:00:01 AM   3711188    213500      5.44     10612    122448     71592      1.19
03:10:01 AM   3711180    213508      5.44     10652    122456     72168      1.20
03:20:01 AM   3711180    213508      5.44     10676    122456     72168      1.20
03:30:01 AM   3710932    213756      5.45     10708    122464     72168      1.20
03:40:01 AM   3710692    213996      5.45     10740    122464     72168      1.20
03:50:01 AM   3710700    213988      5.45     10928    122604     71592      1.19
04:00:01 AM   3710568    214120      5.46     10960    122608     71592      1.19
04:10:01 AM   3710816    213872      5.45     11000    122608     71592      1.19
04:20:01 AM   3710576    214112      5.46     11032    122616     71592      1.19
04:30:01 AM   3710576    214112      5.46     11068    122616     71592      1.19
04:40:01 AM   3710708    213980      5.45     11096    122624     71592      1.19
04:50:01 AM   3710452    214236      5.46     11128    122628     71592      1.19
05:00:01 AM   3710576    214112      5.46     11160    122636     71592      1.19
05:10:01 AM   3710452    214236      5.46     11200    122636     71592      1.19
05:20:01 AM   3710576    214112      5.46     11232    122644     71592      1.19
05:30:01 AM   3710708    213980      5.45     11268    122644     71592      1.19

05:30:01 AM kbmemfree kbmemused  %memused kbbuffers  kbcached  kbcommit   %commit
05:40:01 AM   3710320    214368      5.46     11296    122652     71592      1.19
05:50:01 AM   3710072    214616      5.47     11320    122652     71592      1.19
06:00:01 AM   3710204    214484      5.46     11364    122660     71592      1.19
06:10:01 AM   3710072    214616      5.47     11400    122664     71592      1.19
06:20:01 AM   3710204    214484      5.46     11424    122668     71592      1.19
06:30:01 AM   3710204    214484      5.46     11464    122676     71592      1.19
06:40:02 AM   3709948    214740      5.47     11488    122680     71592      1.19
06:50:01 AM   3710072    214616      5.47     11512    122680     71592      1.19
07:00:01 AM   3710204    214484      5.46     11552    122680     71592      1.19
07:10:01 AM   3709824    214864      5.47     11584    122692     71592      1.19
07:20:01 AM   3709948    214740      5.47     11612    122688     71592      1.19
07:30:01 AM   3709956    214732      5.47     11648    122700     71592      1.19
07:40:01 AM   3709956    214732      5.47     11680    122704     71592      1.19
07:50:01 AM   3709824    214864      5.47     11704    122708     71592      1.19
08:00:01 AM   3709832    214856      5.47     11744    122712     71592      1.19
08:10:01 AM   3709700    214988      5.48     11784    122724     71592      1.19
08:20:01 AM   3709700    214988      5.48     11812    122720     71592      1.19
Average:      3720306    204382      5.21     10794    113138     71580      1.19

5. Other sar options :

[root@oss ~]# sar -h
Usage: sar [ options ] [  [  ] ]
Main options and reports:
        -b      I/O and transfer rate statistics
        -B      Paging statistics
        -d      Block device statistics
        -I {  | SUM | ALL | XALL }
                Interrupts statistics
        -m      Power management statistics
        -n {  [,...] | ALL }
                Network statistics
                Keywords are:
                DEV     Network interfaces
                EDEV    Network interfaces (errors)
                NFS     NFS client
                NFSD    NFS server
                SOCK    Sockets (v4)
                IP      IP traffic      (v4)
                EIP     IP traffic      (v4) (errors)
                ICMP    ICMP traffic    (v4)
                EICMP   ICMP traffic    (v4) (errors)
                TCP     TCP traffic     (v4)
                ETCP    TCP traffic     (v4) (errors)
                UDP     UDP traffic     (v4)
                SOCK6   Sockets (v6)
                IP6     IP traffic      (v6)
                EIP6    IP traffic      (v6) (errors)
                ICMP6   ICMP traffic    (v6)
                EICMP6  ICMP traffic    (v6) (errors)
                UDP6    UDP traffic     (v6)
        -q      Queue length and load average statistics
        -r      Memory utilization statistics
        -R      Memory statistics
        -S      Swap space utilization statistics
        -u [ ALL ]
                CPU utilization statistics
        -v      Kernel table statistics
        -w      Task creation and system switching statistics
        -W      Swapping statistics
        -y      TTY device statistics

6. Other sysstat utilities:

sar collects and displays ALL system activities statistics.
sadc stands for “system activity data collector”. This is the sar backend tool that does the data collection.
sa1 stores system activities in binary data file. sa1 depends on sadc for this purpose. sa1 runs from cron.
sa2 creates daily summary of the collected statistics. sa2 runs from cron.
sadf can generate sar report in CSV, XML, and various other formats. Use this to integrate sar data with other tools.
iostat generates CPU, I/O statistics
mpstat displays CPU statistics.
pidstat reports statistics based on the process id (PID)
nfsiostat displays NFS I/O statistics.
cifsiostat generates CIFS statistics.

How to Check Disk Read Write Speed in Linux

hard diskMeasuring sequential disk performance is easy in linux as every modern Linux distribution comes with a command line utility called hdparm. Hdparm is a free tool to measure sequential disk performance which is primarily used to tune and optimize disk parameters. It is important to read hdparm man page and perform full backup before using hdparm command line utility. The popular option for hdparm command line utility are -t and -T.

-t is to perform buffered disk read which is the data transfer rate directly from the disk to memory.

-T is to perform timing cached read which is the data transfer rate from a memory buffer without reading the hard drive.

If hdparm is not installed then follow this steps to install hdparm on CentOS 6.3.

Type hdparm without option to get full hard disk parameters as a reference.

[root@centos63 ~]# hdparm

hdparm - get/set hard disk parameters - version v9.16

Usage:  hdparm  [options] [device] ..

Options:
 -a   get/set fs readahead
 -A   get/set the drive look-ahead flag (0/1)
 -b   get/set bus state (0 == off, 1 == on, 2 == tristate)
 -B   set Advanced Power Management setting (1-255)
 -c   get/set IDE 32-bit IO setting
 -C   check drive power mode status
 -d   get/set using_dma flag
 -D   enable/disable drive defect management
 -E   set cd/dvd drive speed
 -f   flush buffer cache for device on exit
 -F   flush drive write cache
 -g   display drive geometry
 -h   display terse usage information
 -H   read temperature from drive (Hitachi only)
 -i   display drive identification
 -I   detailed/current information directly from drive
 -k   get/set keep_settings_over_reset flag (0/1)
 -K   set drive keep_features_over_reset flag (0/1)
 -L   set drive doorlock (0/1) (removable harddisks only)
 -M   get/set acoustic management (0-254, 128: quiet, 254: fast)
 -m   get/set multiple sector count
 -N   get/set max visible number of sectors (HPA) (VERY DANGEROUS)
 -n   get/set ignore-write-errors flag (0/1)
 -p   set PIO mode on IDE interface chipset (0,1,2,3,4,...)
 -P   set drive prefetch count
 -q   change next setting quietly
 -Q   get/set DMA queue_depth (if supported)
 -r   get/set device  readonly flag (DANGEROUS to set)
 -R   obsolete
 -s   set power-up in standby flag (0/1) (DANGEROUS)
 -S   set standby (spindown) timeout
 -t   perform device read timings
 -T   perform cache read timings
 -u   get/set unmaskirq flag (0/1)
 -U   obsolete
 -v   defaults; same as -acdgkmur for IDE drives
 -V   display program version and exit immediately
 -w   perform device reset (DANGEROUS)
 -W   get/set drive write-caching flag (0/1)
 -x   obsolete
 -X   set IDE xfer mode (DANGEROUS)
 -y   put drive in standby mode
 -Y   put drive to sleep
 -Z   disable Seagate auto-powersaving mode
 -z   re-read partition table
 --dco-freeze      freeze/lock current device configuration until next power cycle
 --dco-identify    read/dump device configuration identify data
 --dco-restore     reset device configuration back to factory defaults
 --direct          use O_DIRECT to bypass page cache for timings
 --drq-hsm-error   crash system with a "stuck DRQ" error (VERY DANGEROUS)
 --fibmap          show device extents (and fragmentation) for a file
 --fibmap-sector   show absolute LBA of a specfic sector of a file
 --fwdownload            Download firmware file to drive (EXTREMELY DANGEROUS)
 --fwdownload-mode3      Download firmware using min-size segments (EXTREMELY DANGEROUS)
 --fwdownload-mode3-max  Download firmware using max-size segments (EXTREMELY DANGEROUS)
 --fwdownload-mode7      Download firmware using a single segment (EXTREMELY DANGEROUS)
 --idle-immediate  idle drive immediately
 --idle-unload     idle immediately and unload heads
 --Istdin          read identify data from stdin as ASCII hex
 --Istdout         write identify data to stdout as ASCII hex
 --make-bad-sector deliberately corrupt a sector directly on the media (VERY DANGEROUS)
 --prefer-ata12    use 12-byte (instead of 16-byte) SAT commands when possible
 --read-sector     read and dump (in hex) a sector directly from the media
 --security-help   display help for ATA security commands
 --trim-sectors    tell SSD firmware to discard unneeded data sectors (lba and count)
 --verbose         display extra diagnostics from some commands
 --write-sector    repair/overwrite a (possibly bad) sector directly on the media (VERY DANGEROUS)

Note : Substitute /dev/sda with the name of the disk device.

Example on how to check disk Disk Read Write Speed for /dev/sda on CentOS6.3 :

[root@centos63 ~]# hdparm -tT /dev/sda

/dev/sda:
 Timing cached reads:   4128 MB in  2.00 seconds = 2065.62 MB/sec
 Timing buffered disk reads:  276 MB in  3.06 seconds =  90.30 MB/sec

Example on how to check disk Disk Read Write Speed for /dev/sdb on CentOS6.3 :

[root@centos63 ~]# hdparm -tT /dev/sdb

/dev/sdb:
 Timing cached reads:   3410 MB in  2.00 seconds = 1705.84 MB/sec
 Timing buffered disk reads:  364 MB in  3.00 seconds = 121.29 MB/sec

How to Fix “-bash: hdparm: command not found” on Linux CentOS 6.3

hard diskQuestion:
When i try to run hdparm command to check to check the hard drive read and writing speed, the following error has been returned. Hdparm is a command line utility to measure sequential disk performance which is primarily used to tune and optimize disk parameters. Hdparm is free software under the BSD license.

[root@centos63 ~]# hdparm -t /dev/sda
-bash: hdparm: command not found

To Fix this, install hdparm command line utility on CentOS 6.3 using two method :

Method 1: Install hdparm using yum command :

[root@centos63 ~]# yum install hdparm -y

Example :

[root@centos63 ~]# yum install hdparm -y
Loaded plugins: fastestmirror, presto, priorities
Loading mirror speeds from cached hostfile
 * base: mirrors.sin3.sg.voxel.net
 * extras: mirrors.sin3.sg.voxel.net
 * updates: mirror.issp.co.th
Setting up Install Process
Resolving Dependencies
--> Running transaction check
---> Package hdparm.i686 0:9.16-3.4.el6 will be installed
--> Finished Dependency Resolution

Dependencies Resolved

====================================================================================================
 Package               Arch                Version                        Repository           Size
====================================================================================================
Installing:
 hdparm                i686                9.16-3.4.el6                   base                 72 k

Transaction Summary
====================================================================================================
Install       1 Package(s)

Total download size: 72 k
Installed size: 134 k
Downloading Packages:
Setting up and reading Presto delta metadata
Processing delta metadata
Package(s) data still to download: 72 k
hdparm-9.16-3.4.el6.i686.rpm                                                 |  72 kB     00:00
Running rpm_check_debug
Running Transaction Test
Transaction Test Succeeded
Running Transaction
  Installing : hdparm-9.16-3.4.el6.i686                                                         1/1
  Verifying  : hdparm-9.16-3.4.el6.i686                                                         1/1

Installed:
  hdparm.i686 0:9.16-3.4.el6

Complete!

Method 2 : Install hdparm using RPM packages as per OS and bit :

For 32 Bit :

[root@centos63 ~]# rpm -Uvh http://mirrors.hostemo.com/CentOS/6.3/os/i386/Packages/hdparm-9.16-3.4.el6.i686.rpm

For 64 Bit :

[root@centos63 ~]# rpm -Uvh http://mirrors.hostemo.com/CentOS/6.3/os/x86_64/Packages/hdparm-9.16-3.4.el6.x86_64.rpm

Verify if the hdparm RPM is installed or not by using the following command :

[root@centos63 ~]# rpm -qa | grep hdparm
hdparm-9.16-3.4.el6.i686

How to Check Realtime RAM Memory Usage Available in Linux VPS

rhelOn the previous post, i have teach you on how to check the memory usage on linux virtual Private Server (VPS) but only top command had provide real-time information and updates . This quick post will covers on how to check realtime RAM memory usage available in Linux VPS using watch command. Watch runs command repeatedly, displaying its output change over time or at regular intervals. Watch will run until interrupted. This command has been tested on Redhat Linux Enterprise 6 (RHEL 6) and may working on CentOS 6.x as well.

The basic syntax of watch is :

# watch [option(s)] command

1. Check memory usage using “top” command. Watch command not required for top command as top will update the result periodically. :

[root@rhel6 ~]# top

2. Check memory usage using “/proc/meminfo” with watch command :

[root@rhel6 ~]# watch -n 1 cat /proc/meminfo

Output :

Every 1.0s: cat /proc/meminfo                                               Mon Oct 15 13:48:17 2012

MemTotal:        1031320 kB
MemFree:          626372 kB
Buffers:           58576 kB
Cached:           217004 kB
SwapCached:            0 kB
Active:           148516 kB
Inactive:         164708 kB
Active(anon):      37816 kB
Inactive(anon):       84 kB
Active(file):     110700 kB
Inactive(file):   164624 kB
Unevictable:           0 kB
Mlocked:               0 kB
HighTotal:        141256 kB
HighFree:            280 kB
LowTotal:         890064 kB
LowFree:          626092 kB
SwapTotal:       2064376 kB
SwapFree:        2064376 kB
Dirty:                 4 kB
Writeback:             0 kB
AnonPages:         37652 kB
Mapped:            19096 kB
Shmem:               264 kB
Slab:              81048 kB
SReclaimable:      62096 kB
SUnreclaim:        18952 kB
..
..
..

3. Check memory usage using “free” with watch command :

[root@rhel6 ~]# watch -n 1 free

Output :

Every 1.0s: free                                                            Mon Oct 15 13:47:26 2012

             total       used       free     shared    buffers     cached
Mem:       1031320     404548     626772          0      58564     217004
-/+ buffers/cache:     128980     902340
Swap:      2064376          0    2064376

4. Check memory usage using “vmstat” with watch command :

[root@rhel6 ~]# watch -n 1 vmstat

Output :

Every 1.0s: vmstat                                                          Mon Oct 15 13:46:44 2012

procs -----------memory---------- ---swap-- -----io---- --system-- -----cpu-----
 r  b   swpd   free   buff  cache   si   so    bi    bo   in   cs us sy id wa st
 0  0      0 626280  58552 217004   30    0     0    14   46   73  0  1 98  1  0

4 Top Command Howto on Linux RHEL 6/CentOS 6

In this article, i will help you to explore most frequently used top commands that linux system administrator use when analyzing the linux performance and use for daily system administrative jobs. Top command displays system summary information such as tasks currently being managed by the Linux kernel, displays ongoing look at processor activity in real time and will displays a listing of the most CPU-intensive tasks on the system. It also will show the processor and memory are being used and other information like running processes. It will help you to summarize how much of your system’s resources are taking up.

1. How to display top command result :

[root@rhel6 ~]# top

top
This command will show information like tasks, memory, cpu load average, swap and number of users. Press ‘q’ to quit window.

2. How to display selected user using top -u :

[root@rhel6 ~]# top -u apache

top

3. How to display specific process with given PIDs Using top -p(e.g PID 2449, 2450) :

[root@rhel6 ~]# top -p 2449,2450

top

4. How to quit top command after a specified number of iterations :

[root@rhel6 ~]# top -n 10

This top command will automatically exit after 10 number of repetition.

How to Check Swap Space/Memory/Size on Linux VPS Server

Linux uses swap space as a back up to its physical memory with an overflow area. Swapping is necessary for two important reasons. First, when the system requires more memory than is physically available, the swap space will gives memory to the current application (process) that needs the memory immediately. Second, a significant number of the pages used by an application during its startup phase may only be used for initialization and then never used again. The system can swap out those pages and free the memory for other applications or even for the disk cache. All these four commands will show the allocated swap disk or disks, if any and have been tested on linux CentOS 6, CentOS 7, RHEL 6 and RHEL 7. Your output should look like the followings :

1. Check Swap Space/Memory/Size using “swapon -s

[root@centos63 ~]# swapon -s
Filename                                Type            Size    Used    Priority
/dev/dm-1                               partition       2064376 0       -1

2. Check Swap Space/Memory/Size using “cat /proc/swaps

[root@centos63 ~]# cat /proc/swaps
Filename                                Type            Size    Used    Priority
/dev/dm-1                               partition       2064376 0       -1

3. Check Swap Space/Memory/Size using “free -m

[root@centos63 ~]# free -m
             total       used       free     shared    buffers     cached
Mem:          1006        121        885          0          7         71
-/+ buffers/cache:         41        964
Swap:         2015          0       2015

4. Check Swap Space/Memory/Size using “cat /proc/meminfo | grep Swap

[root@centos63 ~]# cat /proc/meminfo | grep Swap
SwapCached:            0 kB
SwapTotal:       2064376 kB
SwapFree:        2064376 kB

How to Install sysstat on CentOS 5.7 Linux Server

In this post i will show how to install sysstat on Linux CentOS 5.7 server. Sysstat package is on of the method to monitor the linux server. sysstat package contains the sar, iostat and mpstat which are the system performance tools for Linux operating system. The sar command collects and reports system activity information. The iostat command reports CPU utilization and I/O statistics for disks. The mpstat command reports global and per-processor statistics. The statistics reported by sar concern I/O transfer rates, paging activity, process-related activities, interrupts, network activity, memory and swap space utilization, CPU utilization, kernel activities and TTY statistics. All these tools can be schedule via cron to collect and historize performance and activity data. These are the list of sysstat’s available packages :

a) iostat(1) reports CPU statistics and input/output statistics for devices, partitions and network filesystems.
b) mpstat(1) reports individual or combined processor related statistics.
c) pidstat(1) reports statistics for Linux tasks (processes) : I/O, CPU, memory, etc.
d) sar(1) collects, reports and saves system activity information (CPU, memory, disks, interrupts, network interfaces, TTY, kernel tables,etc.)
e) sadc(8) is the system activity data collector, used as a backend for sar.
f) sa1(8) collects and stores binary data in the system activity daily data file. It is a front end to sadc designed to be run from cron.
g) sa2(8) writes a summarized daily activity report. It is a front end to sar designed to be run from cron.
h) sadf(1) displays data collected by sar in multiple formats (CSV, XML, etc.) This is useful to load performance data into a database, or import them in a spreadsheet to make graphs.
i) nfsiostat(1) reports input/output statistics for network filesystems (NFS).
j) cifsiostat(1) reports CIFS statistics.

To install sysstat, simply run the following command :

[root@CentOS57 ~]# yum install sysstat -y
Loaded plugins: fastestmirror
Loading mirror speeds from cached hostfile
 * base: mirror.oscc.org.my
 * extras: mirror.oscc.org.my
 * rpmforge: fr2.rpmfind.net
 * updates: mirror.oscc.org.my
Setting up Install Process
Resolving Dependencies
--> Running transaction check
---> Package sysstat.i386 0:7.0.2-11.el5 set to be updated
--> Finished Dependency Resolution

Dependencies Resolved

====================================================================================================
 Package                Arch                Version                       Repository           Size
====================================================================================================
Installing:
 sysstat                i386                7.0.2-11.el5                  base                182 k

Transaction Summary
====================================================================================================
Install       1 Package(s)
Upgrade       0 Package(s)

Total download size: 182 k
Downloading Packages:
sysstat-7.0.2-11.el5.i386.rpm                                                | 182 kB     00:00
Running rpm_check_debug
Running Transaction Test
Finished Transaction Test
Transaction Test Succeeded
Running Transaction
  Installing     : sysstat                                                                      1/1

Installed:
  sysstat.i386 0:7.0.2-11.el5

Complete!

To check the installed sysstat’s package using rpm command :

[root@CentOS57 ~]# rpm -qa sysstat
sysstat-7.0.2-11.el5

To see sysstat manual simply run any of the sisgle package command. As an example sa1 :

[root@CentOS57 ~]# man sa1
SA1(8)                        Linux Userâs Manual                       SA1(8)

NAME
       sa1 - Collect and store binary data in the system activity daily data file.

SYNOPSIS
       /usr/lib/sa/sa1 [ interval count ]

DESCRIPTION
       The sa1 command is a shell procedure variant of the sadc command and handles all of
       the flags and parameters of that command. The sa1 command collects and store binary
       data  in  the  /var/log/sa/sadd  file, where the dd parameter indicates the current
       day. The interval and count parameters specify that the record  should  be  written
       count  times  at interval seconds. If you do not specify these parameters, a single
       record is written.

       The sa1 command is designed to be started automatically by the cron command.

EXAMPLES
       To create a daily record of sar activities, place the following entry in your  root
       or adm crontab file:

       0 8-18 * * 1-5 /usr/lib/sa/sa1 1200 3 &

FILES
       /var/log/sa/sadd
              Indicate  the daily data file, where the dd parameter is a number represent-
              ing the day of the month.

AUTHOR
       Sebastien Godard (sysstat  wanadoo.fr)

SEE ALSO
       sar(1), sadc(8), sa2(8), sadf(1), mpstat(1), iostat(1), vmstat(8)

       http://perso.orange.fr/sebastien.godard/

Linux                              JUNE 2006                            SA1(8)

How to Check Memory and I/O with vmstat on Linux CentOS 5/CentOS 6/RHEL 5/RHEL 6 server – System Performance

vmstat command is a utility that will provides interesting information abaout processes, memory, i/O and CPU activity. When you run vmstat utility command without any arguments, you will see the output as below. This post will guide you on how to check memory and I/O with vmstat on Linux CentOS 5/CentOS 6/RHEL 5/RHEL 6 server to optimize the linux system performance.

[root@rhel6 ~]# vmstat
procs -----------memory---------- ---swap-- -----io---- --system-- -----cpu-----
 r  b   swpd   free   buff  cache   si   so    bi    bo   in   cs us sy id wa st
 0  0      0 888252  28976  63004    5    0     0    11   15   18  0  0 99  0  0

Below are the description of above vmstat output :
1. The procs fields show the number of processes
– Waiting for run time (r)
-Blocked (b)
-Swapped out (w)

2. The memory fields show the kilobytes of swap memory,free memory, buffered memory and Cached memory

3. The swap fields show the kilobytes per second of memory
-Swapped in from disk (si)
-Swapped out to disk (so)

4. The io fields show the number of blocks per second
-Sent to block devices (bi)
-Received from block devices (bo)

5. The system shows the number of
-Interrupts per second (in)
-Context switches per second (cs)

6. The cpu field shows the percentage of total CPU time as
-User time (us)
-System time (sy)
-Idle (id) time

To prints the vmstat version, run this command :

[root@rhel6 ~]# vmstat -V
procps version 3.2.8

We can let the vmstat to run automatically by execute this command :

[root@rhel6 ~]# vmstat 

Where nsec is the number of seconds you want it to wait before another update.

To run vmstat on every 10 seconds :

[root@rhel6 ~]# vmstat 10
procs -----------memory---------- ---swap-- -----io---- --system-- -----cpu-----
 r  b   swpd   free   buff  cache   si   so    bi    bo   in   cs us sy id wa st
 0  0      0 888072  29088  63004    4    0     0    11   14   18  0  0 99  0  0
 0  0      0 888064  29088  63004    0    0     0     0   12   15  0  0 100  0  0
 0  0      0 888064  29088  63004    0    0     0     0   13   16  0  0 100  0  0

Other vmstat usage as below :

usage: vmstat [-V] [-n] [delay [count]]
              -V prints version.
              -n causes the headers not to be reprinted regularly.
              -a print inactive/active page stats.
              -d prints disk statistics
              -D prints disk table
              -p prints disk partition statistics
              -s prints vm table
              -m prints slabinfo
              -t add timestamp to output
              -S unit size
              delay is the delay between updates in seconds.
              unit size k:1000 K:1024 m:1000000 M:1048576 (default is K)
              count is the number of updates.