Awstats is a free, powerful and classy tool for Logs analytics. In this blog i would explain you on how to get started with awstats and it's full use and implementation . This is one of the most useful tool for analyzing Apache logs (but can be used to analyse any sort of logs). I will be explaining things here w.r.t Apache logs. Installing, configuring and using awstats is very easy. But problem occurs when you have distributed system and you want to bring all the logs to one location and apply the analytic on a merged log file. I had tried to explain whatever problems I faced along with solution. Hopefully you don't have to fight much after going through this blog :).
Getting started:
Install awstats on your linux box using yum commnad :
sudo yum install awstats
Vim the domain’s configuration file (located at /etc/awstats) and edit following line via text editor:
Vim awstats.xyz.com.conf
LogFile="/var/log/httpd/xyz.log"
SiteDomain="xyz.com"
HostAliases="servername (e.g: dev.home.lan)"
DirCgi="/var/www/cgi-bin"
DirIcons="../awstats/icon"
|
[kulshresht@dev:/etc/awstats]$ll
-rw-r--r-- 1 root root 61936 Aug 29 13:51 awstats.xyz.com.conf
-rw-r--r-- 1 root root 61665 Feb 21 2013 awstats.localhost.localdomain.conf
-rw-r--r-- 1 root root 61665 Feb 21 2013 awstats.model.conf
/var/lib/awstats --> Database location for awstats
Merging Logs:
One of the solution for aggregating logs from different servers is to get a nfs mounted on all the production servers. Ship all the logs from individual production servers to the NFS (or any common location).
Rsync individual server's Apache logs to a common location (nfs in my case)
On server 1 : rsync -avrL /var/log/httpd/access.log /nfs/Apache_logs/access.log
On server 2 : rsync -avrL /var/log/httpd/access.log /nfs/Apache_logs/access1.log
On server 3 : rsync -avrL /var/log/httpd/access.log /nfs/Apache_logs/access2.log
Use below command to merge all the 3 log files.
sudo /usr/share/awstats/tools/logresolvemerge.pl access.log access1.log access2.log > access_final.log
More detail about logresolvemerge.pl at below link : http://awstats.sourceforge.net/docs/awstats_tools.htm
Linux sometime doesn't understand too many parameters. So it's better we should put the commands inside shell script and give shell script path inside crontab. All cron and shell script has been mentioned below:Merging Logs:
One of the solution for aggregating logs from different servers is to get a nfs mounted on all the production servers. Ship all the logs from individual production servers to the NFS (or any common location).
Rsync individual server's Apache logs to a common location (nfs in my case)
On server 1 : rsync -avrL /var/log/httpd/access.log /nfs/Apache_logs/access.log
On server 2 : rsync -avrL /var/log/httpd/access.log /nfs/Apache_logs/access1.log
On server 3 : rsync -avrL /var/log/httpd/access.log /nfs/Apache_logs/access2.log
Use below command to merge all the 3 log files.
sudo /usr/share/awstats/tools/logresolvemerge.pl access.log access1.log access2.log > access_final.log
More detail about logresolvemerge.pl at below link : http://awstats.sourceforge.net/docs/awstats_tools.htm
Cron Entries:
Crontab -e
*/5 * * * * /home/kulshresht/awstats.sh > /dev/null
2>&1
*/5 * * * * /home/kulshresht/logmerge.sh > /dev/null
2>&1
1 0 * * * /home/kulshresht/logrotate.sh > /dev/null
2>&1
|
Contents of all shell scripts mentioned in Cron above:
awstats.sh [Awstats run to generate the statistic data:]
#! /bin/bash
/usr/share/awstats/wwwroot/cgi-bin/awstats.pl -config=xyz.com
-update
|
#! /bin/bash
set -x
cd /nfs/Apache_logs
/usr/share/awstats/tools/logresolvemerge.pl access.log access1.log
access2.log > access_final.log
echo "The files have been Merged successfully"
|
logrotate.sh [Log rotation and deleting old logs at 12 AM midnight ]
#! /bin/bash
cd /nfs/Apache_logs
rm -rf *
touch access1.log access2.log access.log access_final.log
chmod a+rwx *
echo "Merged"
|
Remeber to change the permission of scripts:
-rwxrwxrwx 1 kulshresht kulshresht 90 Aug 29 17:28 awstats.sh
-rwxrwxrwx 1 kulshresht kulshresht 175 Aug 29 17:27 logmerge.sh
-rwxrwxrwx 1 kulshresht kulshresht 90 Aug 29 17:28 logrotate.sh
You are done and awstats is ready for use.
Use below link (IP address of the server on which awstats is installed) to access the awstats UI.
http://115.11.82.11/awstats/awstats.pl?config=xyz.com
For hourly basis stats: http://115.11.82.11/awstats/awstats.pl?month=11&year=2013&output=main&config=homeshop18.com&framename=index&databasebreak=hour&day=19&hour=11
You can have better view through jawstats (built on top of awstats). Snapshot below:
Reference:
http://blog.secaserver.com/2011/12/linux-install-jawstats-beautiful-statistic-awstats-core/
http://awstats.sourceforge.net/docs/awstats_setup.html
http://awstats.sourceforge.net/
http://awstats.sourceforge.net/docs/awstats_faq.html#DAILY
http://ebasso.net/wiki/index.php/AWSTATS:_Configurando_o_AWSTATS
