How to implement and use Awstats and Jawstats !!!!!! Alternative to weblog expert!!!!! Analytic tool for logs

Awstats is a free, powerful and classy tool for Logs analytics.  In this blog i would explain you on how to get started with awstats and it's full use and implementation . This is one of the most useful tool for analyzing Apache logs (but can be used to analyse any sort of logs). I will be explaining things here w.r.t Apache logs. Installing, configuring and using awstats is very easy. But problem occurs when you have distributed system and you want to bring all the logs to one location and apply the analytic on a merged log file. I had tried to explain whatever problems I faced along with solution. Hopefully you don't have to fight much after going through this blog :).

Getting started:

Install awstats on your linux box using yum commnad :
sudo yum install awstats

Vim the domain’s configuration file (located at /etc/awstats) and edit following line via text editor:
Vim awstats.xyz.com.conf

LogFile="/var/log/httpd/xyz.log" SiteDomain="xyz.com" HostAliases="servername (e.g: dev.home.lan)" DirCgi="/var/www/cgi-bin" DirIcons="../awstats/icon"

[kulshresht@dev:/etc/awstats]$ll
-rw-r--r-- 1 root root 61936 Aug 29 13:51 awstats.xyz.com.conf
-rw-r--r-- 1 root root 61665 Feb 21  2013 awstats.localhost.localdomain.conf

-rw-r--r-- 1 root root 61665 Feb 21  2013 awstats.model.conf

/var/lib/awstats --> Database location for awstats


Merging Logs:

One of the solution for aggregating logs from different servers is to get a nfs mounted on all the production servers. Ship all the logs from individual production servers to the NFS (or any common location).

Rsync individual server's Apache logs to a common location (nfs in my case)

On server 1 : rsync -avrL /var/log/httpd/access.log /nfs/Apache_logs/access.log
On server 2 : rsync -avrL /var/log/httpd/access.log /nfs/Apache_logs/access1.log
On server 3 : rsync -avrL /var/log/httpd/access.log /nfs/Apache_logs/access2.log

Use below command to merge all the 3 log files.

sudo /usr/share/awstats/tools/logresolvemerge.pl access.log access1.log access2.log > access_final.log

More detail about  logresolvemerge.pl at below link : http://awstats.sourceforge.net/docs/awstats_tools.htm

Linux sometime doesn't understand too many parameters. So it's better we should put the commands inside shell script and give shell script path inside crontab. All cron and shell script has been mentioned below:

Cron Entries:

Crontab -e

*/5 * * * * /home/kulshresht/awstats.sh > /dev/null 2>&1 */5 * * * * /home/kulshresht/logmerge.sh > /dev/null 2>&1 1 0 * * * /home/kulshresht/logrotate.sh > /dev/null 2>&1

Contents of all shell scripts mentioned in Cron above:

awstats.sh  [Awstats run to generate the statistic data:]

#! /bin/bash /usr/share/awstats/wwwroot/cgi-bin/awstats.pl -config=xyz.com -update

 logmerge.sh [Merging logs]

#! /bin/bash set -x cd /nfs/Apache_logs /usr/share/awstats/tools/logresolvemerge.pl access.log access1.log access2.log > access_final.log echo "The files have been Merged successfully"

logrotate.sh [Log rotation and deleting old logs at 12 AM midnight ]

#! /bin/bash cd /nfs/Apache_logs rm -rf * touch access1.log access2.log access.log access_final.log chmod a+rwx * echo "Merged"

Remeber to change the permission of scripts:

-rwxrwxrwx 1 kulshresht kulshresht        90 Aug 29 17:28 awstats.sh
-rwxrwxrwx 1 kulshresht kulshresht       175 Aug 29 17:27 logmerge.sh
-rwxrwxrwx 1 kulshresht kulshresht        90 Aug 29 17:28 logrotate.sh

You are done and awstats is ready for use. 

Use below link (IP address of the server on which awstats is installed) to access the awstats UI.
http://115.11.82.11/awstats/awstats.pl?config=xyz.com

For hourly basis stats: http://115.11.82.11/awstats/awstats.pl?month=11&year=2013&output=main&config=homeshop18.com&framename=index&databasebreak=hour&day=19&hour=11


You can have better view through jawstats (built on top of awstats). Snapshot below:

Reference:
http://blog.secaserver.com/2011/12/linux-install-jawstats-beautiful-statistic-awstats-core/
http://awstats.sourceforge.net/docs/awstats_setup.html
http://awstats.sourceforge.net/
http://awstats.sourceforge.net/docs/awstats_faq.html#DAILY
http://ebasso.net/wiki/index.php/AWSTATS:_Configurando_o_AWSTATS