Shell script to find total request vs error count in Apache logs

Tweak the below script to suit your log format .

#!/bin/sh echo "" > /tmp/error.csv outFile="/tmp/Error.txt" outFile1="/tmp/Error_Count.txt" outFile2="/tmp/Total.txt" outFile3="/tmp/Total_Request.txt" a=`cat /var/log/access.log|awk '{print $10}' | wc -l` cat /var/log/access.log|awk '{print $4":"$10}' | awk -F':' '{print $2}' | sort | uniq -c > $outFile2 cat $outFile2 | awk '{for(i=1;i<NF;i++){ print $2 "  " $i*3}}' > $outFile3 b=`cat /var/log/access.log|awk '{print $10}' | grep "^5" | wc -l` cat /var/log/access.log| awk '{print $4":"$10}' | awk -F':' '{if($NF>=500){print $2}}' | sort | uniq -c > $outFile cat $outFile | awk '{for(i=1;i<NF;i++){ print $2 "  " $i*3}}' > $outFile1 mailsend -smtp mailserver.production.xyz.lan -port 25 -M "Total request count/Error count for `date +%m:%d:%Y` : $(expr $a \* 3) / $(expr $b \* 3) " -t kulshresht.gautam@xyz.com -f kulshresht.gautam@xyz.com -sub "Request / Error count" -attach $outFile1,text,a -attach $outFile3,text,a

Log format Sample:

1.2.3.4 - - [31/Jul/2013:00:00:00 +0530] 0 "GET  HTTP/1.1" 200 20 0 "http://www.xyz.com/adsjskds" "Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 5.1; Trident/4.0; InfoPath.2)" "8D8E535ED8CBE15ED60C365.jvmnode1"

**Mailsend should be configured on your system.

http://www.linuxnix.com/2010/12/how-to-configure-and-install-sendmail-in-linux.html

**I am mutlipying the output by 3 because we have 3 apache web servers , and all 3 are under one Load Balancer.