Перейти на Новый сайт

Общая информация

В данном разделе находится общая информация.

Материалы по Linux

В данном разделе находятся примеры различных Linux shell скриптов.

Материалы по Windows

В данном разделе находятся примеры различных скриптов на WSH.

Общие материалы

В данном разделе находятся материалы по разным темам.

Ссылки

В данном разделе находятся ссылки.

Для корректного отображения страницы, рекомендуется использовать браузер поддерживающий JavaScript.

Для навигации пользуйтесь боковым меню и кнопками "Описание" и "Подробно".

Linux cкрипт free-sa-report2 для запуска Free-SA - анализатора логов Squid, в планировщике заданий cron.

(Версия 2.1, оптимизированная для корректного создания накопительных отчетов за день, неделю и месяц. Использует промежуточный лог файл.)

Основные возможности и описание см. Linux cкрипт free-sa-report для запуска Free-SA

Пример скрипта free-sa-report2:

(Скачать файл можно будет здесь. ЗЫ Не тыкать! Жать правой кнопкой: сохранить ссылку как...)

Условия см. Linux cкрипт free-sa-report для запуска Free-SA, за исключением:

LOG_NAME - выбирается автоматически в соответствии с именами лог файлов, указанных в ACCESSLOGS.
В каталоге HTMLOUTDIR/LOG_NAME/day автоматически создаются подкаталоги за период,
типа: dd.mm.yyyy-dd.mm.yyyy-n
В каталоге HTMLOUTDIR/LOG_NAME/month HTMLOUTDIR/LOG_NAME/week автоматически создаются подкаталоги за месяц и неделю,
типа: lastdate-curdate-M и lastdate-curdate-W
, где M - номер месяца, W - номер недели.
Промежуточные накопительные логи за месяц и неделю
соответственно: /var/log/squid/free-sa_LOG_NAME.m.log и /var/log/squid/free-sa_LOG_NAME.w.log
Промежуточные рабочие логи за месяц и неделю
соответственно: /var/log/squid/free-sa_LOG_NAME.mv.log и /var/log/squid/free-sa_LOG_NAME.wv.log

Текст:

#!/bin/sh
#
# free-sa-report2
# Report generate script for Free-SA
# Author: Konstantin Nadezhdin nk_rec@mail.ru
# Version: 2.1
#
# Read sysconfig
if [ -s /etc/sysconfig/free-sa ]; then
    . /etc/sysconfig/free-sa
else
    echo "Error: /etc/sysconfig/free-sa not exist!"
    exit
fi

FREE_SA=/usr/bin/free-sa
if [ "$USE_FREE_SA_REPORT" != "yes" ]; then
    echo "Error: free-sa-report disabled in /etc/sysconfig/free-sa"
    exit
fi

if [ "$FREE_SA_CONF" = "" ]; then
    FREE_SA_CONF=/usr/share/free-sa/free-sa-report.conf
elif [ ! -s $FREE_SA_CONF ]; then
    echo "$FREE_SA_CONF not exist!"
    exit
fi

if [ "$TMPDIR" = "" ]; then
    TMPDIR=/var/cache/free-sa
elif [ ! -d $TMPDIR ]; then
    echo "$TMPDIR not exist or isn't directory!"
    exit
fi

if [ "$HTMLOUTDIR" = "" ]; then
    HTMLOUTDIR=/srv/www/htdocs/free-sa
elif [ ! -d $HTMLOUTDIR ]; then
    echo "$HTMLOUTDIR not exist or isn't directory!"
    exit
fi

if [ "$ACCESSLOGS" = "" ]; then
    ACCESSLOGS=/var/log/squid/access.log
fi

MONTH_CONF=$TMPDIR/THISMONTH.conf
WEEK_CONF=$TMPDIR/THISWEEK.conf
DAY_CONF=$TMPDIR/TODAY.conf

make_index ()
{
    if [ ! -s $HTMLOUT/index.html ]; then
        echo -e "<html>\n\
             <head>\n\
             <META http-equiv=\"content-type\" content=\"text/html; charset=utf-8\">\n\
             <script language=Javascript>\n\
             function change(select) {\n\
                 if( select.options[ select.selectedIndex].value) {\n\
                     select.form.action = select.options[ select.selectedIndex].value;\n\
                     select.form.submit();\n\
                 }\n\
             }\n\
             </script>\n\
             </head>\n\
             <body>\n\
             <h3>Free-SA report</h3>\n\
             <form target='page' name='menuform'>\n\
                <select name='slmenu' onchange='change(this)'>\n\
                    <option selected value=''>-Select reports-</option>\n\
                    <option value = 'day'>Free-SA reports by day</option>\n\
                    <option value = 'week'>Free-SA reports by week</option>\n\
                    <option value = 'month'>Free-SA reports by month</option>\n\
                </select>\n\
             </form>\n\
             </body>\n\
             </html>\n" > $HTMLOUT/index.html
    fi
}

check_archive_log ()
{
    FINDTO=0
    FINDTOTYPE=""
    for archdate in $1
    do
        if [ -s $ACCESSLOG-$archdate.bz2 ]; then
            FINDTO=1
            FINDTOTYPE="bz2"
        elif [ -s $ACCESSLOG-$archdate.gz ]; then
            FINDTO=1
            FINDTOTYPE="gz"
        fi
    done
}

correct_logbase ()
{
    case $1 in
      month)
        $FREE_SA -l $logbasemonth -r -d -$TOTHISMONTH
        ;;
      week)
        $FREE_SA -l $logbaseweek -r -d -$TOTHISWEEK
        ;;
      *)
        ;;
    esac
}

add_archive_log ()
{

    check_archive_log "$TODAY"
    if [ $FINDTO -eq 1 ]; then
        if [ "$VERBOSE" = "yes" ]; then echo "Daily archive found"; fi
        case $FINDTOTYPE in
            bz2) bunzip2 -k $ACCESSLOG-$TODAY.$FINDTOTYPE;;
            gz) gunzip -c $ACCESSLOG-$TODAY.$FINDTOTYPE > $ACCESSLOG-$TODAY;;
        esac
        if [ "$VERBOSE" = "yes" ]; then echo "Add archive to monthly log"; fi
        if [ $MONTHDAY1 -eq 0 ]; then
            if [ "$VERBOSE" = "yes" ]; then echo -e "\tNew month - rewrite monthly log"; fi
            cat $ACCESSLOG-$TODAY > $logbasemonth
        else
            cat $ACCESSLOG-$TODAY >> $logbasemonth
        fi
        if [ "$VERBOSE" = "yes" ]; then echo -e "\tCorrect logbasemonth"; fi
        correct_logbase month
        if [ "$VERBOSE" = "yes" ]; then echo "Add archive to weekly log"; fi
        if [ $WEEKDAY1 -eq 0 ]; then
            if [ "$VERBOSE" = "yes" ]; then echo -e "\tNew week - rewrite weekly log"; fi
            cat $ACCESSLOG-$TODAY > $logbaseweek
        else
            cat $ACCESSLOG-$TODAY >> $logbaseweek
        fi
        if [ "$VERBOSE" = "yes" ]; then echo -e "\tCorrect logbaseweek"; fi
        correct_logbase week
        if [ "$VERBOSE" = "yes" ]; then echo "Create daily log"; fi
        cat $ACCESSLOG >> $ACCESSLOG-$TODAY
        ACCESSLOGDAY=$ACCESSLOG-$TODAY
    else
        if [ "$VERBOSE" = "yes" ]; then echo "Daily archive not found"; fi
        ACCESSLOGDAY=$ACCESSLOG
        if [ $MONTHDAY1 -eq 0 ]; then
            if [ "$VERBOSE" = "yes" ]; then echo -e "\tNew month - clear monthly log"; fi
            rm -f $logbasemonth && touch $logbasemonth
        fi
        if [ $WEEKDAY1 -eq 0 ]; then
            if [ "$VERBOSE" = "yes" ]; then echo -e "\tNew week - clear weekly log"; fi
            rm -f $logbaseweek && touch $logbaseweek
        fi
    fi
    if [ "$VERBOSE" = "yes" ]; then echo "Use daily log: $ACCESSLOGDAY"; fi

    check_archive_log "$TOWEEK"
    if [ $FINDTO -eq 1 ]; then
        if [ "$VERBOSE" = "yes" ]; then echo "Weekly archive found"; fi
        cat $logbaseweek $ACCESSLOG > $logbaseweev
        ACCESSLOGWEEK=$logbaseweev
    else
        if [ "$VERBOSE" = "yes" ]; then echo "Weekly archive not found"; fi
        ACCESSLOGWEEK=$ACCESSLOG
    fi
    if [ "$VERBOSE" = "yes" ]; then echo "Use weekly log: $ACCESSLOGWEEK"; fi

    check_archive_log "$TOMONTH"
    if [ $FINDTO -eq 1 ]; then
        if [ "$VERBOSE" = "yes" ]; then echo "Monthly log found"; fi
        cat $logbasemonth $ACCESSLOG > $logbasemontv
        ACCESSLOGMONTH=$logbasemontv
    else
        if [ "$VERBOSE" = "yes" ]; then echo "Monthly log not found"; fi
        ACCESSLOGMONTH=$ACCESSLOG
    fi
    if [ "$VERBOSE" = "yes" ]; then echo "Use monthly log: $ACCESSLOGMONTH"; fi

}

del_bunzip2ed_log ()
{
    if [ -s $ACCESSLOG-$TODAY ]; then
        if [ "$VERBOSE" = "yes" ]; then echo "Delete archive log"; fi
        rm -f $ACCESSLOG-$TODAY
    fi
}

sed_conf ()
{
sed -e 's/^rotate=/#rotate=/' \
    -e 's/^overwrite=/#overwrite=/' \
    -e 's/^name=/#name=/' < $FREE_SA_CONF > $1
}

check_conf ()
{
    case $1 in
      month)
        sed_conf "$MONTH_CONF"
        echo -e "\nrotate=\"quarter\"\noverwrite=\"2\"\nname=\"THISMONTH\"" >> $MONTH_CONF
        ;;
      week)
        sed_conf "$WEEK_CONF"
        echo -e "\nrotate=\"month\"\noverwrite=\"2\"\nname=\"THISWEEK\"" >> $WEEK_CONF
        ;;
      day)
        sed_conf "$DAY_CONF"
        echo -e "\nrotate=\"week\"\noverwrite=\"2\"\nname=\"TODAY\"" >> $DAY_CONF
        ;;
      *)
        ;;
    esac
}

check_dir ()
{
    if [ ! -d $HTMLOUT/$1 ]; then
        mkdir $HTMLOUT/$1
    fi
    if [ "$2" = "1" ]; then
        for i in css png js
        do
            cp -pfuv /srv/www/htdocs/free-sa/*.$i $HTMLOUT/$1
        done
    fi
}

make_report ()
{
    case $1 in
      day)
        check_conf day
        $FREE_SA -l $ACCESSLOGDAY -f $DAY_CONF -o $HTMLOUT/day -d day
        check_conf day
        $FREE_SA -F -f $DAY_CONF -o $HTMLOUT/day
        ;;
      week)
        check_conf week
        $FREE_SA -l $ACCESSLOGWEEK -f $WEEK_CONF -o $HTMLOUT/week/var -d $THISWEEK-
        rm -rf $HTMLOUT/week/lastdate-curdate-$THISWEEKNUM && mv -f $HTMLOUT/week/var/*.*.*-*.*.* $HTMLOUT/week/lastdate-curdate-$THISWEEKNUM
        check_conf week
        $FREE_SA -F -f $WEEK_CONF -o $HTMLOUT/week
        ;;
      month)
        check_conf month
        $FREE_SA -l $ACCESSLOGMONTH -f $MONTH_CONF -o $HTMLOUT/month/var -d $THISMONTH-
        rm -rf $HTMLOUT/month/lastdate-curdate-$THISMONTHNUM && mv -f $HTMLOUT/month/var/*.*.*-*.*.* $HTMLOUT/month/lastdate-curdate-$THISMONTHNUM
        check_conf month
        $FREE_SA -F -f $MONTH_CONF -o $HTMLOUT/month
        ;;
      *)
        ;;
    esac
}

export LC_TIME=ru_RU.UTF-8

TODAY=`date "+%Y%m%d"`

let WEEKDAY=`date "+%u"`
let WEEKDAY1=$WEEKDAY-1
THISWEEK=`date -d -${WEEKDAY1}day '+%d.%m.%Y'`
TOTHISWEEK=`date -d -${WEEKDAY}day '+%d.%m.%Y'`
THISWEEKNUM=`date "+%V"`
while [ $WEEKDAY -gt 0 ]
do
    let WEEKDAY=$WEEKDAY-1
    TOWEEK="$TOWEEK "`date -d -${WEEKDAY}day '+%Y%m%d'`
done

let MONTHDAY=`date "+%-d"`
let MONTHDAY1=$MONTHDAY-1
THISMONTH=`date "+01.%m.%Y"`
TOTHISMONTH=`date -d -${MONTHDAY}day '+%d.%m.%Y'`
THISMONTHNUM=`date "+%m"`
while [ $MONTHDAY -gt 0 ]
do
    let MONTHDAY=$MONTHDAY-1
    TOMONTH="$TOMONTH "`date -d -${MONTHDAY}day '+%Y%m%d'`
done

for ACCESSLOG in $ACCESSLOGS;
do
    if [ -s $ACCESSLOG ]; then
        # Start script

HTMLOUT="$HTMLOUTDIR/"`echo $ACCESSLOG | sed -e 's/^.*\///' -e 's/\./_/'`
check_dir

logbase="/var/log/squid/free-sa_"`echo $ACCESSLOG | sed -e 's/^.*\///' -e 's/\./_/'`
logbasemonth=$logbase.m.log
logbasemontv=$logbase.mv.log
logbaseweek=$logbase.w.log
logbaseweev=$logbase.wv.log

if [ "$VERBOSE" = "yes" ]; then echo "Week log start at: $TOTHISWEEK"; fi
if [ "$VERBOSE" = "yes" ]; then echo "Month log start at: $TOTHISMONTH"; fi
if [ "$VERBOSE" = "yes" ]; then echo "Find logs"; fi
add_archive_log

chmod 640 $logbasemonth
chmod 640 $logbasemontv
chmod 640 $logbaseweek
chmod 640 $logbaseweev

if [ "$VERBOSE" = "yes" ]; then echo "Create daily report"; fi
#Day
check_dir day 1
make_report day

if [ "$VERBOSE" = "yes" ]; then echo "Create weekly report"; fi
#Week
check_dir week 1
check_dir week/var
make_report week

if [ "$VERBOSE" = "yes" ]; then echo "Create monthly report"; fi
#Month
check_dir month 1
check_dir month/var
make_report month

del_bunzip2ed_log

if [ "$VERBOSE" = "yes" ]; then echo "Create index"; fi
make_index

rm -rf $TMPDIR/*
#rm -f $logbasemontv
#rm -f $logbaseweev

if [ "$VERBOSE" = "yes" ]; then echo "Create archive"; fi
#Archive free-sa
FREE_SA_ARCH=$HTMLOUT/archive/free-sa.tar.bz2
check_dir archive
rm -f $FREE_SA_ARCH
tar -cjf $FREE_SA_ARCH --exclude=archive* -C $HTMLOUT .

#Stop script
    else
        if [ "$VERBOSE" = "yes" ]; then echo "$ACCESSLOG not exist! Skip it!"; fi
    fi
done

exit

Пример /etc/sysconfig/free-sa:

## Path:        Applications/Free-SA
## Description: Configuration of free-sa
## Type:        yesno
## Default:     no
#
# Should free-sa-report be enabled to start by cron ("yes" or "no").
# Read /usr/share/doc/packages/free-sa/report.README before add
# free-sa-report to crontab.
#
USE_FREE_SA_REPORT="yes"

## Type:        string
## Default:     "/var/log/squid/access.log"
#
# Path to squid access.log
#
ACCESSLOGS="/var/log/squid/access.log /var/log/squid/access-icap.log"

## Type:        string
## Default:     "/srv/www/htdocs/free-sa"
#
# Where free-sa should save reports
#
HTMLOUTDIR="/srv/www/htdocs/free-sa"

## Type:        string
## Default:     "/usr/share/free-sa/free-sa-report.conf"
#
# Path to default free-sa-report configuration file
#
FREE_SA_CONF="/etc/free-sa/free-sa.conf"

## Type:        string
## Default:     "/var/cache/free-sa"
#
# Where free-sa-report should create temp configuration
# files for day, week and month reports
#
TMPDIR="/var/cache/free-sa"

## Type:        yesno
## Default:     no
#
# Should free-sa-report be verbose.
#
VERBOSE="yes"

Пример настройки logrotate:

# /etc/logrotate.d/free-sa
# $Id$

/srv/www/htdocs/free-sa/*/archive/free-sa.tar.bz2 {
        daily
        rotate 3
        missingok
        nocreate
        sharedscripts
        postrotate
        endscript
}

Что нового в этой версии:

Версия 2.1 (Срд Ноя 18 2009):
- Добавлена возможность обработки нескольких логов, указанных в ACCESSLOGS.
Версия 2.0:
- Релиз версии 2.0.

Page modification: Птн Сен 10 16:48:10 MSD 2010
Используется Quanta+ 3.5.10 Under the GPL v2 license.