Logwatch Filtering for Apache

[Ben Trussell]

PPS:  Last, but not least =)

If you want to not see it in logwatch reports, just tell logwatch to ignore it..

http://www.google.com/search?q=logwatch+ignore.conf

Ben

On Fri, Feb 11, 2011 at 5:11 PM, Ben Trussell <azlobo73 at gmail.com> wrote:
> Note that email prematurely wrapped the examples.  rules need to be
> all on one line in rsyslog.conf
>
> On Fri, Feb 11, 2011 at 5:08 PM, Ben Trussell <azlobo73 at gmail.com> wrote:
>> This is by far not the only solution, but its one that can be used to
>> filter, combine, etc logs based on your needs.
>>
>> Switch to  rsyslog if not already using it.  The default configuration
>> for Distros like RHEL, CentOS, Debian etc shouldn't be any different
>> than sysklog etc when installed via a package maintained by the
>> Distro, so its not that difficult to switch to (probably want to
>> verify that for your env in a testing scenario first, of course, but
>> it was painless and simple to do so in my case).
>>
>> Add your log entries in the apache conf file(s) like so:
>>
>> CustomLog can be specified once for a file, then again for this, but
>> you might want to send all to rsyslog then use the rsyslog config to
>> parse out or combine as needed based on its abilities (explained
>> further below)
>>
>>   CustomLog "|/usr/bin/logger -t httpd_vhost_tag -p local6.info" combined
>>
>> Error logs can not be specified twice, so they need to be handled
>> mostly in the rsyslog config
>>
>>   ErrorLog "|/usr/bin/logger -t httpd_vhost_tag_error -p local6.notice"
>>
>> in /etc/rsyslog.conf, you can put things like:
>>
>>   local6.notice
>> /var/log/httpd/http_combined_error_log
>>
>>
>> or, a little more handy in this case
>>
>>   :syslogtag, contains, "_error"
>> /var/log/httpd/combined_error_log
>>   :syslogtag, startswith, "httpd_vhost_tag"
>> /var/log/httpd/vhost_combined_log
>>   :syslogtag, isequal, "httpd_vhost_tag"
>> /var/log/httpd/vhost_access_log
>>   :syslogtag, isequal, "httpd_vhost_tag_error"
>> /var/log/httpd/vhost_error_log
>>
>> So far this is useful with regard to *combining*, or getting back to
>> what you'd expect without rsyslog configuration-based logging, but for
>> how to filter based on rsyslog, try this in the rsyslog.conf file
>> (each are on per line)..
>>
>> So now for more useful stuff in this case..
>>
>>  if ($syslogfacility-text == 'local6' and $syslogseverity-text ==
>> 'notice') and ($syslogtag contains 'httpd_audit_') then
>> /var/log/httpd/httpd_audit_log
>>
>> or
>>
>>  if ($syslogfacility-text == 'local6' and $syslogseverity-text ==
>> 'info') and ($msg contains 'w00tw00t') then
>> /var/log/httpd/httpd_alert_log
>>
>> or
>>
>>  if ($syslogfacility-text == 'local6' and $syslogseverity-text ==
>> 'info') and not ($msg contains 'do_not_care_about_me.jpg') then
>> /var/log/httpd/httpd_access_log
>>
>> or (even more useful in your case)
>>
>>  if ($syslogtag == 'httpd_vhost_tag' and $syslogseverity-text ==
>> 'info') and not ($msg regex '.jpg .*404 .*') then
>> /var/log/httpd/httpd_access_log
>>
>> Now you have a real syntax for filtering etc from within your logging service.
>>
>>
>> Add the mysql (or another database backend) functionality and then..
>>
>>   *.*
>> :ommysql:127.0.0.1,Syslog,username,password
>>
>>
>> lets you query your logs in a SQL environment.  loganalyzer is a nice
>> option depending on your scale.
>>
>>
>> And of course this still works fine:
>>
>> *.*                    @loghost.example.net
>>
>> Or practically any combination of the above to get the job done..
>>
>>
>> More information: http://www.rsyslog.com/doc/rsyslog_conf_filter.html
>>
>>
>> - Ben
>> PS: Yeah I'm a fan of rsyslog - how'd you know ? =)
>>
>> On Wed, Feb 9, 2011 at 12:52 PM, Jason Holtzapple <ml at bitflip.net> wrote:
>>> On 02/09/2011 12:20 PM, Tim Noeding wrote:
>>>
>>>> I have servers that I monitor and was hoping to cut the apache sections
>>>> of the logwatch down a bit. These servers have had website changes which
>>>> leave links that people have made to images come up as failed access
>>>> attempts in logwatch. Most of these are a known issue. I do not want to
>>>> add these to the regex ignore file for logwatch, as they may become a
>>>> real issue in the future. The one consistent bit of information that
>>>> defines the true problems from the false positives is the number of
>>>> times the problem happens. Generally, if the failure happens more than
>>>> 100 times, I want to know about it. The rest I don't want in the e-mail.
>>>
>>> Disclaimer: I don't use logwatch so I don't know if you can accomplish
>>> what you want there or not. If I need to flag an event that involves a
>>> certain number of errors in a certain amount of time I will usually use
>>> the simple event correlator - http://simple-evcorr.sourceforge.net
>>>
>>> There's a bit of a learning curve but it's a useful tool.
>>>
>>>
>>> ---------------------------------------------------
>>> PLUG-discuss mailing list - PLUG-discuss at lists.plug.phoenix.az.us
>>> To subscribe, unsubscribe, or to change your mail settings:
>>> http://lists.PLUG.phoenix.az.us/mailman/listinfo/plug-discuss
>>>
>>
>