Re: Logwatch Filtering for Apache

Note that email prematurely wrapped the examples. rules need to be
all on one line in rsyslog.conf

On Fri, Feb 11, 2011 at 5:08 PM, Ben Trussell <azlobo73@gmail.com> wrote:
> This is by far not the only solution, but its one that can be used to
> filter, combine, etc logs based on your needs.
>
> Switch to  rsyslog if not already using it.  The default configuration
> for Distros like RHEL, CentOS, Debian etc shouldn't be any different
> than sysklog etc when installed via a package maintained by the
> Distro, so its not that difficult to switch to (probably want to
> verify that for your env in a testing scenario first, of course, but
> it was painless and simple to do so in my case).
>
> Add your log entries in the apache conf file(s) like so:
>
> CustomLog can be specified once for a file, then again for this, but
> you might want to send all to rsyslog then use the rsyslog config to
> parse out or combine as needed based on its abilities (explained
> further below)
>
>   CustomLog "|/usr/bin/logger -t httpd_vhost_tag -p local6.info" combined
>
> Error logs can not be specified twice, so they need to be handled
> mostly in the rsyslog config
>
>   ErrorLog "|/usr/bin/logger -t httpd_vhost_tag_error -p local6.notice"
>
> in /etc/rsyslog.conf, you can put things like:
>
>   local6.notice
> /var/log/httpd/http_combined_error_log
>
>
> or, a little more handy in this case
>
>   :syslogtag, contains, "_error"
> /var/log/httpd/combined_error_log
>   :syslogtag, startswith, "httpd_vhost_tag"
> /var/log/httpd/vhost_combined_log
>   :syslogtag, isequal, "httpd_vhost_tag"
> /var/log/httpd/vhost_access_log
>   :syslogtag, isequal, "httpd_vhost_tag_error"
> /var/log/httpd/vhost_error_log
>
> So far this is useful with regard to *combining*, or getting back to
> what you'd expect without rsyslog configuration-based logging, but for
> how to filter based on rsyslog, try this in the rsyslog.conf file
> (each are on per line)..
>
> So now for more useful stuff in this case..
>
>  if ($syslogfacility-text == 'local6' and $syslogseverity-text ==
> 'notice') and ($syslogtag contains 'httpd_audit_') then
> /var/log/httpd/httpd_audit_log
>
> or
>
>  if ($syslogfacility-text == 'local6' and $syslogseverity-text ==
> 'info') and ($msg contains 'w00tw00t') then
> /var/log/httpd/httpd_alert_log
>
> or
>
>  if ($syslogfacility-text == 'local6' and $syslogseverity-text ==
> 'info') and not ($msg contains 'do_not_care_about_me.jpg') then
> /var/log/httpd/httpd_access_log
>
> or (even more useful in your case)
>
>  if ($syslogtag == 'httpd_vhost_tag' and $syslogseverity-text ==
> 'info') and not ($msg regex '.jpg .*404 .*') then
> /var/log/httpd/httpd_access_log
>
> Now you have a real syntax for filtering etc from within your logging service.
>
>
> Add the mysql (or another database backend) functionality and then..
>
>   *.*
> :ommysql:127.0.0.1,Syslog,username,password
>
>
> lets you query your logs in a SQL environment.  loganalyzer is a nice
> option depending on your scale.
>
>
> And of course this still works fine:
>
> *.*                    @loghost.example.net
>
> Or practically any combination of the above to get the job done..
>
>
> More information: http://www.rsyslog.com/doc/rsyslog_conf_filter.html
>
>
> - Ben
> PS: Yeah I'm a fan of rsyslog - how'd you know ? =)
>
> On Wed, Feb 9, 2011 at 12:52 PM, Jason Holtzapple <ml@bitflip.net> wrote:
>> On 02/09/2011 12:20 PM, Tim Noeding wrote:
>>
>>> I have servers that I monitor and was hoping to cut the apache sections
>>> of the logwatch down a bit. These servers have had website changes which
>>> leave links that people have made to images come up as failed access
>>> attempts in logwatch. Most of these are a known issue. I do not want to
>>> add these to the regex ignore file for logwatch, as they may become a
>>> real issue in the future. The one consistent bit of information that
>>> defines the true problems from the false positives is the number of
>>> times the problem happens. Generally, if the failure happens more than
>>> 100 times, I want to know about it. The rest I don't want in the e-mail.
>>
>> Disclaimer: I don't use logwatch so I don't know if you can accomplish
>> what you want there or not. If I need to flag an event that involves a
>> certain number of errors in a certain amount of time I will usually use
>> the simple event correlator - http://simple-evcorr.sourceforge.net
>>
>> There's a bit of a learning curve but it's a useful tool.
>>
>>
>> ---------------------------------------------------
>> PLUG-discuss mailing list - PLUG-discuss@lists.plug.phoenix.az.us
>> To subscribe, unsubscribe, or to change your mail settings:
>> http://lists.PLUG.phoenix.az.us/mailman/listinfo/plug-discuss
>>
>
---------------------------------------------------
PLUG-discuss mailing list - PLUG-discuss@lists.plug.phoenix.az.us
To subscribe, unsubscribe, or to change your mail settings:
http://lists.PLUG.phoenix.az.us/mailman/listinfo/plug-discuss