added syslog scripts

This commit is contained in:
Peter Baumann 2013-06-04 15:30:42 +02:00
parent 1798dd8761
commit 7d52a9eed1
3 changed files with 395 additions and 0 deletions

BIN
syslog/.AppleDouble/.Parent Normal file

Binary file not shown.

147
syslog/relogger.pl Executable file
View file

@ -0,0 +1,147 @@
#!/usr/bin/perl -w
# take syslog messages from stdin - push them through syslog again
# by Ed Ravin <eravin@panix.com>. Made available to the
# public courtesy of PANIX (http://www.panix.com).
# This script is licensed under the GPL.
# Requires Date::Parse and Time::HiRes modules
my $usage=
"relogger.pl [--facility fac] [--priority pri] [--replayspeed factor]\n";
use strict;
use Sys::Syslog qw(:DEFAULT setlogsock);
use Getopt::Long;
use Date::Parse; # for str2time
use Time::HiRes qw ( sleep );
my %opt;
die $usage unless
GetOptions (\%opt, "debug", "facility=s", "priority=s", "replayspeed=s");
setlogsock('unix')
if grep /^ $^O $/xo, ("linux", "openbsd", "freebsd", "netbsd");
my $facility= $opt{'facility'} || "mail";
my $priority= $opt{'priority'} || "info";
my $replayspeed= $opt{'replayspeed'} || 0;
my $debug= $opt{'debug'} || 0;
die "$0: Option 'replayspeed' must be a valid floating point number\n"
unless $replayspeed =~ /^[0-9]*\.?[0-9]*$/;
my $progname= "";
# Aug 5 20:28:17 grand-central postfix/qmgr[4389]: AC2BB7F9A: removed
# my $thistime= str2time($date);
# warn "$0: cannot parse date '$date'\n" if !$thistime;
my $lasttimestamp= 0;
my $timestamp;
my $timestep= 0;
while(<>)
{
if ( ((my ($timestr, $process, $msg))= /^(.*) \S+ ([^ []*)\[\d+\]: (.*)$/ ) == 3)
{
$timestamp= str2time($timestr) ||
warn "$0: cannot parse timestamp '$timestr'\n";
if ($progname ne $process)
{
closelog;
openlog "$process", 'ndelay,pid', $facility or die "$0: openlog: $!\n";
$progname= $process;
}
$timestep= $timestamp - $lasttimestamp;
if ($replayspeed and $timestep > 0 and $lasttimestamp > 0)
{
warn "sleeping for " . $timestep * $replayspeed . " seconds...\n" if $debug;
sleep( $timestep * $replayspeed);
}
syslog $priority, "%s", $msg unless $debug;
warn "$process $facility/$priority $msg\n" if $debug;
$lasttimestamp= $timestamp;
}
else
{
warn "$0: cannot parse input line $.: $_\n";
}
}
__END__
=head1 NAME
relogger.pl - re-inject syslog log files back into syslog
=head1 SYNOPSIS
B<relogger.pl> [I<--facility fac>] [I<--priority pri>] [I<--replayspeed factor>] [I<--debug]>]
=head1 DESCRIPTION
B<relogger.pl> takes syslog-formatted messages on standard input and re-sends
them via the default syslog mechanism. The existing timestamps are stripped
off the message before it is re-sent. Delays between messages can be enabled
with the I<--replayseed> option (see B<OPTIONS> below to simulate the
arrival times of the original messages.
<relogger.pl> was written to help test configurations for programs
like B<logsurfer> or B<swatch> that parse log output and take
actions based on what messages appear in the logs.
=head1 OPTIONS
=item B<--facility> I<fac> specify the syslog facility to log the messages
to. Standard syslog messages do not store the facility the message was
logged on, so this cannot be determined from the input. The default is the
B<mail> facility.
=item B<--priority> I<pri> specify the syslog priority to log the messages
to. The default is the B<info> priority. As with B<--facility>, this
information cannot be discovered from the input.
=item B<--replayspeed> I<factor> attempt to parse the timestamps
of the input messages, and simulate the original arrival times by sleeping
between each message. The sleep time is multiplied by I<factor>. To send
simulated log events with time spacing at the same time as the original
arrival times, use a I<factor> of 1. To send simulated log events at twice
the speed of the original logs, use a I<factor> of 0.5 (i.e. sleep only
half the original time between log messages).
=item B<--debug> send all output to standard error, rather than to syslog.
Also prints an extra diagnostic message or two.
=head1 BUGS
B<relogger.pl> is a beta-quality tool for testing logging configurations.
It is not yet recommended for production use.
It would be nice to be able to specify the input filename on the command
line, instead of requiring it to be on standard input.
It would be nice to be able to control the syslog mechanism on the
command line (i.e. specify whether to use a local or remote host)
rather than just using the system default.
The original PID in the message is replaced by the current PID of
B<relogger.pl> in the simulated message. Also, the PID of B<relogger.pl>
will appear in the simulated message even if the original one did not
supply a PID.
In spite of using Time::HiRes to enable sleeping in fractional seconds,
some environments seem to still round off to seconds. This needs a bit
more investigation.
=head1 AUTHOR
B<relogger.pl> was written by Ed Ravin <eravin@panix.com>, and is made
available to the public by courtesy of PANIX (http://www.panix.com).
This script is licensed under the GPL. B<relogger.pl> requires the
Date::Parse and the Time::HiRes Perl modules.

248
syslog/replay-log.pl Executable file
View file

@ -0,0 +1,248 @@
#!/usr/bin/perl -w
#
# $Id: replay-log,v 1.9 2005/08/06 21:00:00 jmates Exp $
#
# The author disclaims all copyrights and releases this script into the
# public domain.
#
# Replays logs, simulating the time delays present in the source
# data. Only supports the standard Unix syslogd(8) format at present.
#
# Run perldoc(1) on this script for additional documentation.
use POSIX qw(strftime);
use Time::Local qw(timelocal);
use Time::HiRes qw(sleep);
use Getopt::Std;
my %opts;
getopts 'h?r:f:o:s:', \%opts;
print_help() if exists $opts{h} or exists $opts{'?'};
my ( $previous_epoch, $random_seed, $speed_factor );
# increase playback speed
if ( exists $opts{f} and defined $opts{f} and $opts{f} > 1 ) {
$speed_factor = $opts{f};
} else {
$speed_factor = 1;
}
# use random seed to determine delay for next log entry
if ( exists $opts{r}
and defined $opts{r}
and $opts{r} > 0 ) {
$random_seed = $opts{r};
$previous_epoch = time;
}
# use custom output file
if ( exists $opts{o} ) {
open STDOUT, '>', $opts{o}
or die "error: could not open: errno=$!, file=$opts{o}\n";
}
# leading delay (good if need time to get something else running)
if ( exists $opts{s} and defined $opts{s} and $opts{s} > 0 ) {
sleep $opts{s};
}
# force flush on output for better replay of logs
$| = 1;
while (<>) {
chomp;
# TODO will need to read different log formats (standard syslog, squid,
# apache, custom...): split into "time, rest of data" and have means to
# output "time with format, rest of data"
my %data;
( $data{rawtime}, $data{suffix} ) =
$_ =~ m/^ (\w{3} \s\s? \d\d? \s [\d:]{8}) (.*) $/x;
$data{epoch} =
$random_seed
? $previous_epoch + rand $random_seed
: syslog_to_epoch( $data{rawtime} );
if ( defined $previous_epoch ) {
my $delay = ( $data{epoch} - $previous_epoch ) / $speed_factor;
# KLUGE syslog messages may have different timezones, which results
# in negative time differences. Workaround: detect and sleep random
# time value. (TODO use moving average instead?)
if ( $delay < 0 ) {
$delay = rand ($random_seed || 3);
}
sleep $delay;
}
# TODO for output, either want original string, or rework with new
# timestamp based on the current time. Allow templating off %data,
# different time formatting via strftime.
if ($random_seed) {
print epoch_to_str( $data{epoch} ), $data{suffix}, $/;
} else {
print $_, $/;
}
$previous_epoch = $data{epoch};
}
sub epoch_to_str {
# epoch may have .123 microseconds (squid logs), so whack that off
strftime "%b %d %H:%M:%S", localtime( int $_[0] || 0 );
}
# convert default syslog timestamp into epoch seconds. Uses current
# year, as syslog does not record the year (nor the timezone, but that's
# another rant).
sub syslog_to_epoch {
my $date = shift;
my $epoch;
my %month_map;
@month_map{qw(Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec)} = 0 .. 11;
my ( $month, $day, $hour, $min, $sec ) =
$date =~ m/^(\w{3}) \s\s? (\d\d?) \s (\d\d):(\d\d):(\d\d)/x;
$epoch = timelocal $sec, $min, $hour, $day, $month_map{$month},
1900 + (localtime)[5];
return $epoch;
}
sub print_help {
print <<"HELP";
Usage: $0 [-o output-file] [-f factor] [-r random-seed] < input-file
Replays logs, simulating the time delays present in the source data.
Options:
-h/-? Display this message.
-o ff Write output to this file instead of standard out.
-f nn Numeric factor to speed replay up by, such as 3600.
-r nn Specify random seed to rand() for random replay speed.
-s nn Sleep this long before starting.
Run perldoc(1) on this script for additional documentation.
HELP
exit 100;
}
__DATA__
=head1 NAME
replay-log - replays logs with simulated time delays
=head1 SYNOPSIS
Replay logs from C</var/log/messages> (may take some time!):
$ replay-log </var/log/messages
A three second random seed causes faster playback:
$ </var/log/messages replay-log -r 3
As does using a higher replay factor:
$ replay-log -f 3600 </var/log/messages
Send C<sshd> logs to a named pipe, read them with C<sec.pl>:
$ mkfifo logfile
$ grep sshd </var/log/messages | replay-log -f 1000 -o logfile &
$ sec.pl --conf=sshd.conf --input=logfile
=head1 DESCRIPTION
=head2 Overview
Replays log data, simulating the time delays of the log data. Faster
playback possible via the random seed and factor options. Good for
replaying logs to tools such as C<swatch> or SEC:
http://swatch.sourceforge.net/
http://kodu.neti.ee/~risto/sec/
Only supports Unix syslogd(8) data at present.
=head2 Normal Usage
$ replay-log [-o output-file] [-f factor] [-r random-seed] < input-file
See L<"OPTIONS"> for details on the command line switches supported.
Log data is read from standard input, and sent to standard output,
unless a custom output file is specified.
=head1 OPTIONS
This script currently supports the following command line switches:
=over 4
=item B<-h>, B<-?>
Prints a brief usage note about the script.
=item B<-o> I<filename>
Send output to I<filename> instead of standard out.
=item B<-r> I<random-seed>
Use I<random-seed> as the random seed to C<rand()>; logs will playback
using a random delay based on the seed.
=item B<-f> I<factor>
Speed playback of logs. I<factor> must be a number greater than one;
higher values lead to faster replays.
=item B<-s> I<leading-sleep>
Sleep I<leading-sleep> seconds before replaying any logs.
=back
=head1 BUGS
=head2 Reporting Bugs
If the bug is in the latest version, send a report to the author.
Patches that fix problems or add new features are welcome.
=head2 Known Issues
Only supports Unix syslogd(8) data at present. Output could be formatted
in different ways and offer better control of timestamp display.
=head1 SEE ALSO
perl(1)
=head1 AUTHOR
Jeremy Mates, http://sial.org/contact/
=head1 COPYRIGHT
The author disclaims all copyrights and releases this script into the
public domain.
=head1 VERSION
$Id: replay-log,v 1.9 2005/08/06 21:00:00 jmates Exp $
=cut