added syslog scripts
This commit is contained in:
parent
1798dd8761
commit
7d52a9eed1
3 changed files with 395 additions and 0 deletions
BIN
syslog/.AppleDouble/.Parent
Normal file
BIN
syslog/.AppleDouble/.Parent
Normal file
Binary file not shown.
147
syslog/relogger.pl
Executable file
147
syslog/relogger.pl
Executable file
|
|
@ -0,0 +1,147 @@
|
|||
#!/usr/bin/perl -w
|
||||
|
||||
# take syslog messages from stdin - push them through syslog again
|
||||
|
||||
# by Ed Ravin <eravin@panix.com>. Made available to the
|
||||
# public courtesy of PANIX (http://www.panix.com).
|
||||
# This script is licensed under the GPL.
|
||||
# Requires Date::Parse and Time::HiRes modules
|
||||
|
||||
|
||||
my $usage=
|
||||
"relogger.pl [--facility fac] [--priority pri] [--replayspeed factor]\n";
|
||||
|
||||
use strict;
|
||||
use Sys::Syslog qw(:DEFAULT setlogsock);
|
||||
use Getopt::Long;
|
||||
|
||||
use Date::Parse; # for str2time
|
||||
use Time::HiRes qw ( sleep );
|
||||
|
||||
my %opt;
|
||||
die $usage unless
|
||||
GetOptions (\%opt, "debug", "facility=s", "priority=s", "replayspeed=s");
|
||||
|
||||
setlogsock('unix')
|
||||
if grep /^ $^O $/xo, ("linux", "openbsd", "freebsd", "netbsd");
|
||||
|
||||
my $facility= $opt{'facility'} || "mail";
|
||||
my $priority= $opt{'priority'} || "info";
|
||||
my $replayspeed= $opt{'replayspeed'} || 0;
|
||||
my $debug= $opt{'debug'} || 0;
|
||||
|
||||
die "$0: Option 'replayspeed' must be a valid floating point number\n"
|
||||
unless $replayspeed =~ /^[0-9]*\.?[0-9]*$/;
|
||||
my $progname= "";
|
||||
|
||||
# Aug 5 20:28:17 grand-central postfix/qmgr[4389]: AC2BB7F9A: removed
|
||||
# my $thistime= str2time($date);
|
||||
# warn "$0: cannot parse date '$date'\n" if !$thistime;
|
||||
|
||||
my $lasttimestamp= 0;
|
||||
my $timestamp;
|
||||
my $timestep= 0;
|
||||
|
||||
while(<>)
|
||||
{
|
||||
if ( ((my ($timestr, $process, $msg))= /^(.*) \S+ ([^ []*)\[\d+\]: (.*)$/ ) == 3)
|
||||
{
|
||||
$timestamp= str2time($timestr) ||
|
||||
warn "$0: cannot parse timestamp '$timestr'\n";
|
||||
if ($progname ne $process)
|
||||
{
|
||||
closelog;
|
||||
openlog "$process", 'ndelay,pid', $facility or die "$0: openlog: $!\n";
|
||||
$progname= $process;
|
||||
}
|
||||
|
||||
$timestep= $timestamp - $lasttimestamp;
|
||||
if ($replayspeed and $timestep > 0 and $lasttimestamp > 0)
|
||||
{
|
||||
warn "sleeping for " . $timestep * $replayspeed . " seconds...\n" if $debug;
|
||||
sleep( $timestep * $replayspeed);
|
||||
}
|
||||
|
||||
syslog $priority, "%s", $msg unless $debug;
|
||||
warn "$process $facility/$priority $msg\n" if $debug;
|
||||
$lasttimestamp= $timestamp;
|
||||
}
|
||||
else
|
||||
{
|
||||
warn "$0: cannot parse input line $.: $_\n";
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
__END__
|
||||
|
||||
=head1 NAME
|
||||
|
||||
relogger.pl - re-inject syslog log files back into syslog
|
||||
|
||||
=head1 SYNOPSIS
|
||||
|
||||
B<relogger.pl> [I<--facility fac>] [I<--priority pri>] [I<--replayspeed factor>] [I<--debug]>]
|
||||
|
||||
=head1 DESCRIPTION
|
||||
|
||||
B<relogger.pl> takes syslog-formatted messages on standard input and re-sends
|
||||
them via the default syslog mechanism. The existing timestamps are stripped
|
||||
off the message before it is re-sent. Delays between messages can be enabled
|
||||
with the I<--replayseed> option (see B<OPTIONS> below to simulate the
|
||||
arrival times of the original messages.
|
||||
|
||||
<relogger.pl> was written to help test configurations for programs
|
||||
like B<logsurfer> or B<swatch> that parse log output and take
|
||||
actions based on what messages appear in the logs.
|
||||
|
||||
=head1 OPTIONS
|
||||
|
||||
=item B<--facility> I<fac> specify the syslog facility to log the messages
|
||||
to. Standard syslog messages do not store the facility the message was
|
||||
logged on, so this cannot be determined from the input. The default is the
|
||||
B<mail> facility.
|
||||
|
||||
=item B<--priority> I<pri> specify the syslog priority to log the messages
|
||||
to. The default is the B<info> priority. As with B<--facility>, this
|
||||
information cannot be discovered from the input.
|
||||
|
||||
=item B<--replayspeed> I<factor> attempt to parse the timestamps
|
||||
of the input messages, and simulate the original arrival times by sleeping
|
||||
between each message. The sleep time is multiplied by I<factor>. To send
|
||||
simulated log events with time spacing at the same time as the original
|
||||
arrival times, use a I<factor> of 1. To send simulated log events at twice
|
||||
the speed of the original logs, use a I<factor> of 0.5 (i.e. sleep only
|
||||
half the original time between log messages).
|
||||
|
||||
=item B<--debug> send all output to standard error, rather than to syslog.
|
||||
Also prints an extra diagnostic message or two.
|
||||
|
||||
=head1 BUGS
|
||||
|
||||
B<relogger.pl> is a beta-quality tool for testing logging configurations.
|
||||
It is not yet recommended for production use.
|
||||
|
||||
It would be nice to be able to specify the input filename on the command
|
||||
line, instead of requiring it to be on standard input.
|
||||
|
||||
It would be nice to be able to control the syslog mechanism on the
|
||||
command line (i.e. specify whether to use a local or remote host)
|
||||
rather than just using the system default.
|
||||
|
||||
The original PID in the message is replaced by the current PID of
|
||||
B<relogger.pl> in the simulated message. Also, the PID of B<relogger.pl>
|
||||
will appear in the simulated message even if the original one did not
|
||||
supply a PID.
|
||||
|
||||
In spite of using Time::HiRes to enable sleeping in fractional seconds,
|
||||
some environments seem to still round off to seconds. This needs a bit
|
||||
more investigation.
|
||||
|
||||
=head1 AUTHOR
|
||||
|
||||
B<relogger.pl> was written by Ed Ravin <eravin@panix.com>, and is made
|
||||
available to the public by courtesy of PANIX (http://www.panix.com).
|
||||
This script is licensed under the GPL. B<relogger.pl> requires the
|
||||
Date::Parse and the Time::HiRes Perl modules.
|
||||
|
||||
248
syslog/replay-log.pl
Executable file
248
syslog/replay-log.pl
Executable file
|
|
@ -0,0 +1,248 @@
|
|||
#!/usr/bin/perl -w
|
||||
#
|
||||
# $Id: replay-log,v 1.9 2005/08/06 21:00:00 jmates Exp $
|
||||
#
|
||||
# The author disclaims all copyrights and releases this script into the
|
||||
# public domain.
|
||||
#
|
||||
# Replays logs, simulating the time delays present in the source
|
||||
# data. Only supports the standard Unix syslogd(8) format at present.
|
||||
#
|
||||
# Run perldoc(1) on this script for additional documentation.
|
||||
|
||||
use POSIX qw(strftime);
|
||||
use Time::Local qw(timelocal);
|
||||
use Time::HiRes qw(sleep);
|
||||
|
||||
use Getopt::Std;
|
||||
my %opts;
|
||||
getopts 'h?r:f:o:s:', \%opts;
|
||||
|
||||
print_help() if exists $opts{h} or exists $opts{'?'};
|
||||
|
||||
my ( $previous_epoch, $random_seed, $speed_factor );
|
||||
|
||||
# increase playback speed
|
||||
if ( exists $opts{f} and defined $opts{f} and $opts{f} > 1 ) {
|
||||
$speed_factor = $opts{f};
|
||||
} else {
|
||||
$speed_factor = 1;
|
||||
}
|
||||
|
||||
# use random seed to determine delay for next log entry
|
||||
if ( exists $opts{r}
|
||||
and defined $opts{r}
|
||||
and $opts{r} > 0 ) {
|
||||
|
||||
$random_seed = $opts{r};
|
||||
$previous_epoch = time;
|
||||
}
|
||||
|
||||
# use custom output file
|
||||
if ( exists $opts{o} ) {
|
||||
open STDOUT, '>', $opts{o}
|
||||
or die "error: could not open: errno=$!, file=$opts{o}\n";
|
||||
}
|
||||
|
||||
# leading delay (good if need time to get something else running)
|
||||
if ( exists $opts{s} and defined $opts{s} and $opts{s} > 0 ) {
|
||||
sleep $opts{s};
|
||||
}
|
||||
|
||||
# force flush on output for better replay of logs
|
||||
$| = 1;
|
||||
|
||||
while (<>) {
|
||||
chomp;
|
||||
|
||||
# TODO will need to read different log formats (standard syslog, squid,
|
||||
# apache, custom...): split into "time, rest of data" and have means to
|
||||
# output "time with format, rest of data"
|
||||
my %data;
|
||||
|
||||
( $data{rawtime}, $data{suffix} ) =
|
||||
$_ =~ m/^ (\w{3} \s\s? \d\d? \s [\d:]{8}) (.*) $/x;
|
||||
|
||||
$data{epoch} =
|
||||
$random_seed
|
||||
? $previous_epoch + rand $random_seed
|
||||
: syslog_to_epoch( $data{rawtime} );
|
||||
|
||||
if ( defined $previous_epoch ) {
|
||||
my $delay = ( $data{epoch} - $previous_epoch ) / $speed_factor;
|
||||
|
||||
# KLUGE syslog messages may have different timezones, which results
|
||||
# in negative time differences. Workaround: detect and sleep random
|
||||
# time value. (TODO use moving average instead?)
|
||||
if ( $delay < 0 ) {
|
||||
$delay = rand ($random_seed || 3);
|
||||
}
|
||||
|
||||
sleep $delay;
|
||||
}
|
||||
|
||||
# TODO for output, either want original string, or rework with new
|
||||
# timestamp based on the current time. Allow templating off %data,
|
||||
# different time formatting via strftime.
|
||||
if ($random_seed) {
|
||||
print epoch_to_str( $data{epoch} ), $data{suffix}, $/;
|
||||
} else {
|
||||
print $_, $/;
|
||||
}
|
||||
|
||||
$previous_epoch = $data{epoch};
|
||||
}
|
||||
|
||||
sub epoch_to_str {
|
||||
# epoch may have .123 microseconds (squid logs), so whack that off
|
||||
strftime "%b %d %H:%M:%S", localtime( int $_[0] || 0 );
|
||||
}
|
||||
|
||||
# convert default syslog timestamp into epoch seconds. Uses current
|
||||
# year, as syslog does not record the year (nor the timezone, but that's
|
||||
# another rant).
|
||||
sub syslog_to_epoch {
|
||||
my $date = shift;
|
||||
my $epoch;
|
||||
|
||||
my %month_map;
|
||||
@month_map{qw(Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec)} = 0 .. 11;
|
||||
|
||||
my ( $month, $day, $hour, $min, $sec ) =
|
||||
$date =~ m/^(\w{3}) \s\s? (\d\d?) \s (\d\d):(\d\d):(\d\d)/x;
|
||||
$epoch = timelocal $sec, $min, $hour, $day, $month_map{$month},
|
||||
1900 + (localtime)[5];
|
||||
|
||||
return $epoch;
|
||||
}
|
||||
|
||||
sub print_help {
|
||||
print <<"HELP";
|
||||
Usage: $0 [-o output-file] [-f factor] [-r random-seed] < input-file
|
||||
|
||||
Replays logs, simulating the time delays present in the source data.
|
||||
|
||||
Options:
|
||||
-h/-? Display this message.
|
||||
|
||||
-o ff Write output to this file instead of standard out.
|
||||
|
||||
-f nn Numeric factor to speed replay up by, such as 3600.
|
||||
-r nn Specify random seed to rand() for random replay speed.
|
||||
-s nn Sleep this long before starting.
|
||||
|
||||
Run perldoc(1) on this script for additional documentation.
|
||||
|
||||
HELP
|
||||
exit 100;
|
||||
}
|
||||
|
||||
__DATA__
|
||||
|
||||
=head1 NAME
|
||||
|
||||
replay-log - replays logs with simulated time delays
|
||||
|
||||
=head1 SYNOPSIS
|
||||
|
||||
Replay logs from C</var/log/messages> (may take some time!):
|
||||
|
||||
$ replay-log </var/log/messages
|
||||
|
||||
A three second random seed causes faster playback:
|
||||
|
||||
$ </var/log/messages replay-log -r 3
|
||||
|
||||
As does using a higher replay factor:
|
||||
|
||||
$ replay-log -f 3600 </var/log/messages
|
||||
|
||||
Send C<sshd> logs to a named pipe, read them with C<sec.pl>:
|
||||
|
||||
$ mkfifo logfile
|
||||
$ grep sshd </var/log/messages | replay-log -f 1000 -o logfile &
|
||||
$ sec.pl --conf=sshd.conf --input=logfile
|
||||
|
||||
=head1 DESCRIPTION
|
||||
|
||||
=head2 Overview
|
||||
|
||||
Replays log data, simulating the time delays of the log data. Faster
|
||||
playback possible via the random seed and factor options. Good for
|
||||
replaying logs to tools such as C<swatch> or SEC:
|
||||
|
||||
http://swatch.sourceforge.net/
|
||||
|
||||
http://kodu.neti.ee/~risto/sec/
|
||||
|
||||
Only supports Unix syslogd(8) data at present.
|
||||
|
||||
=head2 Normal Usage
|
||||
|
||||
$ replay-log [-o output-file] [-f factor] [-r random-seed] < input-file
|
||||
|
||||
See L<"OPTIONS"> for details on the command line switches supported.
|
||||
|
||||
Log data is read from standard input, and sent to standard output,
|
||||
unless a custom output file is specified.
|
||||
|
||||
=head1 OPTIONS
|
||||
|
||||
This script currently supports the following command line switches:
|
||||
|
||||
=over 4
|
||||
|
||||
=item B<-h>, B<-?>
|
||||
|
||||
Prints a brief usage note about the script.
|
||||
|
||||
=item B<-o> I<filename>
|
||||
|
||||
Send output to I<filename> instead of standard out.
|
||||
|
||||
=item B<-r> I<random-seed>
|
||||
|
||||
Use I<random-seed> as the random seed to C<rand()>; logs will playback
|
||||
using a random delay based on the seed.
|
||||
|
||||
=item B<-f> I<factor>
|
||||
|
||||
Speed playback of logs. I<factor> must be a number greater than one;
|
||||
higher values lead to faster replays.
|
||||
|
||||
=item B<-s> I<leading-sleep>
|
||||
|
||||
Sleep I<leading-sleep> seconds before replaying any logs.
|
||||
|
||||
=back
|
||||
|
||||
=head1 BUGS
|
||||
|
||||
=head2 Reporting Bugs
|
||||
|
||||
If the bug is in the latest version, send a report to the author.
|
||||
Patches that fix problems or add new features are welcome.
|
||||
|
||||
=head2 Known Issues
|
||||
|
||||
Only supports Unix syslogd(8) data at present. Output could be formatted
|
||||
in different ways and offer better control of timestamp display.
|
||||
|
||||
=head1 SEE ALSO
|
||||
|
||||
perl(1)
|
||||
|
||||
=head1 AUTHOR
|
||||
|
||||
Jeremy Mates, http://sial.org/contact/
|
||||
|
||||
=head1 COPYRIGHT
|
||||
|
||||
The author disclaims all copyrights and releases this script into the
|
||||
public domain.
|
||||
|
||||
=head1 VERSION
|
||||
|
||||
$Id: replay-log,v 1.9 2005/08/06 21:00:00 jmates Exp $
|
||||
|
||||
=cut
|
||||
Loading…
Add table
Add a link
Reference in a new issue