# HG changeset patch # User Matti Hamalainen # Date 1250266334 -10800 # Node ID b05d0f0ff10633d2afc77655f8605c466c1cd4b7 # Parent 3d18fdeabf90e619b7777a07fb08d540a17c3efe Cleanups in progress, does not work. diff -r 3d18fdeabf90 -r b05d0f0ff106 README --- a/README Fri Aug 14 03:19:54 2009 +0300 +++ b/README Fri Aug 14 19:12:14 2009 +0300 @@ -1,4 +1,4 @@ -Malicious Attack Livid Termination Filter daemon (maltfilter) v0.8 +Malicious Attack Livid Termination Filter daemon (maltfilter) v0.9 ================================================================== Programmed by Matti 'ccr' Hämäläinen (C) Copyright 2009 Tecnic Software productions (TNSP) @@ -67,6 +67,7 @@ settings to enable it, for example in Debian/Ubuntu you can use rcconf(8) or chkconfig(8). + Reports ======= Automatic report generation can be enabled from configuration. diff -r 3d18fdeabf90 -r b05d0f0ff106 maltfilter --- a/maltfilter Fri Aug 14 03:19:54 2009 +0300 +++ b/maltfilter Fri Aug 14 19:12:14 2009 +0300 @@ -10,7 +10,7 @@ use Date::Parse; use Net::IP; -my $progversion = "0.8"; +my $progversion = "0.9"; my $progbanner = "Malicious Attack Livid Termination Filter daemon (maltfilter) v$progversion\n". "Programmed by Matti 'ccr' Hamalainen \n". @@ -54,18 +54,27 @@ ############################################################################# ### Script code ############################################################################# -my $report = 0; -my @scanfiles = (); -my @noblock_ips = (); -my %filehandles = (); -my %hitcount = (); -my %iplist = (); -my %reason = (); -my %reason_n = (); -my %ignored = (); -my %ignored_d = (); -my $pid_file = ""; -my $LOGFILE; +my $reportmode = 0; # Full report mode +my @scanfiles = (); # Files to scan +my @noblock_ips = (); # IPs not to block +my %filehandles = (); # Global hash holding opened scanned log filehandles +my $pid_file = ""; # Name of Maltfilter daemon pid file +my $LOGFILE; # Maltfilter logfile handle + +my %blocklist = (); # IPs currently blocked in Netfilter $blocklist{$ip} = date_blocked + +# Gathered information about hosts +# $statlist{$ip}-> +# "date" = latest change +# "hits" = number of hits +# "reason" = latest reason why added +# "reasons" = array of reasons (only set when $reportmode == 1) +my %statlist = (); + +# Gathered information about ignored hits (e.g. hits for tests that are not enabled) +# Same fields as in %statlist +my %ignorelist = (); + ### Check given logfile line for matches sub check_log_line($) @@ -127,110 +136,8 @@ ############################################################################# -### Script code +### Status output functionality ############################################################################# -sub mlog -{ - my $level = shift; - my $msg = shift; - if (defined($LOGFILE)) { - print $LOGFILE "[".scalar localtime()."] ".$msg if ($settings{"VERBOSITY"} > $level); - } elsif ($settings{"DRY_RUN"}) { - print $msg if ($settings{"VERBOSITY"} > $level); - } -} - -### Host and IP matching functions -sub check_hosts_array($$) -{ - my $chk_host = $_[1]; - my $chk_ip = new Net::IP($chk_host); - foreach my $host (@{$_[0]}) { - if ($chk_host eq $host) { - return 1; - } - my $ip = new Net::IP($host); - if (defined($chk_ip) && defined($ip)) { - if ($chk_ip->binip() eq $ip->binip()) { - return 1; - } - } - } - return 0; -} - -sub check_hosts($$) -{ - my @tmp = split(/\s*\|\s*/, $_[0]); - return check_hosts_array(\@tmp, $_[1]); -} - - -### Execute iptables -sub exec_iptables(@) -{ - my @args = ($settings{"IPTABLES"}, @_); - if ($settings{"DRY_RUN"}) { - mlog(3, ":: ".join(" ", @args)."\n"); - } else { - system(@args) == 0 or print join(" ", @args)." failed: $?\n"; - } -} - -### Get current Netfilter INPUT table entries we manage -sub update_iplist($) -{ - open(STATUS, $settings{"IPTABLES"}." -v -n -L INPUT |") or - die("Could not execute ".$settings{"IPTABLES"}."\n"); - while () { - chomp; - if (/^\s*(\d+)\s+\d+\s+$settings{"ACTION"}\s+all\s+--\s+\*\s+\*\s+(\d+\.\d+\.\d+\.\d+)\s+0\.0\.0\.0\/0\s*$/) { - if (!defined($iplist{$2})) { - $hitcount{$2} = $settings{"TRESHOLD"}; - $iplist{$2} = $_[0]; - if ($_[0] >= 0) { mlog(2, "* $2 appeared in iptables, adding.\n"); } - } - } - } - close(STATUS); -} - -### Weed out old entries -sub check_time($) -{ - return ($_[0] >= time() - ($settings{"WEEDPERIOD"} * 60 * 60)); -} - -sub weed_do($) -{ - if (defined($iplist{$_[0]})) { - mlog(2, "* Weeding $_[0] ($iplist{$_[0]})\n"); - exec_iptables("-D", "INPUT", "-s", $_[0], "-d", "0.0.0.0/0", "-j", $settings{"ACTION"}); - undef($reason{$_[0]}); - undef($reason_n{$_[0]}); - undef($ignored{$_[0]}); - undef($ignored_d{$_[0]}); - undef($iplist{$_[0]}); - } -} - -sub weed_entries() -{ - # Don't weed in report mode. - return if ($report); - - foreach my $mip (keys %iplist) { - if (defined($iplist{$mip})) { - if ($iplist{$mip} >= 0) { - if (!check_time($iplist{$mip})) { weed_do($mip); } - } else { - weed_do($mip); - } - } - } -} - -### Output status file sub cmp_ips($$) { my @ipa = split(/\./, $_[0]); @@ -353,9 +260,9 @@ printP($m, $f, "Generated ".bb($m).$mtime.eb($m).". Data computed from ". - ($report ? "complete logfile scan" : "a period of last $period").".\n"); + ($reportmode ? "complete logfile scan" : "a period of last $period").".\n"); - printH($m, $f, 2, $report ? "Detailed report" : "Blocked entries"); + printH($m, $f, 2, $reportmode ? "Detailed report" : "Blocked entries"); printElem($m, $f, "\n". ""."\n"); my $nexcluded = 0; my $ntotal = 0; @@ -370,7 +277,7 @@ my @s = (); foreach my $cond (sort keys %{$reason{$mip}}) { my $str; - if ($report) { + if ($reportmode) { my @tmp = reverse(@{$reason{$mip}{$cond}}); $#tmp = 5 if ($#tmp > 5); $str = join(" | ", @tmp); @@ -447,6 +354,104 @@ close(STATUS); } + +############################################################################# +### Entry management / handling functions +############################################################################# +### Host and IP matching functions +sub check_hosts_array($$) +{ + my $chk_host = $_[1]; + my $chk_ip = new Net::IP($chk_host); + foreach my $host (@{$_[0]}) { + if ($chk_host eq $host) { + return 1; + } + my $ip = new Net::IP($host); + if (defined($chk_ip) && defined($ip)) { + if ($chk_ip->binip() eq $ip->binip()) { + return 1; + } + } + } + return 0; +} + +sub check_hosts($$) +{ + my @tmp = split(/\s*\|\s*/, $_[0]); + return check_hosts_array(\@tmp, $_[1]); +} + +### Execute iptables +sub exec_iptables(@) +{ + my @args = ($settings{"IPTABLES"}, @_); + if ($settings{"DRY_RUN"}) { + mlog(3, ":: ".join(" ", @args)."\n"); + } else { + system(@args) == 0 or print join(" ", @args)." failed: $?\n"; + } +} + +### Get current Netfilter INPUT table entries we manage +sub update_iplist($) +{ + my $mdate = $_[0]; + open(STATUS, $settings{"IPTABLES"}." -v -n -L INPUT |") or + die("Could not execute ".$settings{"IPTABLES"}."\n"); + while () { + chomp; + if (/^\s*(\d+)\s+\d+\s+$settings{"ACTION"}\s+all\s+--\s+\*\s+\*\s+(\d+\.\d+\.\d+\.\d+)\s+0\.0\.0\.0\/0\s*$/) { + my $mcount = $1; + my $mip = $2; + if (!defined($blocklist{$mip}) && $mdate >= 0) { + mlog(2, "* $mip appeared in iptables, adding.\n"); + $hitcount{$mip} = $settings{"THRESHOLD"}; + check_add_entry($mip, $mdate, "?", "added from iptables", 1); + } + } + } + close(STATUS); +} + +### Weed out old entries +sub check_time($) +{ + return ($_[0] >= time() - ($settings{"WEEDPERIOD"} * 60 * 60)); +} + +sub weed_do($) +{ + if (defined($blocklist{$_[0]})) { + mlog(2, "* Weeding $_[0] ($iplist{$_[0]})\n"); + exec_iptables("-D", "INPUT", "-s", $_[0], "-d", "0.0.0.0/0", "-j", $settings{"ACTION"}); + } + undef($reason{$_[0]}); + undef($reason_n{$_[0]}); + undef($ignored{$_[0]}); + undef($ignored_d{$_[0]}); + undef($iplist{$_[0]}); + undef($blocklist{$_[0]}); +} + +sub weed_entries() +{ + # Don't weed in report mode. +# return if ($reportmode); + + foreach my $mip (keys %iplist) { + if (defined($iplist{$mip})) { + if ($iplist{$mip} >= 0) { + if (!check_time($iplist{$mip})) { weed_do($mip); } + } else { + weed_do($mip); + } + } + } + mlog(-1, "hmm\n"); +} + ### Check if given "try count" exceeds treshold and if entry ### is NOT in Netfilter already, then add it if so. sub check_add_entry($$$$$) @@ -459,12 +464,12 @@ my $cnt = $hitcount{$mip}++; $reason_n{$mip}{$mclass}++; - if ($report) { + if ($reportmode) { push(@{$reason{$mip}{$mclass}}, $mreason); } else { $reason{$mip}{$mclass} = $mreason; } - if ($report || ($cnt >= $settings{"TRESHOLD"} && check_time($mdate))) { + if ($reportmode || ($cnt >= $settings{"TRESHOLD"} && check_time($mdate))) { my $pat; if (!$mcond) { $ignored{$mip}{$mclass} = $mreason; @@ -480,14 +485,30 @@ $iplist{$mip} = $mdate; } else { # Over treshold, but is added, check if we can update the timedate - $iplist{$mip} = $mdate if ($mdate > $iplist{$mip}); + if ($mdate > $iplist{$mip}) { + $iplist{$mip} = $mdate; + } } } } -### + +############################################################################# ### Main helper functions -### +############################################################################# +### Print log entry +sub mlog +{ + my $level = shift; + my $msg = shift; + if (defined($LOGFILE)) { + print $LOGFILE "[".scalar localtime()."] ".$msg if ($settings{"VERBOSITY"} > $level); + } elsif ($settings{"DRY_RUN"}) { + print STDERR $msg if ($settings{"VERBOSITY"} > $level); + } +} + +### Initialize sub malt_init { mlog(0, "Updating initial blocklist from netfilter.\n"); update_iplist(-1); @@ -507,42 +528,13 @@ weed_entries(); } +### Quick cleanup (not complete shutdown) sub malt_cleanup { - # Close open filehandles foreach my $filename (keys %filehandles) { close($filehandles{$filename}); } } -sub malt_scan { - ### Keep on reading - mlog(1, "Entering main scanning loop.\n"); - my $counter = -1; - while (1) { - my %filepos = (); - foreach my $filename (keys %filehandles) { - for ($filepos{$filename} = tell($filehandles{$filename}); $_ = <$filehandles{$filename}>; $filepos{$filename} = tell($filehandles{$filename})) { - chomp; - check_log_line($_); - } - } - if ($counter < 0 || $counter++ >= 30) { - # Every once in a while, update known IP list from iptables - # (in case entries have appeared there from "outside") - # and perform weeding of old entries. - $counter = 0; - update_iplist(time()); - weed_entries(); - generate_status($settings{"STATUS_FILE_PLAIN"}, 0); - generate_status($settings{"STATUS_FILE_HTML"}, 1); - } - sleep(5); - foreach my $filename (keys %filehandles) { - seek($filehandles{$filename}, $filepos{$filename}, 0); - } - } -} - sub malt_finish { # Unlink pid-file if ($pid_file ne "" && -e $pid_file) { @@ -574,45 +566,44 @@ mlog(-1, "Reinitialization finished, resuming scanning.\n"); } - -### -### Main program -### -# Setup signal handlers -$SIG{'INT'} = 'malt_int'; -$SIG{'TERM'} = 'malt_term'; -$SIG{'HUP'} = 'malt_hup'; - -# Banner -my $argc = $#ARGV + 1; -if ($argc < 1) { - print $progbanner. - "\n". - "Usage: maltfilter [config filename]\n". - " maltfilter -f [config filename]\n". - "-f turns on the full report mode.\n"; - exit; +### Main scanning function +sub malt_scan { + mlog(1, "Entering main scanning loop.\n"); + my $counter = -1; + while (1) { + my %filepos = (); + foreach my $filename (keys %filehandles) { + for ($filepos{$filename} = tell($filehandles{$filename}); $_ = <$filehandles{$filename}>; $filepos{$filename} = tell($filehandles{$filename})) { + chomp; + check_log_line($_); + } + } + if ($counter < 0 || $counter++ >= 30) { + # Every once in a while, update known IP list from iptables + # (in case entries have appeared there from "outside") + # and perform weeding of old entries. + $counter = 0; + update_iplist(time()); + weed_entries(); + generate_status($settings{"STATUS_FILE_PLAIN"}, 0); + generate_status($settings{"STATUS_FILE_HTML"}, 1); + } + sleep(5); + foreach my $filename (keys %filehandles) { + seek($filehandles{$filename}, $filepos{$filename}, 0); + } + } } -# Test pid file existence -$pid_file = shift; -if ($pid_file eq "-f") { - $report = 1; -} else { - die("'$pid_file' already exists, not starting.\n". - "If the daemon is NOT running, remove the pid-file and re-start.\n") - if (-e $pid_file); -} +sub malt_read_config($) +{ + my $filename = $_[0]; + my $errors = 0; + my $line = 0; -# Read configuration file -if (defined(my $config_file = shift)) { - my $errors = 0; - - # Let user define his/her own logfiles to scan - undef(@scanfiles_def); - - open(CONFFILE, "<", $config_file) or die("Could not open configuration '".$config_file."'!\n"); + open(CONFFILE, "<", $filename) or die("Could not open configuration '".$filename."'!\n"); while () { + $line++; chomp; if (/(^\s*#|^\s*$)/) { # Ignore comments and empty lines @@ -622,7 +613,7 @@ if (defined($settings{$key})) { $settings{$key} = $value; } else { - print STDERR "Unknown setting '$key' = $value\n"; + print STDERR "[$filename:$line] Unknown setting '$key' = $value\n"; $errors = 1; } } elsif (/^\s*\"?([a-zA-Z0-9_]+)\"?\s*=>?\s*\"(.*?)\",?\s*$/) { @@ -635,20 +626,60 @@ } elsif (defined($settings{$key})) { $settings{$key} = $value; } else { - print STDERR "Unknown setting '$key' = '$value'\n"; + print STDERR "[$filename:$line] Unknown setting '$key' = '$value'\n"; $errors = 1; } } else { - print STDERR "Syntax error: $_\n"; + print STDERR "[$filename:$line] Syntax error: $_\n"; $errors = 1; } } close(CONFFILE); - die("Errors in configuration file '$config_file', bailing out.\n") unless ($errors == 0); + return $errors; +} + + +############################################################################# +### +### Main program +### +############################################################################# +# Setup signal handlers +$SIG{'INT'} = 'malt_int'; +$SIG{'TERM'} = 'malt_term'; +$SIG{'HUP'} = 'malt_hup'; + +# Print banner and help if no arguments +my $argc = $#ARGV + 1; +if ($argc < 1) { + print $progbanner. + "\n". + "Usage: maltfilter [config filename] [config filename...]\n". + " maltfilter -f [config filename] [config filename...]\n". + "-f turns on the full report mode.\n"; + exit; +} + +# Test pid file existence unless report mode +$pid_file = shift; +if ($pid_file eq "-f") { + $reportmode = 1; +} else { + die("'$pid_file' already exists, not starting.\n". + "If the daemon is NOT running, remove the pid-file and re-start.\n") + if (-e $pid_file); +} + +# Read configuration files +if (defined(my $filename = shift)) { + # Let user define his/her own logfiles to scan + undef(@scanfiles_def); + die("Errors in configuration file '$filename', bailing out.\n") + unless (malt_read_config($filename) == 0); } # Force dry run mode if we are reporting only -if ($report) { +if ($reportmode) { $settings{"DRY_RUN"} = 1; $settings{"VERBOSITY"} = 1; } @@ -686,7 +717,7 @@ # Fork to background, unless dry-running if ($settings{"DRY_RUN"}) { - if ($report) { + if ($reportmode) { mlog(-1, "Outputting report files.\n"); generate_status($settings{"STATUS_FILE_PLAIN"}, 0); generate_status($settings{"STATUS_FILE_HTML"}, 1);
HitsIP-addressDate of last hitReason(s)