Php hook for all my emails

Prev: Postfix architecture + Ldap + Courier IMAP
Next: ?best practice to stop " root+:|" exploits

From: "Mark Scholten" on 8 May 2010 08:37

---

> -----Original Message-----
> From: owner-postfix-users(a)postfix.org [mailto:owner-postfix-
> users(a)postfix.org] On Behalf Of mouss
> Sent: Friday, May 07, 2010 11:51 PM
> To: postfix users
> Subject: Re: Php hook for all my emails
>
> Andrés Gattinoni a écrit :
> > On Fri, May 7, 2010 at 5:29 PM, Noel Jones <njones(a)megan.vbhcs.org>
> wrote:
> >> On 5/7/2010 2:53 PM, Andrés Gattinoni wrote:
> >> Here's an example of a shell script that receives all mail.
> Shouldn't be
> >> much of a stretch to use PHP instead.
> >> http://www.postfix.org/FILTER_README.html#simple_filter
> >>
> >> If you still want to use amavisd-new, then use multiple postfix
> instances
> >> for each filter hop.
> >>
> >> Another alternative is to integrate your code into amavisd-new,
> either as a
> >> custom hook or as a pseudo antivirus scanner. Check the amavis-
> users
> >> archives for other folks doing similar things.
> >
> > Thanks for your response Noel.
> > I've checked out that link. The thing is that I don't actually to do
> > any filtering inside my script, I just need to gather some
> > information.
>
> What kind of information? if it's just about envelope, then logs are
> enough.
I am now doing it by parsing the logs, but I want to get the following information in a MySQL database (currently I get it from the logs, but that takes a few hours to parse for 1 day logs). I currently log the following data in a MySQL database:
- sender (email address)
- receiver (email address)
- IP sender (from where did postfix receive it)
- date/time (when)
I would like to add: subject (but I didn't find a solution yet for that). Also doing it "realtime" and not with a delay (as we don't do parse it every minute and putting it in a database when we receive the information would be great).

What do you think is the best solution?
>
> > Maybe there's an easier way that I can do that without having to call
> > sendmail again.
> >
> > [snip]

From: mouss on 8 May 2010 11:00

---

Mark Scholten a écrit :
> [snip]
> I am now doing it by parsing the logs, but I want to get the following information in a MySQL database (currently I get it from the logs, but that takes a few hours to parse for 1 day logs).
> I currently log the following data in a MySQL database:
> - sender (email address)
> - receiver (email address)
> - IP sender (from where did postfix receive it)
> - date/time (when)
> I would like to add: subject (but I didn't find a solution yet for that).

if you keep using logs, then you can use header_checks to log the subject:

/^Subject:/ WARN blahblah

DISCLAIMER: The above is a technical answer to a technical question. It
is not a recommendation. Subject logging may introduce privacy issues.

> Also doing it "realtime" and not with a delay (as we don't do parse it every minute and putting it in a database when we receive the information would be great).
>

you have many options:

- keep using a log parser. you can "tail -F $logfile | yourscript" or
call the parser often enough (use seek() to avoid re-parsing).
alternatively, you can mkfifo and log a copy to the fifo... (People do
this with syslog-ng).

- write a content_filter. you can use a pipe based filter (see the
CONTENT FILTER README). but you need to be careful... (yes, you need to
resubmit mail via sendmail...)
(if you already use amavisd-new and feel confortable writing perl code,
then you can do that from inside amavisd-new).

- write a milter.

- you can deliver a copy of mail to a program that does what you want.
for example

bcc_recipient_maps = pcre:/etc/postfix/bcc_recipient.pcre

== bcc_recipient.pcre:
/(.*)@(example\.com)$/ $1(a)bcc.example.com

== transport
bcc.example.com loggit:

and define "loggit" in master.cf.



> What do you think is the best solution?

I prefer log parsing because it doesn't interfere with mail flow. you
can stop it when you want and start it again, without touching
postfix... but it's just my opinion.

>>> Maybe there's an easier way that I can do that without having to call
>>> sendmail again.

with a content_filter, you need to resubmit mail, either via sendmail or
via smtp (if you chose tha letter, don't resubmit to the same port as
that would create an infinite loop. add a new smtpd listener which
doesn't trigger the content filter. as you see, this is more work than
with sendmail, and is probably not worth the pain in your case).

>>>
>>> [snip]
>

From: Reinaldo de Carvalho on 8 May 2010 11:14

---

On Sat, May 8, 2010 at 9:37 AM, Mark Scholten <mark(a)streamservice.nl> wrote:
> I am now doing it by parsing the logs, but I want to get the following information in a MySQL database (currently I get it from the logs, but that takes a few hours to parse for 1 day logs). I currently log the following data in a MySQL database:
> - sender (email address)
> - receiver (email address)
> - IP sender (from where did postfix receive it)
> - date/time (when)
> I would like to add: subject (but I didn't find a solution yet for that). Also doing it "realtime" and not with a delay (as we don't do parse it every minute and putting it in a database when we receive the information would be great).
>

You can log headers as subject with header_checks.

> What do you think is the best solution?
>>

LogProcessor (http://sf.net/projects/logprocessor) provide a cool
engine to load logfile in real-time to a database.

Each module is a class, then is easy to implement support for new log
formats. Postfix module isn't complete, and you can contribute with
the project. Some modules like to Iptables are available.


--
Reinaldo de Carvalho
http://korreio.sf.net
http://python-cyrus.sf.net

"Don't try to adapt the software to the way you work, but rather
yourself to the way the software works" (myself)

From: David Touzeau on 8 May 2010 18:54

---

On 07/05/2010 23:51, mouss wrote:
> Andrés Gattinoni a écrit :
>> On Fri, May 7, 2010 at 5:29 PM, Noel Jones<njones(a)megan.vbhcs.org> wrote:
>>> On 5/7/2010 2:53 PM, Andrés Gattinoni wrote:
>>> Here's an example of a shell script that receives all mail. Shouldn't be
>>> much of a stretch to use PHP instead.
>>> http://www.postfix.org/FILTER_README.html#simple_filter
>>>
>>> If you still want to use amavisd-new, then use multiple postfix instances
>>> for each filter hop.
>>>
>>> Another alternative is to integrate your code into amavisd-new, either as a
>>> custom hook or as a pseudo antivirus scanner. Check the amavis-users
>>> archives for other folks doing similar things.
>>
>> Thanks for your response Noel.
>> I've checked out that link. The thing is that I don't actually to do
>> any filtering inside my script, I just need to gather some
>> information.
>
> What kind of information? if it's just about envelope, then logs are
> enough.
>
>> Maybe there's an easier way that I can do that without having to call
>> sendmail again.
>>
>> [snip]


Noel Jones help you on the right way... better is to hook amavis instead
adding a new command-line in php that will reduce dramatically Postfix
processing.

I know is not very well documented on Amavis, but this a starting help,

you will see it easy.

Add in your amavisd.conf

include_config_files('path-to-your-perl-script.conf');




In path-to-your-perl-script.conf

package Amavis::Custom;
use strict;
#use Net::LDAP;
#use Config::IniFiles;
#use Geo::IP;
use File::Copy;
use File::stat;
use POSIX qw(ceil floor);
use Unix::Syslog qw(:macros); # Syslog macros
use Unix::Syslog qw(:subs); # Syslog functions

BEGIN {
import Amavis::Conf qw(:platform :confvars c cr ca $myhostname);
import Amavis::Util qw(do_log untaint safe_encode safe_decode);
import Amavis::rfc2821_2822_Tools;
import Amavis::Notify qw(build_mime_entity);
}



sub new {
my($class,$conn,$msginfo) = @_;
bless {}, $class;


}

sub before_send {
my($self,$conn,$msginfo) = @_;
## do what you want before routing message to final destination.
$self;
}

# main function that allows you to hook amavis filtering

sub checks {
#$r->recip_addr_modified($new_addr); # replaces delivery address!
my($self,$conn,$msginfo) = @_;
my($client_ip) = $msginfo->client_addr;
my $bann_reason='';
my $sender_domain='';
my($log_id)=$msginfo->log_id;
my($tempdir) = $msginfo->mail_tempdir;
my $recipient_domain;
my($received)=$msginfo->get_header_field_body('received');
my($kasrate)= trim($msginfo->get_header_field_body('X-SpamTest-Rate'));
my($mail_file_name) = $msginfo->mail_text_fn;
my($sender_address)=$msginfo->sender;
$message_id=Mydecode($msginfo->get_header_field_body('message-id'));
$message_id=~ s/<//g;
$message_id=~ s/>//g;
#Add header token
$msginfo->header_edits->add_header('X-YOUR-scanner','0.00');


# ($country_name,$region_name,$city)=ScanGeoIP($last_recived); Example
to using geoip

$sender_address=~ m/(.+?)@(.+)/;
$sender_domain=$2;

#Example log trough amavis loger
do_log(0, "%s my-plugin: client's IP [%s], last [%s] sender: <%s>,
X-SpamTest-Rate:
%s",$message_id,$client_ip,$last_recived,$sender_address,'');

# change the Amavis behavior using specific class
Amavis::load_policy_bank('killAll');

$self;
}

1; # insure a defined return

First  |  Prev  | 
Pages: 1 2
Prev: Postfix architecture + Ldap + Courier IMAP
Next: ?best practice to stop " root+:|" exploits