From: Michael Tokarev (no email)
Date: Wed Oct 02 2002 - 13:52:26 EDT
Craig Sanders wrote:
> On Tue, Oct 01, 2002 at 03:40:30PM +0400, Michael Tokarev wrote:
>
>>Craig Sanders wrote:
>>
>>>On Fri, Sep 27, 2002 at 12:35:25AM +0400, Michael Tokarev wrote:
>>>
>>>
>>>>4. It's unclear what to do with multiple content_filters.
>>>>This very issue currently exists as well and it may be
>>>>very confused: when one defines *different* content_filters
>>>>in different places (global parameter, smtpd maps, header/body
>>>>checks, whatever).
>>>
>>>
>>>it occurs to me that perhaps the best thing to do here is for postfix to
>>>support just ONE global content-filter, and for that content-filter to
>>>be a wrapper which is "smart" enough to make decisions about which other
>>>content-filter(s) to call.
>>
>>Yes - this is how I see it too. The problem however is that queue file
>>may contain multiple filter records (example: one from global content_filter
>>parameter, it is placed on top of qfile, and another from header_checks,
>>it is placed at the end of qfile). The question was about what to do
>>with those multiple records, and which one to use (currently, postfix
>>uses LAST one).
>
> i'm not sure i see the problem. maybe i'm missing something here.
>
> if you take it outside of postfix, then (as far as postfix is concerned)
> there aren't multiple records, there's just one. all postfix needs is a
> fairly simple facility for getting an external process (pipe, daemon,
> whatever) to inspect and/or filter a message. postfix doesn't need to
> know and shouldn't know what the filter does or how or why, it should
> only care what the *final result* from the content-filter is - "OK",
> "DUNNO", "REJECT", or "FILTER".
[I'm not trimming context for a purpose]
Well, there is no problem per se - one just needs to make a descision.
And yes, Craig, - it seems you're missed something here.
The "problem" is that currently, queue file, when qmgr will open it
for a first time, already may contain *multiple* content_filter
records. Try that yourself: define global content_filter=something
in main.cf, and configure header_checks that contains e.g.
/./ FILTER other
and look at resulting qfile with postcat. You'll see *two* content_filter
records in it, not just one.
Having in mind postfix's security model (many programs that doesn't
trust each other), we can't assume that there will be one filter record
at max in queue file (wich is already not true but may be fixed by
modifying smtpd, pickup and cleanup daemons).
The "problem" is what to do with multiple filter records that for
some reason may be present in qfile. Not how to stack multiple
content_filters together, not how to react to their return codes,
but exactly this - what to do with *already present* *multiple*
FILTER records in a single queue file.
We should either choose one based on some criteria and ignore the
rest, or process every one in turn.
Currenly, it's irrelevant how many FILTER records a queue file
contains, since any such record triggers a call to content_filter
transport and message (and qfile with any other FILTER records)
will be deleted when transport completes. But in "inspector mode",
when message may be left in queue after it will be processed by
inspector, there is a question - what to do with other FILTER
records that *already* exists in this same qfile.
My proposed solution is to use *last* FILTER record and ignore
others -- exactly what postfix currently does (first record may
be choosen instead, and I'm in fact not sure what is preferred -
from programming point of view, using first one is a bit more
efficient).
> "OK" is for whitelist entries that the CF approves of. e.g. for a
> spam-lovers map for recipients who want to eat all their spam.
>
> "DUNNO" is if the filter has no reason to reject the message...other
> rules within postfix may cause a later rejection.
>
> "REJECT"'s meaning is obvious.
>
> "FILTER" is "I'LL TAKE IT FROM HERE" aka "DISCARD". the filter is going
> to transform the message and reinject it back to postfix.
>
> optionally, there could be a "DISCARD" code...but that would only be
> useful for postfix's logging as any filter could easily just discard a
> message rather than re-inject it.
It seems you're looking at this from some strange point of view,
or I just don't understand you. Please at least avoid using the
same keywords (OK, DUNNO, REJECT etc) as used in smtpd maps - this
increases confusion (and indeed there IS some comfusion at either
my or your side).
Look at the current content_filter logic. Note we're NOT talking
about header/body checks (maybe this is the source of confusion?):
1. Message enters postfix and gets stored in queue with a flag
indicating that it needs to be "content filtered", by adding
appropriate record into qfile.
2. Queue manager notices this record, and instead of delivering
this message as usual, it delivers message to a dedicated
content filtering transport. Currently, this is the last
step - in your terminology, current content_filter is *always*
working in "DISCARD" or "FILTER" or "I'll TAKE IT" mode
(except of "REJECT" mode - but this should happen if something
goes wrong only).
3, and this is where content_INSPECTOR arrives. A program working
as content filter may tell postfix "ok, sir, I did my work but
I don't want to take control over the message - please continue
delivery as usual" - that is, it returns "OK" instead of (the
only currently possible "DISCARD" (not counting bounces)).
4, now, when all recipients are looked for, qmgr can't just delete
qfile (as it currently does) since some recipients may require
further processing. But it should mark a flag/record in qfile
that triggered all the above processing as done - so not to repeat
the whole process again, - and rescan qfile from the beginning.
But at this time, it may find other, *already existing*, records
that also indicates that this message needs content filtering.
And the question actually was - what to do in this situation.
5. at this point, message was processed by a content filter, some
recipients are left, and it now gets delivered in a usual way -
without reinjection, without removing and recreating qfile, and
with consistent logging. Voila. ;)
>
>
>>>(btw, for the reinjection it would be nice if there was some way to
>>>connect the new queue-id to the old queue-id, to make it easier to
>>>trace what happened to any given message in the logs)
>>
>>That will be difficult at least. And in fact, this whole issue with
>>content inspector is "inspired" by this -- inability to connect two
>>"different" emails/qids that are in fact the same message.
>
>
> yes. the only way i can think of to do it is a kludge using a temporary
> header to store the old queue-id. e.g.
>
> - the filter adds a header "X-Postfix-Old-QUEUE-ID" to the message
[...]
I think there is an easy (?) and more clean (?) to do that. For example,
smtpd writes a line similar to "250 Ok (id 12345)" at the end of SMTP
transaction, where "12345" is a queue id. And this (new) queue id,
together with current (old) queue id, may be written into log by content
filter itself. There are 2 problems:
- in order to determine current (old) qid, one needs to parse top-level
received: line (why there is no ${queueid} macro in pipe(8)? ;)
- /usr/sbin/sendmail does not print qid (but postdrop does)
I thought about all this quite long time ago, way before any talk about
content INSPECTOR... Heh, even WITH real content_inspector in place
(like this one currently discussed or via other means), connecting
2 queue IDs maybe useful anyway - think about e.g. local's logging
"1234: status=sent (forwarded as 2345)", or bounces - content inspector
anyway needs to be able to take control over the message, and thus
it will be useful if it will be able to tell us how it "controlled"
the message (i.e. sent a notify to a postmaster, or "cured" on-the-fly
from a virus infection, or whatever).
/mjt
-
To unsubscribe, send mail to with content
(not subject): unsubscribe postfix-users
|
|
|