Prev: Preventing outgoing local mail being delivered to certain domains
Next: multiple IPs in and out
From: Victor Duchovni on 26 Jan 2010 17:36
On Tue, Jan 26, 2010 at 04:45:25PM -0600, adrian ilarion ciobanu wrote:

> > Instead of using a DOMAIN_PORT kludge which requires "reserving"
> > a TCP port or UNIX-domain pathname per customer, it would make
> > sense to use the existing Postfix connection caching mechanism.
> >
> > The idea is to push an open socket into the scache daemon (with a
> > suitable time to live) under the name of the customer's domain.
> > Then, the Postfix SMTP client would automatically find that open
> > socket and start talking SMTP over it.
>
>
> So I would push the socket to scache after I'm done setting it up
> from SMTPD (auth, policy checks) and forget about it. If it times
> out before local smtp will start deliver then the client is welcome to reconnect.
> This will happen if it has to happen in SMTPD or in SCACHE the same way.
> In fact it's a descriptor passing tweaked for smtp deliveries. Nice! :)

Yes, but in this case, because the client is authenticated, and requesting
a service that it is willing to wait for with typical SMTP command timeouts,
the scache lifetime of the cache entry needs to be (significantly) higher.
I would suggest ~300s. There may need to be some minor scache code tweaks
to support a new class of longer-lived cache slots.

--
Viktor.

P.S. Morgan Stanley is looking for a New York City based, Senior Unix
system/email administrator to help architect and sustain our perimeter
email environment. If you are interested, please drop me a note.

From: Wietse Venema on 26 Jan 2010 17:42
adrian ilarion ciobanu:
>
> > In both clases the client connects to the standard SMTP port. The
> > biggest difference is that ETRN creates new SMTP connections for
> > delivery, whereas ATRN delivers over the existing connection.
>
> So I should understand that RFC specifying port 366 as the ODMR
> port is just a "should" and not a "must"? And the odmr client is
> supposed to try ATRN on smtp port? The rfc is exactly not so clear
> on if this is a requirement or ... plus i see odmr ports poisoning
> my etc/services file. Now I don't know what the smtp client
> expectations are, but I assume windows will use port 366 :)

I think that the port 366 requirement alone does not justify the
effort of maintaining two Postfix SMTP servers (one full server
and one that provides ODMR only), especially considering that

- It will have to support STARTTLS, and thus all the things that
come with it. Duplicating SMTP server code is not an option;
mutilating a shared source file source with #ifdefs makes testing
and maintenance harder; and librarifying portions of the SMTP
server so it can be shared by different programs would suck up
a lot of quality assurance cycles to ensure nothing gets broken.

- Once the Postfix SMTP server has negotiated an encrypted session
and authenticated the client, the remainder of the ATRN work can
be delegates to a different process that I'll call atrnd(8) for
brevity.

> > UNIX-domain
> > sockets would provide a more private name space.
>
> I agree. And somehow faster, by dropping the tcp stack. But in
> this case one should use lmtp service for honoring flushes and
> that would involve some more headache on proxying the data to
> client (lmtp to smtp translations). Or is it possible to use unix
> addresses with smtp (manpage says no)? If not, no one stops me to
> alias the loopback interface (127.0.0.2, 0.0.3, ... ). Although
> it sounds like a windows hotfix.

I think that one TCP port or UNIX-domain "name" per customer can
be avoided. atrnd(8) could store an open socket in scache(8),
under the customer's domain name, so that the Postfix SMTP client
can find it there.

> > It is to be expected that the customer will log in over a TLS
> > session, so the SMTP server needs to open a socket to an ATRN server
> > and tell it what customer to receive mail for. The ATRN server then
> > does all the SMTP-specific stuff, like sending "220 servername" to
> > the Postfix SMTP client, and dropping QUIT commands from the Postfix
> > SMTP client so that the connection is not closed too early. The
> > Postfix SMTP server just reads/writes bytes between ATRN daemon
> > and customer until both parties close a connection, or until some
> > error or timeout (while respecting the built-in watchdog timer and
> > other safety mechanisms).
> >
> > Customer <--> Postfix SMTPD <--> ATRND <--> Postfix SMTP client
> >
>
> I understand, you would rather have some operations (like data
> proxying) duplicated instead of risking more bugs in the core and
> thats why there's a new ATRND service. The less code added to
> SMTPD, the less the chance of new bugs. The only problem i see
> with this is it only gets more complicated. If I would really
> want to free smtpd from ATRN code then after a successfull
> authentication the ATRN specific talk should be totally routed to
> ATRND service and forgotten, by passing the socket descriptor. I

You can't pass a TLS session from one process into another one,
without closing the connection and re-negotiating TLS.

> my scenario, now that I'm talking SMTPD patching:
>
> smtpd: if client_auth OK && request_policy_check OK then:
> smtpd: listen(transport_socket) #be prepared for local smtp deliveries
> smtpd: begin_foreach_domain_loop
> smtpd: send_flush_request(domain) #this is just etrn flush request multiplied
> smtpd: end_foreach_domain_loop
> #this is where the time gets spent
> smtpd: proxy from transport_socket to client_socket (filter quits, additional helos,etc)
> everybody_ends_conversations
>
> yours (did I get it right?):

My preference is that smtpd(8) connects to the atrn port (specified
in master.cf) and sends the customer domain name. The atrnd(8)
daemon then pushes an open socket into scache(8) and sends a flush
request.

> #request_policy_check sits here pretty neat, still (can reuse etrn policy check service)
> smtpd: if client_auth OK && request_policy_check OK then:
> smtpd: connect(ATRN service)
> smtpd: send userinfo
> atrnd: listen(transport_socket)

Better: send open socket into scache(8) daemon under the customer
domain name and wait for events.

> atrnd: begin_foreach_domain_loop
> atrnd: send_flush_request(domain)
> atrnd: end_foreach_domain_loop
> #this is where the time gets spent, 2 ops
> atrnd: proxy from transport_socket to smtpd_atrnd_socket (filter quits, additional helos,etc)
> smtpd: proxy from smtpd_atrnd_socket to client_socket
> everybody_ends_conversations

Yes, roughly.

Wietse

From: adrian ilarion ciobanu on 26 Jan 2010 18:40
> >
> > So I would push the socket to scache after I'm done setting it up
> > from SMTPD (auth, policy checks) and forget about it. If it times
> > out before local smtp will start deliver then the client is welcome to reconnect.
> > This will happen if it has to happen in SMTPD or in SCACHE the same way.
> > In fact it's a descriptor passing tweaked for smtp deliveries. Nice! :)
>
> Yes, but in this case, because the client is authenticated, and requesting
> a service that it is willing to wait for with typical SMTP command timeouts,
> the scache lifetime of the cache entry needs to be (significantly) higher.
> I would suggest ~300s. There may need to be some minor scache code tweaks
> to support a new class of longer-lived cache slots.

The client still enjoys typical smtp timeouts on smtpd side. What is going to
scache is a ready to go descriptor for the local smtp client to start delivering
for a domain. Some sort of shortcircuit so the smtp client wont lookup transports
for the domains and connect to atrnd(8) plus it will solve the many-UNIX-addrs problem.

My question now is if i can register with scache the same socket for multiple domains.


>
> --
> Viktor.
>

--
adrian ilarion ciobanu
adrian.i(a)ciobanu.name
http://pub.mud.ro/~cia
+40 788 319 497

From: Victor Duchovni on 26 Jan 2010 18:23
On Tue, Jan 26, 2010 at 05:40:40PM -0600, adrian ilarion ciobanu wrote:

> > >
> > > So I would push the socket to scache after I'm done setting it up
> > > from SMTPD (auth, policy checks) and forget about it. If it times
> > > out before local smtp will start deliver then the client is welcome to reconnect.
> > > This will happen if it has to happen in SMTPD or in SCACHE the same way.
> > > In fact it's a descriptor passing tweaked for smtp deliveries. Nice! :)
> >
> > Yes, but in this case, because the client is authenticated, and requesting
> > a service that it is willing to wait for with typical SMTP command timeouts,
> > the scache lifetime of the cache entry needs to be (significantly) higher.
> > I would suggest ~300s. There may need to be some minor scache code tweaks
> > to support a new class of longer-lived cache slots.
>
> The client still enjoys typical smtp timeouts on smtpd side. What is going to
> scache is a ready to go descriptor for the local smtp client to start delivering
> for a domain. Some sort of shortcircuit so the smtp client wont lookup transports
> for the domains and connect to atrnd(8) plus it will solve the many-UNIX-addrs problem.
>
> My question now is if i can register with scache the same socket for
> multiple domains.

You associate a fixed nexthop with each authenticated client, and their
entire set of domains. You flush either all their domains, or the subset
they requested. The scache entry is for the client-specific nexthop, not
the recipient domain.

example.com atrn:[client1.atrn.invalid]
example.net atrn:[client1.atrn.invalid]
example.org atrn:[client2.atrn.invalid]

The scache slot is for "[client1.atrn.invalid]".

--
Viktor.

P.S. Morgan Stanley is looking for a New York City based, Senior Unix
system/email administrator to architect and sustain our perimeter email
environment. If you are interested, please drop me a note.

From: adrian ilarion ciobanu on 26 Jan 2010 18:57
>
> You associate a fixed nexthop with each authenticated client, and their
> entire set of domains. You flush either all their domains, or the subset
> they requested. The scache entry is for the client-specific nexthop, not
> the recipient domain.
>
> example.com atrn:[client1.atrn.invalid]
> example.net atrn:[client1.atrn.invalid]
> example.org atrn:[client2.atrn.invalid]
>
> The scache slot is for "[client1.atrn.invalid]".

Got it. I mixed scache manpage missreadings with flush records.

Thank you!

>
> --
> Viktor.
>
> P.S. Morgan Stanley is looking for a New York City based, Senior Unix
> system/email administrator to architect and sustain our perimeter email
> environment. If you are interested, please drop me a note.

--
adrian ilarion ciobanu
adrian.i(a)ciobanu.name
http://pub.mud.ro/~cia
+40 788 319 497

First  |  Prev  |  Next  |  Last
Pages: 1 2 3 4 5
Prev: Preventing outgoing local mail being delivered to certain domains
Next: multiple IPs in and out