[Sussex] PHPSMTPD - tests anyone?

[Steve Dobson]

Ronan

I'm enjoying this little exchange of ours.

On Fri, Sep 16, 2005 at 10:01:17AM +0100, Ronan Chilvers wrote:
> On Thu, 15 Sep 2005 21:59:01 +0100 Steve Dobson <steve at dobson.org> wrote:
> Neither, I'm a wine drinker, don't you know!!!

Red I hope!
 
> > > This in itself doesn't mean that an SMTP
> > > interface is required.  However the reasoning behind the SMTP
> > > interface was that it would gain me a lot of flexibility.
> > 
> > I'm curious to know what the advantages of an SMTP interface are over
> > a pipe interface - I can't see that it gives you anything.
> 
> Actually, having argued the toss I'm now leaning towards LMTP, rather
> than SMTP.  Simpler and designed for local network exchange.  Anyhoo,
> your point remains cos its still not a pipe.
> 
> So...
> 
> Postfix, of course, supports pipes and many autoresponders are built
> and work with postfix this way, including ones that are used with
> virtual domains.  However, if I use a pipe I'm tied to that single
> machine.  If I have 2 mail servers, far better to standardise on
> something like LMTP / SMTP and then both servers can use a single
> autoresponder service.  Then, lets say I have a web interface that
> allows users to turn their autoresponder on and off and customise the
> message (which is where I'm heading), you have single point of
> reference for your user information.  You can also scale very easily -
> if I add a third mail server, I can simply attach it to my
> autoresponder LMTP interface and we're off!

Okay, I now see where you are going with the SMTP (maybe LMTP) interface.
But pipes don't exclude you from that either.  I am going to assume that
your autoresponder extracts content from the e-mail - it would appear
stupid to send an e-mail with that 1.5M word attachment over the network
just to look at the "From" header.  Centralising servers can be a good
thing.

A pipe interface would work better here.  The pipe command running on
the SMTP server could just extract the information from the mail needed
and just transmit that over the wire to your autoresponder daemon.

But as you pointed out, you autoresponder is expected to have a low
hit rate on e-mails that it should generate a response from.  Your
SMTP/LMTP interface requires that *all* e-mails get transfered over
the network (which maybe local to a single machine, but on on the
scaled version).  This is not good system design.  You need to filter
out as early as possible those e-mails that don't need responses.

> > > The primary one is, as you rightly point out, the
> > > efficiency of the system.  Don't know if you know Postfix, but it
> > > uses its own virtual MDA and I'm not sure how hookable it is (am
> > > reading about it as we scribble).
> > 
> > This is probably why I've never used Postfix.  Exim and SendMail use
> > standard interfaces and are thus easy to write standalone programs
> > that work with them.
> 
> Postfix also uses standard interfaces.  That's why I have the option to
> use a pipe/SMTP/LMTP, etc.  The bit that isn't so satisfactory is its
> virtual domain support.  Virtual domains work well, except for this one
> area.

As I have never used Postfix I can't comment on whether it's virtual
domain support is better or worse that exim4.  I had a little problem
configuring virtual domains to work - but that was my silly mistake.
 
> > I;m not saying that you're wrong, but my gut reaction is "Get a better
> > MTA".
> 
> Right, then!!! MTA's at dawn on Brighton beach and I'll beat your mail
> server into submission.
> 
> But seriously, lets not get into the whole mine is better than yours...
 
I was not trying to get into a "mines better than yours" debate.  I'm
just pointing out that using a different MTA may make the problem you're
trying to solve easier.  Of course it may make other areas more difficult,
so there is a balance to be found.

> > > Since I'll probably be defining a second transport, my
> > > feelings were that to allow some scalability (ie: across hosts) an
> > > SMTP interface is going to be most flexible.
> > 
> > While that is true for a single binary, in exim you could configure
> > a pipe transport that tunnels across SSH for example:
> > 
> >    remote_pipe:
> >      driver = pipe
> >      command = ssh user at remote.host cat > /var/autoresponder/file
> 
> You can do this in postfix too, but see above about SMTP/LMTP.  What if
> someone is using an MTA that doesn't do this?  It might support pipes
> but may not allow this level of flexibility...  what then?
> 
> Secondly, I'm now communicating with my autoresponder via 2 more
> processes (ssh and cat) plus the autoresponder itself rather than a
> native standard interface spoken by both sides of the equation which
> requires only a couple of sockets in addition to the autoresponder.  I
> also have to have a shell account for each user on the remote host
> don't I?  What if I haven't?

Good point.

> > If /var/autoresponder/file is a fifo then your autoresponder program
> > on the other end could just be sitting there waiting for the next
> > e-mail to hit.
> > 
> > The really, really, really nice thing about *nix are the pipes - you
> > can do just about anything with them if you just think outside the
> > box. 
> 
> Absolutely and I'm not chucking them in the junkbin.  I'm thinking of
> using network standards rather than *nix black magic...  (Steve, what
> are you doing?  PUT DOWN THAT PLATE!!  DON'T THROW IT FOR GOD'S SAKE!!)

A better point, using network standards if you need your system to
support non-*nix type systems.

> > > The point about the sequential sockets is not to do with PHP5
> > > itself, only with my implementation of the socket server in
> > > PHPSMTPD.  There's no requirement for lots of concurrent
> > > connections since the software is intended as a development tool
> > > for 1 user (at the moment).  The current implementation does mean
> > > that the sockets will block (as you point out) but this isn't
> > > important at the moment because the load is _very_ low. Does that
> > > make sense?
<snip>
> > examples of plugging in spamassissin that way.  So ask yourself this
> > question:  If spamassissin doesn't need an SMTP interface why does
> > your autoresponder?  Unless there is something in the SMTP protocol
> > itself (not the e-mail content) then the answer has to be that it
> > doesn't.
> 
> True that spamassassin doesn't but then why does Amavisd-new
> (http://www.ijs.si/software/amavisd/) use an SMTP/LMTP interface and not
> a pipe?  Precisely because of the scalability advantages I'm discussing
> above...

Okay, scalability is the one big advantage of the SMTP/LMTP interface.
But you can't use it.  You're using PHP which doesn't scale because of
the way it handles sockets sequentially.  As most *nix MTAs can handle
multiple e-mail transports at a time to scale you're system you would
have to have multiple autoresponders running to handle the load.

Because the processes running via a pipe interface would be autonomous
you'd get much better scalability from the pipe interface with PHP than
with SMTP/LMTP.

> > Also your autoresponder has to be installed by root as the MTA has
> > to forward the e-mails onto it.  So a user that hasn't root access
> > can't use your system.  If it used a pipe interface then procmail
> > (if configured) would allow him to use your system.
> 
> Yup, but I've got no user system accounts and no procmail...  Also no
> shell users apart from admin shells...

Okay, you're designing for a particular deployment setup, nothing wrong
with that - it means you can ignore some complexities and thus keep the
system simpler.

> Supporting an SMTP interface doesn't equal autobloat!!

Yes it does, because of the extra complication of protocol which is designed
to allow MTAs to accept/reject/defer e-mail deliveries.  Your autoresponder
only wants good e-mails, the pipe interface only gets good e-mails.  If the
MTA rejects or defers the e-mail you're autoresponder never gets to see it.

> The
> autoresponder I have in mind is relatively complex but is in several
> bits.  The actual responder is pretty simple and the user interface is
> a separate set of scripts to a db table (or whatever the user info is
> stored in).  The server script only does
> 
> SMTP/LMTP receive -> parse email -> check mysql -> inject response if
> appropriate.
> 
> Job done!  Yours probably does the same thing just has a pipe at the
> front instead...

Of course we just replace the "SMTP/LMTP receive" with a "pipe receiver".

Steve
-- 
QOTD:
	Flash!  Flash!  I love you! ...but we only have fourteen hours to
	save the earth!
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: Digital signature
Url : http://mailman.lug.org.uk/pipermail/sussex/attachments/20050916/dbfe5942/attachment.pgp