[HLUG] Content filtering server, email server, domain controller

[Julian Robbins]

Paul Stenning wrote:
> Hi Julian, all,
>
>   
>> You can have roaming profiles in Samba, but we found it was a bit prone
>> to problems. Windows profiles can get a bit big, which is the problem,
>> and thus more likely to suffer corruption, so good user training, ie not
>> putting large files on their desktops, all helps.
>>     
>
> I think the biggest files for most of our users would be the local IMAP 
> caches of Thunderbird.  I am trying to train users to clear out their 
> sent mail folder periodically.  That can be fixed using Thunderbird's 
> disk space options though.
>
> Only one user tends to store big files on the desktop and he is 
> trainable. :)
>   
Roaming profiles can work - but if you have different pc's and different 
applications on them,it can get confusing.
>   
>> This is exactly what we have use.  fetchmail running  doing a POP, then
>> delivering the mail so that IMAP clients  can pick it up. Thunderbird
>> works well with a IMAP server- we started with Thunderbird 0.8, and
>> Washigton Uni IMAP server, now use Cyrus.
>>     
>
> So is fetchmail the mail fetcher and the IMAP server?  Or is Cyprus the 
> IMAP server?
>   
Fetchmail is the small utility that simply does a POP from your ISPs 
mailbox, then stores the mail. Cyrus is a IMAP server. You also need 
something to send mails, ie Postfix/Sendmail are popular choices.
>   
>> Other options are Dovecot. It may be better not to use fetchmail. It
>> works well enough but these days instantaneous emails being delivered is
>> expected a bit more . That said, this would require putting your mail
>> server in a Firewall DMZ and security hardening it. Which if you're
>> trying to do all of this with one box isnt easily possible.
>>     
>
> Yes, the budget is fairly limited and it's all got to run on one box, 
> though it will probably be a fairly decent spec one.
>   
Should be ok. AV scanning can slow down a box though.
> I can't really claim much cost saving by going for Linux as we have 
> Microsoft Action Pack already which contains most of the Windows based 
> stuff.
>
>   
>> The main differences are whether you want to use mbox, or maildir
>> formats, ie whether  every email is stored as a file, or whether a
>> folder in your email client is stored as a single file. Each has its
>> pros and cons.
>>     
>
> Can you elaborate please?  The vast majority of emails are text only (we 
> have HTML email disabled in Thunderbird by default) and have no 
> attachments.  Most are customer queries and replies so are relatively 
> disposable.
>   
mbox and maildir are the formats that unix mail systems store the actual 
mails (and their attachments) in. I can never remember which is which, 
but maildir I think effectively is 1 file for a whole users email folder 
such as their inbox. This is good in terms of simplicity, but if you 
need to restore a mail, it will be in a probably huge maildir file to 
retrieve. Whereas, mbox (assuming i've got it the right way around), 
stores every single users emails as flat file single files, so if you 
have 1000 mails in a folder, that folder in the filesystem will have 
1000 mbox files.  Smaller files, but lots of them !
>   
>> Mail servers are still IMHO the most complicated to setup once you have
>> added, Antispam, and/or antivirus. Adding either or both increases the
>> complexity. Its a well trodden path though, and there are lots of good
>> Howto guides out there. Spamassasin and Clam work excellently, but do
>> need some setting up.
>>
>> If you have your antispam / AV done upstream its much easier, but
>> riskier. 
>>     
>
> We have anti-spam upstream with SpamAssassin which works reasonably 
> well.  There is ClamAV on the web server but I am not sure whether it 
> scans all incoming mail or not so it would be good to scan it on the our 
> server if possible.  Not a big deal if not though as it gets scanned by 
> Eset NOD32 on the clients.
>
>   
best to scan on the server if you can too. That way you can have belt 
and braces as not every AV scanner picks up new viruses as quick as the 
others.
>> mail backups are easy enough, mails are all files anyway in Linux .... ;-)
>>     
>
> Good!  A previous company that I worked for used Exchange from their 
> data centre in Manchester for all sites in Europe.  It crashed and 
> trashed its mail stores, and it took over a week for them to get it all 
> back online and backups restored (Exchange has to rebuild every mailbox 
> as data is restored which is painfully slow).  This is why I have that 
> over-blown Microsoft solution so much!  Plus it's a resource hog.
>   
mail will always comprises lots of files, or smaller numbers of large 
files, but at least in Linux, the files, are plain text of the emails 
within. Attachments are encoded and also stored within. No odd formats - 
even if your mail gets corrupted, you can still cut out the bad bits and 
restore the rest, especially with mbox format.
>   
>> We use IPCop with various addons. It works ok, but there are odd
>> annoyances sometimes. Dan's guardian may be worth looking at , or
>> Smoothwall/Astaro if you want to pay money.
>>     
>
> OK, I'll look at those.  We don't mind spending a bit of money for the 
> right products.
>   
Dans Guardian is free. The others cost money ...
>   
>>> Managing the whole thing:  Probably Webmin.  Remote access to this would 
>>> be very useful but that will probably be handled by VPN routers.
>>>   
>>>       
>> webmin is ok, and is the way we went to start with. But there is the
>> possibility for some of the third party modules that they overwrite your
>> own custom tweaks if not tested thoroughly. This was a few years ago
>> though so it may have improved though ... Best to learn it yourself if
>> possible - unless you've got other staff to train who dont want Linux
>>     
>
> It is just me managing it but they will want to consider the "in event 
> of death" stuff and have something that someone else could take over. 
> There is nobody else IT literate in the company though so it'll have to 
> be someone external.  HLUG will be first port of call for them.  :)
>   
You can also use SWAT to configure and diagnose Samba. There arent many 
GUI tools for mail systems, (unless you pay), but they are usually ultra 
reliable and dont need much looking at, apart from the odd bit of log 
checking.
>   
>>> Eset anti-virus management:  That will have to be done with Windows in 
>>> vmware (or virtualbox if I can get it to work).
>>>   
>>>       
>> Will have to be windows, but it may possibly work in WINE. WINE works
>> pretty well with a lot of software nowaways.
>>     
>
> The management is a service running on the server all the time plus a 
> GUI management client that is run when needed.  So I think it will need 
> to be a Windows server in vmware.
>   
May still run in WINE - but WINE is still a little flimsy sometimes, and 
takes a lot of the box over - worth checking though - the ESET admin 
tool, looks quite straightfordward. check winehq.com for compatibility
>
>   
>> tar is the star. There are a lot of tap backup still out there. 
>>     
>
> Simple Backup (which I think was a Ubuntu sponsored summer of code 
> project) is basically a front-end to tar.  It creates a tar file for 
> each backup together with index files to allow the restore part to 
> display the backup contents quickly.
>
> I have restored files with it a few times and it works as expected.  I 
> haven't done a full restore yet though.
>
>   
I havent seen that one.
>> many sys
>> admins still like scripts with tar / SSH etc. I would consider using
>> backing up to disk instead though these days with the abundance of disk
>> based cheap disks around. 
>>     
>
> Big USB drives are certainly not expensive now.  Plus it's easier to 
> plug them into another system and access the data.  The ones I am using 
> for my home server backup are NTFS which Linux writes to fine and can be 
> plugged into a Windows box too.  7-Zip opens the tar files in Windows.
>   
Although linux works better than ever with NTFS, I wouldnt use it as the 
mail filesystem type. Its still too poorly documented and will always 
not work as well as EXT3/4 etc on Linux. Admittedly, if you want to plug 
it in to a winblows box, you cant easily read it then. FAT32 is an 
option, but the file storgae limit is getting a bit more of a problem 
sometimes.
>> Rsync works really well, extremely useful, and
>> there are loads of utilities based around it. BackupPC is what I would
>> recommend.
>>     
>
> I'll look at BackupPC.
>   
Yes do. But there are others too worth looking at. Amanda etc
>   
>>> If I can do most of this with Linux I will probably go for Ubuntu Server 
>>> 8.04 LTS as that's what I'm familiar with.  CentOS is another possibility.
>>>   
>>>       
>> Ubuntu server is fine. There's not a great deal of difference really.
>> Package Mgt is still better in Debian based systems though. Dont forget
>> the Ubuntu server stuff isnt GUI based though. But there are some tools
>> to help.
>>     
>
> Ubuntu it is then.  Would you go with the 8.04 LTS version?  I see no 
> reason to go bleeding-edge with this and would rather it was stable and 
> kept working.  My home server uses 8.04.
>   
If you wont be changing the server for a couple of years its worth it, 
as you'll get 5 years of security updates on the LTS server version I 
believe. Server apps dont chnagev as much as desktop ones.
>> All of this can be done in Linux !! and work extremely well too !! But
>> it can be intimidating and need some initial help. The great thing with
>> using Linux here is the transparency and openness - backing up is easy
>> with open formats - it gives you so many options and flexibility. You
>> can do much more with your network too, with SSH and rsync utilities.
>> even backing up over the net !
>>
>> Linux may be free, but there is a learning curve, it does/can cost money
>> too and time. But you do get a system YOU control, with high levels of
>> reliability and flexibility.
>>     
>
> Fortunately I have a reasonable amount of time to get this sorted.  It 
> is needed as part of a move to a new office which is not likely to 
> happen until late summer.
>   
If you need any help, give me a call ;-)
>   
>> Please email me offlist if you would like to come and see our linux
>> server setup at work.
>>     
>
> I will do, thanks Julian.
>
> Paul
>   
Julian