[Gllug] Getting linux ipsec to vpn to smoothwall server - problems!

[Andy McGarty]

Hi,

I need to get a linux server (2.6 kernal) to set up a vpn to a smoothwall  
server so I can access one of the PCs behind it.

I've set it up as per the smoothwall instructions.
https://support.smoothwall.net/index.php?_m=knowledgebase&_a=viewarticle&kbarticleid=228

The ipsec-tools are loading and the varios RSA/MD5 etc encryption modules  
show up in lsmod.

I'm using roadwarrior mode as the standalone server has no local lan  
connected.

My secrets file says its using RSA, includes the .pem file and the second  
password used in creating it with ipsec, as describe in the above  
instructions.

The smoothwall certificate .p12 file has been processed by ipsec to create  
the two .pem files (put in /etc/ipsec.d) and the cacert.pem from the  
smoothwall has been put in /etc/ipsec.d/cacerts.

Using tcpdump I can see the two starting to talk and the secure.log shows  
its trying to connect, but then gives errors about  "malformed payload"  
which searching the web tends to mean the encryption isn't taking place.
I notice that the secure log also says "I did not send a certificate  
because I do not have one.".  I couldn't find anywhere on the net that  
stated this was a problem!

On the smoothwall side its saying
no RSA public key known for 'C=UK, ST=West Yorkshire, L=Keighley, O=J S  
Ramsbottom, CN=Andy Cert, E=andy at mac1systems.com'
sending encrypted notification INVALID_KEY_INFORMATION to 87.106.19.26:500

Which is the certificate I'm using, but where would it get the public key  
from?  Is it a smoothwall setting or should it appear somewhere in my  
config files.  I can't see that anywhere.

Anyone know where I should start to look for what to do next.  I've been  
banging my head against this most of the day and I'm sure its just  
something stupid I'm doing.

Cheers

Andy
-- 
Gllug mailing list  -  Gllug at gllug.org.uk
http://lists.gllug.org.uk/mailman/listinfo/gllug