imap troubleshooting (was Re: martian source)

[George Toft]

Hi Nathan,

I don't agree with your assessment.

Swagging: 100 users each reading 1 100K e-mail per second = 10000K 
traffic = 10 mbps - well within a single 100mbps NIC on a saturated 
network.

You can get a better idea of your NIC traffic with something like this:
netstat -i; sleep 60; netstat -i
take the difference and divide by 60


I think you have a server performance issue.  You need to determine if 
it's the OS or the application or a dependence on some external server.



I suggest the following:
Remove one NIC
Turn on HT - Why is HT turned off?
Install package sysstat
Install package ethereal, libpcap (and any other dependencies is may 
have) or tcpdump

Review the data 1-2 days later using the sar command.  During a period 
of degraded operation:
- run iostat
- run vmstat
- run netstat -an | wc -l and compare it to ulimit -n
- run top, set refresh to 1 sec, and note the process(es) using the most CPU
- review system logs and imap logs.  I'm not familiar with your imap 
server, but I hope it has logs.  I hope you can change the logging level 
to really verbose.
- If possible, turn off the firewall (I've seen many problems caused by 
firewalls).
- run ethereal on your workstation and the server and trace your imap 
traffic.  When you find lags in the traffic, investigate the cause.  Lag 
between your workstation and the server is a network issue.  Lag in the 
server can be caused by imap, name resolver, firewalls, etc.
- You may also want to review your TCP parameters.  Your iptables script 
might have made some changes (like stuffing a 1 in 
/proc/sys/net/ipv4/conf/eth0/log_martians).  Make sure all the changes 
it makes are necessary.  I still like the idea of turning it off until 
you resolve your problem.
- Drastic move - use strace, attach to your imap server and watch the 
output to see where it chokes (network call, file not found, permission 
denied, etc).  I was able to troubleshoot a clamav problem using this 
trick - turned out to be a directory permission issue (700 vice 755).

Regards,

George Toft, CISSP, MSIS
My IT Department
www.myITaz.com
480-544-1067

Confidential data protection experts for the financial industry.


Nathan England wrote:
> I have looked online and found this is not a problem.
> I decided that since my email bottle neck will most likely be network, I added 
> a second nic to the machine. They are configured as such:
> 
> eth0 10.0.1.3 netmask 255.255.255.0
> eth1 10.0.1.4 netmask 255.255.255.0
> 
> route -n shows
> Kernel IP routing table
> Destination     Gateway         Genmask         Flags Metric Ref    Use Iface
> 10.0.1.0        0.0.0.0         255.255.255.0   U     0      0        0 eth0
> 10.0.1.0        0.0.0.0         255.255.255.0   U     0      0        0 eth1
> 169.254.0.0     0.0.0.0         255.255.0.0     U     0      0        0 eth1
> 0.0.0.0         10.0.1.1        0.0.0.0         UG    0      0        0 eth1
> 
> 
> All is well, I can point some to .3 and some to .4 specified by mail1 or mail2
> But Fedora is giving me the martian source error.
> Is there a better way to do this? I have never seen a martian error like this, 
> though I have never had two nics on the same network on a Fedora machine 
> before. I've done it with other ditros and never saw this error...
> 
> Nathan
>