pointless select()s and IMAP connections in CLOSE_WAIT

• Previous message: Henrique de Moraes Holschuh: "Re: Cyrus POP3 Issue"
• Messages sorted by: [ date ] [ thread ] [ subject ] [ author ] [ attachment ]

Hi

We're running Cyrus imapd version 2.1.17 on a FreeBSD server
(4.11-RELEASE) with Cyrus SASL 2.1.19 and saslauthd. Everything
is installed from the ports tree. In the middle of the day we
get up to about 2000 concurrent connections to the IMAP port.

Usually these connections are all ESTABLISHED according to
netstat/lsof etc., but sometimes we suddenly get a huge spike in
connections. If I look at netstat at that point, all the
"extra" connections are in the CLOSE_WAIT state.

# netstat -n | grep CLOSE_WAIT
[...]
tcp4 102 0 xxx.yyy.zzz.www.993 xxx.yyy.65.101.1480 CLOSE_WAIT
tcp4 105 0 xxx.yyy.zzz.www.993 xxx.yyy.5.211.4289 CLOSE_WAIT
tcp4 0 0 xxx.yyy.zzz.www.993 xxx.yyy.35.232.1721 CLOSE_WAIT
tcp4 84 0 xxx.yyy.zzz.www.993 xxx.yyy.28.3.3480 CLOSE_WAIT
tcp4 98 0 xxx.yyy.zzz.www.993 xxx.yyy.66.14.50285 CLOSE_WAIT
tcp4 0 0 xxx.yyy.zzz.www.993 xxx.yyy.28.34.1373 CLOSE_WAIT
tcp4 0 0 xxx.yyy.zzz.www.993 xxx.yyy.35.232.3552 CLOSE_WAIT
tcp4 57 32147 xxx.yyy.zzz.www.993 xxx.yyy.5.182.1316 CLOSE_WAIT

The only fix for the problem appears to be to restart Cyrus.

It I have a look at one of the processes with lsof, this is what
it looks like:

# lsof -i @xxx.yyy.65.101:1480
COMMAND PID USER FD TYPE DEVICE SIZE/OFF NODE NAME
imapd 85462 cyrus 0u IPv4 0xef4a0c00 0t0 TCP xxx.yyy.zzz.www:imaps->xxx.yyy.65.101:1480 (CLOSE_WAIT)
imapd 85462 cyrus 1u IPv4 0xef4a0c00 0t0 TCP xxx.yyy.zzz.www:imaps->xxx.yyy.65.101:1480 (CLOSE_WAIT)
imapd 85462 cyrus 2u IPv4 0xef4a0c00 0t0 TCP xxx.yyy.zzz.www:imaps->xxx.yyy.65.101:1480 (CLOSE_WAIT)

If I look at the process with truss, it seems to be stuck in a
loop calling select with no file descriptors!

# truss -p 85462
(null)() = 0 (0x0)
select(0x0,0x0,0x0,0x0,0xbfbfe808) = 0 (0x0)
select(0x0,0x0,0x0,0x0,0xbfbfe808) = 0 (0x0)
select(0x0,0x0,0x0,0x0,0xbfbfe808) = 0 (0x0)
select(0x0,0x0,0x0,0x0,0xbfbfe808) = 0 (0x0)
select(0x0,0x0,0x0,0x0,0xbfbfe808) = 0 (0x0)
select(0x0,0x0,0x0,0x0,0xbfbfe808) = 0 (0x0)
select(0x0,0x0,0x0,0x0,0xbfbfe808) = 0 (0x0)
select(0x0,0x0,0x0,0x0,0xbfbfe808) = 0 (0x0)
select(0x0,0x0,0x0,0x0,0xbfbfe808) = 0 (0x0)
select(0x0,0x0,0x0,0x0,0xbfbfe808) = 0 (0x0)

It seems as if some of the imapds suddenly forget which fds they
are supposed to be reading from/writing to and just wait around
for something to happen that never will.

I've had a look around on the Cyrus mailing list archives, but
haven't managed to find a solution to this problem. Is it a
known problem?

Thanks.

-- 
Michael Wood <>
---
Cyrus Home Page: http://asg.web.cmu.edu/cyrus
Cyrus Wiki/FAQ: http://cyruswiki.andrew.cmu.edu
List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html

• Previous message: Henrique de Moraes Holschuh: "Re: Cyrus POP3 Issue"
• Messages sorted by: [ date ] [ thread ] [ subject ] [ author ] [ attachment ]