Mailing List Archive: connected but connection died ?

connected but connection died ?

Jun 19, 2008, 1:36 PM

Post #1 of 9 (5332 views)

I'm in the process of setting up a set of qmail & dovecot servers
distributed across our company's network, with one dmz system as the
single entry and exit point for mail. It's mostly all working now with
two sites, but I found an odd problem.

If I send a message from the remote site to my gmail account, and then
reply to that message, it is accepted by the dmz server but then when it
is forwarded to the remote internal server I get this:

Jun 19 16:18:14 buster qmail-send: 1213906694.836116 delivery 15908:
deferral:
Connected_to_192.168.1.219_but_connection_died._Possible_duplicate!_(#4.4.2)/

I don't see any evidence of the connection on the destination server.
Successful incoming messages are logged as usual.

Yet when I just send a message from my gmail account directly to the
user at the remote site it works fine. This is reproducable. A reply
to a second message fails, and incoming mail from gmail works fine.

What could be happening here? Thanks in advance.

One message "stuck" in the queue looks like this:

--------------
MESSAGE NUMBER 94624
--------------
Received: (qmail 27738 invoked by alias); 19 Jun 2008 16:16:44 -0400
Delivered-To: robm@m2.seamanpaper.com
Received: (qmail 27735 invoked from network); 19 Jun 2008 16:16:44 -0400
Received: from rv-out-0506.google.com (209.85.198.230)
by buster.seamanpaper.com with SMTP; 19 Jun 2008 16:16:44 -0400
Received-SPF: pass (buster.seamanpaper.com: SPF record at
_spf.google.com designates 209.85.198.230 as permitted sender)
Received: by rv-out-0506.google.com with SMTP id f6so7319385rvb.5
for <robm@m2.seamanpaper.com>; Thu, 19 Jun 2008 13:16:42
-0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
d=gmail.com; s=gamma;

h=domainkey-signature:received:received:message-id:date:from:reply-to
:to:subject:in-reply-to:mime-version:content-type:references;
bh=7eb4+qyKj7gnc31TVvEkrV/arN9VXFNySe5rfb+u21M=;

b=gvVeGWT5IwXBRAjXAxvSsVtHwgcUuqoNHCp2pDVEvabgSyBNbfM2lm72/3R9pcJkXo

iop0Shj+UdnoGON2XtmHNo516sOaSvhCLV40RQ7dKM4rItXZl79grZQw/d792Dsgx8bm
xDutCLJ3Ccfe5ynp8FlhEq9iHzBZZjxat0Ci8=
DomainKey-Signature: a=rsa-sha1; c=nofws;
d=gmail.com; s=gamma;

h=message-id:date:from:reply-to:to:subject:in-reply-to:mime-version
:content-type:references;

b=Bf7VKxnEiIuF7Wj9YH7eEc6RPKqF3qsjlzi5/CMG3UMFkcQ6VWf/2MEyXQnkm5pTQn

MesiHGLgWeFRWOfyF+ntcpyU5QxBHMB6YGJ7/0yqVfdmhJ+eYX99fzdb9IwT9NbkrpGB
ksr0PJc4tWYk1pEjnsEGc+S38/FfopQNpOlAA=
Received: by 10.141.89.13 with SMTP id
r13mr6959097rvl.177.1213906602574;
Thu, 19 Jun 2008 13:16:42 -0700 (PDT)
Received: by 10.141.206.17 with HTTP; Thu, 19 Jun 2008 13:16:42
-0700 (PDT)
Message-ID:
<588667830806191316g7d2e1b9ev26be3cbb320fa1b6@mail.gmail.com>
Date: Thu, 19 Jun 2008 16:16:42 -0400
From: "Jeff Dickens" <dreamgear@gmail.com>
Reply-To: dickens@dreamgear.com
To: "robm test" <robm@m2.seamanpaper.com>
Subject: Re: test e1
In-Reply-To: <485ABA56.9040805@m2.seamanpaper.com>
MIME-Version: 1.0
Content-Type: multipart/alternative;
boundary="----=_Part_6593_2398558.1213906602576"
References: <485ABA56.9040805@m2.seamanpaper.com>

------=_Part_6593_2398558.1213906602576
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: 7bit
Content-Disposition: inline

test f2

On Thu, Jun 19, 2008 at 3:58 PM, robm test <robm@m2.seamanpaper.com>
wrote:

> not really rob
>

------=_Part_6593_2398558.1213906602576
Content-Type: text/html; charset=ISO-8859-1
Content-Transfer-Encoding: 7bit
Content-Disposition: inline

test f2<br><br><div class="gmail_quote">On Thu, Jun 19, 2008 at 3:58
PM, robm test <<a
href="mailto:robm@m2.seamanpaper.com">robm@m2.seamanpaper.com</a>>
wrote:<br><blockquote class="gmail_quote" style="border-left: 1px
solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left:
1ex;">
not really rob<br>
</blockquote></div><br>

------=_Part_6593_2398558.1213906602576--

Re: connected but connection died ? [ In reply to ]

jeff at m2

Jun 19, 2008, 1:41 PM

Post #2 of 9 (5225 views)

Permalink

I accidently posted from a test account, so please reply to the list or
to me at this address.

Thanks.

Re: connected but connection died ? [ In reply to ]

jeff at m2

Jun 19, 2008, 2:03 PM

Post #3 of 9 (5249 views)

Permalink

I figured out why smtpd wasn't logging. Now I get this:

Jun 19 17:00:43 cardinal qmail-smtpd: 1213909243.173962 tcpserver: pid
44309 from 172.17.2.2
Jun 19 17:00:43 cardinal qmail-smtpd: 1213909243.176586 tcpserver: ok
44309 :192.168.1.219:25 :172.17.2.2::62104
Jun 19 17:00:43 cardinal qmail-smtpd: 1213909243.486045 tcpserver: end
44309 status 256

So I guess "status 256" is no good. Where should I go for more information?

robm test wrote:
> I'm in the process of setting up a set of qmail & dovecot servers
> distributed across our company's network, with one dmz system as the
> single entry and exit point for mail. It's mostly all working now
> with two sites, but I found an odd problem.
>
> If I send a message from the remote site to my gmail account, and then
> reply to that message, it is accepted by the dmz server but then when
> it is forwarded to the remote internal server I get this:
>
> Jun 19 16:18:14 buster qmail-send: 1213906694.836116 delivery 15908:
> deferral:
> Connected_to_192.168.1.219_but_connection_died._Possible_duplicate!_(#4.4.2)/
>
> I don't see any evidence of the connection on the destination server.
> Successful incoming messages are logged as usual.
>
> Yet when I just send a message from my gmail account directly to the
> user at the remote site it works fine. This is reproducable. A reply
> to a second message fails, and incoming mail from gmail works fine.
>
> What could be happening here? Thanks in advance.
>
>
> One message "stuck" in the queue looks like this:
>
> --------------
> MESSAGE NUMBER 94624
> --------------
> Received: (qmail 27738 invoked by alias); 19 Jun 2008 16:16:44 -0400
> Delivered-To: robm@m2.seamanpaper.com
> Received: (qmail 27735 invoked from network); 19 Jun 2008 16:16:44
> -0400
> Received: from rv-out-0506.google.com (209.85.198.230)
> by buster.seamanpaper.com with SMTP; 19 Jun 2008 16:16:44 -0400
> Received-SPF: pass (buster.seamanpaper.com: SPF record at
> _spf.google.com designates 209.85.198.230 as permitted sender)
> Received: by rv-out-0506.google.com with SMTP id f6so7319385rvb.5
> for <robm@m2.seamanpaper.com>; Thu, 19 Jun 2008 13:16:42
> -0700 (PDT)
> DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
> d=gmail.com; s=gamma;
>
> h=domainkey-signature:received:received:message-id:date:from:reply-to
> :to:subject:in-reply-to:mime-version:content-type:references;
> bh=7eb4+qyKj7gnc31TVvEkrV/arN9VXFNySe5rfb+u21M=;
>
> b=gvVeGWT5IwXBRAjXAxvSsVtHwgcUuqoNHCp2pDVEvabgSyBNbfM2lm72/3R9pcJkXo
>
> iop0Shj+UdnoGON2XtmHNo516sOaSvhCLV40RQ7dKM4rItXZl79grZQw/d792Dsgx8bm
> xDutCLJ3Ccfe5ynp8FlhEq9iHzBZZjxat0Ci8=
> DomainKey-Signature: a=rsa-sha1; c=nofws;
> d=gmail.com; s=gamma;
>
> h=message-id:date:from:reply-to:to:subject:in-reply-to:mime-version
> :content-type:references;
>
> b=Bf7VKxnEiIuF7Wj9YH7eEc6RPKqF3qsjlzi5/CMG3UMFkcQ6VWf/2MEyXQnkm5pTQn
>
> MesiHGLgWeFRWOfyF+ntcpyU5QxBHMB6YGJ7/0yqVfdmhJ+eYX99fzdb9IwT9NbkrpGB
> ksr0PJc4tWYk1pEjnsEGc+S38/FfopQNpOlAA=
> Received: by 10.141.89.13 with SMTP id
> r13mr6959097rvl.177.1213906602574;
> Thu, 19 Jun 2008 13:16:42 -0700 (PDT)
> Received: by 10.141.206.17 with HTTP; Thu, 19 Jun 2008 13:16:42
> -0700 (PDT)
> Message-ID:
> <588667830806191316g7d2e1b9ev26be3cbb320fa1b6@mail.gmail.com>
> Date: Thu, 19 Jun 2008 16:16:42 -0400
> From: "Jeff Dickens" <dreamgear@gmail.com>
> Reply-To: dickens@dreamgear.com
> To: "robm test" <robm@m2.seamanpaper.com>
> Subject: Re: test e1
> In-Reply-To: <485ABA56.9040805@m2.seamanpaper.com>
> MIME-Version: 1.0
> Content-Type: multipart/alternative;
> boundary="----=_Part_6593_2398558.1213906602576"
> References: <485ABA56.9040805@m2.seamanpaper.com>
>
> ------=_Part_6593_2398558.1213906602576
> Content-Type: text/plain; charset=ISO-8859-1
> Content-Transfer-Encoding: 7bit
> Content-Disposition: inline
>
> test f2
>
> On Thu, Jun 19, 2008 at 3:58 PM, robm test
> <robm@m2.seamanpaper.com> wrote:
>
> > not really rob
> >
>
> ------=_Part_6593_2398558.1213906602576
> Content-Type: text/html; charset=ISO-8859-1
> Content-Transfer-Encoding: 7bit
> Content-Disposition: inline
>
> test f2<br><br><div class="gmail_quote">On Thu, Jun 19, 2008 at
> 3:58 PM, robm test <<a
> href="mailto:robm@m2.seamanpaper.com">robm@m2.seamanpaper.com</a>>
> wrote:<br><blockquote class="gmail_quote" style="border-left: 1px
> solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left:
> 1ex;">
> not really rob<br>
> </blockquote></div><br>
>
> ------=_Part_6593_2398558.1213906602576--
>

Re: connected but connection died ? [ In reply to ]

kyle-qmail at memoryhole

Jun 19, 2008, 2:40 PM

Post #4 of 9 (5237 views)

Permalink

On Thursday, June 19 at 05:03 PM, quoth Jeff Dickens:
> I figured out why smtpd wasn't logging. Now I get this:
>
> Jun 19 17:00:43 cardinal qmail-smtpd: 1213909243.173962 tcpserver: pid
> 44309 from 172.17.2.2
> Jun 19 17:00:43 cardinal qmail-smtpd: 1213909243.176586 tcpserver: ok
> 44309 :192.168.1.219:25 :172.17.2.2::62104
> Jun 19 17:00:43 cardinal qmail-smtpd: 1213909243.486045 tcpserver: end
> 44309 status 256
>
> So I guess "status 256" is no good. Where should I go for more information?

Status 256 means, if I'm not mistaken, that whatever program tcpserver
ran exited with a return code of 1 (which doesn't really tell you
much).

Try and re-create sending a message from the one computer to the
other. Use telnet and see what happens (you can use my example
conversation here:
http://www.memoryhole.net/~kyle/index.php?section=reference&inc=smtp).

You could also try making your recipient qmail system to do more
verbose logging, either with recordio or with a logging patch (such as
http://www.memoryhole.net/qmail/logging.patch), or both, in order to
find out exactly what's going on.

~Kyle
--
We live at the intersection of mysterious freedoms, God's and our own.
-- Jesuit Fr. Daniel Berrigan

Re: connected but connection died ? [ In reply to ]

jeff at m2

Jun 20, 2008, 8:33 AM

Post #5 of 9 (5215 views)

Permalink

The latest is that it doesn't happen when I receive a reply from another
system. So far it only happens with replies from gmail.

I verified that I'm running an identical set of qmail binaries and a
nearly identical configuration as on a system at a different site that
does work.

Re: connected but connection died ? [ In reply to ]

kyle-qmail at memoryhole

Jun 20, 2008, 8:57 AM

Post #6 of 9 (5220 views)

Permalink

On Friday, June 20 at 11:33 AM, quoth Jeff Dickens:
> The latest is that it doesn't happen when I receive a reply from another
> system. So far it only happens with replies from gmail.
>
> I verified that I'm running an identical set of qmail binaries and a
> nearly identical configuration as on a system at a different site that
> does work.

Hmmm... So, what's your run file look like on the system that dies?

~Kyle
--
The most important thing a father can do for his children is to love
their mother.
-- Fr. Theodpre Hesburgh

Re: connected but connection died ? [ In reply to ]

jeff at m2

Jun 20, 2008, 10:09 AM

Post #7 of 9 (5225 views)

Permalink

Kyle Wheeler wrote:
> On Friday, June 20 at 11:33 AM, quoth Jeff Dickens:
>> The latest is that it doesn't happen when I receive a reply from
>> another system. So far it only happens with replies from gmail.
>>
>> I verified that I'm running an identical set of qmail binaries and a
>> nearly identical configuration as on a system at a different site
>> that does work.
>
> Hmmm... So, what's your run file look like on the system that dies?
>
> ~Kyle
/service/qmail-smtpd/run:

#!/bin/sh
# qmail-smtpd/run
CONLIMIT=30
exec 2>&1
echo "*** Starting qmail-smtpd..."
exec \
envuidgid qmaild \
softlimit -m 3000000 -f 20000000 \
tcpserver -v -HR \
-c ${CONLIMIT} \
-x /etc/tcprules/smtp.cdb \
0 25 \
/var/qmail/bin/qmail-smtpd

/service/qmail-smtpd/log/run:

#!/bin/sh
exec setuidgid qmaill /var/qmail/bin/splogger qmail-smtpd

I'm running qmail-1.03_7 from the FreeBSD ports collection on the
internal servers.

Re: connected but connection died ? [ In reply to ]

kyle-qmail at memoryhole

Jun 20, 2008, 10:47 AM

Post #8 of 9 (5232 views)

Permalink

On Friday, June 20 at 01:09 PM, quoth Jeff Dickens:
>> Hmmm... So, what's your run file look like on the system that dies?
> /service/qmail-smtpd/run:
>
> #!/bin/sh
> # qmail-smtpd/run
> CONLIMIT=30
> exec 2>&1
> echo "*** Starting qmail-smtpd..."
> exec \
> envuidgid qmaild \
> softlimit -m 3000000 -f 20000000 \
> tcpserver -v -HR \
> -c ${CONLIMIT} \
> -x /etc/tcprules/smtp.cdb \
> 0 25 \
> /var/qmail/bin/qmail-smtpd

Hrm... nothing obvious. Your softlimit might be a bit too low (for
example, the DNS resolver library often requires quite a bit of
memory)... try increasing your softlimit dramatically (say, 100x)
temporarily to see whether that changes anything; if you can then
receive mail properly, then we know you need to increase your memory
limit). Have you patched qmail with anything?

Is there a reason you use softlimit’s -f patch instead of just setting
/var/qmail/control/databytes?

~Kyle
--
It is not bigotry to be certain we are right; but it is bigotry to be
unable to imagine how we might possibly be wrong.
-- Gilbert Chesterton

Re: connected but connection died ? [ In reply to ]

jeff at m2

Jul 3, 2008, 12:04 PM

Post #9 of 9 (4993 views)

Permalink

Kyle Wheeler wrote:
> On Thursday, June 19 at 05:03 PM, quoth Jeff Dickens:
>> I figured out why smtpd wasn't logging. Now I get this:
>>
>> Jun 19 17:00:43 cardinal qmail-smtpd: 1213909243.173962 tcpserver:
>> pid 44309 from 172.17.2.2
>> Jun 19 17:00:43 cardinal qmail-smtpd: 1213909243.176586 tcpserver: ok
>> 44309 :192.168.1.219:25 :172.17.2.2::62104
>> Jun 19 17:00:43 cardinal qmail-smtpd: 1213909243.486045 tcpserver:
>> end 44309 status 256
>>
>> So I guess "status 256" is no good. Where should I go for more
>> information?
>
> Status 256 means, if I'm not mistaken, that whatever program tcpserver
> ran exited with a return code of 1 (which doesn't really tell you much).
>
> Try and re-create sending a message from the one computer to the
> other. Use telnet and see what happens (you can use my example
> conversation here:
> http://www.memoryhole.net/~kyle/index.php?section=reference&inc=smtp).
>
> You could also try making your recipient qmail system to do more
> verbose logging, either with recordio or with a logging patch (such as
> http://www.memoryhole.net/qmail/logging.patch), or both, in order to
> find out exactly what's going on.
Your logging patch helped me track this down.. it turns out that an
inspection rule on a Cisco ASA is sporadically blocking TCP/SMTP packets
with "no connection". I've got a TAC call open to track down why.
Until then I can just have it not inspect SMTP, I guess.

Thanks.
>
> ~Kyle

Mailing List Archive

Attached Files:

Attached Files:

Attached Files:

Attached Files: