Apache 2 high weirdness

Mon, 18 Nov 2002 16:36:07 -0700

I have apache 2 working in several production environments and am now 
ready to move it into production on a new bank of servers. The servers 
are unusual as they are virtualized by a product called virtuozzo, think 
usermode linux. Anyway apache 2 runs fine on the test servers I have set 
up but when moved to these environments exhibit some very stange behavior.

The environment is a updated, modified and hardened redhat 7.1/7.3 and 
the apache 2 packages I have tried are internal (50km version 2.0.43) 
and the redhat 8 2.0.40 compiled for the 7.3 environment. Both work fine 
in a test environment under kernel 2.4.2-2 or 2.4.9. The servers are 
running a virtuozzo provided 2.4.9 (I have no easy control over this) so 
really the only difference is in the kernel from test production 
systems. Here is where things get strange.

Apache installs and works fine through https (mod_ssl 2.0.40 or 2.0.43) 
or using php (4.2.3) to generate pages. However when it comes to serving 
regular static pages over http it fails. The nature of this failure is such:

cannot serve a static page over 255 bytes

The telnet log shows this behavior:

Escape character is '^]'.
GET /index5.html HTTP/1.0

HTTP/1.1 200 OK
Date: Mon, 18 Nov 2002 22:58:10 GMT
Server: Apache/2.0.40 (Red Hat Linux)
Last-Modified: Mon, 18 Nov 2002 07:48:15 GMT
ETag: "544064-940-66de0dc0"
Accept-Ranges: bytes
Content-Length: 2368
Connection: close
Content-Type: text/html
X-Pad: avoid browser bug

with a file under 255 bytes:

Escape character is '^]'.
GET /index.html HTTP/1.0

HTTP/1.1 200 OK
Date: Mon, 18 Nov 2002 23:01:13 GMT
Server: Apache/2.0.40 (Red Hat Linux)
Last-Modified: Mon, 18 Nov 2002 07:46:20 GMT
ETag: "544068-9b-60034b00"
Accept-Ranges: bytes
Content-Length: 155
Connection: close
Content-Type: text/html

<html>
<body>
This is test content. [other sentence redacted]
</body>
</html>
Connection closed by foreign host.

So the difference is the X-pad: avoid browser bug line ??
after exhaustive searching I haven't been able to find out much about 
that other than it is normal in some circumstances. Anyway apache feels 
the need to erroneously close the connection. This behavior also 
generates an error in the logs as follows

[info] (22)Invalid argument: core_output_filter: writing data to the network

This error would typically occur if the browser user were to hit the 
stop button, but that is not the case here.

It gets weirder, mozilla and internet explorer can get http pages over 
255 bytes if keepalives are set on, however gets for images are closed 
at 0 byte, with the above error. I can't for the life of me figure out 
what is going on, except to conclude it is realted to very subtle things 
in the kernel, the problem is that it would be a huge pain for me to 
change the kernel again or an apache bug. I am happy to try anything 
version/compile/package wise as I would really like to move these 
servers over to apache 2. I have reinstalled, recompiled, upgraded glibc 
(currently 2.2.4-31), upgrade the compiler I compiled apache with 
(currently gcc-2.96-112.7.1). Let me know if you have any leads.

I have pretty exhaustively search google, deja, apache mailng lists etc.
All i found was this deja link 
http://groups.google.com/groups?q=%22apache+2%22+%22avoid+browser+bug%22&hl=en&lr=&ie=UTF-8&oe=UTF-8&filter=0 
to an earlier bug with php and apache. That post leads me to believe 
this is an apache bug. But that illustrates the real weirdness here that 
php and anything over ssl work fine, it is just static html that has the 
problem.

Thanks for the help.

Sincerely,
David Uhlman
CTO 50km Inc.