The "keep-alive" token is only for HTTP/1.0, and even then the meaning is backwards from HTTP/1.1, which hints at the old nature of your test cases.
I see no HTTP version tokens after the request URI in the example requests. (But i see it being parsed, I need to look into that log output on our side to see if its our fault)
That shows you that the connection went from being normal "Open,in,out" to having its input closed "Open,ISHUT,out", due to the client closing the connection.
The parser then detected this "HttpParser.parseNext() -> HttpChannelOverHttp.earlyEOF()" and closed the connection.
It looks like that is the one that triggered the rest of the issues you had.
As for your description of the testing "consecutive requests are fired one after the other without any idle time in between"
That doesn't convey enough detail to know how you are producing those requests.
We can assume you are attempting to do it all pipelined as we can see the bad "Connection: keep-alive" header.
The client you are using might support that, but it also might be sending them as separate request on different connections.
If you have a client that does request/response/exchange timeouts, you might be hitting an issue with the pipelined nature of your testing. The requests on a pipelined connection are processed sequentially meaning that the response latency for each request increases with the number of outstanding requests you have on that pipelined connection, resulting in high probability of timeout/connection close by the client.
Also, seeing a DELETE in your logs makes me think your testing is not replicating your real world scenario.
DELETE is usually seen from non-browser clients, which typically don't use persistent/pipelined connections.
If you want to use persistent/pipelined connections, make sure your testing uses it properly.
Use HTTP/1.1 only.
Only add the "Connection" header for the last request via a "Connection: close".
(or if you have upgraded connections, then "Connection: upgrade" is valid, but that also means the Connection is no longer HTTP after that request)
Understand the nature of the connection state in HTTP/1.1 (who closes and why).
Try to limit your outstanding requests on that one Connection to something reasonable, or use multiple Connections.
- Joakim