| occasional spike in 502 while running jetty behind AWS elb [message #1777968] |
Thu, 07 December 2017 04:01 |
 Abhishek Gupta
Messages: 1
Registered: December 2017 |
Junior Member |
|
|
Please excuse my lack of proper terminology here. I am trying to debug something that I don't understand well enough.
We are running about 18 Java server with 9.4.0.v20161208 and jersey 2.25. We have configured 50 threads(both min and max value) for a 4 core CPU ( of which 3 acceptors and 4 selectors) .
AWS elb show occasional spike in 502 error from these servers, about 0.5% of that traffic (800k rpm). From ELB logs we infer that the instance is not serving accepted requests or the request gets mal formed response. (within seconds).
These spikes happen more when our traffic is high but there is no sign of lack of worker threads or cpu resources , adding extra boxes did not make any difference at all. Java api processing time for all the served requests remains almost the same(~30 ms)
Jetty configuration is
CPUs: 4
Acceptors : 3
Selectors : 4
Thread pool capacity = 50 (min and max)
IdleTimeout = 30000
netstat shows that then number of incoming established connections to the process is ~350.
Looking though the code it looks like number of pending quests can only be size of the blocking queue , which should be 50. So I would expect about 100 established connections (50 pending and 50 being served) .
So what does this high number of established connections indicate ?
Also I see that number of connection is TIME_WAIT varies between 200-1900 , what could that indicate ?
Any suggestion on how do avoid/debug elb 502 responses.
|
|
|
Powered by
FUDForum. Page generated in 0.02900 seconds
] 
Search
Help
Register
Login
Home
