Skip to main content
[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index] [List Home]
Re: [jetty-users] Customizing jetty worker threadpool
Hi Joakim,

Thanks a lot for the feedback.

>>In real world practice though, this sort of tweaking of threading tweaking is a form of premature optimization.
Are you sure this can happen?

Yes. We have seen such deadlocks in real-life. The application in this context is the distributed version of Apache Solr search engine. https://wiki.apache.org/solr/DistributedSearch

>>If it has happened, were you using the recommended dynamic sizing thread pool configuration? (like the default QueuedThreadPool does) Fixed size ThreadPool/Executors are not a good choice in your situation.

I am not sure if dynamic sizing can entirely solve this problem. It has the same problem as the QoSFilter in the sense that external requests could potentially occupy the entire thread-pool thereby blocking the internal requests to complete. Thoughts?

At this point I think there are couple of alternatives,

(a) Use separate Connector instances (each backed by a dedicated thread-pool) for internal & external requests.
(b) Implement a Filter similar to QoSFilter - which would provide the reservation capability for different priority levels.

Thanks again :)
-Hrishikesh

--
Joakim Erdfelt <joakim@xxxxxxxxxxx>
Expert advice, services and support fromÂfrom the Jetty & CometD experts

On Thu, May 21, 2015 at 4:44 PM, Hrishikesh Gadre <gadre.solr@xxxxxxxxx> wrote:
Hi Joakim,

Thanks for the feedback. I think QoSFilter is an interesting alternative but I think it does not quite fit the requirement. Let me elaborate my use-case for better understanding.

Every external request typically spawns multiple internal requests for multiple servers (e.g. for scatter/gather queries). Hence it is important to reserve a certain % of thread-pool capacity for internal requests. Without this reservation, it is quite possible that the thread-pool is consumed entirely by external requests - thereby creating a distributed deadlock (since the external request depends upon the result of internal request which can not be processed due to unavailability of threads). 

After reading the docs, I am not sure if QosFilter will be able to provide this reservation capability out-of-box. i.e. it ensures to pick a higher priority request from the backlog but doesn't prevent the system to occupy the entire thread-pool with low-priority requests.

Is my understanding correct? Also can you please elaborate the threading model used with the jetty continuation?

>>Or use the QoSFilter to setup a higher priority for internal vs external requests. :-)

--
Joakim Erdfelt <joakim@xxxxxxxxxxx>
Expert advice, services and support fromÂfrom the Jetty & CometD experts

On Thu, May 21, 2015 at 3:10 PM, Hrishikesh Gadre <gadre.solr@xxxxxxxxx> wrote:
Hi,

I am currently working on an application which comprises of multiple servers. Each server is deployed using a dedicated Jetty instance. Each server accepts HTTP requests from external clients as well as other servers in the system. i.e. the servers form the peer-to-peer system using HTTP protocol.

I am currently working on a feature to separate out internal vs external requests such that a % of worker threads are reserved for internal requests and remaining threads for external ones.

I could think of couple of approaches to solve this problem. Can you please take a look and provide feedback?


(1) Using Servlet 3 specification

The idea would be to define two separate thread-pools internally and submit the asynchronous request based on request type. My understanding here is that it would require an additional thread switch as compared to synchronous request processing (jetty_acceptor -> jetty_selector -> jetty_worker -> app_thread).  Is this accurate? If yes is there a way to avoid this?

(2) Somehow customizing the Jetty implementation such that we reserve a % of jetty worker threads for internal requests and other for external requests. The flow would look like this,
  
  jetty_acceptor -> jetty_selector (Demux) ----> jetty_worker_pool_for_internal
                                                                   |
                                                                   -----> jetty_worker_pool_for_external

The demux here would look at the HTTP request to figure out its type and submit it to appropriate thread-pool. Is this possible? If yes, any pointers?

Any other approach I may have missed?

Thanks in advance,
-Hrishikesh





  


Back to the top