You can write your own custom ErrorHandler for these kinds of errors, but it has to be associated with the Server, not a ServletContextHandler.
Remember, there is no Request object.
Using the Server.setErrorHandler(ErrorHandler) is sufficient to set your custom error handler.
Note that you will need to override and provide your own .badMessageError() implementation.
Keep in mind that it should only produce a body if the status code allows it, and it should only produce 1 ByteBuffer of response body content.
There is absolutely no support for ErrorPage techniques in these kinds of cases.
Do not expect to have headers or anything else in these cases. (a status code only is the majority case).
Also note that this Server level error handler will be called if the incoming legitimate request does not match your servlet context, and that request cannot be dispatched to your context.
(Yes, there are many flavors of HTTP requests that fit this category too)
So it's a good idea to also customize the server level ErrorHandler for these cases too.