Thanks everyone. What do you think about the effectiveness of using Jetty to tarpit HTTP requests under the following assumptions:
- Jetty is serving a Java web application
- This application has a fairly regular URL pattern which can be used as an "allow-known-good" filter for valid requests.
- Anything failing the known-good test yields a delayed (the "tar pit") 400 response.
I think that at the very least, failing-fast with a very small 400 response (no tarpit) is probably a good thing. Also that tarpitting failed login attempts is always recommended.
If tarpitting has a place for more than failed logins, what do you think of Silvio's suggestion? Would you recommend using ContinuationSupport.getContinuation(request) as detailed here:
It seems that with tarpitting, you are testing your tarpit resources against the attacker's resources. If that's a botnet, Jetty or any Java-based web server is probably the wrong tool. Botnets must be mostly blocked before DoS'ing the web server. But what about the HTTP attacks that get through that? Maybe a slow server (tar-pitted requests) could dissuade some attackers before an attack starts in earnest?
Sounds like you are looking for a tar pit like solution. Although
the effectiveness of that is questionable I have implemented such a
thing in a quite basic way that might suit you.
Put the requests you don't like in async mode with something like
10s timeout to decouple them from the handler thread and push them
on a limited size FIFO queue (I use a simple circular buffer) to
prevent them from hogging your memory. Every 5 seconds or so have a
dedicated thread eat the sufficiently (couple of seconds) old head
part from the queue and complete the requests. When the buffer
overflows purged items are completed also (this makes the handler
thread complete older requests as it pushes the newer ones onto the
queue). This construct places limited constraint on your application
and somewhat stalls the requests.
I use this for tar pitting failed login attempts. To be honest, we
only do this because some customers that perform regular pen tests
on our system prefer we "slow down" rogue login attempts. Perhaps it
will fit your needs also.
Kind regards,
Silvio
On 4/3/20 1:38 AM, Glen Peterson wrote:
Thanks for your help. Sorry to take so long to get
back. All three seem to return some kind of blank response, or
maybe they just close the connection instantly. I was imagining
that there might be a way for the server to literally just leave
the client hanging there waiting for a response and not getting
one.
I'm looking for a response for any request that ends with
".php" or asks for "/user/register." That's just not even a
vaguely valid request for our app and indicates some kind of
script or hacker.
I had previously approximated a truly ignored request by
doing a thread.sleep() for a somewhat random amount of time
before returning 503 - Service Unavailable
SC_SERVICE_UNAVAILABLE, which was the most ambiguous response
I could give. As in, "Yeah, maybe we're a wordpress site
that's just having issues right now."
But that temporarily dedicates a thread for an attacker
that wed rather ignore. Maybe this is why people use a Web
Application Firewall - so that their app doesn't even get
these requests in the first place. The WAF that blocks it
can withstand a DDoS. I'm a little wary of letting a 3rd
party WAF inspect our traffic in the name of security. It's
also another point of failure. Maybe that's as good as it
gets? The good outweighs the bad?
Simone Bordet's is nice and quiet:
baseRequest.httpChannel.endPoint.close()
The other two show exceptions in the logs:
response.sendError(-1, "We don't dignify hacking
attempts with a response.") // abruptly close the
connection
java.lang.NullPointerException
at
org.eclipse.jetty.http2.server.HttpTransportOverHTTP2.retrieveTrailers(HttpTransportOverHTTP2.java:214)
at
org.eclipse.jetty.http2.server.HttpTransportOverHTTP2.send(HttpTransportOverHTTP2.java:179)
at
org.eclipse.jetty.server.HttpChannel.sendResponse(HttpChannel.java:829)
at
org.eclipse.jetty.server.HttpChannel.write(HttpChannel.java:901)
at
org.eclipse.jetty.server.HttpOutput.channelWrite(HttpOutput.java:283)
at
org.eclipse.jetty.server.HttpOutput.complete(HttpOutput.java:479)
at
org.eclipse.jetty.server.Response.completeOutput(Response.java:842)
at
org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:512)
at
org.eclipse.jetty.server.HttpChannel.run(HttpChannel.java:335)
at
org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:336)
at
org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:313)
at
org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:171)
at
org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produce(EatWhatYouKill.java:135)
at
org.eclipse.jetty.http2.HTTP2Connection.produce(HTTP2Connection.java:170)
at
org.eclipse.jetty.http2.HTTP2Connection.onFillable(HTTP2Connection.java:125)
at
org.eclipse.jetty.http2.HTTP2Connection$FillableCallback.succeeded(HTTP2Connection.java:348)
at
org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:103)
at
org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:543)
at
org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:398)
at
org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:161)
at
org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:103)
at
org.eclipse.jetty.io.ChannelEndPoint$2.run(ChannelEndPoint.java:117)
at
org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:336)
at
org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:313)
at
org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:171)
at
org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:129)
at
org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:388)
at
org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:806)
at
org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:938)
at java.base/java.lang.Thread.run(Thread.java:834)
baseRequest.httpChannel.abort(Throwable("Bogus"))
java.lang.NullPointerException
at
org.eclipse.jetty.http2.server.HttpTransportOverHTTP2.retrieveTrailers(HttpTransportOverHTTP2.java:214)
at
org.eclipse.jetty.http2.server.HttpTransportOverHTTP2.send(HttpTransportOverHTTP2.java:179)
at
org.eclipse.jetty.server.HttpChannel.sendResponse(HttpChannel.java:829)
at
org.eclipse.jetty.server.HttpChannel.write(HttpChannel.java:901)
at
org.eclipse.jetty.server.HttpOutput.channelWrite(HttpOutput.java:283)
at
org.eclipse.jetty.server.HttpOutput.complete(HttpOutput.java:479)
at
org.eclipse.jetty.server.Response.completeOutput(Response.java:842)
at
org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:512)
at
org.eclipse.jetty.server.HttpChannel.run(HttpChannel.java:335)
at
org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:336)
at
org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:313)
at
org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:171)
at
org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produce(EatWhatYouKill.java:135)
at
org.eclipse.jetty.http2.HTTP2Connection.produce(HTTP2Connection.java:170)
at
org.eclipse.jetty.http2.server.HTTP2ServerConnection.onOpen(HTTP2ServerConnection.java:150)
at
org.eclipse.jetty.io.AbstractEndPoint.upgrade(AbstractEndPoint.java:442)
at
org.eclipse.jetty.server.NegotiatingServerConnection.onFillable(NegotiatingServerConnection.java:130)
at
org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311)
at
org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:103)
at
org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:543)
at
org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:398)
at
org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:161)
at
org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:103)
at
org.eclipse.jetty.io.ChannelEndPoint$2.run(ChannelEndPoint.java:117)
at
org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:806)
at
org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:938)
at java.base/java.lang.Thread.run(Thread.java:834)
On Tue, Mar 17, 2020 at 4:47
AM Greg Wilkins <gregw@xxxxxxxxxxx> wrote:
and just to add some more options...
if you are using Jetty APIs then calling
abort(Throwble) on the HttpChannel is he best transport
independent way to do a GO_AWAY style semantic. Doing a
sendError(-1) will also trigger an abort under the scenes
(and can be done from a servlet).
Finally, throwing BadMessageException is another way to
handle this. It will send a 400 and then close the
connection.
On Mon, Mar 16, 2020 at 5:13 PM Glen Peterson <glen.k.peterson@xxxxxxxxx>
wrote:
>
> My first choice would be to decide not to respond
from within an AbstractHandler's handle() method, after
examining the (HttpServlet)Request. But if there's
another place we can examine the request (ideally in
Java), that would work too. Right now I've got some code
like:
>
> object MyHandler: AbstractHandler() {
>
> override fun handle(target: String,
> baseRequest: Request,
> request: HttpServletRequest,
> response:
HttpServletResponse) {
>
> val rawPath = request.getPathInfo()
>
> // We don't have any PHP files. Any attempt
to access one is hacking.
> if ( rawPath.endsWith(".php") ) {
> logger.info("BOGUS
Request: [${request.pathInfo}]")