Skip to main content
[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index] [List Home]
Re: [jetty-users] curl: (56) Unexpected EOF
I'm guessing the problem is the way you make your fat jar you must not be correctly combining the META-INF/services/org.eclipse.jetty.http.HttpFieldPreEncoder files

To run both http1 and http2, you need that file to exist in the jar and to contain:

org.eclipse.jetty.http.Http1FieldPreEncoder
org.eclipse.jetty.http2.hpack.HpackFieldPreEncoder

On Sat, 5 Oct 2019 at 07:35, Joakim Erdfelt <joakim@xxxxxxxxxxx> wrote:
Interesting, I wonder if you are hitting the response headers issues with regards to HTTP/2.

See:
and https://github.com/eclipse/jetty.project/issues/4115

Can you run curl again, but with HTTP/1.1?

$ curl --http1.1 --verbose ....

What kind of headers are you actually using?

Joakim Erdfelt / joakim@xxxxxxxxxxx


On Fri, Oct 4, 2019 at 4:04 PM Glen Peterson <glen.k.peterson@xxxxxxxxx> wrote:
I changed the logging configuration from INFO to DEBUG and found this error when Jetty tries to serve the request:

Sending HeadersFrame@676f5d6c#1{end=false}
Appended HeadersFrame@676f5d6c#1{end=false}, entries=1
Flushing HTTP2ServerSession@74a58163{l:/0:0:0:0:0:0:0:1:8443 <-> r:/0:0:0:0:0:0:0:1:53320,sendWindow=15728640,recvWindow=1048576,streams=1,NOT_CLOSED,null}
Processing HeadersFrame@676f5d6c#1{end=false}
CtxTbl[3319f232] encoding
encode IdxFieldS1:':status: 200' to '88'
Failure generating HeadersFrame@676f5d6c#1{end=false}

java.lang.ArrayIndexOutOfBoundsException: Index 1 out of bounds for length 1

at org.eclipse.jetty.http.PreEncodedHttpField.putTo(PreEncodedHttpField.java:118)
at org.eclipse.jetty.http2.hpack.HpackEncoder.encode(HpackEncoder.java:316)
at org.eclipse.jetty.http2.hpack.HpackEncoder.encode(HpackEncoder.java:204)
at org.eclipse.jetty.http2.generator.HeadersGenerator.generateHeaders(HeadersGenerator.java:72)
at org.eclipse.jetty.http2.generator.HeadersGenerator.generate(HeadersGenerator.java:56)
at org.eclipse.jetty.http2.generator.Generator.control(Generator.java:80)
at org.eclipse.jetty.http2.HTTP2Session$ControlEntry.generate(HTTP2Session.java:1250)
at org.eclipse.jetty.http2.HTTP2Flusher.process(HTTP2Flusher.java:187)
at org.eclipse.jetty.util.IteratingCallback.processing(IteratingCallback.java:241)
at org.eclipse.jetty.util.IteratingCallback.iterate(IteratingCallback.java:223)
at org.eclipse.jetty.http2.HTTP2Session.frame(HTTP2Session.java:757)
at org.eclipse.jetty.http2.HTTP2Session.frames(HTTP2Session.java:727)
at org.eclipse.jetty.http2.HTTP2Stream.headers(HTTP2Stream.java:112)
at org.eclipse.jetty.http2.server.HttpTransportOverHTTP2.sendHeadersFrame(HttpTransportOverHTTP2.java:270)
at org.eclipse.jetty.http2.server.HttpTransportOverHTTP2.send(HttpTransportOverHTTP2.java:142)
at org.eclipse.jetty.server.HttpChannel.sendResponse(HttpChannel.java:852)
at org.eclipse.jetty.server.HttpChannel.write(HttpChannel.java:929)
at org.eclipse.jetty.server.HttpOutput.write(HttpOutput.java:250)
at org.eclipse.jetty.server.HttpOutput.write(HttpOutput.java:226)
at org.eclipse.jetty.server.HttpOutput.close(HttpOutput.java:308)
at org.eclipse.jetty.server.HttpWriter.close(HttpWriter.java:47)
at org.eclipse.jetty.server.ResponseWriter.close(ResponseWriter.java:161)
at org.eclipse.jetty.server.Response.closeOutput(Response.java:844)
at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:517)
at org.eclipse.jetty.server.HttpChannel.run(HttpChannel.java:314)
at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:336)
at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:313)
at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:171)
at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produce(EatWhatYouKill.java:135)
at org.eclipse.jetty.http2.HTTP2Connection.produce(HTTP2Connection.java:170)
at org.eclipse.jetty.http2.server.HTTP2ServerConnection.onOpen(HTTP2ServerConnection.java:150)
at org.eclipse.jetty.io.AbstractEndPoint.upgrade(AbstractEndPoint.java:442)
at org.eclipse.jetty.server.NegotiatingServerConnection.onFillable(NegotiatingServerConnection.java:130)
at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311)
at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:103)
at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:426)
at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:320)
at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:158)
at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:103)
at org.eclipse.jetty.io.ChannelEndPoint$2.run(ChannelEndPoint.java:117)
at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:336)
at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:313)
at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:171)
at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:129)
at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:367)
at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:782)
at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:918)
at java.base/java.lang.Thread.run(Thread.java:834)
Failing, entries processed/pending/queued=0/1/0

I re-posted this version here:

On Wed, Oct 2, 2019 at 5:40 PM Glen Peterson <glen.k.peterson@xxxxxxxxx> wrote:
Yes Joakim.  I just made one and posted it here:

On Wed, Oct 2, 2019 at 2:43 PM Joakim Erdfelt <joakim@xxxxxxxxxxx> wrote:
Do you have a project that you can share that replicates this?

Joakim Erdfelt / joakim@xxxxxxxxxxx


On Wed, Oct 2, 2019 at 1:16 PM Glen Peterson <glen.k.peterson@xxxxxxxxx> wrote:
P.S. I was able to get some interesting output from wget2 (successor to wget).  It works fine running with the classes.  With the jar file, I get the following which seems jetty-related:

$ wget2 -d --no-check-certificate https://dev.memoryjoggerlibrary.com:8443/
02.140606.876 name=check-certificate value=https://dev.memoryjoggerlibrary.com:8443/ invert=1
02.140606.876 name=check-certificate value=https://dev.memoryjoggerlibrary.com:8443/ invert=1
02.140606.876 Local URI encoding = 'UTF-8'
02.140606.876 Input URI encoding = 'UTF-8'
02.140606.876 Fetched HSTS data from '/home/gpeterso/.wget-hsts'
02.140606.876 Fetched HPKP data from '/home/gpeterso/.wget-hpkp'
02.140606.877 add TLS session data for dev.memoryjoggerlibrary.com (maxage=64800, size=1260)
02.140606.877 Fetched TLS session data from '/home/gpeterso/.wget-session'
02.140606.877 Fetched OCSP hosts from '/home/gpeterso/.wget-ocsp_hosts'
02.140606.877 Fetched OCSP fingerprints from '/home/gpeterso/.wget-ocsp'
02.140606.877 *url = "" href="https://dev.memoryjoggerlibrary.com:8443/" target="_blank">https://dev.memoryjoggerlibrary.com:8443/
02.140606.877 *3 https://dev.memoryjoggerlibrary.com:8443/
02.140606.877 local filename = 'index.html'
02.140606.877 host_add_job: job fname index.html
02.140606.877 host_add_job: 0x55b15a6bfb60 https://dev.memoryjoggerlibrary.com:8443/
02.140606.877 host_add_job: qsize 1 host-qsize=1
02.140606.877 queue_size: qsize=1
02.140606.877 queue_size: qsize=1
02.140606.877 queue_size: qsize=1
02.140606.877 [0] action=1 pending=0 host=0x0
02.140606.877 qsize=1 blocked=0
02.140606.877 pause=-1570039566877
02.140606.877 dequeue job https://dev.memoryjoggerlibrary.com:8443/
02.140606.877 resolving dev.memoryjoggerlibrary.com:8443...
02.140606.877 has 127.0.0.1:8443
02.140606.877 Add dns cache entry dev.memoryjoggerlibrary.com
02.140606.877 trying 127.0.0.1:8443...
02.140606.877 GnuTLS init
02.140606.877 Certificates loaded: -1
02.140606.877 GnuTLS init done
02.140606.877 TLS False Start requested
02.140606.877 ALPN offering h2
02.140606.877 ALPN offering http/1.1
02.140606.877 found cached session data for dev.memoryjoggerlibrary.com
WARNING: The certificate is NOT trusted. The certificate issuer is unknown.
02.140606.885 TLS False Start: on
02.140606.885 ALPN: Server accepted protocol 'h2'
02.140606.885 Handshake completed
02.140606.885 established connection dev.memoryjoggerlibrary.com
02.140606.885 cookie_create_request_header for host=dev.memoryjoggerlibrary.com path=
02.140606.885 HTTP2 stream id 1
02.140606.885 [0] action=1 pending=1 host=0x55b15a6bf9c0
02.140606.885 qsize=1 blocked=0
02.140606.885 pause=-1570039566885
02.140606.885 [0] action=2 pending=1 host=0x55b15a6bf9c0
02.140606.885   ##  pending_requests = 1
02.140606.885   ##  loop responses=0
02.140606.885 [FRAME 0] > SETTINGS
02.140606.885 [FRAME 1] > HEADERS
02.140606.885 [FRAME 1] > :method: GET
02.140606.885 [FRAME 1] > :path: /
02.140606.885 [FRAME 1] > :scheme: https
02.140606.885 [FRAME 1] > :authority: dev.memoryjoggerlibrary.com
02.140606.885 [FRAME 1] > accept-encoding: gzip, deflate, bzip2, xz, lzma
02.140606.885 [FRAME 1] > accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
02.140606.885 [FRAME 1] > user-agent: wget2/1.0.0
02.140606.885   ##  loop responses=0
02.140606.886 Got delayed session data
02.140606.886 found TLS session data for dev.memoryjoggerlibrary.com
02.140606.886 removed TLS session data for dev.memoryjoggerlibrary.com
02.140606.886 add TLS session data for dev.memoryjoggerlibrary.com (maxage=64800, size=1260)
02.140606.886   ##  loop responses=0
02.140606.886 [FRAME 0] < SETTINGS
02.140606.886 [FRAME 0] < WINDOW_UPDATE
02.140606.886   ##  loop responses=0
02.140606.886 [FRAME 0] > SETTINGS
02.140606.886   ##  loop responses=0
02.140606.886 [FRAME 0] < SETTINGS
02.140606.886   ##  loop responses=0
02.140608.260   ##  loop responses=0
02.140608.260   ##  loop responses=0
02.140608.260   ##  loop responses=0
02.140608.260   ##  loop responses=0
02.140608.260   ##  loop responses=0
02.140608.260   ##  loop responses=0
02.140608.260   ##  loop responses=0
02.140608.260   ##  loop responses=0
02.140608.260   ##  loop responses=0
02.140608.260   ##  loop responses=0
02.140608.260   ##  loop responses=0

That scrolled really fast in a loop until I hit CTRL-C

02.140608.260 host_increase_failure: dev.memoryjoggerlibrary.com failures=1
02.140608.260 closing connection

It just sat there doing nothing, so I hit CTRL-C again to exit


On Wed, Oct 2, 2019 at 1:43 PM Glen Peterson <glen.k.peterson@xxxxxxxxx> wrote:
Thank you for jetty.  I've been using it for a few years now and love it.  Especially that I can configure it programmatically (with Kotlin code) and that you have updated the SSL/TLS protocol defaults over time.  Such an improvement over Tomcat!

Problem
My problem may be jar-file/classpath related and not jetty related, but then why does jetty actually call my code, service the request, but not return the response?

I recently enabled HTTP/2 (Google wants that for a Progressive Web App).  Now if I run my app the way IntelliJ does, giving the compiled classes directory, then adding all the jar dependencies it works. `java -classpath classes/:bunch-of-jar-files... MainClassName` it works.  But if I run from the fat-jar file it doesn't work.

curl

Jar File Failure:
$ curl --insecure https://dev.memoryjoggerlibrary.com:8443 -D headers.txt
curl: (56) Unexpected EOF
$ ls -s
total 0
0 headers.txt

Class Files Success:
That same curl command returns the HTML file I was expecting when running with the massive classpath (no fat-jar), plus the following headers.txt:
HTTP/2 200
server: Jetty(9.4.20.v20190813)
content-type: text/html;charset=utf-8

I have determined that the jar files included by maven exactly match the jar files specified on the command line.

Chrome
With chrome, the class files work great, but the Jar file produces, "This site can’t be reached... unexpectedly closed the connection... ERR_CONNECTION_CLOSED"

Unaffected
Log Files:
I can see in the log files in both cases that the request is being processed, but there is no exception thrown and no response apparently written.

Nmap Cipher Test:
Whether it yields HTML or not, the server gives the same response to my cipher test:
$ nmap --script ssl-cert,ssl-enum-ciphers -p 8443 dev.memoryjoggerlibrary.com

Starting Nmap 7.60 ( https://nmap.org ) at 2019-10-02 13:32 EDT
Nmap scan report for dev.memoryjoggerlibrary.com (127.0.0.1)
Host is up (0.000053s latency).
rDNS record for 127.0.0.1: localhost

PORT     STATE SERVICE
8443/tcp open  https-alt
| ssl-cert: Subject: commonName=dev.memoryjoggerlibrary.com/organizationName=PlanBase Inc./stateOrProvinceName=Massachusetts/countryName=US
| Issuer: commonName=dev.memoryjoggerlibrary.com/organizationName=PlanBase Inc./stateOrProvinceName=Massachusetts/countryName=US
| Public Key type: rsa
| Public Key bits: 2048
| Signature Algorithm: sha256WithRSAEncryption
| Not valid before: 2019-09-25T20:57:00
| Not valid after:  2022-09-25T20:57:00
| MD5:   483e 988f 709f 7300 3f3d 1d51 4f95 81cb
|_SHA-1: abe8 c872 c7d7 dfb1 4152 8bf3 2fee 5617 7613 94cd
| ssl-enum-ciphers:
|   TLSv1.2:
|     ciphers:
|       TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256 (rsa 2048) - A
|       TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (rsa 2048) - A
|       TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305_SHA256 (rsa 2048) - A
|     compressors:
|       NULL
|     cipher preference: server
|_  least strength: A

Nmap done: 1 IP address (1 host up) scanned in 0.19 seconds

Details:
I'm using a self-signed certificate.

Currently using (from maven .pom):
<jetty.version>9.4.20.v20190813
jetty-server
jetty-alpn-conscrypt-server
http2-common
http2-hpack
http2-server

Not currently using: conscrypt-openjdk (seems to be overridden by the conscrypt-openjdk-uber included in jetty-alpn-conscrypt-server - same issues with or without this dependency)


--
Glen K. Peterson
(828) 393-0081


--
Glen K. Peterson
(828) 393-0081
_______________________________________________
jetty-users mailing list
jetty-users@xxxxxxxxxxx
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://www.eclipse.org/mailman/listinfo/jetty-users
_______________________________________________
jetty-users mailing list
jetty-users@xxxxxxxxxxx
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://www.eclipse.org/mailman/listinfo/jetty-users


--
Glen K. Peterson
(828) 393-0081


--
Glen K. Peterson
(828) 393-0081
_______________________________________________
jetty-users mailing list
jetty-users@xxxxxxxxxxx
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://www.eclipse.org/mailman/listinfo/jetty-users
_______________________________________________
jetty-users mailing list
jetty-users@xxxxxxxxxxx
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://www.eclipse.org/mailman/listinfo/jetty-users


--
Greg Wilkins <gregw@xxxxxxxxxxx> CTO http://webtide.com

Back to the top