HI Matthieu,
do you have any more log information from before that crash? It
looks like it happened during a young generation garbage collection.
Your eden space (where all new objects are placed in memory) was
full and there was not much space (probably not enough) left in the
old generation for objects which survived the young generation gc.
The interesting part of your crash dump indicating this, is:
Heap
PSYoungGen total 417088K, used 417056K [0x00007f04eb080000,
0x00007f050a3f0000, 0x00007f0515b20000)
eden space 416960K, 100% used
[0x00007f04eb080000,0x00007f05047b0000,0x00007f05047b0000)
from space 128K, 75% used
[0x00007f05055d0000,0x00007f05055e8000,0x00007f05055f0000)
to space 47232K, 0% used
[0x00007f05075d0000,0x00007f05075e8000,0x00007f050a3f0000)
PSOldGen total 529920K, used 377402K [0x00007f0495b20000,
0x00007f04b60a0000, 0x00007f04eb080000)
object space 529920K, 71% used
[0x00007f0495b20000,0x00007f04acbae9b0,0x00007f04b60a0000)
PSPermGen total 21504K, used 20440K [0x00007f048b320000,
0x00007f048c820000, 0x00007f0495b20000)
object space 21504K, 95% used
[0x00007f048b320000,0x00007f048c7163a8,0x00007f048c820000)
As solr/lucene usage can be quite memory intensive it's likely that
your app simply ran out of memory. Maybe during indexing. So your
next steps will be to find out what the application was doing before
the crash and think about what might have caused it to run oom. If
you can't find the root cause this way, you should try to get a heap
dump and analyze what was consuming memory.
Try setting these JVM options
(http://www.oracle.com/technetwork/java/javase/tech/vmoptions-jsp-140102.html)
in your startup script:
-XX:HeapDumpPath=./java_pid<pid>.hprof Path to directory or
filename for heap dump. Manageable. (Introduced in 1.4.2
update 12, 5.0 update 7.) -XX:-HeapDumpOnOutOfMemoryError Dump heap
to file when java.lang.OutOfMemoryError is thrown. Manageable.
(Introduced in 1.4.2 update 12, 5.0 update 7.)
-XX:-PrintGCDetails Print more details at garbage collection. Manageable.
(Introduced in 1.4.0.) -XX:-PrintGCTimeStamps Print timestamps at
garbage collection. Manageable (Introduced in 1.4.0.)
Again it might be normal and that your application simply needs a
bigger heap. Your current settings are: "jvm_args: -Xms512M
-Xmx2048M". If you don't have more memory available you might need
to rethink your indexing strategy (if the oom happened during
indexing). But it might as well be that you've got a memory leak of
some kind. You'll have to read your log files and try to get a heap
dump.
As indexing is memory intensive, proper tuning of your jvm memory
settings is crucial. Normally you want a big newsize and the overall
heap space as big as you have physical memory. But that heavily
depends on your indexing strategy.
Hope I did not confuse you too much. Please not that this is most
certainly NOT a jetty issue.
Cheers,
Thomas
On 04/04/2011 12:20, Matthieu Huin wrote:
Greetings
all,
I am currently using solr as the backend behind a log aggregation
and
search system my team is developing. All was well and good until I
noticed a test server crashing quite unexpectedly. We'd like to
dig more
into the incident but none of us has much experience with Jetty
crash
logs - not to mention that our Java is very rusty.
The crash log is joined as an attachment.
Could anyone help us with understanding what went wrong there ?
Also, would it be possible and/or wise to automatically restart
the
server in case of such a crash ?
Thanks for your help. If you need any extra info about that case,
do not
hesitate to ask !
Matthieu Huin
( Resending this message as I used to wrong e-mail to send it,
sorry for the inconvenience)
_______________________________________________
jetty-users mailing list
jetty-users@xxxxxxxxxxx
https://dev.eclipse.org/mailman/listinfo/jetty-users
|