Hi all,
Yesterday Jetty 7.3.1 on Windows Server 2008 crashed (see below),
and at the same time false
system alerts on disk space occurred (seen by email alerts).
Disk capacity was only at 50% of total 80GB capacity. I run a
Shibboleth IdP
(version 2.3.8) under Jetty. I need to understand what
happened in the hope of preventing further re-occurrence.
There didn't appear to
be any general network issues. Nothing unusual in the Shibboleth
logs, and nothing about heap size in the Jetty logs.
The event logs
contained:
Error - Application
Error - 14:28:52
Faulting application
name: JettyService.exe, version: 1.0.10.0, time stamp: 0x4f4600a2
Faulting module name:
ntdll.dll, version: 6.1.7601.17725, time stamp: 0x4ec4aa8e
Exception code:
0xc0000005
Fault offset:
0x0000000000053210
Faulting process id:
0x214
Faulting application
start time: 0x01cedf8048792d0c
Faulting application
path: c:\jetty-distribution-7.3.1.v20110307\bin\JettyService.exe
Faulting module path:
C:\Windows\SYSTEM32\ntdll.dll
Report Id:
12552a2a-56a7-11e3-b01b-0050568b0d9a
Warning - srv -
26/11/2013 14:36
The C: disk is at or
near capacity. You may need to delete some files.
Error - WMI -
26/11/2013 14:34:50
Event filter with query
"SELECT * FROM __InstanceModificationEvent WITHIN 60 WHERE
TargetInstance ISA "Win32_Processor" AND
TargetInstance.LoadPercentage > 90" could not be reactivated
in namespace "//./root/CIMV2" because of error 0x80041003.
Events cannot be delivered through this filter until the problem is
corrected.
And this one which I
assumed to be as a result of temporary insufficient disk capacity:
Error - VSS -
26/11/2013 14:32:38
Volume Shadow Copy
Service information: The COM Server with CLSID
{4e14fba2-2e22-11d1-9964-00c04fbbb345} and name CEventSystem cannot
be started. [0x800705aa, Insufficient system resources exist to
complete the requested service.
In Jettyservice-stderr
lots of "SSL renegotiate denied", which I understand aren't
serious:
2013-11-26
11:55:49.368:WARN::o.e.j.io.nio.SslSelectChannelEndPoint#checkRenegotiate(SslSelectChannelEndPoint.java:628):SSL
renegotiate
denied: java.nio.channels.SocketChannel[connected
local=/143.117.16.127:443 remote=/84.13.201.152:54003]
and:
2013-11-26
11:43:41.103:WARN::o.e.j.io.nio.SslSelectChannelEndPoint#fill(SslSelectChannelEndPoint.java:444):javax.net.ssl.SSLException:
Unrecognized
SSL message, plaintext connection?
It looks like these jetty errors go
back for days and nothing that stands out as different at the time
of
failure.
Can anyone shed any
further insight as to what may have happened ?
JVM heap space if configured to 4096M,
Any help in understanding what caused this or how I can avoid it in
future would be much appreciated,
Many thanks,
Karen
--
Karen Murphy
Systems Analyst -
Bibliographic Services
The Library at Queen's
Queen's University
Belfast
Belfast BT7 1LP
Tel: 028 90976260
Email:
k.l.murphy@xxxxxxxxx
|