Skip to main content
[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index] [List Home]
Re: [jetty-users] Error with shibboleth-idp and jetty
A simple solution it to move the java tmp directory location out of /tmp.  Other than that, find and disable the tmp sweeper.

cheers

On 23 May 2017 at 08:06, Pritha Srivastava <prsrivas@xxxxxxxxxx> wrote:

----- Original Message -----
> From: "Pritha Srivastava" <prsrivas@xxxxxxxxxx>
> To: "JETTY user mailing list" <jetty-users@xxxxxxxxxxx>
> Sent: Tuesday, May 23, 2017 10:33:24 AM
> Subject: Re: [jetty-users] Error with shibboleth-idp and jetty
>
>
> ----- Original Message -----
> > From: "Joakim Erdfelt" <joakim@xxxxxxxxxxx>
> > To: "JETTY user mailing list" <jetty-users@xxxxxxxxxxx>
> > Sent: Tuesday, May 23, 2017 4:20:58 AM
> > Subject: Re: [jetty-users] Error with shibboleth-idp and jetty
> >
> > This is the source of your Apache Jasper warning ...
> >
> > 2017-04-20 05:50:06.059:WARN:oaj. EmbeddedServletOptions: qtp987405879-11:
> > The scratchDir you specified: /tmp/jetty-0.0.0.0-443-idp. war-_idp-any-
> > 6580006879402156844.dir/jsp is unusable.
> >
> > https://github.com/apache/tomcat85/blob/trunk/java/org/apache/jasper/EmbeddedServletOptions.java#L688-L691
> >
> > Your /tmp/jetty-0.0.0.0-443-idp. war-_idp-any- 6580006879402156844.dir/jsp
> > path fails one of the following checks ....
> >
> >
> >     * must exist
> >     * is a directory
> >     * jetty user should be able to read from that directory
> >     * jetty user should be able to write to that directory
> > Does this happen immediately? or after some time?
> >
> > If it happens after some time, then you likely have a process on your unix
> > server that is periodically cleaning up the /tmp/ directory (which broke
> > jetty).
> >
>
> Thank you for your reply.
>
> The error happened after some days. Initially things were working fine.
>
> After seeing this error, shibboleth-idp was restarted which in turn restarted
> the jetty server as well, the server comes up fine as can be seen in the
> logs below:
>
> 2017-04-21 09:29:14.347:INFO:oejs.Server:main: jetty-9.4.0.M0
> 2017-04-21 09:29:14.347:WARN:oejs.Server:main: THIS IS NOT A STABLE RELEASE!
> DO NOT USE IN PRODUCTION!
> 2017-04-21 09:29:14.347:WARN:oejs.Server:main: Download a stable release from
> http://download.eclipse.org/jetty/
> 2017-04-21 09:29:14.396:INFO:oejdp.ScanningAppProvider:main: Deployment
> monitor [file:///opt/shibboleth-idp/jetty-base/webapps/] at interval 1
> 2017-04-21 09:29:14.641:INFO:root:main: Warning: No
> org.apache.tomcat.JarScanner set in ServletContext. Falling back to default
> JarScanner implementation.
> 2017-04-21 09:29:14.837:INFO:oajs.TldScanner:main: At least one JAR was
> scanned for TLDs yet contained no TLDs. Enable debug logging for this logger
> for a complete list of JARs that were scanned but no TLDs were found in
> them. Skipping unneeded JARs during scanning can improve startup time and
> JSP compilation time.
> 2017-04-21 09:29:14.856:WARN:oejs.session:main: No workerName configured for
> DefaultSessionIdManager, using node0
> 2017-04-21 09:29:14.856:WARN:oejs.session:main: No SessionScavenger set,
> using defaults
> 2017-04-21 09:29:14.900:INFO:oejsh.ContextHandler:main: Started
> o.e.j.w.WebAppContext@5383967b{/,file:///opt/shibboleth-idp/jetty-base/webapps/root/,AVAILABLE}{/root}
> 2017-04-21 09:29:22.723:INFO:/idp:main: No Spring WebApplicationInitializer
> types detected on classpath
> 2017-04-21 09:29:22.734:INFO:/idp:main: Warning: No
> org.apache.tomcat.JarScanner set in ServletContext. Falling back to default
> JarScanner implementation.
> 2017-04-21 09:29:22.974:INFO:oajs.TldScanner:main: At least one JAR was
> scanned for TLDs yet contained no TLDs. Enable debug logging for this logger
> for a complete list of JARs that were scanned but no TLDs were found in
> them. Skipping unneeded JARs during scanning can improve startup time and
> JSP compilation time.
> 2017-04-21 09:29:23.026:WARN:oejs.SecurityHandler:main:
> ServletContext@o.e.j.w.WebAppContext@5db45159{/idp,[file:///tmp/jetty-0.0.0.0-443-idp.war-_idp-any-4503664939735826572.dir/webinf/,
> jar:file:///opt/shibboleth-idp/war/idp.war!/],STARTING}{/opt/shibboleth-idp/war/idp.war}
> has uncovered http methods for path: /*
> 2017-04-21 09:29:23.291:INFO:/idp:main: Initializing Spring root
> WebApplicationContext
> 2017-04-21 09:29:31.291:INFO:/idp:main: Initializing Spring FrameworkServlet
> 'idp'
> 2017-04-21 09:29:32.483:INFO:oejsh.ContextHandler:main: Started
> o.e.j.w.WebAppContext@5db45159{/idp,[file:///tmp/jetty-0.0.0.0-443-idp.war-_idp-any-4503664939735826572.dir/webinf/,
> jar:file:///opt/shibboleth-idp/war/idp.war!/],AVAILABLE}{/opt/shibboleth-idp/war/idp.war}
> 2017-04-21 09:29:32.495:INFO:oejs.AbstractNCSARequestLog:main: Opened
> /opt/shibboleth-idp/jetty-base/logs/2017_04_21.request.log
> 2017-04-21 09:29:32.549:INFO:oejus.SslContextFactory:main:
> x509=X509@214b342f(creek,h=[creek.eng.arb.redhat.com],w=[]) for
> SslContextFactory@5db0003d(file:///opt/shibboleth-idp/credentials/idp-backchannel.p12,null)
> 2017-04-21 09:29:32.572:INFO:oejs.AbstractConnector:main: Started
> ServerConnector@1c0c9c99{SSL,[ssl, http/1.1]}{0.0.0.0:443}
> 2017-04-21 09:29:32.573:INFO:oejs.Server:main: Started @19519ms
>
>
> But again, as soon as I issue a request, I start getting the following
> errors:
> java.lang.IllegalStateException: Not valid for write:
> id=node0wcbt9cl1f24p11zowsnp88kb0 not resident
>         at
>         org.eclipse.jetty.server.session.Session.checkValidForWrite(Session.java:587)
>         at
>         org.eclipse.jetty.server.session.Session.setAttribute(Session.java:722)
>         at
>         org.eclipse.jetty.server.session.SessionHandler.newHttpSession(SessionHandler.java:796)
>         at org.eclipse.jetty.server.Request.getSession(Request.java:1520)
>         at org.eclipse.jetty.server.Request.getSession(Request.java:1493)
>         at
>         net.shibboleth.idp.log.SLF4JMDCServletFilter.doFilter(SLF4JMDCServletFilter.java:66)
>
> Is it still a problem with the /tmp directory getting wiped off?
>

Ok, i checked the logs and the contents of /tmp again, as soon as jetty server is restarted the jetty-0.0.0.0-443-idp.war-_idp-any-4503664939735826572 folder appears fine under /tmp, but after sometime it goes away. I'll check what has to be done to solve this.

Thanks again for your help,
Pritha

> Thanks,
> Pritha
>
> >
> > Joakim Erdfelt / joakim@xxxxxxxxxxx
> >
> > On Fri, May 19, 2017 at 3:26 AM, Pritha Srivastava < prsrivas@xxxxxxxxxx >
> > wrote:
> >
> >
> > Hi,
> >
> > We have a setup that has a shibboleth-idp with a jetty server. Any requests
> > to the shibboleth-idp is returning an error. I issued the following
> > command:
> >
> > curl -kvL https://<hostname>/idp and instead of seeing the expected result,
> > I
> > see errors like:
> >
> > <html>
> >
> > http-equiv="Content-Type" content="text/html;charset=utf-8"/>
> > Error 500 Server Error
> >
> > HTTP ERROR 500
> > Problem accessing /idp/. Reason:
> > <pre> Server Errorpre>Caused
> > by:<pre>java.lang.IllegalStateException: Not valid for write:
> > id=node0xzkusdg1igdt4hol48y607u66 not resident
> > at
> > org.eclipse.jetty.server.session.Session.checkValidForWrite(Session.java:587)
> > at
> > org.eclipse.jetty.server.session.Session.setAttribute(Session.java:722)
> >
> >
> > Looking at some older logs, I could see an error, after which similar
> > errors
> > started appearing:
> >
> > 2017-04-20 05:50:06.059:WARN:oaj.EmbeddedServletOptions:qtp987405879-11:
> > The scratchDir you specified:
> > /tmp/jetty-0.0.0.0-443-idp.war-_idp-any-6580006879402156844.dir/jsp is
> > unusable.
> > 2017-04-20 05:50:09.289:WARN:/idp:qtp987405879-11: unavailable
> > java.lang.ClassNotFoundException:
> > org.apache.jsp.WEB_002dINF.jsp.metadata_jsp
> > at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
> > at org.apache.jasper.servlet.JasperLoader.loadClass(JasperLoader.java:131)
> > at org.apache.jasper.servlet.JasperLoader.loadClass(JasperLoader.java:62)
> >
> >
> > I have checked permissions for the scratch directory, and they seem to be
> > fine. I have also restarted shibboleth-idp several times, but that doesn't
> > seem to solve the problem.
> > Everytime I issue a request to the idp, I get back the same error as above
> >
> > Does anyone have any idea as to what has gone wrong and what can be done to
> > solve this issue? (Maybe, rebuild the WAR file?)
> >
> > Thanks in advance,
> > Pritha
> >
> > _______________________________________________
> > jetty-users mailing list
> > jetty-users@xxxxxxxxxxx
> > To change your delivery options, retrieve your password, or unsubscribe
> > from
> > this list, visit
> > https://dev.eclipse.org/mailman/listinfo/jetty-users
> >
> >
> > _______________________________________________
> > jetty-users mailing list
> > jetty-users@xxxxxxxxxxx
> > To change your delivery options, retrieve your password, or unsubscribe
> > from
> > this list, visit
> > https://dev.eclipse.org/mailman/listinfo/jetty-users
> _______________________________________________
> jetty-users mailing list
> jetty-users@xxxxxxxxxxx
> To change your delivery options, retrieve your password, or unsubscribe from
> this list, visit
> https://dev.eclipse.org/mailman/listinfo/jetty-users
>
_______________________________________________
jetty-users mailing list
jetty-users@xxxxxxxxxxx
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://dev.eclipse.org/mailman/listinfo/jetty-users



--

Back to the top