Hi Virgo devs,
As you might remember, I have contacted you previously with issues and queries around Virgo. My name is Aleksandar and my team is using the Virgo server extensively for one of our
projects.
We are currently using latest Virgo (3.7.2), but we have identified a few security flaws in the embedded Tomcat (v8.5.16), so we’ve been trying to upgrade to latest Tomcat. The
change is trivial – we have just changed the Tomcat version in the main gradle.properties file and rebuilt the project to produce an updated Virgo (diff is attached).
However, after this change our application won’t start. We are seeing the following error:
[2018-02-08T14:31:56.043+02:00] [ERROR] start-signalling-1 org.apache.catalina.core.ContainerBase ContainerBase.addChild:
start: org.apache.catalina.LifecycleException: Failed to start component [StandardEngine[Catalina].StandardHost[localhost].ExtendedStandardContext[/ui]]
at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:167)
at org.apache.catalina.core.ContainerBase.addChildInternal(ContainerBase.java:752)
at org.apache.catalina.core.ContainerBase.addChild(ContainerBase.java:728)
at org.apache.catalina.core.StandardHost.addChild(StandardHost.java:734)
at org.eclipse.gemini.web.tomcat.internal.TomcatServletContainer.startWebApplication(TomcatServletContainer.java:125)
at org.eclipse.gemini.web.internal.StandardWebApplication.start(StandardWebApplication.java:109)
at org.eclipse.virgo.web.core.internal.WebBundleLifecycleListener.onStarted(WebBundleLifecycleListener.java:122)
at org.eclipse.virgo.kernel.install.artifact.internal.StandardArtifactStateMonitor.onStarted(StandardArtifactStateMonitor.java:271)
at org.eclipse.virgo.kernel.install.artifact.internal.AbstractInstallArtifact.asyncStartSucceeded(AbstractInstallArtifact.java:319)
at org.eclipse.virgo.kernel.install.artifact.internal.AbstractInstallArtifact.access$0(AbstractInstallArtifact.java:316)
at org.eclipse.virgo.kernel.install.artifact.internal.AbstractInstallArtifact$StateMonitorSignal.signalSuccessfulCompletion(AbstractInstallArtifact.java:252)
at org.eclipse.virgo.nano.core.internal.BundleStartTracker$1.run(BundleStartTracker.java:140)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalStateException: java.io.FileNotFoundException: bundleentry:\265.fwk1301352406\WEB-INF\lib\jstl-1.2.jar (The filename,
directory name, or volume label syntax is incorrect)
at org.apache.catalina.webresources.AbstractSingleArchiveResourceSet.isMultiRelease(AbstractSingleArchiveResourceSet.java:121)
at org.apache.catalina.webresources.AbstractArchiveResourceSet.getResource(AbstractArchiveResourceSet.java:258)
at org.apache.catalina.webresources.StandardRoot.getResourcesInternal(StandardRoot.java:327)
at org.apache.catalina.webresources.CachedResource.validateResources(CachedResource.java:127)
at org.apache.catalina.webresources.Cache.getResources(Cache.java:147)
at org.apache.catalina.webresources.StandardRoot.getResources(StandardRoot.java:315)
at org.apache.catalina.webresources.StandardRoot.getClassLoaderResources(StandardRoot.java:231)
at org.apache.catalina.util.ExtensionValidator.validateApplication(ExtensionValidator.java:147)
at org.apache.catalina.core.StandardContext.startInternal(StandardContext.java:5015)
at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:150)
... 14 common frames omitted
Caused by: java.io.FileNotFoundException: bundleentry:\265.fwk1301352406\WEB-INF\lib\jstl-1.2.jar (The filename, directory name, or volume label
syntax is incorrect)
at java.util.zip.ZipFile.open(Native Method)
at java.util.zip.ZipFile.<init>(ZipFile.java:219)
at java.util.zip.ZipFile.<init>(ZipFile.java:149)
at java.util.jar.JarFile.<init>(JarFile.java:166)
at java.util.jar.JarFile.<init>(JarFile.java:130)
at org.apache.tomcat.util.compat.JreCompat.jarFileNewInstance(JreCompat.java:188)
at org.apache.tomcat.util.compat.JreCompat.jarFileNewInstance(JreCompat.java:173)
at org.apache.catalina.webresources.AbstractArchiveResourceSet.openJarFile(AbstractArchiveResourceSet.java:316)
at org.apache.catalina.webresources.AbstractSingleArchiveResourceSet.isMultiRelease(AbstractSingleArchiveResourceSet.java:116)
... 23 common frames omitted
This happens even though the above jar exists and both the name and the path are the same as the ones mentioned above. Furthermore, this error occurs only for bundles, that have
their dependencies (other jars) packaged inside them (e.g inside the WEB-INF/lib folder).
After spending a considerable amount of time debugging this issue, it seems that Tomcat is now executing some extra code that it wasn’t executing before. As part of a so-called
“multi-release” check it is trying to open the above jar and check if it is a multi-release jar (a Java 9 notion). This is where things fail. I have tried running the nested jar scenario on a vanilla Tomcat and it seems to work. The reason is that in a vanilla
Tomcat the path to the nested jar is passed in as an absolute path (e.g. C:\<tomcat-install-dir>\webapps\<my-app>\WEB-INF\lib\<jar-file>), while as you can see in the case of Virgo it is being passed in as a bundle entry URI (e.g. bundleentry:\265.fwk1301352406\WEB-INF\lib\jstl-1.2.jar).
For reference see:
So, I have a couple of questions:
-
Have you guys tried upgrading to latest Tomcat? Have you hit similar issues? Do you plan on releasing a new version with newer Tomcat anytime soon?
-
Could I get some help with this issue? Seems like the latest changes in Tomcat have caused a regression in Virgo.
Thanks,
Aleksandar Kanchev
