Thank you for doing this analysis, Snjezana. I'll comment on individual
items inline, but generally please submit git-format-patch patches for
separate problems as separate bugzillas. For each bugzilla/patch please
provide scenario that demonstrates the problem and before/after
performance metrics. Where practical, please provide corresponding
test(s) as a patch against m2e-core-test.
--
Regards,
Igor
On 11-04-11 07:59 PM, Snjezana Peco wrote:
I have tested m2eclipse performance.
These are my first impressions:
1) ProjectRegistryRefreshJob changes a workspace and fires a lot of
resource change listeners.
When creating a marker, for instance, this job fires five resource
change listeners (createMarker and four the setAttribute methods).
This job should be WorkspaceJob.
Workspace project "refresh" is a two-stage process. First, m2e resolves
dependencies of all affected workspace projects. Depending on network
connection speed and number/size of dependencies, this can take
arbitrary long time (I saw builds that took well over 30 minutes) so m2e
does not acquire any workspace locks during this stage. After all
dependencies are resolved, m2e acquires workspace lock and broadcasts
MavenProjectChangeEvents to all registered listeners. m2e.jdt, one of
the listeners, updates classpath and triggers (re)build of affected
projects upon reception of MavenProjectChangeEvents.
Frankly, I am not convinced we need to keep workspace lock for
entire time of project refresh but I am open for discussion. To help me
understand pros and cons better
1a) Is marker creation the only cause of resource change events fired by
the job or there are other sources?
1b) If marker creation is the only cause, what is the real overhead of
firing them as separate events vs one bulk event?
Depending on answers to these two questions, delaying marker creation
until the second stage of refresh may be a better solution.
2) ProjectRegistryRefreshJob and MavenBuilder (a builder job) aren't
synchronized what often causes StaleMutableProjectRegistryException
(ProjectRegistryManager, line 319) and that is the cause of starting
this job again.
I need to think some more about this, it looks like can disable
ProjectRegistryRefreshJob altogether when workspace autobuild is
enabled...
We still need to run the job when autobuild is off, but this should only
cause StaleMutableProjectRegistryException if the user manually runs the
build when refresh job is running, so should not be a big deal
3) LifecycleMappingFactory.getBundleMetadataSources is too often called.
Every time it is called it reads the extension point registry and
lifecycle mapping xml files. Caching would speed up the overall
performance.
We *guessed* that proper cache implementation, which deals with dynamic
bundle installation/uninstallation, is not worth the performance
benefits, but we did not actually measure it. Please provide
before/after numbers that demonstrate performance benefits, proper cache
implementation with adequate test coverage and I will be happy to review
and merge this change (and admit we guessed wrong ;-) ).
4) The BuildPathManager.configureAttchedSourcesAndJavadoc method is very
slow. I have improved it a little (DownloadSourcesJob isn't started when
the download preferences are off), but it is possible to improve it
more.
DownloadSourcesJob should not be started if download sources and javadoc
are disabled in maven configuration (see #getAttachedSourcesAndJavadoc
implementation and how its return value is used in scheduleDownload
around line 817). If you see DownloadSourcesJob start when
source/javadoc download is off, please provide a patch and corresponding
unit test(s) and I will review and merge it.
As for BuildPathManager.configureAttchedSourcesAndJavadoc performance in
general, I briefly looked at it about a month ago. At that time Yourkit
was telling me that MavenImpl.getSettings was the problem, but I did not
investigate it further. Do you know what makes
#configureAttchedSourcesAndJavadoc slow in your case?
5) NexusIndexManager.mavenProjectChanged(...) always removes and adds
back an artifact to the index repository. Performance will be much
better if the index repository is updated only when the artifact doesn't
exist or is changed. Searching the index repository is much faster than
removing and adding artifacts.
Please provide a patch with before/after performance numbers and I
will review and merge it.
6) MarkerLocationService.addEditorHintMarkers calls two methods that
acquire the WTP's IDOMModel
(StructuredModelManager.getModelManager().getModelForRead(...)) and
release it. WTP's methods can be slow for larger files and it would be
better to acquire/release the model once for
MarkerLocationService.addEditorHintMarkers.
MarkerLocationService.addEditorHintMarkers is used only once.
Please provide a patch with before/after performance numbers and I
will review and merge it.
7) ProjectRegistryManager.applyMutableProjectRegistry calls
stateReader.writeWorkspaceState(projectRegistry) when refreshing a
project. Since the workspaceState.ser file is only used when starting a
workspace, this method can be used only when stopping the
org.eclipse.m2e.core bundle
Please provide a patch with before/after performance numbers and I
will review and merge it.
8) BuildPathManager.mavenProjectChanged updates Maven classpath
containers of all projects no matter they are changed or not. They
should be updated only for relevant events (event.getFlags() != 0).
Please provide a patch with before/after performance numbers and I
will review and merge it.
9) ProjectRegistryManager.refresh(MutableProjectRegistry newState,
DependencyResolutionContext context, IProgressMonitor monitor)
aggressively
read Maven projects.
I have tested JBoss AS 7 (https://github.com/jbossas/jboss-as). When
calling Maven>Update Project Configuration on the jboss-as-parent
project, the MavenImpl.readProject is called for about 2-6 times for
each project in the workspace. Sometimes this method lasts short time
(Maven cache is active), but sometimes takes a longer time.
I have tried to cache MavenProjectFacade in each of the two phases so
that MavenImpl.readProject is called two times for every project (once
per each phase). Not sure if it is possible to make this method to be
run only once.
Let me put it this way. I am not aware of any cases when m2e unnecessary
calls MavenImpl.readProject during project refresh... "Extra"
MavenImpl.readProject invocations are needed to support workspace
dependency resolution properly. When workspace project project changes,
everything that directly and indirectly depends on this project needs to
refresh. For parent pom project, this also includes all child modules.
Unit tests coverage should be pretty good for this part of the code, so
if you are able to reduce number of MavenImpl.readProject invocations
without breaking any tests, I review and apply your patch.
10) The Maven Pom Editor has a huge memory leak. Just opening/closing
jboss-as-parent/pom.xml will cause the JVM heap size to increase by
2-10MB which can cause Eclipse to crash (OOM).
Please provide a patch with before/after heap size numbers and I
will review and merge it.
I have implemented 1-9 enhancements. The Maven>Update Project
Configuration action on the jboss-as-parent project is much faster (2-3
times). Editing the pom.xml file of the jboss-as-parent project is 2-3
times faster. The Maven builder is improved 2-3 times.
Would you like me to create a patch?
Snjeza
Snjezana Peco wrote:
Hello Steve,
I would like to test this a bit more. Could you provide me with more
details?
Steve Cohen wrote:
and, sad to say, it's mostly because of the combination of m2eclipse,
helios, subversive and the whole fracking mess.
Eclipse has become less and less usable. I don't know who's to blame,
which plugin is the cause or whether it's the whole platform or some
combination. Rare is the day of heavy development where Eclipse
doesn't crash many times.
Every crash is different. Sometimes I have to delete the .lock file.
Other times I don't. Other times I can't do it without killing
processes.
Could you check if there are files named hs_err_pidXXXX.log in your
working directory (your ECLIPSE_HOME probably)?
If so, could you attach any of them?
eclipse-jee-helios-SR2
m2eclipse, whatever the latest that's available today
subversive
hibernate plugin
Could you attach your configuration : Help>About Eclipse
SDK>Installation Details>Configuration
There are still user interface actions that provoke crashes.
What actions are causing the crash? Editing pom.xml, cleaning the
project, svn checkout/merge or something else.
Thanks,
Snjeza
_______________________________________________
m2e-dev mailing list
m2e-dev@xxxxxxxxxxx
https://dev.eclipse.org/mailman/listinfo/m2e-dev
