Re: [equinox-dev] Huge performance loss between Kepler and Luna/Mars on

[Tom Schindl <tom.schindl@xxxxxxxxxxxxxxx>]

Hi,

Just to give you an update.

Today we looked closer at the code and added some debug statistics and
found that although we can replace the used ArrayList through HashSet
but the real problem is the algorithm used to detect bundles with reexports!

The statistics showed us that the 2 loops:

for (Wire w : candWiring.getRequiredResourceWires(null))

for (Requirement req : candCap.getResource().getRequirements(null))

came to a total count of ~90,000,000 iterations! We now went a head and
did the *right* and removed most of the reexports.

That brought us back to a resonable boot time including a reasonable
allocation count!

Tom

On 20.02.15 21:30, Thomas Watson wrote:
> 
>> From: Tom Schindl <tom.schindl@xxxxxxxxxxxxxxx>
>> Hi,
>>
>> I've been with a customer today who wants to update from Eclipse 3.x
>> Kepler to the 4.x code base and so he naturally inherits the latest
>> Equinox implementation who has changed in between those releases.
>>
>> The intial boot time increased from about 2 seconds to ~20 seconds on
>> Mars (on Luna Equinox or better said Felix-Resolver crashes).
> 
> I think the crash may be fixed in Luna SR2, at least the version of the
> felix resolver was updated in both SR2 and Mars.
> From a cached restart I assume you do not see this slow start.
> 
>>
>> We profiled the bootstrap process and the whole time (95%) is eaten up
>> the by felix resolver ResolverImpl#mergeCandidatePackages
>>
>> Looking at tbe hot methods we see:
>> * 15% of the time is spend in ArrayList.contains()
>>
>> * 30% of the time in HashMap.getNode()
>>   - called from ResolverImpl#mergeCanidatePackages 7%
>>   - called from ResolverImpl#calculateExportedPackages 5%
>>   - called from ResolverImpl#mergeUses 5%
>>
>> Looking Allocation counts we see the biggest amount (1.9GB) being caused
>> by ArrayList.grow where we could track back 1.84GB to
>> mergeCandidatePackages once more which suggests that the initial sizes
>> choosen for the arrays there are probably not optimal.
>>
>> One very interesting to this piece of software is that it uses:
>> a) only require bundle - not very unnatural for Eclipse apps
>> b) a loooot of reexports!
> 
> Lots of reexports did cause issues in Luna, but should be fixed in SR2.
>  We got a fix for Felix issue
> _https://issues.apache.org/jira/browse/FELIX-4762_which got released to
> SR2 and Mars.
> 
>>
>>
>> Would it be possible to:
>> a) replace the lists through sets? This should improve the contains check
>> b) use better initial array sizes?
> 
> I'm open to these types of changes if we can show they help improve the
> resolution time.  We will need to get the contributions back to apache,
> but I can handle that if you have some suggested changes.  Also note
> that we are having other issues in Mars that are not in Luna with
> 'batch' resolution.  This was supposed to help with performance but can
> cause the felix resolver algorithm to explode.  Any help there would be
> appreciated also.  See https://bugs.eclipse.org/bugs/show_bug.cgi?id=460393
> 
>>
>> Tom
>>
>> --
>> Thomas Schindl, CTO
>> BestSolution.at EDV Systemhaus GmbH
>> Eduard-Bodem-Gasse 5-7, A-6020 Innsbruck
>> http://www.bestsolution.at/
>> Reg. Nr. FN 222302s am Firmenbuchgericht Innsbruck
>> _______________________________________________
>> equinox-dev mailing list
>> equinox-dev@xxxxxxxxxxx
>> To change your delivery options, retrieve your password, or
>> unsubscribe from this list, visit
>> https://dev.eclipse.org/mailman/listinfo/equinox-dev
>>
> 
> 
> _______________________________________________
> equinox-dev mailing list
> equinox-dev@xxxxxxxxxxx
> To change your delivery options, retrieve your password, or unsubscribe from this list, visit
> https://dev.eclipse.org/mailman/listinfo/equinox-dev
> 


-- 
Thomas Schindl, CTO
BestSolution.at EDV Systemhaus GmbH
Eduard-Bodem-Gasse 5-7, A-6020 Innsbruck
http://www.bestsolution.at/
Reg. Nr. FN 222302s am Firmenbuchgericht Innsbruck