Re: [equinox-dev] Service Lookup by GUID very Slow - the Framework Scala

[<Stanley_Poon@xxxxxxxx>]

Thank you Neil.

 

But the exact same example runs 50X faster on the other two OSGi runtime. Any thoughts? And we observed the other runtime used more memory. Will they be using some indexing (in memory)?

 

Thanks,

Stanley

 

From: equinox-dev-bounces@xxxxxxxxxxx [mailto:equinox-dev-bounces@xxxxxxxxxxx] On Behalf Of Neil Bartlett
Sent: Tuesday, May 08, 2012 11:02 AM
To: Equinox development mailing list
Subject: Re: [equinox-dev] Service Lookup by GUID very Slow - the Framework Scalability

 

* No, services are not created lazily by default. You are creating the services yourself in your code sample, i.e., the 'serviceObject' variable.

There are some frameworks such as Declarative Services that create service instances lazily, however the creation of the service (and therefore the associated cost) would not be triggered by merely calling getServiceReferences(). It would be triggered by calling getService() on one of the ServiceReference objects.

* By creating a ServiceTracker, you can be notified synchronously as each service is registered. You could then very cheaply extract the ID property from each service and store it in a ConcurrentHashMap for later reference.

I'm pretty sure that the poor performance of your example is due to the fact that the framework is forced to evaluate the filter _expression_ "(ID=x)" against every entry in the registry. You don't even allow it to return when it finds the first match, because you have called getServiceReferences(), plural, so it must find all of the matches not just the first one. If you called getServiceReference() then the search time would be halved, on aggregate.

Regards
Neil

Stanley_Poon@xxxxxxxx

8 May 2012 18:44

The key question is about the scalability of the runtime, in terms of

 

-        Number of services, 100K, 1M, 10M, etc

-        Registering and unregistering them, churn over long period of time

-        Is it primarily memory bound? Assuming that services are not CPU heavy or thread hungry

 

Some details of the experiment we had:

 

The code is pretty straightforward. We only have a 5 Service Classes. We add an ID property to each service object we register and later look it up using that ID. The look up through getServiceReferences() took 4-5 seconds when we have 200K services registered.

 

Also, can someone shed some light for the following questions:

 

-        Do you think lazy service creation may be the reason? Is lazy creation the default? How to configure it?
-        Can you outline the steps to use ServiceTrackerCustomizer to build index? Do you mean trapping the registration events and build the needed indexes?

 

Service Registration: we register all the 200K service instances in a loop. They are one of 5 different server classes.

 

BundleContext bc,

 

            Dictionary<String, String> props = new Hashtable<String, String>(1);

            props.put(“ID”, “ID0000001”);

            bc.registerService(ServiceClass, serviceObject, createProperties(props));

 

 

Service Look up:

 

            BundleContext bc,

            bc.getServiceReferences(ServiceClass, "(ID=ID0000001)”);

 

 

Thank you,

Stanley

 

From: equinox-dev-bounces@xxxxxxxxxxx [mailto:equinox-dev-bounces@xxxxxxxxxxx] On Behalf Of BJ Hargrave
Sent: Friday, May 04, 2012 10:19 AM
To: Equinox development mailing list
Subject: Re: [equinox-dev] Service Lookup by GUID very Slow

 

Equinox also indexes by objectClass alone. So I am not sure what the discrepancy is here. Would be nice to have the test case code to analyze. Stanley, can you post a gist with the code?

--

BJ Hargrave Senior Technical Staff Member, IBM OSGi Fellow and CTO of the OSGi Alliance hargrave@xxxxxxxxxx

office: +1 386 848 1781 mobile: +1 386 848 3788

From:        "Richard S. Hall" <heavy@xxxxxxxxxxxxxx>
To:        Equinox development mailing list <equinox-dev@xxxxxxxxxxx>,
Date:        2012/05/04 13:16
Subject:        Re: [equinox-dev] Service Lookup by GUID very Slow
Sent by:        equinox-dev-bounces@xxxxxxxxxxx

_______________________________________________
equinox-dev mailing list
equinox-dev@xxxxxxxxxxx
https://dev.eclipse.org/mailman/listinfo/equinox-dev