Re: [ecf-dev] Race conditions with remote provider

[Alex Blewitt <alex.blewitt@xxxxxxxxx>]

On 6 Jun 2011, at 21:28, Scott Lewis wrote:

> Hi Alex,
> 
> On 6/6/2011 1:04 PM, Alex Blewitt wrote:
>> <stuff deleted>
>> Well, I'm writing a discovery container. So I'm using r-osgi and the example hello consumer/producer examples, just with my container instead of ZeroConf. In turn, I'm instantiating this and connecting to it (with connect(id,connectContext)) in the bundle activator's start method.
> 
> I see...that's interesting (using r-osgi to write a discovery API provider).

Well, I'm not using r-osgi to write a discovery provider. I'm writing a discovery provider, and I'm then using R-OSGi to talk between two VMs. So I'm pretty much using the r-{osgi,zookeeper} example product, except ^zookeeper^myContainer.

>> <stuff deleted>
>> So I had something like (excuse the exact method calls, this is from memory):
>> 
>> class Activator {
>>   void start() {
>>     id = IDFactory.getDefault().createId(namespace, new String[] { "a","b","c" });
>>     myContainer = new MyContainer();
>>     myContainer.connect(id,null);
>>     context.registerService(myContainer, {IDiscoveryListener, IDiscoveryAdvertiser});
>>   }
>> }
>> 
>> The problem was that the ID Factory was using the namespace (either from the plugin.xml where I have some links, or from a service I may have registered prior to that call ... think the plugin.xml is how it's finding it) and using bundle.getClass() to load the namespace's specific class provider.
>> 
>> Since the start() method hasn't completed at that point, an external bundle doing bundle.loadClass(some.class.name) fails with some OSGi 'bundle not started' type exception.
> 
> I see.  Markus Kuppe is the lead committer on the discovery API, so I'll defer to him...but one approach you might take is to create a ServiceFactory (rather than registering the locator/advertiser service directly in the start method).  This will defer the container creation and the call to IDFactory.getDefault() (the IDFactory...and the container factory for that matter...defer the processing of the appropriate extensions until the *first* time that IDFactory.getDefault() is called).

Right, I see. Somewhat like registering a service factory instead of registering a service. I still need to plug in some data at ID creation tme though.

>                        jdc = new JMDNSDiscoveryContainer();   // <-- the jmdns id is created in this constructor
>                        jdc.connect(null, null);

This creates the JMDNS' ID, true. But I need to pass in an ID in the jdc.connect(id,null) instead of a 'null' - because it can be configured with different properties. 

> Hopefully this will help with your case.  Discovery providers are naturally kind of tricky...since they typically 'want' to be started very early (upon start of ECF itself), while still having lazy processing of the ECF extensions (id factories and container factories).  The ServiceFactory allows/supports this without creating timing/classload/bundle state exception with another thread.


I managed to find out what I was doing wrong.

The 'connect' method was setting up a callback to be notified of new changes. When (my) discovery provider starts, it doesn't broadcast what's there already (just waits for changes). So I had put a call in to do a dummy notify, which would cause the current state of the world to be broadcast out.

However, I discovered that the listener being registered with my discovery container was being done *after* I'd done that initial state of the world. So my setup looked like:

start
- connect
-- refresh
-- allListeners.notify
- addListener(r-osgi)

So what was happening was I was broadcasting the initial set of the world prior to the listener being registered, and therefore appeared that the client was failing to work. (It's also why the 'registerService' was working before the call to connect - because the 'registerService' triggers the listener to be added.)

I changed my code so that the refresh doesn't happen in the connect/setup stage:

start
- connect
- addListener(r-osgi)
-- super.addListener(r-osgi)
-- if (listeners wasEmpty) refresh

That way, I can delay any time between the discovery provider setup and the (first) listener being registered. Hooray!

On the plus side, I've now got a much better idea of how the discovery stuff hangs together. I think there's also a lot of opportunity to make an abstract container to make it easier to provide discovery containers in the future. Quite a lot of the specific provider calls boil down to 'make this available' and 'tell me when a service arrives'. But there's a lot of glue code (turning ServiceProperties into a Map<String,String>, managing the set of services, standard implementations of notifying when services come and go etc.)

I'll have a go sometime at putting up a proof of concept on GitHub when I get a chance.

Thanks,

Alex