Re: [iot-pmc] Concierge 1.0 release review

["Jan S. Rellermeyer" <rellermeyer@xxxxxxxxxxx>]

Hi Kai, 

I will start inlining my comments if you don't mind. 

> If I get you right, your main argument is:
> "Concierge passes the OSGi R5 TCK and therefore there won't be any further
> work on its codebase necessary" (besides adding some not-in-spec features
> and working on R6 compliance in future).

No, that's not what I meant. There is always work on the code that needs to
be done in terms of bug fixes and--you know us, we are obsessed with
performance--optimization. The question that I wanted to bring up was more
along the lines, is there enough incremental potential that would make a
difference between a, let's say, 0.7 version and a 1.0 version. I would
think that this is not the case. There is, as you point out, a high
probability for bug fix releases (which would be 1.0.X to me) or maybe even
1.X releases that would bring new bundles into the Concierge ecosystem.
However, my impression is that the project will primarily be judged by the
maturity of its core framework and in this sense a 1.0 version feels right
to me. 

> You are the OSGi expert, so you can probably answer this: Does passing the
> TCK mean that the implementation is bug-free and usable in the wild?

Usable in the wild: Yes, I think so. We are talking about a body of tests
that has evolved over many years and for R5 includes 400+ test cases (of
which many tests multiple things) for the framework and launch alone.  
Bug-free: No, that would not be a TCK, that would be pure magic :-) A TCK is
never perfect but I would say, approaching the problem from the opposite
direction, passing all the tests severely limits the degree to which me
might have screwed up without noticing. So I would say, the risk of having
done something that would require another major release on R5 is very close
to zero. 

> I remember you saying one year ago that Concierge passes almost all TCK
> tests (with only 5-8 minor ones failing). This sounded already as if the
> implementation is "how a good implementation should be". During the past
> year though, this list had been created as well:
> https://github.com/JochenHiller/concierge-tests#closed-bugs-in-concierge
> Were all the reported and fixed bugs in the context of the few failing TCK
> tests?

Well, the list is more close to two years old but the point is valid.
Looking at the bugs, some of them definitely were known issues and covered
by the TCK. Some were platform-specific problems (only happens on Mac,
etc.). I see some that did not affect the framework TCK but surfaced in the
launch TCK that was not that close to complete until a few months ago. A few
were even optional features that I did not care to implement until it was
clear that someone (here: SmartHome) would use it. 

> From my personal experience, the TCK is a hint that something might be
> usable, but not a guarantee. Especially, afaik it does not say much about
> performance and memory usage, right?
> Do you have any real life statistics on these?

We do have performance data:
https://oracleus.activeevents.com/2014/connect/fileDownload/session/6E511CBA
FF03CC48B49F5314E4AC607A/CON3007_Rellermeyer-java_one_2014.pdf

I should re-run these experiments and very likely going to do it for ECE. 
 
> I think you underestimate your work here. IF it is a replacement for
Equinox,
> Felix, Knopflerfish, ProSyst mBS, etc, which is smaller, has a better
> performance and a modern architecture, I think you will see MANY people
> jumping on it. And this is what I fear: Only when these people start using
it,
> you will find out about all the special tricky use cases that you might
not have
> thought about yet and where you might need to adapt the implementation.
> Therefore - and in your own interest - I think it is a much wiser move not
to
> claim that everything is already perfect and cannot be improved anymore.

While I generally agree, I don't see the immediate connection to a 1.0
version. I am not sharing the opinion that 1.0 would have to be perfect
(which I guess was for a long time Google's rationale for some things never
going 1.0 but I feel this is more lawyer-talk than software-engineering
rationale). What I can say is that whatever problems people might run into
when running Concierge in situations that we could not anticipate and that
were not adequately covered by the TCK there is almost no chance that it
would affect the API and this is what makes a major version number for me
(semantic versioning). 

(Okay, at this point I am really embarrassed by myself because I had almost
the entire email written 5 minutes after you replied and now so many hours
have passed until I finally finished it. IBM = I've Been in a Meeting...)

--Jan.