Re: [cross-project-issues-dev] On the issue of building with the latest

["David M Williams" <david_williams@xxxxxxxxxx>]

I think I still owe a response to this
note. 

There are so many things I do not like
about it, I hardly know where to begin. So, I will start at the beginning.

The Sim. Release site is intended to
be simply an aggregation of what project's provide. It should not add or
subtract from what project's provide (if projects are truly being "simultaneous").
 
There is no way for there to be a "single
authority" to provide which Orbit bundles are correct for a given
project. Only the project can decide that. 

Someone should be able to install or
build against only a project's site, and get exactly what the project intends.
That should be the same as the Sim. Release repo (after release) if everyone
stays current and specifies ranges or pre-reqs correctly.  

Your proposal would have the effect
of making the contents of a project's software site indeterminate. If someone
installs or builds one day (even after a release), they might get something
different than their colleague who follows the exact same procedure on
a different day. This should not occur and can be the source of bugs that
are nearly impossible to diagnosis and provide long term maintenance for.

The act of "building an update
release" should be easy to do. Easy enough if your pre-reqs change
then you need to rebuild and re-test. That is part of being in the Simultaneous
Release train.  Everyone needs to do that, even if their own code
does not change. By having "pre-reqs" you are implicitly stating
a contract that they are "part of your code" and therefore the
project is responsible for them. Orbit is intended simply to provide a
consistent, correct version for Eclipse projects to use. Projects must
still decide which one from Orbit is appropriate. 

I realize there are two schools of thought
on this issue. One that says a build and install should be completely determinant
and reproducible. And this should be true no matter what day of the week
it's done, and should be true, ideally, no matter which repository they
"point at". The other school of thought, seems to me, to be more
that things are kinds of loose. Users and adopters sort of "get what
they get" and it is up to them, the Users and adopters to decide if
it is correct and what they want. I know that often works, and many smaller
projects work that way, but I do not think it is appropriate when you are
trying to produce "enterprise quality" code. 

I do know that I am over simplifying
many things in my reply, but by doing so, I hope I am getting across the
main ideas. And, hope my remarks are constructive. I am all for thinking
of making things better, but do not see how these suggestions accomplish
that. 

I do recognize that everyone wants "release
engineering" to be easier, but, I do not think your proposal accomplishes
that for anyone except a few "small number of committers" teams.
It actually complicates things for Users and adopters. In fact, in the
past, the "looseness" of "what went with what" was
the whole reason the Simultaneous Release got started! 

In short, if you are in the Simultaneous
Release that means you are also in the Simultaneous Update Releases and
you must rebuild if your pre-reqs change even if your own code does not.
In theory, some of your idea might apply at the project level. That is,
in theory, a project could "build a new repository site" -- without
literally re-compiling their code -- and the site would have the correct,
new, prerequisites in it. Of course, there are lots of advantages to recompiling,
even if you expect no changes -- since, who knows how a third party pre-req
might change from x.y.1 to x.y.2 (they all follow different rules). But,
in any case, it is up to the project do decide what method is best for
them, as long as they end up with the correct deliverables. 

I know that many projects work pretty
hard at "getting around the rules" and making things easier for
themselves, but also believe that most projects live by the spirit and
goals of the Simultaneous Release and provide consistently correct deliverables
for every Release and Update Release. I suggest a the cross-project level
we focus on other ways to make things easier. 






From:      
 Alexander Nyßen <nyssen@xxxxxxxxx>
To:      
 Cross project issues
<cross-project-issues-dev@xxxxxxxxxxx>, 
Date:      
 02/06/2016 09:17 AM
Subject:    
   Re: [cross-project-issues-dev]
On the issue of building with        thelatest
Orbit repository
Sent by:    
   cross-project-issues-dev-bounces@xxxxxxxxxxx

---

What about the following policy?

1) Ensure that none of the features contributed to simrel
include any Orbit bundles, but only refer to them via dependencies (and
ensure that proper version constraints are used in addition).
2) Use specific all-in-one features that include the required
Orbit bundles (and the features deployed to simrel), but only use them
to produce self-contained local drop files (zip).
3) Do not publish all-in-one features on your local update
sites either, but rather have your local update-sites refer to the simrel
update-site via an associate-site entry (so Orbit dependencies get pulled
in from there).

With 1) we would have a single authority that controls
which Orbit bundles are actually provided in a release, namely the simrel
aggregator. 2) would cover your use case, Ed. 3) would ensure that newer
Orbit bundles (added in a maintenance release) would also get picked up
when installing from a local update-site.

Cheers
Alexander

Am 05.02.2016 um 20:22 schrieb Ed Willink <ed@xxxxxxxxxxxxx>:

Hi Konstantin

I have no idea how a download ZIP can easily mirror something.

But I presume what you really mean is that we should produce two ZIPs/categories.

- A contribution ZIP/category that replicates the features contributed
to SimRel.
- An All-in-One ZIP/category that additionally bundles Orbit facilities
avoiding the need for users to learn how to include Orbit in the available
sites.

    Regards

        Ed Willink

On 05/02/2016 17:33, Konstantin Komissarchik wrote:
I would advise strongly against using
the feature includes directive for Orbit bundles. While doing so provides
a cheap way to mirror the Orbit bundle into the produced project repository,
it also locks you down to using that one version of the bundle and we end
up with multiple versions of various bundles in the install, often for
no good reason. It’s easy enough to mirror the required Orbit bundles
using a separate step at the end of the project build and be free to require
the bundle via whatever version range is most appropriate.
 
If everyone did this and simrel aggregation
included the latest Orbit repository, the result of aggregation would contain
fewer duplicate Orbit bundles without requiring projects to issue a new
release just to move to a newer version of an Orbit bundle.
 
Thanks,
 
- Konstantin
 
 
From: Ed
Willink
Sent: Thursday, February 4, 2016 11:11 PM
To: cross-project-issues-dev@xxxxxxxxxxx
Subject: Re: [cross-project-issues-dev] On the issue of building with
thelatest Orbit repository
 
Hi David

Interesting if you're right. I thought projects needed to bundle what wasn't
bundled upstream. Thus OCL bundles LPG since no one else does. OCL includes
ASM, Guava that someone else bundles.

You suggest that OCL could stop bundling LPG since LPG would appear in
the SimRel repo automatically.

However OCL still needs to bundle LPG in order for the install from ZIPs
to work offline. AFAIK there is no SimRel ZIP distribution. If there was,
it would probably be so big that few would use it. However if there was
a used-in-SimRel-Orbit ZIP distribution that could be useful and could
help you enforce limited usage.

    Regards

        Ed Willink
On 04/02/2016 21:35, David M Williams
wrote:
Off the top of my head, I think features
are suppose to 'include' them, since that is the only way to have a reproducible
build or install. If it was left up to "requires" then who knows
what you would get. 
Granted, there should not be anything breaking, if you simply took a what
ever was there, within some specified range, but especially with third
party bundles, you never know. Some are real good at following good versioning
practices, some are not.  Plus, keep in mind, the "aggregated
repository" is supposed to be a simple grouping of a subset of what
ever is in the project repositories. We do not want a situation where if
someone installs directly from "your" repository, they get one
set of things, and if they install from the Sim. Release repository they
get another set of things. Maintenance would be very difficult, then. To
repeat, that's off the top of my head. Maybe you meant something else.




From:        Alexander
Nyßen <nyssen@xxxxxxxxx>
To:        Cross project
issues <cross-project-issues-dev@xxxxxxxxxxx>,
Date:        02/04/2016
04:20 PM
Subject:        Re:
[cross-project-issues-dev] On the issue of building with the    
   latest Orbit repository
Sent by:        cross-project-issues-dev-bounces@xxxxxxxxxxx




Hi David,

could you please clarify why exactly updates would be needed from projects
because of changes to Orbit bundles? Does it result from the fact that
Orbit bundles are actually re-bundled by project features? Or from the
fact that requirements on them are specified too restrictive within project
bundles or features?

I’m not sure if this is already covered by some simrel reports, but IMHO
we would be pretty safe if we ensured that

1) no Orbit bundles were actually re-bundled in project features, but only
required by them, and that
2) dependencies on Orbit bundles or packages would be specified in line
with the respective Orbit main releases (based on proper version ranges),

because the aggregation could then pretty much control which Orbit bundles
get pulled in. If we would in addition impose the same restrictions on
Orbit releases as on project releases (namely that updates including breaking
changes are not allowed in maintenance releases), I would assume no project
should actually have to update its contribution for a maintenance release.

Cheers
Alexander

Am 04.02.2016 um 21:43 schrieb Ed Willink <ed@xxxxxxxxxxxxx>:

HI

"commons.collections" doesn't seem that well used. No version
of it is my workspaces, so QVTd, (Xtext, EGIT, UML, QVTo, OCL) cannot have
a dependency on it. No re-contribution needed.

   Regards

       Ed Willink

On 04/02/2016 20:19, David M Williams wrote:
Ed, 

Thanks for bringing this "no maintenance, no new Orbit" issue
to my attention. 

While the Planning Council does not like to "make" people do
extra work they would not normally do, I believe it was the intent of one
of our requirements [1] that the latest Orbit be consumed every update
release -- if there has been a new Orbit "released". Most often
there is not a new Orbit release, since we in Orbit do that only for significant
issues. This time, it was only for the 'commons.collections' security bug,
and a bad bug in Ant 1.9.4 that drove us to provide Ant 1.9.6. [2]. 

While I will not say you *have* to update and provide a new build, I would
encourage you to, as well as anyone else who uses "commons.collections"
since we don't want to "spread around" a package that has known
security flaw in it. 

As far as I know, in most cases of installing and updating people will
get the correct, fixed version of that bundle, but am not positive that
is always true so I hate for it to be the available from any of our "most
recent repositories" (Simultaneous Release or not) -- after all, the
b3 aggegator is including it for some reason -- so someone must say they
require it?  

But I am also not the "security policeman" that will say that
bundle must be expunged from all current downloads. (If I recall, the security
issue only applied to specialized cases ... but, if you were running in
that case, it was a bad security bug possibly leading to a malicious person
"executing arbitrary commands". 

I have opened bug 487285 to investigate or discuss this issue further.
[3] And,  I will put this on future Planning Council agendas to see
if we can word that requirement [1] better so that all projects know better
what is expected of them.

Thanks again, 

[1] https://wiki.eclipse.org/SimRel/Simultaneous_Release_Requirements#Re-use_and_share_common_third_party_code_.28partially_tested.29
[2] https://dev.eclipse.org/mhonarc/lists/orbit-dev/msg04419.html
[3] https://bugs.eclipse.org/bugs/show_bug.cgi?id=487285






From:        Ed Willink
<ed@xxxxxxxxxxxxx>
To:        cross-project-issues-dev@xxxxxxxxxxx,
Date:        02/04/2016
01:12 AM
Subject:        Re:
[cross-project-issues-dev] Ready for Mars.2 ?
Sent by:        cross-project-issues-dev-bounces@xxxxxxxxxxx




Hi David

On 03/02/2016 22:29, David M Williams wrote:
- Every contribution file has changed since Mars.1. Also good. (i.e. no
projects are just sleeping and forgot to update :) 

You might want to review your query. qvtd.b3aggrcon was last changed by
me on 26 June, and by you on 14 July.

We are certainly not sleeping, and did not forget to update. Just working
very hard to support the functionality required for graduation to 1.0.0.
And ... worst of all, IMHO, some "old" third party jars are still
being used, which implies to me someone is not using the latest version
of Orbit (R20151221205849). 
But if a project has no maintenance to contribute, I thought no rebuild/contribution
was required and so of course an old Orbit would be in use. (I don't think
that QVTd imposes tight bounds on Orbit contributions.)

   Regards

       Ed Willink_______________________________________________
cross-project-issues-dev mailing list
cross-project-issues-dev@xxxxxxxxxxx
To change your delivery options, retrieve your password, or unsubscribe
from this list, visit
https://dev.eclipse.org/mailman/listinfo/cross-project-issues-dev



_______________________________________________
cross-project-issues-dev mailing list
cross-project-issues-dev@xxxxxxxxxxx
To change your delivery options, retrieve your password, or unsubscribe
from this list, visit
https://dev.eclipse.org/mailman/listinfo/cross-project-issues-dev

_______________________________________________
cross-project-issues-dev mailing list
cross-project-issues-dev@xxxxxxxxxxx
To change your delivery options, retrieve your password, or unsubscribe
from this list, visit
https://dev.eclipse.org/mailman/listinfo/cross-project-issues-dev

--
Dr. Alexander Nyßen
Dipl.-Inform.
Principal Engineer

Telefon: +49 (0) 231 / 98 60-202
Telefax: +49 (0) 231 / 98 60-211
Mobil: +49 (0) 151 /  17396743

http://www.itemis.de
alexander.nyssen@xxxxxxxxx

itemis AG
Am Brambusch 15-24
44536 Lünen

Rechtlicher Hinweis:

Amtsgericht Dortmund, HRB 20621

Vorstand: Jens Wagener (Vors.), Wolfgang Neuhaus, Dr. Georg Pietrek, Jens
Trompeter, Sebastian Neus

Aufsichtsrat: Prof. Dr. Burkhard Igel (Vors.), Michael Neuhaus, Jennifer
Fiorentino


[attachment "signature.asc" deleted by David M Williams/Raleigh/IBM]
_______________________________________________
cross-project-issues-dev mailing list
cross-project-issues-dev@xxxxxxxxxxx
To change your delivery options, retrieve your password, or unsubscribe
from this list, visit
https://dev.eclipse.org/mailman/listinfo/cross-project-issues-dev




_______________________________________________
cross-project-issues-dev mailing list
cross-project-issues-dev@xxxxxxxxxxx
To change your delivery options, retrieve
your password, or unsubscribe from this list, visit
https://dev.eclipse.org/mailman/listinfo/cross-project-issues-dev
 
 

_______________________________________________
cross-project-issues-dev mailing list
cross-project-issues-dev@xxxxxxxxxxx
To change your delivery options, retrieve your password, or unsubscribe
from this list, visit
https://dev.eclipse.org/mailman/listinfo/cross-project-issues-dev

--
Dr. Alexander Nyßen
Dipl.-Inform.
Principal Engineer

Telefon: +49 (0) 231 / 98 60-202
Telefax: +49 (0) 231 / 98 60-211
Mobil: +49 (0) 151 /  17396743

http://www.itemis.de
alexander.nyssen@xxxxxxxxx

itemis AG
Am Brambusch 15-24
44536 Lünen

Rechtlicher Hinweis:

Amtsgericht Dortmund, HRB 20621

Vorstand: Jens Wagener (Vors.), Wolfgang Neuhaus, Dr. Georg Pietrek, Jens
Trompeter, Sebastian Neus

Aufsichtsrat: Prof. Dr. Burkhard Igel (Vors.), Michael Neuhaus, Jennifer
Fiorentino


[attachment "signature.asc"
deleted by David M Williams/Raleigh/IBM] _______________________________________________
cross-project-issues-dev mailing list
cross-project-issues-dev@xxxxxxxxxxx
To change your delivery options, retrieve your password, or unsubscribe
from this list, visit
https://dev.eclipse.org/mailman/listinfo/cross-project-issues-dev