[
Date Prev][
Date Next][
Thread Prev][
Thread Next][
Date Index][
Thread Index]
[
List Home]
|
Re: [iot-pmc] top-level GitHub project
|
Hi,
I have been following this discussion for a while. firstly if I can learn something for my project and secondly I had Eclipse Cyclone on my todo list for
getting it into 4diac. After today I must say it moved way down to the bottom of my list. Because it looks now so complicated to get it build for the many
different platforms we would like to use it on. Is this really needed when I want to use Eclipse Cyclone?
BR,
Alois
On Thu, 2018-11-08 at 22:10 +0100, Jeroen Koekkoek wrote:
> (1) Yes, we could. But we've just setup everything wrt to Travis and
> AppVeyor and it works really well. The process and the services (GitHub,
> Travis, AppVeyor, Bintray) work great, is it really that big of a
> problem if we don't use JIPP for now?
> (3) There are multiple recipes. There's two direct dependencies, but
> those have dependencies as well. All-in-all there's currently five and
> more are to follow when we start work on the C++ language binding,
> support for security, etc. Yes, you can build all of it manually and set
> it up that way, but it requires an awful lot of work that needs to be
> repeated every time we want to upgrade etc. Conan more-or-less requires
> a repository per dependency. You could view the recipe as an RPM/DEB
> source package, but with instructions for multiple operating systems. Is
> that really something we want to create a repositories for directly
> under the Eclipse GitHub organization?
>
> It's vendor neutral to use Conan and Bintray. It's currently managed by
> me and owned by the atolab GitHub organization (division of ADLINK).
> Anyone with the right privileges on the repository can push a new
> version and it automatically gets uploaded if the build is successful,
> all the tests pass and it's in a stable branch. If anyone wants to fix a
> bug, add a platform etc, all they need to do is submit a pull request.
> Of course if they rather maintain their own copy, they simply fork the
> repository and push to their own Bintray repository or internal
> Conan/Artifactory server and use that.
>
> All of these services are tied to GitHub organizations. For example, the
> binary artifacts in Bintray are hosted per organization. e.g. the Conan
> repository that we use is now https://bintray.com/atolab/public-conan/
> that would become https://bintray.com/cyclonedds/public-conan/ for
> instance. But descriptions and that sort of thing need to be updated,
> perhaps older versions discarded and that's something that would then
> require action by people with privileges, aka the Eclipse webmaster.
> With a TLOrg all of that becomes just that much easier.
>
>
> On 2018-11-08 18:16, Jens Reimann wrote:
>
> > So summing this all up, and only focusing on the topic of a dedicated
> > organization, I don't really see the need for that.
> >
> > (1) You can build with JIPP, so that should be fine. You can still have
> > a Travis build in addition, so that others can use this. But the
> > project default can be JIPP. And it can validate PRs as well.
> > (3) I am not really sure if those recipes belong in cyclonedds or not.
> > In any case the project must be buildable without relying on conan, or
> > bintray IMHO. And even if you want to store the recipes in cyclone,
> > then you can do this in e.g. a dedicated cyclonedds-dependencies
> > repository.
> >
> > Aside from the question about the organization, a question to me would
> > be how the project would ensure a vendor neutral approach to the
> > management of binary artifacts? I, as a committer of cyclonedds, not
> > being an adlink employee: how would I push new binaries into this
> > bintray repository? Or set up a new job to do so?
> >
> > On Thu, Nov 8, 2018 at 5:33 PM Jeroen Koekkoek <jeroen@xxxxxxxxxxx>
> > wrote:
> >
> > > Hi Jens,
> > >
> > > (3a) Yes, it's possible if they have the libraries installed on their
> > > system and CMake can find them. But it's tricky to create them
> > > yourself
> > > especially on Windows (Debug, Release, x86, x86_64, patches).
> > > Something
> > > we rather not have potential contributors (or colleagues) go through.
> > >
> > > (3b) The recipes are rather small and certainly wouldn't qualify as an
> > > open source project in their own right. However, they are purposely
> > > created for cyclonedds (CUnit got adopted by Bincrafters) and I think
> > > the project should be able to exercise some level of control which
> > > versions, binaries etc are used as long as they're not "official"
> > > packages (as in adopted by Bincrafters or the Conan project itself).
> > >
> > > (3c) The recipes themselves are automatically build and tested on both
> > > Travis and AppVeyor, if successful, the artifacts are automatically
> > > pushed to Bintray. There are a lot of custom scripts provided by
> > > Bincrafters, an initiative to create recipes for many open source
> > > libraries, that require a special repository layout etc. Something I
> > > don't want to recreate myself.
> > >
> > > To give an idea of how/when Conan comes into play.
> > >
> > > ---- with conan ----
> > > $ pip install conan
> > > $ mkdir cyclonedds/build
> > > $ cd cyclonedds/build
> > > $ conan install ..
> > > $ cmake -DBUILD_TESTING=on ../src
> > > $ cmake --build .
> > > ---- /with conan ----
> > >
> > > ---- without conan ----
> > > > > Download, build and install CUnit (mostly available in distro
> > > > > repositories, but not for Windows or macOS, Windows even needs some
> > > > > patches)
> > > > > Download, build and install Criterion (ppa available for Ubuntu, but
> > > > > a
> > > > > nightmare for every other platform, official source tarball doesn't
> > > > > include everything to build)
> > > > > The same as above for all future dependencies, let's hope the user
> > > > > built everything with the right flags etc.
> > > > > Export paths for all dependencies
> > > $ mkdir cyclonedds/build
> > > $ cd cyclonedds/build
> > > $ cmake -DBUILD_TESTING=on ../src
> > > $ cmake --build .
> > > ---- /without conan ----
> > >
> > > I know the "without conan" doesn't seem like a lot of work, it's just
> > > two extra lines :), but trust me when I say it saves A LOT of time and
> > > effort.
> > >
> > > Cheers,
> > > Jeroen
> > >
> > > On 2018-11-08 16:37, Jens Reimann wrote:
> > >
> > > > Hi Jeroen,
> > > >
> > > > regarding (3), I would like to understand:
> > > >
> > > > (a) if it is possible for people to build cyclonedds without having
> > > > access to precompiled dependencies?
> > > > (b) what would be the benefit of having an additional organization?
> > > > (c) As this organization would be part of your project, why can't you
> > > > store those dependencies in your project repository instead?
> > > >
> > > > Thanks
> > > >
> > > > Jens
> > > >
> > > > On Thu, Nov 8, 2018 at 1:57 PM Jeroen Koekkoek <jeroen@xxxxxxxxxxx>
> > > > wrote:
> > > >
> > > > > Hi Kai,
> > > > >
> > > > > I too am sorry for taking this long to respond. It's not lack of
> > > > > interest, but rather to get an idea first on how to proceed.
> > > > >
> > > > > (1) While it's not impossible for cyclonedds to use JIPP, using
> > > > > Travis
> > > > > and AppVeyor is easier for developers that want to send us pull
> > > > > requests
> > > > > because they build their sources on the exact same platform using
> > > > > the
> > > > > exact same build recipes and dependencies.
> > > > >
> > > > > (2) We're still looking into this, but we're probably going to fork
> > > > > and
> > > > > start the process for the CQ soon.
> > > > >
> > > > > (3) Build recipes for dependencies. We use Conan.io to manage build
> > > > > dependencies for the platforms on which we build. This makes it
> > > > > extremely easy for developers and users to simply get pre-compiled
> > > > > dependencies by pulling them from Bintray (https://bintray.com/) if
> > > > > they're not available on their system. Apart from that we have a
> > > > > guarantee that the product is build on the "exact same" platform
> > > > > every
> > > > > time because even the build dependencies are version controlled.
> > > > > I've
> > > > > written a bunch of these recipes, so the source for the recipe is
> > > > > "owned" by the project already, but it must not (and cannot) be
> > > > > stored
> > > > > in the cyclonedds repository. Currently we provide them through a
> > > > > GitHub
> > > > > organization tied to the company I work for (ADLINK), but it would
> > > > > be
> > > > > better if they're hosted with the project. We're going to be adding
> > > > > more
> > > > > and more of these recipes in the future as the number of
> > > > > dependencies
> > > > > grows. Please see https://github.com/atolab/conan-criterion to get
> > > > > an
> > > > > idea of what such a recipe looks like. I think maintaining them
> > > > > under
> > > > > our own organization would be best.
> > > > >
> > > > > I'll get a bugzilla ticket created, let's continue the discussion
> > > > > there
> > > > > if there's still doubts wrt the validity of the request.
> > > > >
> > > > > Cheers,
> > > > > Jeroen
> > > > >
> > > > > On 2018-10-31 11:37, Kai Kreuzer wrote:
> > > > >
> > > > > > Hi Jeroen,
> > > > > >
> > > > > > Sorry for my late reply on this.
> > > > > > As an outcome of the PMC discussion, I have added a section about
> > > > > > Github orgs here: https://wiki.eclipse.org/IoT/PMC#Project
> > > > > >
> > > > > > As you can see, there are no general requirements that would allow
> > > > > > it,
> > > > > > but decisions have to be taken on a case by case evaluation.
> > > > > >
> > > > > > So please let us again dive into your two arguments, why you are
> > > > > > requesting such an org:
> > > > > >
> > > > > > (1) In general, the PMC is of the opinion that Eclipse projects
> > > > > > should
> > > > > > use the provided build infrastructure (i.e. JIPP). Note that it is
> > > > > > possible to also use Windows nodes for JIPP. External services such
> > > > > > as
> > > > > > Travis should only be a last resort, if it isn't possible to use
> > > > > > JIPP.
> > > > > > It would therefore be nice if you could check whether you can do
> > > > > > the
> > > > > > builds on JIPP and if this isn't possible to give the concrete
> > > > > > reasons
> > > > > > for that.
> > > > > >
> > > > > > (2) As Wayne already stated: Any source code that is added in
> > > > > > Eclipse
> > > > > > project repositories IS source code of the Eclipse project. So you
> > > > > > cannot maintain a fork (as part of your project) and continue to
> > > > > > claim
> > > > > > that it is merely a dependency of your project. The code would
> > > > > > require
> > > > > > to go through a full CQ as a code contribution to Eclipse. In any
> > > > > > case,
> > > > > > this should not require a dedicated Github org, but could be done
> > > > > > as
> > > > > > a
> > > > > > sub-project in its own repo under the eclipse org.
> > > > > >
> > > > > > Best regards,
> > > > > > Kai
> > > > > >
> > > > > > On 30. Oct 2018, at 15:18, Jeroen Koekkoek <jeroen@xxxxxxxxxxx>
> > > > > > wrote:
> > > > > >
> > > > > > Hi All,
> > > > > >
> > > > > > Just curious what the outcome of the discussion regarding TLOrg's
> > > > > > is.
> > > > > > Perhaps I can read about it somewhere or a short summary with
> > > > > > requirements etc can be given?
> > > > > >
> > > > > > Thanks in advance.
> > > > > >
> > > > > > Cheers,
> > > > > > Jeroen
> > > > > >
> > > > > > On 2018-10-23 12:54, Wayne Beaton wrote:
> > > > > >
> > > > > > If you want to keep content under the project's control, it needs
> > > > > > to
> > > > > > become project code. The first step to do that is to convince the
> > > > > > PMC
> > > > > > that it makes sense to do so (i.e. convince them that forking is
> > > > > > desirable and in the best interests of the project and the Eclipse
> > > > > > community). Then you need to engage the IP Team to get their review
> > > > > > and
> > > > > > approval.
> > > > > > HTH,
> > > > > > Wayne
> > > > > > On Mon, Oct 22, 2018 at 6:27 PM Jeroen Koekkoek
> > > > > > <jeroen@xxxxxxxxxxx>
> > > > > > wrote:
> > > > > > Hi Kai,
> > > > > > Well we need it primarily for our own Conan recipes (build
> > > > > > dependency
> > > > > > build recipes), currently six or so, and faster Travis and AppVeyor
> > > > > > builds. The recipes are all written by me btw, so no forking
> > > > > > involved
> > > > > > there.
> > > > > > That being said, and this just out of curiosity and trying to
> > > > > > understand
> > > > > > the reasons, mcpp is very likely to become a dependency for
> > > > > > cyclonedds,
> > > > > > one that we must have to do code generation. mcpp is a fine
> > > > > > project,
> > > > > > but
> > > > > > it's not very actively maintained. I could just write a Conan
> > > > > > recipe
> > > > > > and
> > > > > > keep some patches on the side, or I could fork it under the
> > > > > > ADLINK-IST
> > > > > > GitHub organization, but it all boils down to the same thing
> > > > > > really.
> > > > > > What's wrong with keeping a dependency directly under the projects
> > > > > > control? Keeping some level of control over it is actually why I
> > > > > > thought
> > > > > > it would be a good idea.
> > > > > > Cheers,
> > > > > > Jeroen
> > > > > > On 2018-10-22 21:04, Kai Kreuzer wrote:
> > > > > > Hi Jeroen,
> > > > > > Please wait with creating an according Bugzilla ticket - while it
> > > > > > is
> > > > > > in
> > > > > > general technically possible, we would like to discuss this within
> > > > > > the
> > > > > > PMC (which is together at ECE this week).
> > > > > > Also note that a TLOrg won't allow you to add a (2) forked
> > > > > > sourceforge
> > > > > > project, so this actually won't help you much (EMO, correct me, if
> > > > > > I
> > > > > > am
> > > > > > wrong).
> > > > > > Cheers,
> > > > > > Kai
> > > > > > Am 22.10.2018 um 18:12 schrieb Jeroen Koekkoek
> > > > > > <jeroen@xxxxxxxxxxx>:
> > > > > > Hi Wayne,
> > > > > > Never mind. I saw a GitHub Organization request for a different
> > > > > > project, so I know what to do now. Thanks for your help.
> > > > > > Best regards,
> > > > > > Jeroen
> > > > > > P.S.
> > > > > > Should anyone find this thread in search of the same information,
> > > > > > here's an example of a BugZilla ticket requesting a GitHub
> > > > > > Organization
> > > > > > for a project: https://bugs.eclipse.org/bugs/show_bug.cgi?id=539829
> > > > > > On 2018-10-22 17:45, Jeroen Koekkoek wrote:
> > > > > > Hi Wayne,
> > > > > > I take it you mean requesting another repository under the Eclipse
> > > > > > organization? I can't find any bugs regarding the promotion to to a
> > > > > > top-level GitHub project?
> > > > > > Thanks in advance.
> > > > > > Regards,
> > > > > > Jeroen
> > > > > > On 2018-10-22 16:50, Wayne Beaton wrote: We added this sort of
> > > > > > support
> > > > > > recently. Open a bug against Community/GitHub to ask the webmaster
> > > > > > for
> > > > > > assistance.
> > > > > > Wayne
> > > > > > On Mon, Oct 22, 2018, 14:29 Kai Kreuzer, <kai@xxxxxxxxxxx> wrote:
> > > > > > Hi Jeroen,
> > > > > > I very much doubt that this is possible. Afaik, ALL Eclipse
> > > > > > projects
> > > > > > are under the eclipse github org and that's mandatory. For a
> > > > > > definitive
> > > > > > answer on this, you might want to contact EMO.
> > > > > > Note that it is possible to request additional repositories for
> > > > > > your
> > > > > > sub projects / components, though (see e.g.
> > > > > > https://github.com/eclipse/smarthome-packaging-sample, which
> > > > > > belongs
> > > > > > to
> > > > > > the SmartHome project, where the main repo is
> > > > > > https://github.com/eclipse/smarthome) - simply create a Bugzilla
> > > > > > issue
> > > > > > for it and the webmasters will provision it for you.
> > > > > > Regards,
> > > > > > Kai
> > > > > > Am 22.10.2018 um 12:04 schrieb Jeroen Koekkoek
> > > > > > <jeroen@xxxxxxxxxxx>:
> > > > > > Hi All,
> > > > > > I've got a question regarding the possibility of making cyclonedds
> > > > > > a
> > > > > > top-level GitHub project. I'm not sure if this is the right list to
> > > > > > ask
> > > > > > that question, if it isn't, please let me know the right list. If
> > > > > > it
> > > > > > is, and I'm looking at the problem in the wrong way, please let me
> > > > > > know
> > > > > > too. It's just that making it a top-level project seems like the
> > > > > > best
> > > > > > solution.
> > > > > > There's two reasons for me to want to promote it to a top-level
> > > > > > GitHub
> > > > > > project.
> > > > > > 1. Faster Travis and AppVeyor builds. Right now builds for pull
> > > > > > requests take a long time and that'll only get worse when more
> > > > > > projects
> > > > > > are going to join.
> > > > > > 2. Dependencies. We're planning on implementing a better IDL
> > > > > > compiler
> > > > > > to get quicker build and drop the Java dependency. Because we're
> > > > > > looking to implement support for more language bindings, now seems
> > > > > > like
> > > > > > the right time to do that. Now, the IDL compiler will need a
> > > > > > preprocessor. We've more-or-less decided to go with mcpp
> > > > > > (http://mcpp.sourceforge.net/). The product itself is hosted on
> > > > > > SourceForge, which is a little slow and code is still hosted in
> > > > > > svn.
> > > > > > The biggest problem however is that the original author isn't doing
> > > > > > any
> > > > > > work on the project anymore and nobody can accept patches etc. For
> > > > > > that
> > > > > > reason we want to "fork" it (svn import on GitHub) to fix some
> > > > > > small
> > > > > > bugs etc. In my opinion hosting the project under a top-level
> > > > > > cyclonedds GitHub project is the best place. That way it's neatly
> > > > > > separated from the cyclonedds code while all members/privileges etc
> > > > > > can
> > > > > > be managed by the project leads and committers.
> > > > > > Please let me know if it's possible or if there are better ways to
> > > > > > go
> > > > > > about this.
> > > > > > Best regards,
> > > > > > Jeroen Koekkoek
> > > > > > _______________________________________________
> > > > > > iot-pmc mailing list
> > > > > > iot-pmc@xxxxxxxxxxx
> > > > > > To change your delivery options, retrieve your password, or
> > > > > > unsubscribe
> > > > > > from this list, visit
> > > > > > https://dev.eclipse.org/mailman/listinfo/iot-pmc
> > > > > > _______________________________________________
> > > > > > iot-pmc mailing list
> > > > > > iot-pmc@xxxxxxxxxxx
> > > > > > To change your delivery options, retrieve your password, or
> > > > > > unsubscribe
> > > > > > from this list, visit
> > > > > > https://dev.eclipse.org/mailman/listinfo/iot-pmc
> > > > > > _______________________________________________
> > > > > > iot-pmc mailing list
> > > > > > iot-pmc@xxxxxxxxxxx
> > > > > > To change your delivery options, retrieve your password, or
> > > > > > unsubscribe
> > > > > > from this list, visit
> > > > > > https://dev.eclipse.org/mailman/listinfo/iot-pmc
> > > > > > _______________________________________________
> > > > > > iot-pmc mailing list
> > > > > > iot-pmc@xxxxxxxxxxx
> > > > > > To change your delivery options, retrieve your password, or
> > > > > > unsubscribe
> > > > > > from this list, visit
> > > > > > https://dev.eclipse.org/mailman/listinfo/iot-pmc
> > > > > > _______________________________________________
> > > > > > iot-pmc mailing list
> > > > > > iot-pmc@xxxxxxxxxxx
> > > > > > To change your delivery options, retrieve your password, or
> > > > > > unsubscribe
> > > > > > from this list, visit
> > > > > > https://dev.eclipse.org/mailman/listinfo/iot-pmc
> > > > > > _______________________________________________
> > > > > > iot-pmc mailing list
> > > > > > iot-pmc@xxxxxxxxxxx
> > > > > > To change your delivery options, retrieve your password, or
> > > > > > unsubscribe
> > > > > > from this list, visit
> > > > > > https://dev.eclipse.org/mailman/listinfo/iot-pmc
> > > > > > _______________________________________________
> > > > > > iot-pmc mailing list
> > > > > > iot-pmc@xxxxxxxxxxx
> > > > > > To change your delivery options, retrieve your password, or
> > > > > > unsubscribe
> > > > > > from this list, visit
> > > > > > https://dev.eclipse.org/mailman/listinfo/iot-pmc
> > > > > --
> > > > > Wayne Beaton
> > > > > Director of Open Source Projects | Eclipse Foundation, Inc. Meet us
> > > > > at
> > > > > EclipseCon Europe 2018: LUDWIGSBURG, OCTOBER 23 - 25
> > > > > _______________________________________________
> > > > > iot-pmc mailing list
> > > > > iot-pmc@xxxxxxxxxxx
> > > > > To change your delivery options, retrieve your password, or
> > > > > unsubscribe
> > > > > from this list, visit
> > > > > https://dev.eclipse.org/mailman/listinfo/iot-pmc
> > > > > _______________________________________________
> > > > > iot-pmc mailing list
> > > > > iot-pmc@xxxxxxxxxxx
> > > > > To change your delivery options, retrieve your password, or
> > > > > unsubscribe
> > > > > from this list, visit
> > > > > https://www.eclipse.org/mailman/listinfo/iot-pmc
> > > > > _______________________________________________
> > > > > iot-pmc mailing list
> > > > > iot-pmc@xxxxxxxxxxx
> > > > > To change your delivery options, retrieve your password, or
> > > > > unsubscribe
> > > > > from this list, visit
> > > > > https://www.eclipse.org/mailman/listinfo/iot-pmc
> > > > > _______________________________________________
> > > > > iot-pmc mailing list
> > > > > iot-pmc@xxxxxxxxxxx
> > > > > To change your delivery options, retrieve your password, or
> > > > > unsubscribe from this list, visit
> > > > > https://www.eclipse.org/mailman/listinfo/iot-pmc
> > > >
> > > > --
> > > >
> > > > Jens Reimann
> > > > Senior Software Engineer / EMEA ENG Middleware
> > > > Werner-von-Siemens-Ring 14
> > > > 85630 Grasbrunn
> > > > Germany
> > > > phone: +49 89 2050 71286
> > > > _____________________________________________________________________________
> > > >
> > > > Red Hat GmbH, www.de.redhat.com,
> > > > Registered seat: Grasbrunn, Commercial register: Amtsgericht
> > > > Muenchen,
> > > > HRB 153243,
> > > > Managing Directors: Paul Argiry, Charles Cachera, Michael Cunningham,
> > > > Michael O'Neill
> > > > _______________________________________________
> > > > iot-pmc mailing list
> > > > iot-pmc@xxxxxxxxxxx
> > > > To change your delivery options, retrieve your password, or
> > > > unsubscribe
> > > > from this list, visit
> > > > https://www.eclipse.org/mailman/listinfo/iot-pmc
> >
> > --
> >
> > Jens Reimann
> > Senior Software Engineer / EMEA ENG Middleware
> > Werner-von-Siemens-Ring 14
> > 85630 Grasbrunn
> > Germany
> > phone: +49 89 2050 71286
> > _____________________________________________________________________________
> >
> > Red Hat GmbH, www.de.redhat.com,
> > Registered seat: Grasbrunn, Commercial register: Amtsgericht Muenchen,
> > HRB 153243,
> > Managing Directors: Paul Argiry, Charles Cachera, Michael Cunningham,
> > Michael O'Neill
> _______________________________________________
> iot-pmc mailing list
> iot-pmc@xxxxxxxxxxx
> To change your delivery options, retrieve your password, or unsubscribe from this list, visit
> https://www.eclipse.org/mailman/listinfo/iot-pmc
