Re: [platform-releng-dev] buildable source zips for eclipse.org plugins

[Jeff McAffer <Jeff_McAffer@xxxxxxxxxx>]

Releng folks comment but it seems that
what Ben is after is the mechanism that is similar to what is used to build
things like GEF, EMF, etc.  That is, there is a base set of zips (e.g.,
prebuilt SDK) against which a set of source is compiled.  The source
can be fetched using CVS, untaring tarballs, ... it doesn't really matter.
 PDE build happens to have builtin support for fetching from CVS but
it is not carved in stone.

Ben, I didn't follow all of the patch
(I am blissfully ignorant of the build script details :-).  Have I
described what you want?  What you are doing?  Something completely
different?  Basically you are looking for a way to get/generate the
source tarballs?

Jeff

Ben Konrath <bkonrath@xxxxxxxxxx>
Sent by: platform-releng-dev-admin@xxxxxxxxxxx

01/24/2005 08:10 PM

Please respond to
platform-releng-dev

To	platform-releng-dev@xxxxxxxxxxx
cc
Subject	Re: [platform-releng-dev] buildable source zips for eclipse.org        plugins

Hi Kim, 

I appreciate the thorough response. My comments are interspersed below.

On Fri, 2005-01-21 at 14:50 -0500, Kim Moir wrote:
<snip>
> Eclipse is not built by checking out a src tree and running a
> makefile, the approach that  many other open source projects
take.  We
> use pde build to manage the complexity of determing dependancies and
> generating the appropriate ant scripts that are used to build Eclipse.
> This avoids the problem of having to modify a makefile each time a
> dependancy changes.  Having a static build.xml in the repository
to
> build eclipse does not scale for prerequisite changes.

I understand this process and I think it's a good idea. A lot of open
source projects generate their Makefiles with automake and/or autoconf
for the same reasons you are listing here. Just to clarify, I'm not
suggesting that Eclipse should move to using static build.xml's.  

> proposal>Eclipse does not currently have a standard method of
> releasing buildable source archives; the SDK can be built and packaged
> with the "srcIncluded" zip but the plugins cannot easily
be built.
> Part of the reason for this seems to be that users are intended to
use
> the update site feature of Eclipse to install plugins. The update
site
> functionality, however, does not offer the afore mentioned benefits.
> 
> With respect to building other plugins, there are dependancies that
> need to be managed.  You can't simply checkout the source for
a plugin
> and compile it without first determining the other plugins that it
> depends on.

Right, I'm not proposing that the plugins should be compiled
independently of the Eclipse SDK. What I want is the ability to build
plugins that aren't part of the Eclipse SDK. Using a headless Eclipse
that has all of the dependencies installed is completely acceptable.

> proposal> Eclipse 3.0 introduced a new build method for plugins;
the
> source is fetched as part of the build process by a special releng
> plugin that is responsible for the actual building. To build the
> plugin in a way that can be controlled, the sources are pre-fetched
> into custom tarball. At build time, cvs access is disabled and the
> plugin is built from the pre-fetched tarball in a chrooted
> environment. 
> 
> We are a bit confused by this comment.  I don't think it describes
> what we do.  Could you elaborate further?

Sorry, this is meant to describe how we are currently building the non-
SDK plugins for our Red Hat Developer Suite product. For plugins outside
of the SDK, we use your build method but instead of grabbing the source
from cvs at build time, we have a tarball that has the source pre-
fetched. I have updated the RFC to clarify this point and have posted a
patch that demonstrates how we are doing this for CDT 2.0.

http://people.redhat.com/bkonrath/eclipse/eclipse-cdt-no-cvs2-patch

> Here is a link that may be useful, it describes how to build eclipse
> components using ant and org.eclipse.releng.eclipsebuilder and
> org.eclipse.releng.basebuilder. 
> 
> http://dev.eclipse.org/viewcvs/index.cgi/~checkout~/org.eclipse.releng.eclipsebuilder/readme.html?rev=HEAD&content-type=text/html

Thanks, I am already aware of this document.

> proposal> The main problem, however, is that not all plugin projects
> tag their cvs tree correctly when a release is made. This prohibits
> anybody from building that particular version of the plugin. 
> 
> Each time we run an integration build, we check out the versions of
> plugins and features referred to in org.eclipse.releng/maps as the
> source for the build.   The org.eclipse.releng  project
is tagged each
> integration build with a version that corresponds to the build id
to
> easily identify the version of the builder that was used. This can
be
> used to determine the version of the plugins and features that are
> used in the build. We also rtag all plugins and features with a
> release name shortly following an official release such as 3.0.1.

Right, I've noticed that the plugins in the Eclipse SDK are tagged
consistently but I have encountered plugins that are not tagged in the
manner you describe. My main interest is with plugins outside of the
Eclipse SDK including plugins that are not hosted on eclipse.org.

Looking through the SDK build process I noticed that the tagging you are
referring to is accomplished with the tagMapFiles target in
org.eclipse.releng.eclipsebuilder/sdk/customTargets.xml. This target
does not seem to be required by
org.eclipse.releng.basebuilder/plugins/org.eclipse.pde.build_3.1.0/scripts/build.xml
and therefore is probably not used in plugins outside of the SDK. 

> Build method #2 does not reflect Eclipse design intent.  The
 src.zips
> are included in the SDK downloads to assist  during debugging,
they
> are not designed to be used to  build Eclipse.

I understand that. I included this build method because it is a method
that I tried and because it is similar a method used by jpackage.org to
compile the GEF plugin. Just a little clarification again, I do not want
to build Eclipse; the SDK "srcIncluded" zip allows me to do that.
What I
want is the ability to build plugins that are not included in this
archive. For example, I would like to be able to build plugins such as
the GEF, EMF, CDT, FTP_WebDAV etc.

> The scripts used to create srcIncluded and srcFetch scripts actually
> reside on an internal server for historical reasons for the simple
> reason that we haven't had time to move them to the external server.
> We'll look into moving them as time permits. In summary, it's a zip
of
> the source tree, the generated build scripts that arise from the build
> process described in the link above.  We'll provide more details
when
> we move the scripts to the external server.  I've opened a bug
for
> this issue, please feel free to annonate it with additional comments
> https://bugs.eclipse.org/bugs/show_bug.cgi?id=83443

Great! I've added myself to the cc field of that bug. I'm looking
forward to reviewing the "srcIncluded" and "srcFetch"
zip file
generation process when it become available. I will then be able to
propose a more complete solution.

Thanks again for the comprehensive response,

Ben



_______________________________________________
platform-releng-dev mailing list
platform-releng-dev@xxxxxxxxxxx
http://dev.eclipse.org/mailman/listinfo/platform-releng-dev