Re: [equinox-dev] Maven and all that [was: Jetty and commons logging]

[Jeff McAffer <Jeff_McAffer@xxxxxxxxxx>]

Thanks Tom.  So it seems that people
can get what they want now.  Someone should just run the tool at the
frequency needed.  Wonder if that is how the original content got
into the maven.org repo.

Your point about the mapping is well
taken.  It would be VERY unfortunate if the Maven using community
ended up producing deliverables that had differently named JAR files than
the rest of the Eclipse community.  Personally I would find it hard
to in any way sanction a repo of any kind that resulted in people getting
JARs with names different than what we produced.  As has been said
before, the JAR name technically does not matter to the OSGi framework
but in reality there are lots of other things (tools, lists, doc, books,
newgroup posts, ...) that use the names.  Changing file names is just
a recipe for confusion and disarray.

Jeff

"Tom Huybrechts" <tom.huybrechts@xxxxxxxxx>
Sent by: equinox-dev-bounces@xxxxxxxxxxx

01/07/2008 08:15 AM

Please respond to
Equinox development mailing list <equinox-dev@xxxxxxxxxxx>

To	"Equinox development mailing list" <equinox-dev@xxxxxxxxxxx>
cc
Subject	Re: [equinox-dev] Maven and all that [was: Jetty and commons logging]

On Jan 7, 2008 3:45 AM, Jeff McAffer <Jeff_McAffer@xxxxxxxxxx>
wrote:

[redoing the subject to be more accurate as we are well and truly off the
original topic...] 

That doc talks about "releases" but does not define the term.
 Do you know what the expectation is?  One of our builds can
be 200MB or so and we end up running at least one and sometimes multiple
a day in some (unhappy) cases.  Last I knew ibiblio was not keen on
this sort of publishing.  Our weekly integration builds would be a
candidate though they might prefer only the 6week milestones.  Either
way the process is the same.  We run several such builds during the
integration/milestone week and in the end declare one of them as "good"
and promote it.  We do not apriori know which build is going to be
"the one" as a mess o' testing is done after the build (automated
and manual).  A publishing step such as discussed here would have
to be after the fact and so cannot be built into the build itself.  

What is really needed is a tool that would take an Eclipse build (e.g.,
update site or download zip or ...) and fire it into the desired repo in
the desired form. 

These tools exist. See e.g. http://maven.apache.org/plugins/maven-eclipse-plugin/to-maven-mojo.html
(although I do not agree with the conventions
it uses to map osgi identifiers onto Maven identifiers) or Tycho ( http://svn.m2eclipse.codehaus.org/browse/~raw,r=374/m2eclipse/tycho/trunk/readme.txt
) which I wrote but which could use a little
more documentation.

Tom
 
 This would accommodate the myriad
of different build mechanisms/processes that exist today at Eclipse and
allow anyone who wanted the content in another repo to do so without having
to depend on the particular project build team/process.  With this
in hand we could talk about getting particular build teams to call the
tool when they promote a build.  Alternatively and more generally
it would be interesting to have an RSS feed (forgive me Kim) for build
promotions.  External parties can then start sucking content as soon
as it is "released".   

Jeff

Hal Hildebrand <hal.hildebrand@xxxxxxxxxx>
Sent by: equinox-dev-bounces@xxxxxxxxxxx

01/06/2008 09:08 PM

Please respond to
Equinox development mailing list <equinox-dev@xxxxxxxxxxx>

To	Equinox development mailing list <equinox-dev@xxxxxxxxxxx>
cc
Subject	Re: [equinox-dev] Jetty and commons logging

I guess I'm a little confused...  There's a few sites which will do
the mirroring for you, like ibiblio.org.
 Ibiblio, and the myriad of mirrors for the site, host a plethora
of maven artifacts for projects all over.  I'm not completely up on
the mechanisms for publishing to these repositories - i.e. what are the
protocols, contacts, etc - but considering that plenty of projects and
code bases publish there means it can't be too hard to do.  Further,
tools exist for Ant and Maven to make this publishing part of the build
release process.  Or, at the very least, a target of the build which
can be manually called when appropriate. 

So, it seems like everyone could be happy.  Those actually build outside
of Eclipse (strange, I know) could have access to Eclipse artifacts from
a maven repository hosted by Ibiblio rather than having to provide your
own space, and you Eclipse folks would have this folded into your build
in a painless fashion. 

This link has the Apache documentation on publishing to their site which
is mirrored by Ibiblio: http://www.apache.org/dev/release-publishing.html
Although this wouldn't be what you'd be doing, I'm fairly certain the process
would be similar. 

Hope this helps all involved... 

On Jan 6, 2008, at 5:29 PM, Jeff McAffer wrote: 


Hey Alex, 

Please note that I started my reply by saying "I don't disagree".
 The point is that the team available to do this sort of work is small
and the diskspace/mirror capacity to do it is similarly restricted.  I'm
not happy about it either.  As with many of these things, if there
are folks in the community who care deeply about this and want to do the
work/supply the space, we'd be happy to work with them.

You are also correct in observing that in the p2 work we did indeed decide
to have separately downloadable repository (metadata and artifact) indexing.
 Further our general approach has been to allow for individual bundle
download/install rather than whole feature management.  These moves
have been a big win and should go a long way towards addressing the valid
usecases you point out. 

I completley agree that the update site structure is lacking in these areas
but disagree with you that Maven addresses the issues.  The POM file
certainly can represent the metadata in a bundle but its primary focus
is to drive the build.  So, for example, if the build generates the
dependency information into the output manifest.mf (e.g., using bnd or
mangen style processing) then that dependency info is not going to be in
the POM.  The standard maven repo does not have an index so it still
does not help you find something unless you know exact ids and version
numbers.  If you have that information then you can get it from a
Maven repo or Update site equally well. (yes, I realize that there is a
Maven repo indexer.  This is akin to the update site digest but with
a finer grain) 

I don't mean to slam Maven at all.  It is a fine tool and many people
like it.  In the course of the p2 work we have learned some things
from Maven and a careful eye will observe some Mavenesque characteristics
in p2.  The standard Maven repo structure however does not meet our
requirements (we spent quite a bit of time with various Maven folks going
over the possibilities here).   As a design point we opted out of
defining "one" repo format in favour of allowing people to add
their own.  This opens the door for people with a Maven (or other)
bent to add the support they need.  Similarly, our default repo structure
is not that complicated so someone in the Maven world should be able to
write a Maven repo class to manage p2's simple repos or even use p2 API
to write them (thus mapping onto any p2 repo). 

Anyway, we are way off topic here.  It suffices to say that the orginal
poster's problems were caused by someone creating and publishing a faulty
commons.logging bundle.  I doubt that was created by the eclipse team
or if it was, it was a prerelease version (the Europa version has the necessary
dependency version numbers).  In the ideal world it would actually
be the Apache folks who produced these bundles and we would all just consume
them.  But I'm ahead of myself... 

Jeff

"Alex Blewitt" <alex.blewitt@xxxxxxxxx>
Sent by: equinox-dev-bounces@xxxxxxxxxxx

01/06/2008 01:35 PM

Please respond to
Equinox development mailing list <equinox-dev@xxxxxxxxxxx >

To	"Equinox development mailing list" <equinox-dev@xxxxxxxxxxx>
cc
Subject	Re: [equinox-dev] Jetty and commons logging

On Jan 6, 2008 1:18 AM, Jeff McAffer <Jeff_McAffer@xxxxxxxxxx>
wrote:
>
> I won't disagree and would certainly welcome anyone who wanted to
> create/maintain such a repo.  It seems that someone has done
this at
> maven.org.
 The Maven user community should encourage that person to
> continue doing the work.  I don't really see eclipse.org
or the projects
> hosting another copy of all their bundles laid out in the standard
Maven dir
> structure.  Our mirrors would be upset at the doubling in size
and we have
> to maintain the update site structure for the Eclipse user community
anyway.
> As I understand Maven, it is at least in theory possible to define
different
> kinds of repos so, for example, Eclipse update sites could be directly
> accessed from Maven.  No idea what is involved in that but it
feels like the
> right kind of direction.

The problem with the Eclipse downloads is that there's actually no
real way of knowing where to look for something. Quite apart from the
arbitrary tools/platform/technology project splits, each directory is
versioned in an S-3.4M2-200709210919 directory whose name is computed
based on the time that something was run, rather than anything
detectable. In order to provide my 'direct download links' I had to
manually grep for the random-date-string in order to update my links
each time a new drop was released.

Not only that, but eclipse.org
doesn't provide any really useful way
of finding out what plugins are hosted, or even where. It also ends up
wasting a *heap* of bandwith; the -win32-win32-, -win32-wpf-,
-linux-gtk-, -linux-motif-, combos are all pretty much 95%+ the same,
with only a few native bits different from each of them. There's no
'one' place to download the individual plugins.

Even the update sites aren't helpful. You can't browse to them, and
even when you do (e.g. via a mirror) it's not clear as to which plugin
is stored in which location.

Lastly, there's no metadata available. One of the things which will
have been observed from the P2 work is that it is necessary to have a
(small) downloadable metadata file that can be downloaded without
needing to download the bits that the metadata refers to. In Maven,
this is the POM, and without it, a maven repo is just an HTTP store.

Whether the P2 work can be leveraged to have a Maven->P2 link (or vice
versa) is something that could be investigated, but the problem is
that if you rely on e.g. additions to the maven repository then they
are either done sporadically or erroneously. In the case of Eclipse,
several (different) people have uploaded variants of some of the core
Eclipse plugins, but then never get updated as newer versions are
available. If this process were baked into the Eclipse release
process, more people might be tempted to build upon OSGi systems than
they are at the moment.

For those that are interested in building OGSi with Maven, then take a
look at Pax Construct:
http://wiki.ops4j.org/confluence/display/ops4j/Pax+Construct

There's also Pax Runner, which allows bundles to be launched inside
different OSGi engines, such as Equinox and Felix:
http://wiki.ops4j.org/confluence/display/ops4j/Pax+Runner

Alex
_______________________________________________
equinox-dev mailing list
equinox-dev@xxxxxxxxxxx
https://dev.eclipse.org/mailman/listinfo/equinox-dev

_______________________________________________
equinox-dev mailing list
equinox-dev@xxxxxxxxxxx
https://dev.eclipse.org/mailman/listinfo/equinox-dev

_______________________________________________
equinox-dev mailing list
equinox-dev@xxxxxxxxxxx
https://dev.eclipse.org/mailman/listinfo/equinox-dev


_______________________________________________
equinox-dev mailing list
equinox-dev@xxxxxxxxxxx
https://dev.eclipse.org/mailman/listinfo/equinox-dev

_______________________________________________
equinox-dev mailing list
equinox-dev@xxxxxxxxxxx
https://dev.eclipse.org/mailman/listinfo/equinox-dev