On Jan 7, 2008 3:45 AM, Jeff McAffer <
Jeff_McAffer@xxxxxxxxxx> wrote:
[redoing the subject to be more accurate
as we are well and truly off the original topic...]
That doc talks about "releases"
but does not define the term. Do you know what the expectation is?
One of our builds can be 200MB or so and we end up running at least
one and sometimes multiple a day in some (unhappy) cases. Last I
knew ibiblio was not keen on this sort of publishing. Our weekly
integration builds would be a candidate though they might prefer only the
6week milestones. Either way the process is the same. We run
several such builds during the integration/milestone week and in the end
declare one of them as "good" and promote it. We do not
apriori know which build is going to be "the one" as a mess o'
testing is done after the build (automated and manual). A publishing
step such as discussed here would have to be after the fact and so cannot
be built into the build itself.
What is really needed is a tool that
would take an Eclipse build (e.g., update site or download zip or ...)
and fire it into the desired repo in the desired form.
This would
accommodate the myriad of different build mechanisms/processes that exist
today at Eclipse and allow anyone who wanted the content in another repo
to do so without having to depend on the particular project build team/process.
With this in hand we could talk about getting particular build teams
to call the tool when they promote a build. Alternatively and more
generally it would be interesting to have an RSS feed (forgive me Kim)
for build promotions. External parties can then start sucking content
as soon as it is "released".
Jeff
I guess I'm a little confused... There's a few sites
which will do the mirroring for you, like ibiblio.org. Ibiblio, and
the myriad of mirrors for the site, host a plethora of maven artifacts
for projects all over. I'm not completely up on the mechanisms for
publishing to these repositories - i.e. what are the protocols, contacts,
etc - but considering that plenty of projects and code bases publish there
means it can't be too hard to do. Further, tools exist for Ant and
Maven to make this publishing part of the build release process. Or,
at the very least, a target of the build which can be manually called when
appropriate.
So, it seems like everyone could be happy. Those
actually build outside of Eclipse (strange, I know) could have access to
Eclipse artifacts from a maven repository hosted by Ibiblio rather than
having to provide your own space, and you Eclipse folks would have this
folded into your build in a painless fashion.
This link has the Apache documentation on publishing to
their site which is mirrored by Ibiblio: http://www.apache.org/dev/release-publishing.html
Although this wouldn't be what you'd be doing, I'm fairly
certain the process would be similar.
Hope this helps all involved...
On Jan 6, 2008, at 5:29 PM, Jeff McAffer wrote:
Hey Alex,
Please note that I started my reply by saying "I don't disagree".
The point is that the team available to do this sort of work is small
and the diskspace/mirror capacity to do it is similarly restricted. I'm
not happy about it either. As with many of these things, if there
are folks in the community who care deeply about this and want to do the
work/supply the space, we'd be happy to work with them.
You are also correct in observing that in the p2 work we did indeed decide
to have separately downloadable repository (metadata and artifact) indexing.
Further our general approach has been to allow for individual bundle
download/install rather than whole feature management. These moves
have been a big win and should go a long way towards addressing the valid
usecases you point out.
I completley agree that the update site structure is lacking in these areas
but disagree with you that Maven addresses the issues. The POM file
certainly can represent the metadata in a bundle but its primary focus
is to drive the build. So, for example, if the build generates the
dependency information into the output manifest.mf (e.g., using bnd or
mangen style processing) then that dependency info is not going to be in
the POM. The standard maven repo does not have an index so it still
does not help you find something unless you know exact ids and version
numbers. If you have that information then you can get it from a
Maven repo or Update site equally well. (yes, I realize that there is a
Maven repo indexer. This is akin to the update site digest but with
a finer grain)
I don't mean to slam Maven at all. It is a fine tool and many people
like it. In the course of the p2 work we have learned some things
from Maven and a careful eye will observe some Mavenesque characteristics
in p2. The standard Maven repo structure however does not meet our
requirements (we spent quite a bit of time with various Maven folks going
over the possibilities here). As a design point we opted out of
defining "one" repo format in favour of allowing people to add
their own. This opens the door for people with a Maven (or other)
bent to add the support they need. Similarly, our default repo structure
is not that complicated so someone in the Maven world should be able to
write a Maven repo class to manage p2's simple repos or even use p2 API
to write them (thus mapping onto any p2 repo).
Anyway, we are way off topic here. It suffices to say that the orginal
poster's problems were caused by someone creating and publishing a faulty
commons.logging bundle. I doubt that was created by the eclipse team
or if it was, it was a prerelease version (the Europa version has the necessary
dependency version numbers). In the ideal world it would actually
be the Apache folks who produced these bundles and we would all just consume
them. But I'm ahead of myself...
Jeff
On Jan 6, 2008 1:18 AM, Jeff McAffer <Jeff_McAffer@xxxxxxxxxx>
wrote:
>
> I won't disagree and would certainly welcome anyone who wanted to
> create/maintain such a repo. It seems that someone has done
this at
> maven.org. The Maven user community should encourage that person
to
> continue doing the work. I don't really see eclipse.org or the
projects
> hosting another copy of all their bundles laid out in the standard
Maven dir
> structure. Our mirrors would be upset at the doubling in size
and we have
> to maintain the update site structure for the Eclipse user community
anyway.
> As I understand Maven, it is at least in theory possible to define
different
> kinds of repos so, for example, Eclipse update sites could be directly
> accessed from Maven. No idea what is involved in that but it
feels like the
> right kind of direction.
The problem with the Eclipse downloads is that there's actually no
real way of knowing where to look for something. Quite apart from the
arbitrary tools/platform/technology project splits, each directory is
versioned in an S-3.4M2-200709210919 directory whose name is computed
based on the time that something was run, rather than anything
detectable. In order to provide my 'direct download links' I had to
manually grep for the random-date-string in order to update my links
each time a new drop was released.
Not only that, but eclipse.org doesn't provide any really useful way
of finding out what plugins are hosted, or even where. It also ends up
wasting a *heap* of bandwith; the -win32-win32-, -win32-wpf-,
-linux-gtk-, -linux-motif-, combos are all pretty much 95%+ the same,
with only a few native bits different from each of them. There's no
'one' place to download the individual plugins.
Even the update sites aren't helpful. You can't browse to them, and
even when you do (e.g. via a mirror) it's not clear as to which plugin
is stored in which location.
Lastly, there's no metadata available. One of the things which will
have been observed from the P2 work is that it is necessary to have a
(small) downloadable metadata file that can be downloaded without
needing to download the bits that the metadata refers to. In Maven,
this is the POM, and without it, a maven repo is just an HTTP store.
Whether the P2 work can be leveraged to have a Maven->P2 link (or vice
versa) is something that could be investigated, but the problem is
that if you rely on e.g. additions to the maven repository then they
are either done sporadically or erroneously. In the case of Eclipse,
several (different) people have uploaded variants of some of the core
Eclipse plugins, but then never get updated as newer versions are
available. If this process were baked into the Eclipse release
process, more people might be tempted to build upon OSGi systems than
they are at the moment.
For those that are interested in building OGSi with Maven, then take a
look at Pax Construct:
http://wiki.ops4j.org/confluence/display/ops4j/Pax+Construct
There's also Pax Runner, which allows bundles to be launched inside
different OSGi engines, such as Equinox and Felix:
http://wiki.ops4j.org/confluence/display/ops4j/Pax+Runner
Alex
_______________________________________________
equinox-dev mailing list
equinox-dev@xxxxxxxxxxx
https://dev.eclipse.org/mailman/listinfo/equinox-dev
_______________________________________________
equinox-dev mailing list
equinox-dev@xxxxxxxxxxx
https://dev.eclipse.org/mailman/listinfo/equinox-dev
_______________________________________________
equinox-dev mailing list
equinox-dev@xxxxxxxxxxx
https://dev.eclipse.org/mailman/listinfo/equinox-dev
_______________________________________________
equinox-dev mailing list
equinox-dev@xxxxxxxxxxx
https://dev.eclipse.org/mailman/listinfo/equinox-dev