Re: [udig-devel] A need for branching uDig and Geotools, and how to merg

[Adrian Custer <acuster@xxxxxxxxx>]

Hello Harald,

Thanks for your big post and for your work exploring OSGi. 

There are two issues in your post:
 1) how can you keep working downstream?
 2) how do we support your work upstream?

For the second of these, we have not yet needed OSGi bundling in
Geotools and we are all aware that Java 7 is going to integrate a new
moduling system based on OSGi so we have been waiting for that to
consider refactoring the code into modules. For now, the servers throw
all the jars into the overall war and uDig throws all the jars into the
single plugin. This is why you are on your own in this effort---if you
need it now, you are going to have to code it. That said, we will be
quite interested in learning from your experience and would value your
input. I would think the uDig community would be especially interested.

For the first, you seem indeed better off with a distributed versioning
system, if you can work that way. I have been hoping for over two years
that Geotools itself would move that way but part of the community does
not yet need, or maybe understand, the benefits. For now, then, we are
stuck with an SVN server upstream. 

For your information, we have been maintaining a geotools hg repository
at:
   http://hg.geomatys.fr/geotools/trunk 
which you could pull from instead of from yours. We update it from trunk
periodically by hand; we have not yet bothered to automate it. If you
run your 'hg convert' yourself, please consider using the --datesort
flag so we keep at least that level of sanity in our different
conversion repositories. Not that it really matters, but it would be
more elegant for readers of the commit messages to have the svn commits
in the same order everywhere.

So, I unfortunately expect your work will not re-emerge directly into
the upstream geotools repo unless you are the one to bring it there.
However, I would expect us to be able to benefit from your experience.
Keep us posted, if you can, on your work. If you can't work downstream
on a mercurial conversion for a technical reason that the upstream
system could help out with, then come to one of the weekly IRC meetings
and propose some solution--we are generally willing to do some work to
help others keep working.

cheers,
adrian


On Mon, 2008-11-03 at 21:07 +0100, Wellmann, Harald wrote:
> Sorry for cross-posting, but I think this issue has an impact on both communities. This post is going to be long, so let me start with a summary:
> 
> - For a couple of months, I've been experimenting with uDig to build a viewer for a proprietary car navigation database format on top of it. By now, I am positive that uDig (in itself and through its usage of Geotools) provides an excellent basis for our work, and it would be foolish not to use this basis.
> 
> - uDig binary distributions or SDKs do not work for us due to version clashes in third-party dependencies of our own bundles and of the monolithic net.refractions.udig.libs bundle which includes all required Geotools libraries and external dependencies. 
> 
> - As a consequence, I had to build uDig from source and start modifying the sources.
> 
> - I've now got a quick-and-dirty demo running, but for a more permanent solution the Geotools libraries and all third-party dependencies need to be converted to OSGi bundles, and all uDig plugin manifests need to be revised accordingly. This requires modifying Geotools as well (at least the POMs, not the sources.)
> 
> - The batch builds for uDig and Geotools both do not work without modifications in our environment. (uDig does not build at all, Geotools has a couple of test failures.)
> 
> At this point, I'm stuck and I do not see how to get any further without creating branches both of uDig and Geotools or even cloning the repositories. There are a couple of options:
> 
> 1) Nobody has similar problems and nobody sees the point of what I am trying to do. In that case, I could simply work on private clones, the communities would not see any of my modifications and I would be detached from any fixes and enhancements occurring on the trunks. 
> 
> (No, I don't really mean that, or else I wouldn't be writing this message.)
> 
> 2) Working branches will be created in the uDig and Geotools repositories for my team and myself to do the required modifications, merging changes from the trunk as often as possible. If and when our branches have stabilized and the communities have verified that the results are useful and correct, the working branches can be merged into the trunk and die. 
> 
> (At first sight, this is the approach I would prefer, but I have second thoughts for technical reasons: Both repositories run on Subversion over HTTP, locking out anyone in a corporate IT environment with a web proxy blocking the non-standard HTTP requests used by svn. The only way we can use svn is over HTTPS. Moreover, the repository servers seem to have frequent downtimes. {udig,lists,gtsvn}.refractions.net have been off and on sporadically on Sunday and today, not for the first time.)
> 
> 3) As a variant of 2), I could clone both repositories, have them hosted by an open source provider offering services usable though our proxy, maintain a trunk mirror branch and an OSGi branch in my clones, so that anyone could track what we are doing and pull in any of our modifications into the official trunks at their own discretion.
> 
> (Actually, I've started working with Mercurial and hgimportsvn to mirror the Geotools trunk and start hacking on a osgi branch. So far, this setup seems to work well, and maybe even better than on Subversion which does not support any incremental merges between branches on server versions < 1.5.)
> 
> So much for the problem and the possible scenarios. 
> 
> What I don't want to do is tread on anyone's toes, and I do not expect anyone to solve my problems for me. Neither do I think I could write a better uDig or Geotools on my own. I just have to solve a problem which I believe to be of more than just private interest, and which could be regarded simply as an alternative distribution of (a subset of) udig and Geotools.
> 
> Some more detailed proposals for this task:
> 
> My feeling is that our plugins depend on not more than 25 % of uDig and maybe 5% of Geotools. (We do not need any of the DataStore formats supported by uDig and Geotools, and no editing features either.) 
> 
> So the first step is to identify the minimal subset of uDig and Geotools required by our plugins. For Geotools, this will probably be just modules/library/* and not much else.
> 
> All snapshot dependencies have to be replaced by milestone or real releases. Currently, we are working on a trunk snapshot of uDig using a snapshot version of Geotools using a snapshot version of Geoapi, so we cannot control when Maven pulls in an updated snapshot which might break our build.
> 
> All third-party dependencies of this subset have to be osgified. Some of them can be taken from the SpringSource Enterprise Bundle repository, others can be wrapped automatically using the maven-bundle-plugin, and special cases need some manual treatment. All these osgified artifacts need to be deployed to a public Maven repository with different group or artifact IDs, and the Geotools POMs have to use the osgified dependencies instead of the original ones.
> 
> Then the Geotools libraries will be osgified as outlined in 
> http://docs.codehaus.org/display/GEOTDOC/03+GeoTools+and+Eclipse+or+OSGi, without breaking the META-INF/services mechanism and without losing the capability of being used as plain old JARs on the classpath.
> 
> Finally, net.refractions.udig.libs will be replaced by a collection of individual Geotools and third-party bundles. Other uDig plugins will drop the dependency on bundle n.r.u.libs and declare new dependencies on individual packages org.geotools.foo and any required third-party packages.
> 
> Once we're done with this conversion for our subset, the scope can be extended to the rest of uDig and Geotools.
> 
> That's it... Please let me hear your opinions, or maybe there is even a simple solution I have failed to see so far.
> 
> Regards,
> 
> Harald 
>  
> *******************************************
> innovative systems GmbH Navigation-Multimedia
> Geschaeftsfuehrung: Edwin Summers - Kevin Brown - Regis Baudot 
> Sitz der Gesellschaft: Hamburg - Registergericht: Hamburg HRB 59980 
>  
> *******************************************
> Diese E-Mail enthaelt vertrauliche und/oder rechtlich geschuetzte Informationen. Wenn Sie nicht der richtige Adressat sind oder diese E-Mail irrtuemlich erhalten haben, informieren Sie bitte sofort den Absender und loeschen Sie diese Mail. Das unerlaubte Kopieren sowie die unbefugte Weitergabe dieser Mail ist nicht gestattet.
> This e-mail may contain confidential and/or privileged information. If you are not the intended recipient (or have received this e-mail in error) please notify the sender immediately and delete this e-mail. Any unauthorized copying, disclosure or distribution of the contents in this e-mail is strictly forbidden.
> *******************************************
> _______________________________________________
> User-friendly Desktop Internet GIS (uDig)
> http://udig.refractions.net
> http://lists.refractions.net/mailman/listinfo/udig-devel