Re: [smila-dev] aperture bundles for smila integration

[Leo Sauermann <leo.sauermann@xxxxxxx>]

Hi Smila, Aperture,

In the past.... we intentionally did NOT turn each dependency of Aperture into a separate OSGi bundle because we learned that "special" versions of the dependencies work better than others, so it was easier to just put all compatible ones into the bundle directly as libs. But now.... plan B, that was long prepared by us anyway.

If "you"(=smila) do the work of putting the *needed* libs into ORBIT repo, then we can do the packaging of aperture into microbundles.
That means:
* we all concentrate on the Mimetype Identifier and Exctractors now!
** what libs are needed for the extractors and mimetype identifiers?
(ignore the subcrawlers and crawlers!)

Aperture People (=Antoni, could you  make a ticket for this), do:
* separate core OSGi bundle (= we have this already, just verify that we have a core Aperture bundle that doesn't depend on weird libs)
   *  why is applewrapper-0.2.jar in the core bundle?
   *  why is aduna-commons-xml-2.0 in the core (this is ok I think because its needed in the SAX util package of us)?
   * otherwise, this is clean as a baby face: Require-Bundle: org.semweb4j.rdf2go.api,org.semweb4j.rdf2go.impl.base
* separate all extractors into "individual bundles" (= we planned that this step will happen sometime, now it does, so we have to extend the build.xml a bit. this should be mangeable given our year-long preparations)
* we put all crawlers & subcrawlers into an extra package "the rest"


the "individual extractor bundles" have to have proper OSGi dependencies - on released OSGi Jars of 3rd party libs

* for funny stuff like demork, we ignore it and put it into "the rest"
* For central stuff like PDFBox, we have to have a release - why not join PDFBox and make one? (or we pay them again something to do it, we already outsorced some work to them)
* crawlers and subcrawlers go into "the rest"

==> we have a release of three chunks pretty soon, one core, many good extractors, all the bad ones in "the rest"

It will be a LOT OF WORK for making the individual Manifests of the individual extractors right,
this is a mixture of the selectors.xml and the finegrained activators, can be a bit tricky.
will take time.
I guess it will take more than a month, given the Christmas break in between.

overall, this procedure has the following advantages:
* we follow the guidelines and path we did the last years, no changes in plans
* we have a good release for SMILA/Eclipse/OSGI with core features SOON
* we can clean up later "the rest" but do not make ugly decisions today that will make this harder

best
Leo


Coming to the 5 problems and my views on them:

Apart from that there are 5 problems:
demork-2.1.jar - Gunnar, can you make an official release?
  

if not, we move it to "the rest"
we should move demork into a separate sourceforge project where we ALL join as admins, so that someone can pick up the mess if needed and can make decisions.

DFKIUtils2.jar - this one is LGPL, i'd rather rewrite DeliciousCrawler
to remove the need for it
  

I asked Andreas Lauer, the author, about a license change or a rewrite. He would also have to do a publshing of the libs.
answer pending.

jpim - this project appears to be dead, we may have to fork it, i'll
try to get in touch with the author
  

Fork is ok, or they should add us as admins.
Otherwise - we move it to "the rest"

sesame - will have to integrate bnd in the build process of Aduna
  

bnd?

pdfbox - tricky, there has been no release in years, thousands of
people use trunk, i'm sure that ESF does it too
  

we can bug them further to release, pay them a little, or get ourselves into the admin team.
Do they have sensible Ant scripts to do releases?

best
Leo




It was Antoni Mylka who said at the right time 05.12.2008 11:25 the following words:

2008/12/5  <Daniel.Stucky@xxxxxxxxxxx>:
  

Hi aperture team,

I just took a look at the osgi distribution of release 1.2.0. There are
several issues that need to be addressed to allow integration of
aperture in smila.

- non EPL compatible licenses
There are some 3rd party components which licenses are not compatible
with EPL (e.g. jaudiotagger-1.0.8.jar is LGPL). These libs simply cannot
be contributed to smila. These libs and the dependencies on them have to
be made optional in aperture, perhaps by fine grained bundles (see
below).
    

Does this apply only to LGPL? What about Sun libs (JAI, Javamail etc.)
Please have a look at
http://aperture.wiki.sourceforge.net/Dependencies and write exactly
which dependencies are inacceptable because of their license.

  

- eclipse legal process
we have to create a Contribution Questionnaire (CQ) for each bundle and
for each lib used in the bundles. That means a CQ for every (nested) jar
is needed. Of course we can skip CQs for non EPL compatible libs, as
they will be rejected anyway. At first we should focus on functionality
we want to make use of in smila (this would be MimeTypeIdentifier an
Extractors only). For some jars used in aperture this was already done
and we can simply reuse those CQs (e.g. commons-*.jar).
Eclipse also usually allows only usage of "released" 3rd party software,
not any nightly builds or beta releases.
    

Does this include test dependencies, i.e. jars only used for tests,
the final bundle does not depend on them
(infSail/unionSail/nrlvalidator).

Apart from that there are 5 problems:
demork-2.1.jar - Gunnar, can you make an official release?
DFKIUtils2.jar - this one is LGPL, i'd rather rewrite DeliciousCrawler
to remove the need for it
jpim - this project appears to be dead, we may have to fork it, i'll
try to get in touch with the author
sesame - will have to integrate bnd in the build process of Aduna
pdfbox - tricky, there has been no release in years, thousands of
people use trunk, i'm sure that ESF does it too

  

- fine grained bundles
In order to be able to integrate selected functionality (either because
we don't need it or we can't use it because of license issues) a finer
grained bundleing is needed. In addition, some of the 3rd party jars
used in aperture are already used in smila. We provide each 3rd party
jar as a separate bundle (and contribute them to Orbit). This allows for
easier update of 3rd party dependencies. Perhaps this is a practical
approach for aperture, too.
    

Right now the impl bundle contains the following jars that need to be
turned into bundles if you want to enforce this policy:

 activation-1.0.2-upd2.jar
 ant-compression-utils-1.7.1.jar
 applewrapper-0.2.jar
 bcmail-jdk14-132.jar
 bcprov-jdk14-132.jar
 commons-codec-1.3.jar
 commons-httpclient-3.1.jar
 commons-lang-2.3.jar
 DFKIUtils2.jar
 flickrapi-1.0.jar
 fontbox-0.2.0-dev.jar
 htmlparser-1.6.jar
 ical4j-1.0-beta4.jar
 jacob-1.10.jar
 jacob.dll
 jai_codec-1.1.3.jar
 jai_core-1.1.3.jar
 jaudiotagger-1.0.8.jar
 JempBox-0.2.0.jar
 jpim-0.1-aperture-1.jar
 jutf7-0.9.0.jar
 mail-1.4.jar
 metadata-extractor-2.4.0-beta-1.jar
 mstor-0.9.11.jar
 pdfbox-0.7.4-dev-20071030.jar
 poi-3.0.2-FINAL-20080204.jar
 poi-scratchpad-3.0.2-FINAL-20080204.jar

We have included them inside precisely not to turn them into proper
bundles ourselves. This is something we might use some help for. All
of them need to be equipped with proper osgi manifests and contributed
to orbit am I right?

  

Please let us discuss the listed issues and how both projects can help
each other. Of course we are willing to contribute to aperture to
achieve smila integration!
    

Great, since doing all this will require quite a lot of work.

  

For further communication please send copies to smila-dev@xxxxxxxxxxx.

    

OK


  

_______________________________________________
smila-dev mailing list
smila-dev@xxxxxxxxxxx
https://dev.eclipse.org/mailman/listinfo/smila-dev
  

-- 
____________________________________________________
DI Leo Sauermann       http://www.dfki.de/~sauermann 

Deutsches Forschungszentrum fuer 
Kuenstliche Intelligenz DFKI GmbH
Trippstadter Strasse 122
P.O. Box 2080           Fon:   +49 631 20575-116
D-67663 Kaiserslautern  Fax:   +49 631 20575-102
Germany                 Mail:  leo.sauermann@xxxxxxx

Geschaeftsfuehrung:
Prof.Dr.Dr.h.c.mult. Wolfgang Wahlster (Vorsitzender)
Dr. Walter Olthoff
Vorsitzender des Aufsichtsrats:
Prof. Dr. h.c. Hans A. Aukes
Amtsgericht Kaiserslautern, HRB 2313
____________________________________________________