Home » Archived » Eclipse SmartHome » Change buildsystem to Gradle or alternatives
| Change buildsystem to Gradle or alternatives [message #1769769] |
Thu, 03 August 2017 09:21  |
 David Graeff
Messages: 17
Registered: April 2016 |
Junior Member |
|
|
Hey all,
The pure maven based buildsystem seems to be too slow for the existing codebase. On my system a linux kernel with about 800k lines of code builds in 7:30min while ESH takes 20min on my Core i7 system with a DSL internet connection speed.
A very successful consumer of gradle in the java environment in my opinion is android. It is super easy to integrate unit tests as well as device tests. Gradle works with the common IDEs like Eclipse and IntelliJ IDEA.
Downloading dependencies is a different build step then building and testing whilst with maven it is all mixed together. With gradle I can download dependencies and then develop on the go, while mavens "mvn -o" never worked reliably for me.
There exists osgi gradle plugins according to my research and gradle allows maven dependencies.
Gradle builds a lot faster than maven, it appears to be smarter to decide what to build. Follow up incremental builds are possible in contrast to maven: If i just change a line in a test and want to rebuild, maven takes again 15min, while gradle would probably do this in seconds.
The gradle DSL is much easier to explain in the documentation than the pom.xml.
A gradle task could be used to create a new binding + test from a skeleton, instead of using the cmd/sh files that hide that super long maven command line.
Less useless output while building, maven can be very verbose.
Building a single binding + test instead of building everything if the build command is issued within a binding directory. This is what I would expect from a proper build-system actually.
Maven was published to the public 2004, gradle reached 1.0 in 2012 and maven feels really aged nowadays.
Feedback would be nice.
Cheers, David
|
|
| | | |
| Re: Change buildsystem to Gradle or alternatives [message #1769935 is a reply to message #1769793] |
Sun, 06 August 2017 14:33  |
David Graeff
Messages: 17
Registered: April 2016 |
Junior Member |
|
|
The OSGi world is moving, too. Now there are "subsystem"s in the spec and the karaf specific feature file is not the only way to provision bundles. I understand, that the ESH core might stick to the maven solution for a while, although I recon it will be more difficult in the future to find people to contribute or maintain that solution.
For ESH extensions I propose to change the build system though with the following goals in mind:
- Have the tests together with the extension.
- Independent if necessary. The extension directory builds without being embedded in the ESH git checkout.
- Works offline after initial dependency resolution
- Extensions can have own dependencies (and don't need to distribute embedded jars)
- Incremental builds
- Build process creates the feature file for karaf or other provisioning files (like a subsystem file)
1. Tests together with the extension
The current system is a little off and most tools that I know for creating test stub classes expecting the tests to be in src/test/java.
2. Independent extension directory
This would allow to develop an extension in an own repository, and easily setup a continuous integration or continuous delivery. ESH could split the main repository, so that extensions have their own issue area on github.
3. Works offline
This works awesome with gradle. I issue a build and dependencies are downloaded to the .gradle cache directory (and shared with following builds) and then it builds the jar. If I rebuild, the existing dependencies are used.
I'd like a build system with the same experience.
4. Extensions with own dependencies
OSGi allows this awesome modularity and at the same time the build system is centered around this monolithic target platform. A proper build system should allow an extension to declare it's own dependencies.
Right now: A lot of the target platform deps are in there because of extensions. Other extensions embed their dependencies, which is far from optimal as well, because people will forget to update those. Newer versions of libraries (like apache commons or guava) can't be introduced progressively, one extension at a time.
5. Incremental builds
At least the bndtools built a bundle immediately after saving a file and it is even deployed to the OSGi instance in a run/debug situation.
6. Build process creates the feature file for karaf
If I write a binding for ESH or OH2, I need to edit two additional files outside of my binding directory within the repository. One is the pom for maven, the other is the features file for karaf. Why is a feature file or subsystem file not generated and later collected by the build system.
I have seen Peter Kriens porting effort. I thing it is harder to accomplish the above goals with sticking to the manifest first approach. The bnd guys actually say the manifest file was never meant to be written by humans. Dependencies can be expressed in gradle or the bdn configuration directly.
This is only the user perspective, of course.
Cheers,
David
|
|
|
| Re: Change buildsystem to Gradle or alternatives [message #1770026 is a reply to message #1769793] |
Mon, 07 August 2017 15:07 |
Kai Kreuzer
Messages: 673
Registered: December 2011 |
Senior Member |
|
|
> @Kai: Can you recall where I can find the discussion thread?
There is no thread, it was endless coding and skype sessions...
Some outcomes were:
- Bndtools requires a workspace to be a flat list of projects from a single repo. There is no way to structure it in sub-components or across multiple repos. A setup like the openHAB IDE (which combines projects from ESH, openHAB Core, Addons1, Addons2, maybe Z-Wave and other repos) will not be possible at all. That's for me the biggest blocker.
- ESH should be first refactored to have proper API bundles that can be compiled against and which should be the only dependency when e.g. starting a binding development in a separate repo. This will be a rather huge task to accomplish.
- We would still want to get dependencies from Maven repos and publish our artifacts to Maven repos. Bnd support for that was not ready last year, it has been worked on meanwhile.
- Launching the runtime from within the IDE wasn't as easily possible as with the current setup - this is a main feature for daily development, though.
- The grass always looks greener on the other side, but the devil is in the detail. From my experience, it will be a huge effort to switch the build system and it will involve almost all developers. Looking at what things there are to do around ESH with what little manpower, it felt not to justify the effort, at least for now.
Regarding your points:
1. Unit tests already can be within the bundle itself. And I don't see it as a huge pain to have plugin tests as separate fragments.
2. We once already discussed that the extensions should be moved to a separate repository to allow independent versioning and releases. But this is unrelated from the build system as the example of openhab2-addons shows (which is a separate repo that builds against ESH artifacts).
3. We should investigate ways to get that done with Tycho, afaik this should be possible.
4. This is something we actually try to avoid. As ESH is not an enterprise application, but targeting embedded devices, size matters. Therefore we try to keep the number of dependencies as small as possible and do the best to make everyone using the same libs for certain purposes. The current setup actually helps us doing so.
5. Well, if developing in the IDE, PDE directly takes changed classfiles from the workspace - no need to repackage anything. Agreed, for remote debugging, the auto-deploy mechanism of bndtools is pretty nice.
6. I have no idea how well Karaf feature creation fits in bnd. For the moment, I don't think it is too much pain the way it is, especially as ESH is supposed to be "pure OSGi" and we only maintain Karaf feature information as a side project. I haven't heard anyone asking for subsystem artifacts so far and the overall relevance of subsystems seems to be limited in the OSGi community...
Regards,
Kai
|
|
|
| Re: Change buildsystem to Gradle or alternatives [message #1772044 is a reply to message #1770026] |
Mon, 04 September 2017 14:34 |
|
Hm, I use offline builds (mvn -o) from time to time. Why do you think it does not work? If there is something broken it should be fixed. As it worked for me before there is no "no possible".
As already pointed out by Kai it is not only about using Maven or not. For the whole PDE stuff, the most of the work is done by the Maven Tycho plugin that IMHO does not follow the Maven approach at all. Dependencies are not declared in the pom files as done by pure Maven artifacts but injected by the Tycho plugin that uses p2 repositories (not Maven ones), the Manifest file of the project etc.
|
|
| | |
| Re: Change buildsystem to Gradle or alternatives [message #1785994 is a reply to message #1772461] |
Tue, 24 April 2018 00:17 |
Lukasz Dywicki
Messages: 34
Registered: July 2009 |
Member |
|
|
Jumping into this topic thanks to Kai.
I would like to share my overall thoughts first, and then describe what I did in one of projects I worked for several years. Some statistics of it include over 800 modules grouped in 60 projects and over 500 000 LOCs (summary in all source files). Out of these "only" 60 modules had custom manifests, all other relied on conventions. We were able to build whole thing in one run (still skipping tests and code analysis). On my computer it took - in ~20 minutes with assembly. Below 10 minutes with parallel threads. It was possible to rebuild part of projects selectively and do assembly again when manual update of modules was getting tricky. Developers who didn't require all projects were able to build their assembly without having all sources (platform, core, persistence and so on) compiled locally.
First of all, a very generic statement on behalf of myself - OSGi development is broken, it must be broken big time, if it forces developers to use a IDE specific plugin in order to achieve a "simplest thing". Leaving standard tools boundaries creates big barriers for new comers to contribute. In my opinion both BNDtools and PDE approaches are completely wrong because they fail to utilise existing tools to deliver what is really needed. Most of developers, do not care about manifests and they shouldn't, this is something which can be automated because most of it contents comes from java sources. ESH started to generate declarative components configurations instead of maintaining them manually, why not doing same for manifest? You shouldn't be too worried about it while you write a code, and it drives me nuts when I can not import a class, which is available in workspace just because I didn't declare import statement in manifest. A target platform idea from PDE is great, yet it is killing developer productivity big time when there is a need to bring new dependency. Validation of manifest early on is life saver and allows to fail fast, but is that what you really need while you compile? I don't know all of you by person , maybe most of you say yes, I want manifest validation first. For me most important is to build the thing. Manifest which is important for runtime, can be checked at assembly time. Somehow I prefer this way over doing it each time.
It also happens that assembly fails after 10 minute build just because of one missing package.
The main point I would like to mark is - build time doesn't necessarily reflect runtime. We always compile against interfaces and their actual implementation, once program is launched, underlying implementations can differ from these used for build. This is expected and desired in software. Sometimes there is even a need to use a different version of archive in runtime than in build time, quite often patching of referenced library may happen even without recompiling project (but by reassembling it with patched module). All these things are getting tricky with PDE and libraries embedded inside bundles invisible to target platform. Reason why Smart Home target platform is so limited mainly to its own artifacts (ie. no guava, commons, http client/netty available) is because maintaining external dependencies with it is pure nightmare. I don't know BNDtools good enough to make claims about it, it might be certainly smarter than that.
With java 9 we got some of checks forced by Tycho/PDE in compiler itself, question is - does anyone going to use it?
Tycho keeps refreshing p2 metadata and seems to ignore offline switch (karaf maven plugin fails in same trap), if you are unlucky and metadata "expires", you are in big troubles because build will fail immediately.
I don't know how big adoption of Smart Home Designer is. Keeping Tycho and PDE build everywhere just for designer is something which makes all contributors to face quite steep learning curve in not easy environment such OSGi. Whole communities are bound to something used by very few people (correct me if I'm wrong), and if that's the case then P2 metadata should be generated for Smart Home Designer and not other way around. You want to avoid P2 unless your thing is deployed under Eclipse. From what I see, it is not critical to create Smart Home assembly.
Key take aways which current build do not have:
1. Use standard dependency resolution mechanism based on maven repositories. Most of people understands it, learned it hard way, and knows how to handle it.
2. Make embedding of dependencies as easy as possible, but also provide a way to extend platform with additional libraries.
3. Generate descriptors if they can be built based on metadata available in project sources.
These 3 points are crucial for keeping entry barrier low while having high developer productivity. It can be achieved with maven, possibly with gradle+bnd. This is the way used by projects such Open Day Light, various Apache projects used by Adobe to for CQ5 and many other smaller projects and companies (ie. Jahia Marketing Factory/Apache Unomi), Atricode Identity Bus and so on. For sure these companies don't do eclipse plugins in their projects.
Things such run from IDE are very nice and great for newcomers, but they are difficult to get working across multiple editors and from continuous integration point of view - they are simply not critical as CI server doesn't use IDE for anything. Either you favour one environment and ignore others or make them equal and do not support any. ;-) I been using Eclipse for OSGi development for many years (my Eclipse carrier started in early 2004 with PHP Eclipse), and during this time I never ever used PDE and manifest first approach beside eclipse plugins. I loved Eclipse so much that I was generating .project descriptors from maven metadata and doing everything from console. By accident I let people loving other IDEs do their job in tool of their choice too.
These were 0.02€ from Łukasz. :-)
Cheers!
[Updated on: Tue, 24 April 2018 00:19] Report message to a moderator |
|
|
| Re: Change buildsystem to Gradle or alternatives [message #1787181 is a reply to message #1785994] |
Mon, 21 May 2018 12:42 |
Lukasz Dywicki
Messages: 34
Registered: July 2009 |
Member |
|
|
To throw some more context to the pains current build causes.
I am working on some feature in branch and I do have lots of troubles building them because of following behavior.
When one of modules I added fails due to style checks after I fix them I can not run build from place it failed (-rf option in maven). When I do run build with -rf switch platform resolution simply ignores all changes I have made in core bundles (ie. new types added in org.eclipse.smarthome.core) leading to compilation errors. Each and every time when I want to verify my local changes I have endless loop of
1) platform resolution which takes 30 seconds up to minute, even if everything is cached
2) building of modules which I am sure they are fine, because I didn't touch them nor code used in their dependencies (I add new parts in core, do not modify existing ones).
3) once this is done I have to rebuild all other repos (openhab related).
As you can see a small tweak might result in a 30 minute fight in order to produce new assembly.
Regarding offline mode - it does not exist, simply like that. Just few minutes ago - I built eclipse smarthome, had build openhab core few days ago and had all things cached. So I attempted to build openhab core with -o switch, to speed up target platform resolution, which resulted in following error:
~/e/o/g/openhab-core (esh-authz2) $ mvn clean install -o -nsu 14:06:56
[INFO] Scanning for projects...
[WARNING] No explicit target runtime environment configuration. Build is platform dependent.
[INFO] Computing target platform for MavenProject: org.openhab.core:org.openhab.core:2.3.0-SNAPSHOT @ /home/splatch/eclipse-esh/openhab2-master/git/openhab-core/bundles/org.openhab.core/pom.xml
[INFO] Adding repository https://dl.bintray.com/openhab/p2/openhab-deps-repo/1.0.26
[ERROR] Internal error: java.lang.RuntimeException: Failed to load p2 repository with ID 'p2-openhab-deps-repo' from location https://dl.bintray.com/openhab/p2/openhab-deps-repo/1.0.26: No repository found at https://dl.bintray.com/openhab/p2/openhab-deps-repo/1.0.26. -> [Help 1]
org.apache.maven.InternalErrorException: Internal error: java.lang.RuntimeException: Failed to load p2 repository with ID 'p2-openhab-deps-repo' from location https://dl.bintray.com/openhab/p2/openhab-deps-repo/1.0.26
at org.apache.maven.DefaultMaven.execute (DefaultMaven.java:122)
at org.apache.maven.cli.MavenCli.execute (MavenCli.java:955)
at org.apache.maven.cli.MavenCli.doMain (MavenCli.java:290)
at org.apache.maven.cli.MavenCli.main (MavenCli.java:194)
at sun.reflect.NativeMethodAccessorImpl.invoke0 (Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke (NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke (DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke (Method.java:498)
at org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced (Launcher.java:289)
at org.codehaus.plexus.classworlds.launcher.Launcher.launch (Launcher.java:229)
at org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode (Launcher.java:415)
at org.codehaus.plexus.classworlds.launcher.Launcher.main (Launcher.java:356)
Caused by: java.lang.RuntimeException: Failed to load p2 repository with ID 'p2-openhab-deps-repo' from location https://dl.bintray.com/openhab/p2/openhab-deps-repo/1.0.26
at org.eclipse.tycho.p2.target.TargetPlatformFactoryImpl.loadMetadataRepository (TargetPlatformFactoryImpl.java:302)
at org.eclipse.tycho.p2.target.TargetPlatformFactoryImpl.gatherExternalInstallableUnits (TargetPlatformFactoryImpl.java:270)
at org.eclipse.tycho.p2.target.TargetPlatformFactoryImpl.createTargetPlatform (TargetPlatformFactoryImpl.java:176)
at org.eclipse.tycho.p2.target.TargetPlatformFactoryImpl.createTargetPlatform (TargetPlatformFactoryImpl.java:128)
at org.eclipse.tycho.p2.target.TargetPlatformFactoryImpl.createTargetPlatform (TargetPlatformFactoryImpl.java:1)
at org.eclipse.tycho.p2.manager.ReactorRepositoryManagerImpl.computePreliminaryTargetPlatform (ReactorRepositoryManagerImpl.java:84)
at org.eclipse.tycho.p2.resolver.P2DependencyResolver.computePreliminaryTargetPlatform (P2DependencyResolver.java:223)
at org.eclipse.tycho.core.resolver.DefaultTychoResolver.resolveProject (DefaultTychoResolver.java:109)
at org.eclipse.tycho.core.maven.TychoMavenLifecycleParticipant.afterProjectsRead (TychoMavenLifecycleParticipant.java:95)
at org.apache.maven.DefaultMaven.doExecute (DefaultMaven.java:267)
at org.apache.maven.DefaultMaven.doExecute (DefaultMaven.java:194)
at org.apache.maven.DefaultMaven.execute (DefaultMaven.java:107)
at org.apache.maven.cli.MavenCli.execute (MavenCli.java:955)
at org.apache.maven.cli.MavenCli.doMain (MavenCli.java:290)
at org.apache.maven.cli.MavenCli.main (MavenCli.java:194)
at sun.reflect.NativeMethodAccessorImpl.invoke0 (Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke (NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke (DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke (Method.java:498)
at org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced (Launcher.java:289)
at org.codehaus.plexus.classworlds.launcher.Launcher.launch (Launcher.java:229)
at org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode (Launcher.java:415)
at org.codehaus.plexus.classworlds.launcher.Launcher.main (Launcher.java:356)
Caused by: org.eclipse.equinox.p2.core.ProvisionException: No repository found at https://dl.bintray.com/openhab/p2/openhab-deps-repo/1.0.26.
at org.eclipse.equinox.internal.p2.repository.helpers.AbstractRepositoryManager.fail (AbstractRepositoryManager.java:395)
at org.eclipse.equinox.internal.p2.repository.helpers.AbstractRepositoryManager.loadRepository (AbstractRepositoryManager.java:692)
at org.eclipse.equinox.internal.p2.metadata.repository.MetadataRepositoryManager.loadRepository (MetadataRepositoryManager.java:96)
at org.eclipse.tycho.p2.remote.RemoteMetadataRepositoryManager.loadRepository (RemoteMetadataRepositoryManager.java:61)
at org.eclipse.tycho.p2.remote.RemoteMetadataRepositoryManager.loadRepository (RemoteMetadataRepositoryManager.java:53)
at org.eclipse.tycho.p2.target.TargetPlatformFactoryImpl.loadMetadataRepository (TargetPlatformFactoryImpl.java:298)
at org.eclipse.tycho.p2.target.TargetPlatformFactoryImpl.gatherExternalInstallableUnits (TargetPlatformFactoryImpl.java:270)
at org.eclipse.tycho.p2.target.TargetPlatformFactoryImpl.createTargetPlatform (TargetPlatformFactoryImpl.java:176)
at org.eclipse.tycho.p2.target.TargetPlatformFactoryImpl.createTargetPlatform (TargetPlatformFactoryImpl.java:128)
at org.eclipse.tycho.p2.target.TargetPlatformFactoryImpl.createTargetPlatform (TargetPlatformFactoryImpl.java:1)
at org.eclipse.tycho.p2.manager.ReactorRepositoryManagerImpl.computePreliminaryTargetPlatform (ReactorRepositoryManagerImpl.java:84)
at org.eclipse.tycho.p2.resolver.P2DependencyResolver.computePreliminaryTargetPlatform (P2DependencyResolver.java:223)
at org.eclipse.tycho.core.resolver.DefaultTychoResolver.resolveProject (DefaultTychoResolver.java:109)
at org.eclipse.tycho.core.maven.TychoMavenLifecycleParticipant.afterProjectsRead (TychoMavenLifecycleParticipant.java:95)
at org.apache.maven.DefaultMaven.doExecute (DefaultMaven.java:267)
at org.apache.maven.DefaultMaven.doExecute (DefaultMaven.java:194)
at org.apache.maven.DefaultMaven.execute (DefaultMaven.java:107)
at org.apache.maven.cli.MavenCli.execute (MavenCli.java:955)
at org.apache.maven.cli.MavenCli.doMain (MavenCli.java:290)
at org.apache.maven.cli.MavenCli.main (MavenCli.java:194)
at sun.reflect.NativeMethodAccessorImpl.invoke0 (Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke (NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke (DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke (Method.java:498)
at org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced (Launcher.java:289)
at org.codehaus.plexus.classworlds.launcher.Launcher.launch (Launcher.java:229)
at org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode (Launcher.java:415)
at org.codehaus.plexus.classworlds.launcher.Launcher.main (Launcher.java:356)
[ERROR]
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR]
[ERROR] For more information about the errors and possible solutions, please read the following articles:
None of things in these repos have changed, they are not in snapshot version and they should be cached on my computer already. There is no need, at least shouldn't be, to go there and check if there is anything new.
After passing openhab-core I want to build openhab-addons2. I do that because sometimes it is faster to rebuild this repo, than wait for all maven snapshots checks in openhab-distro (and pulling their binary versions). I'm doing same for openhab1 addons, simply because its faster to build these than fetch from internet. Again, I have a failure which is caused by platform resolution:
[ERROR] Failed to execute goal org.eclipse.tycho:tycho-compiler-plugin:1.0.0:compile (default-compile) on project org.openhab.binding.homematic: Compilation failure: Compilation failure:
[ERROR] /home/splatch/eclipse-esh/openhab2-master/git/openhab2-addons/addons/binding/org.openhab.binding.homematic/src/main/java/org/openhab/binding/homematic/internal/discovery/CcuDiscoveryService.java:[83]
[ERROR] String configuredBroadcastAddress = networkAddressService.getConfiguredBroadcastAddress();
[ERROR] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[ERROR] The method getConfiguredBroadcastAddress() is undefined for the type NetworkAddressService
[ERROR] /home/splatch/eclipse-esh/openhab2-master/git/openhab2-addons/addons/binding/org.openhab.binding.homematic/src/main/java/org/openhab/binding/homematic/handler/HomematicThingHandler.java:[526]
[ERROR] updateStatus(ThingStatus.OFFLINE, ThingStatusDetail.GONE);
[ERROR] ^^^^
[ERROR] GONE cannot be resolved or is not a field
[ERROR] -> [Help 1]
Both errors are pointers to methods which I already have in my local checkout and built them already. These should be available in my earlier local build and I have no clue why compiler fails on these. Maybe because I didn't add -o switch? For next execution I added -o, so target platform computation should get fine, yet it is again wrong and fails with same error. Note, that part which is failed is not the one which uses typical maven project packaging but one which is bound to tycho, meaning with eclipse-plugin packaging. I have to go back, rebuild smarthome,
Because of how dependencies are handled (bundle-classpath in most of the cases), local checkout built in offline mode should always produce artifacts bound to local checkout and consistent with local changes. It's not a case with tycho which forces online mode and there is no way to switch it off.
As you can see its utterly frustrating to build everything to produce assembly with core changes, because almost every project depending on it I have to build at least twice in order to get it done. Build is not deterministic at all and that's biggest issue. If build can not give consistent results it simply doesn't work.
Writing this message and collecting errors took me already 30 minutes and now I have to do it again because of missing -o switch on openhab-addons2..
|
|
| | |
| Re: Change buildsystem to Gradle or alternatives [message #1787229 is a reply to message #1787181] |
Tue, 22 May 2018 12:45 |
Kai Kreuzer
Messages: 673
Registered: December 2011 |
Senior Member |
|
|
The fun fact here is that the issues described above were related to changes to Maven dependencies and not due to Tycho/p2.
1. The offline build didn't work, because the version of the zigbee-libs to be included in the zigbee binding had changed on Sunday night, so these dependencies had to be resolved.
2. I have changed openhab-core to fully build against Maven dependencies instead of p2 repos last week.
So what shall we do?
1. Yes, the build is complex, but this is not necessarily due to a chosen technology.
2. This here is the ESH forum, so you should not argue with openhab distro build. Even if we build ESH in one second, you still have to deal with different openHAB repos to build.
3. If you change ESH to e.g. bnd/gradle, the openHAB IDE will become MORE complex to handle if you want to use it for working on parts of the ESH code. At least nobody was able so far to show me a solution for this - so you would actually have to switch between two IDEs when doing refactorings/enhancements of APIs.
Note that we should discuss ESH here: I doubt that the build will be much faster as most time is spend on tests, static code analysis and Karaf feature verification. The build results are available as Maven artifacts today already, so consumers can theoretically use any build system they like as long as it can consume Maven artifacts.
If your main concern is the openHAB build, this discussion would better be moved to https://community.openhab.org/c/organisation/code.
|
|
|
| Re: Change buildsystem to Gradle or alternatives [message #1787410 is a reply to message #1787229] |
Fri, 25 May 2018 00:14 |
Lukasz Dywicki
Messages: 34
Registered: July 2009 |
Member |
|
|
Kai,
Thank you for picking up topic. Neither offline and online build of openhab2 addons passed on me past Monday. I didn't clean local p2 repo nor do anything to it beside switching offline mode on or off. It might be due to late sunday night builds which didn't catch yet zigbee-libs. I do not have many issues with openhab-core, because plain maven is rarely making troubles and there is a way to control snapshot resolution with -nsu and -U switches. Don't know how, if at all, these influence tycho build due different resolution mechanism.
I'm sharing my thoughts as independent contributor who attempts to do something around. You can try to gather feedback from people who donated code to smarthome and openhab in order to find bottlenecks. I can point myself few:
1. Tycho build is hard to understand in first place because there are multiple models involved: PDE, OSGi, Tycho+Maven. Even seasoned Java guys who knows maven quite well will get into troubles with that. Just think of how many things you actually have to maintain in order to get simple SCR descriptor you made in OSGi-INF included in binary archive. You have to create dir with proper name, append is namet to cryptic syntax of build.properties (for newcomers bin.includes is something they will either guess or just copy without understanding) and update MANIFEST.MF(service-component attribute). There is no a single place in smarthome docs which would describe this process in detail so people either blindly use archetypes or copy stuff over and over hopping it will work. It is completely opposite of convention over configuration way where all resources from given directory are alwas included just because they are in given place.
2. Stuff is not getting published to maven central for whathever reason so people have to start messing with their local maven settings in order to even use smarthome dependencies (see next pont) or rely on projects built locally (having a domino effects like mine above).
3. Tycho and PDE builds are useless when it comes to creation of smaller extensions. Someone who wants to pull few things from smarhome will be forced to use tycho build or append 10 dependencies in order to produce proper dependency tree because PDE and Tycho builds use empty poms files. This means that you can not use transient dependencies and you have to duplicate dependency information in two places (or three if we count sources) - MANIFEST where you declare import packages or required bundles and pom files where you translate same requirements back from runtime to build time. Core assumption of using runtime dependecies for build time gives advantages but it also turns build upside down because as said before build time doesn't have and often will not reflect runtime. Openhab-core is a perfect example showing that missing maven dependency metadata from smart home forces manual maintanance of dependency trees (see openhab-core/features/repo/pom.xml). It's definitelly not a way which is reliable and developer friendly. This makes smarthome maven artifacts as useful as eclipse ones
4. Tycho and PDE mess runtime and build time too much. While it gives benefits of stable (let say) results and extra compiler checks such support slowly gets into Java core with module system. There is already "official" way of forcing compiler to honor module boundaries which requires java 9 module definitions. Obviously Java 9 modules are again redundant. Because runtime and build time is tied together its hard to add repackaged dependencies. In fact Tycho/PDE promotes a ant style way of handling dependencies by putting them in SCM.
I understand reason why you are concerned about IDE setup, however reason why it is so important right now is due to PDE approach supported by Tycho. If you leave manifest first approach and rely on regular maven dependency model then any other IDE will handle it equally good. Both Eclipse, IntelliJ and NetBeans can handle such information and they will understand it in the same way. Maven might be imperfect, however its dependency model is used quite extensively in many projects and it gets job done in most of cases. Code checks area already part of maven build so there is no need for any specific IDE to handle that.
My key proposals to consider:
1) Don't use Tycho for smarthome-core, use it for designer where it will pay off.
2) Declare build dependencies in pom.xml files.
3) Stop manual mainanance of manifests, generate them using maven-bundle-plugin or any other maven plugin which embeds bnd core.
Third point will allow to cut off required-bundle headers everywhere. MANIFEST.MF is just another resource, just like a SCR component descriptor. There is a way to generate manifest by using pom files and no way to generate pom files using manifests. Tycho internally matches everything against P2 metadata which would require another loop to compute maven dependencies (if its even possible).
There are ways to speed up maven builds with paralel switches (ie. -T2C). Most of maven plugins is able to honor these, including clean, compile, surefire (test runner), resources plugins. It does give a performance increase. I don't know Tycho to tell how this affects it.
Cheers,
Lukasz
[Updated on: Fri, 25 May 2018 08:46] Report message to a moderator |
|
| | |
Goto Forum:
Current Time: Fri Dec 27 03:52:48 GMT 2024
Powered by FUDForum. Page generated in 0.05183 seconds |
] 
Search
Help
Register
Login
Home
