Original work item: "Make the build path less sensitive with regard to whether a project is a source or a binary project. This is also related to patching (i.e., adding a source folder to a binary project)."
Related issue: support for extension directories containing many jars.
There was much discussion. The underlying issue is how to structure workspaces for significant Java development efforts. The Eclipse Project is an example of such an effort, so the question is not merely of passing interest. Since much of what is being developed in the Eclipse Project are plug-ins, the question also touches on PDE's domain. However, some aspects of the problem affect all significant Java development efforts and not just ones for developing plug-ins. So we begin to investigating those.
The following is an attempt to capture a familiar problem in a general way that does not presume exactly how developers' workspaces are to be structured. Consider the case of two components, with PUI dependent on Core (imagine that Core is the core component, and UI is the corresponding UI component built atop it). Now imagine that Core and UI are under active development by two teams, with ownership split along component lines. For maximum convenience to both teams, the teams would like to be able to do the following:
Referencing a project's libraries in another project: The project I'm working on uses a fairly large number of libraries (about 20 jar files, IIRC). I had hoped I'd be able to create one project which used these libraries, and make all other projects have that project on their build-paths to bring in the libraries - that way I wouldn't need to change *every* project every time I added or removed a library. (It's not a common operation, but even so...) Unfortunately, I can't get this to work - putting project A on the classpath of project B only seems to put project A's individual source/class files on the classpath for project B. I've tried adding as an external jar and importing the jar file into the project, and had no joy with either :(Here is the arrangement:
project P1It is clear that the customer is looking for a way to deal with a set of libraries that would be required by several projects, and was hoping that required projects would give him that.
build classpath = source /P1/src; library lib1.jar; library lib2.jar; ...; library lib20.jar
build classpath = source /P2/src; library lib1.jar; library lib2.jar; ...; library lib20.jar
One idea would be to allow variables to be bound to a list of paths instead of a single path. This would allow, for example, a single classpath variable "BASE_LIBS" to be bound to the list of the paths of the library jars lib1.jar; lib2.jar; ...; lib20.jar. Each project referencing this classpath variable would thereby gain access to all the libraries:
project P1Changing the workspace binding of the classpath variable affects all projects that reference the variable. This makes it easy to change the set of libraries without having to change the projects individually.
build classpath = source /P1/src; BASE_LIBS
build classpath = source /P2/src; BASE_LIBS
Since classpath variable bindings are local to the workspace, there is no obvious automatic way by which these classpath variables would get their bindings. The bindings would have to be configured for each workspace; loading bindings from a file, or initializing them via a script, are feasible options..
The classpath variables currently bind to paths, and classpath entries can contains paths that begin in a variable. Changing variables to bind to a list of paths would be a major change, and would likely require reworking much of existing API. We generally agreed that we would not pursue this approach since it seems somewhat unlikely to solve much of the problem.
The proposal is to extend the required projects mechanism to allow a required project to contribute more than just its binary output folder. Rather, a project would be able to indicate that any of its libraries are to be exported. Exported libraries also become available to other projects in the workspace that list this project as a required project.
In the above example, the customer could instead have an arrangement like:
project P1What this means is that when P1 (similarly, P2) is built, the libraries on its build classpath consists of /PLib/bin, lib1.jar, ..., lib20.jar.
build classpath = source /P1/source; project PLib
build classpath = source /P2/source; project PLib
build classpath = source /PLib/source; export library lib1.jar; ...; export library lib20.jar
The modified semantics of a project P with a required project R are as follows:
project PEffective build classpath of P:
build classpath = source /P/source; project R; library plib.jar
build classpath = source /R/source; export library rlib1.jar; library rlib2.jar; export project Q; export library rlib3.jar
build classpath = source /Q/source
build classpath = source /P/source; library /R/bin; library rlib1.jar; library /Q/bin; library rlib3.jar; library plib.jar
Should projects be allowed to export arbitrary libraries, or should exported libraries always be internal to the project?
Restricting exports to internal libraries ensures that a project that is to be used by other projects is somewhat self contained. If you load such a project from a repository, you are guaranteed all the libraries it exports will be contained therein.
On the other hand, unrestricted exports are more flexible, and allow a project to export a collection of libraries that are not necessarily contained within the project's resource tree. (However, it is unclear whether this additional flexibility would be useful.)
We opted to allow unrestricted imports, but decided to simplify the error processing by only reporting problems against the project with the missing library (or required project) explicitly on its classpath. Any missing entries would simply be omitted from the effective classpath calculation.for dependent projects.
A project's build classpath may include references to classpath variables that get bound to libraries (or other projects). How do exports and variables interact?
The export flag could be associated either with the variable reference or with the variable binding. If the export flag is associated with the variable reference, it would indicate exporting whatever library the classpath variable happened to be bound to on a given occasion. If the export flag is associated with the variable binding itself, the library the classpath variable happened to be bound to on a given occasion would be exported conditionally on the flag in the binding. The proposal is to go with the former (export flag with variable reference) since its semantics are somewhat simpler and allow the export flag to be shared with other team members (the bindings of classpath variables are not shareable).
Note that explicit exports also address another problem that arises when a project needs to export a pre-built library in addition to the results of compiling its source files (the Eclipse debugger's jdi.jar is a fine instance of this). In Eclipse 1.0, only the class files in the project's binary output folder are exported to dependent projects. This proposal allows a project to export any number of additional libraries as required.
By exporting other projects, a project can consolidate and concentrate the outputs from several other projects:..
project P1Effective build classpath of P1andP2 includes everything exported from both P1 and P2:
build classpath = source /P/source; export library /P1/lib1.jar
build classpath = source /P2/source; export library /P2/lib2.jar
build classpath = source /P1andP2/source; export project P1; export project P2
build classpath = library /P1andP2/source; library /P1/bin; library /P1/lib1.jar; library /P2/bin; library /P2/lib2.jar
One objection to this whole approach is that the Java project is being hijacked. In the original design, a Java project is a buildable container of Java source code. The addition of exported libraries starts to turn the project into something more general. If we are comfortable with this general trend, there are ways that the design can embrace it more whole-heartedly.
The above proposal to expand the semantics of required project creates a new role for a Java project as a container of libraries. This notion of library project is rounded out by allowing Java projects without source or output folders. The build classpath of a library project orders the list of exported libraries and projects (and provides the additional context required for browsing the project itself).
In our example, the library project could be expressed more directly without having to postulate source and output folders which are completely unmotivated in this case:
build classpath = export library lib1.jar; ...; export library lib20.jar
A library project is recognizable simply by the absence of source folders on the build classpath. (We considered adding a new flag to the project to explicitly mark library projects, but decided this was not necessary and would be confusing.)
Note that library projects are not "buildable" in any meaningful sense since they lack source code to compile (and without source code there is no pressing need for a binary output folder). However, there still needs to be a mechanism for detecting and reporting errors in the build classpath for library projects. Except for such checks, the Java incremental project builder should ignore library projects.
The following kinds of entries on the build classpath of a library project would serve these purposes:
The workspace must compute an input delta even for incremental project builders that ignore the delta they are handed. We considered whether we should go one step further and remove the Java builder from a library project's list of incremental project builders. This would make it crystal clear that there is no Java building going on, and it would reduce workspace memory footprint because the workspace would not need to remember the shape of the resource tree at the time of the last build. However, if the builder was removed for library projects, there would need to be mechanism to add it if the classpath was changed to include a source project. We decided that we should simply avoid the additional hassles and leave the Java builder installed for all Java projects.
From Philippe Mulet 08/31/2001 12:13 PM
I also think this has to be a basic mechanism at the JavaCore level,
which the PDE can then surface to plugin writers.
I would like to see this one addressed asap, so that it is possible to share projects transparently at least amongst us.
From Erich Gamma on 08/31/2001 12:43 PM
Here is some more input on our favorite topic "make the build class
path less sensitive
with regard to whether a project is a source or binary project".
Our conclusion in SNZ was that this boils down to a PDE project layout
projects etc). However, the EC discussion from below illustrates a use case for allowing to export
JARs in addition to the output folder from a project, that is independent of PDE.
The scenario is similar to the WSAD scenario with extension dirs. If we support to
contribute contained JARs from a project then this problem would be addressed.
We should therefore reconsider the solution independent of the PDE issue.
If I do that then I come to the conclusion that the proposed mechanism is valuable
and I suggest to go ahead with specifying it in detail.
From Jon Skeet <email@example.com> on Eclipse Corner 20.8.2001
Referencing a project's libraries in another project:
The project I'm working on uses a fairly large number of libraries
(about 20 jar files, IIRC). I had hoped I'd be able to create one
project which used these libraries, and make all other projects have
that project on their build-paths to bring in the libraries - that way I
wouldn't need to change *every* project every time I added or removed a
library. (It's not a common operation, but even so...) Unfortunately, I
can't get this to work - putting project A on the classpath of project B
only seems to put project A's individual source/class files on the
classpath for project B. I've tried adding as an external jar and
importing the jar file into the project, and had no joy with either :(