Extension Directories

Last revised 17:10 Thursday October 25, 2001

Work item: Add support to Java model for JDK 1.2-style extension directories (aka optional packages)

An extension directory is a folder containing any number of JAR files (including 0). Extension directories were originally added in JDK 1.2, and are described in http://java.sun.com/j2se/1.3/docs/guide/extensions/index.html.

The proposal is to allow a library classpath entry to refer to an extension directory. A library classpath entry (CPE_LIBRARY) with a path whose last segment is "*" indicates an extension directory; for example, a library entry with the path "D:/jdk1.4/jre/lib/ext/*" means the extension directory "D:/jdk1.4/jre/lib/ext". Like all library entries, extension directories can be in folders inside the workspace or in directories in the local file system outside the workspace. A variable classpath entry may also resolve to an extension directory following the same rule about ending in "*". Thus a variable classpath entry like "JRE_LIB/ext/*" would resolve to an extension directory; and a simple one like "FOO" could resolve to either a JAR library, a library folder, an extension directory, or a project.

Each of the JARs in an extension directory on a project's class path would give rise to a separate IPackageFragmentRoot (kind K_BINARY); these package fragment roots are all child elements of the IJavaProject. The Java model would not have a Java element corresponding to the extension directory itself. A classpath entry that identifies a extension folder internal to the workspace gives rise to 0, 1, or more binary package fragment roots for JAR file resources internal to the workspace; similarly, a classpath entry that identifies a extension folder external to the workspace gives rise to binary package fragment roots for JAR files outside the workspace.

Every package fragment root that is a child of a project element stems from some entry on that project's original classpath. Given a package fragment root, how does one trace it back to the classpath entry that gave rise to it? In the presence of variable classpath entries and extension directories, the answer is not obvious. Having a simple answer is important to some clients. For instance, the Java UI includes the names of classpath variables in items in its Packages view.

IPackageFragmentRoot
/**
 * Returns the classpath entry that gives rise to this package fragment root.
 * This package fragment root must exist. The returned classpath entry is
 * equivalent to one of the classpath entries of this package fragment
 * root's project.
 *
 * @return the originating classpath entry
 * @exception JavaModelException if this element does not exist
 */
IClasspathEntry getOriginatingClasspathEntry() throws JavaModelException;

This method would replace IJavaProject.getPackageFragmentRoots(IClasspathEntry entry), which has problems and would be removed from the API (via deprecation).

Packages View Clutter

The packages view would show all the JARs from an extension directory as children of a project. Packages view clutter exists for anyone with a large number of JAR libraries on their build classpath. Extension directories exacerbate the problem by making it easy to include a whole set of JAR libraries with a single classpath entry (WSAD reputedly has 70+ JARs in its extension directory).

The packages view supports filtering out JAR libraries, meaning they could be hidden. However, the filter is off by default and not all users are aware that the filter even exists.

Note that extension folders typically contain a bunch of unrelated JARs, making it likely that a user will be interested in a particular subset and uninterested in the rest. The IBM 1.3.0 JRE jre\lib\ext contains indicim.jar and JawBridge.jar. The Sun JDK 1.4.0-beta jre\lib\ext contains dnsns.jar, ldapsec.jar, and sunjre_provider.jar. All these JARs are likely pure noise and do not contain API that a client would write to.  In a J2EE JRE, there would be a diverse collection of JARs in the jre\lib\ext extension directory (e.g., Java Telephony, JavaMail, Java 2D, Java 3D, Java Media Frameworks), only a handful of which would be used within any given project.

One idea is to allow a new JavaElement "ExtensionFolder". This would allow to group the extension JARs and would avoid some clutter. Although an extension folder element could avoid clutter, it may not provide enough of an improvement over the existing filtering mechanism to warrant complicating the Java model API.

[This problem is still open. I recommend living with the currently supported filtering for now. We could come back later and look at ways to reduce clutter as a separate work item.]

Default Build Classpath

The standard JRE implicitly has the jre/lib/ext extension directory on its runtime classpath. By default, the build classpath for a typical project should include entries for both the standard class library (jre/lib/rt.jar) and the standard extension directory (jre/lib/ext/*).

These defaults should be a function of the VM launcher for the project. It should always be possible for the user to override the defaults and set up their build classpath as they see fit.

Build Classpath Ordering

The UI currently allows users to freely reorder package fragment roots for the project. This does not really make sense for package fragments roots that arise from extension directories.

Indeed, the addition of extension directories means that there is no longer a 1-1 correspondence between package fragment roots and classpath entries. The UI should allow the user to determine the order of classpath entries. Here are the guidelines and restrictions on the build classpath for a project (N.B., these apply to the raw classpath, not necessarily to a partially or fully "resolved" classpath):

Build Classpath Duplicates

How are duplicates libraries on the build classpath handled? For instance, if there are two classpath entries on a project's build classpath for exactly the same JAR, should there be one or two package fragment roots? What if a JAR classpath entry explicitly mentions a JAR that is included in an extension folder also on the build classpath?

At the Java model API, package fragment roots are handle objects with well-known identity criteria. It turns out that this dictates the answer to how dupicates are handled.

For library package fragment roots based on a resource in the workspace, the identity criteria includes the resource handle (handle constructor isJavaProject.getPackageFragmentRoot(IResource)) For library package fragment roots based on files outside the workspace, the identity criteria includes the path (handle constructor isJavaProject.getPackageFragmentRoot(String)).

This means the answer is forced: any duplication in the build classpath necessarily washes out in the mapping to package fragment roots. Two classpath entries for exactly the same JAR must map to a single package fragment root. JARs from an extension directory gives rise to the same package fragment roots one would get if each JAR was mentioned explicitly.

In the case of duplicates, we do have a choice for which classpath entry is considered the originating one. IPackageFragmentRoot.getOriginatingClasspathEntry(), described above, will be used to locate an entry in the build classpath carring important source attachment information. A specific classpath entry should always be preferred to a generic classpath entry (extension folders entries carry no source attahment info), and an earlier entry should be preferred over a later one.

For example, given the following (unusual) build classpath:
<classpath>
    <classpathentry kind="lib" path="/jre/lib/ext/*">
    <classpathentry kind="lib"
        path="/jre/lib/ext/servlet.jar"
        sourcepath="/src/servletsrc.zip">
    <classpathentry kind="lib"
        path="/jre/lib/ext/servlet.jar"
        sourcepath="/backup/servletsrc.zip">
</classpath>
the package fragment root for "/jre/lib/ext/servlet.jar" is considered to originate from the second classpath entry: the second is more specific that the first, and is equally specific to, but ordered earlier than, the third.

Extension Directories and the Runtime Classpath

By default, jre/lib/ext is the extension directory at runtime. Overriding this default entails specifying a command line argument to set the value of the java.ext.dirs system property. This system property specifies one or more directories to search for installed extensions, each separated by File.pathSeparatorChar.

We need to determine whether the JAR needs to be added to the runtime class path or not. Since the launcher is JVM-install-type-specific, it feels like the right place to hardcode the knowledge of how to map a build classpath to a runtime classpath. The JAR for the standard class library does not need to be included in the runtime classpath because it is implicitly on the boot classpath. Similarly, any JAR that is included in an extension directory does not need to be mentioned explicitly.

Attaching Source to Library JARs in an Extension Folder

JARs from extension directories are not really any different from other class libraries. It would make debugging difficult if there were no way to attached source code to a JAR in a extension directory.

How does one attach source to JARs in an extension directory? As you recall from R0.9, we consciously opted to put the source attachment information in a place where it would be sharable, rather than leave this information local to the workspace. The API for attaching source is on the classpath entry; this allows this information to be shared between developers. Clearly, this would not work for extension directories, where the names of the JARs in the directory might not even be known in advance. There is no general naming scheme for locating the source code for a given JAR (we do not proposing to invent one).

As discussed in the section on package view clutter, it will be common for an extension directory to contain a diverse set of JARs, only a handful of which would be used within any given project. The proposal is to provide no direct support for attaching source via an extension folder classpath entry. If the user needs to see source code for a JAR in an extension folder, they would need to create an additional explicit classpath entry for that JAR. As discussed above, this would not result in a new package fragment root; however, it would effectively change the originating classpath entry to the explicit one where the source attachment information would be found. This would allow the user to attach source code to JARs on an as-needed basis. The bet is that this will suffice (and finesses the issue of where one might find the source for a JAR in an extension folder).