Proposal for g-Eclipse - an integrated Grid enabled environment (IGE)

Introduction

The g-Eclipse project is a proposed open source project under the Eclipse Technology Project.

This proposal is in the Proposal Phase (as defined in the Eclipse Development Process document) and is written to declare its intent and scope. This proposal is written to solicit input from the Eclipse community and additional participation. You are invited to comment on and join the project.

Please send all feedback to the http://www.eclipse.org/newsportal/thread.php?group=eclipse.technology.g-eclipse newsgroup.

Background

Over the last few years Grid infrastructures have been becoming the backbone of those fields of science and research that require to solve complex computational problems. At the same time the commercial application of Grid technologies has led to new categories of offerings (e.g. on-demand offerings) and will likely play an important role in the "software as a service" landscape. However, the complexity of Grid infrastructures is often discouraging to new and inexperienced users and impedes the use of Grid technologies in new application domains.

By providing an integrated Grid enabled environment based on Eclipse (called g-Eclipse) this project aims at facilitating the use of Grid infrastructures. As an integrating tool g-Eclipse will have impact on the different actors in the Grid domain. Grid users will access the Grid with g-Eclipse in an easy and uniform way, instead of having to use application specific portals or specific Grid consoles. Developers can take advantage of the large variety of languages that are supported by Eclipse and benefit from a standard mechanism to deploy applications on Grid infrastructures.

With the flexible and open plug-in approach of g-Eclipse, the management interface of new Grid applications can be customized and deployed relatively easily. It shall be possible to reuse already existing management components.

g-Eclipse will deliver exemplary support for the EGEE middleware gLite/LCG, which is deployed on the biggest Grid infrastructure currently available. This middleware provides a solid basis for exemplary tools, as the number of generic applications will increase in the second phase of EGEE (a European Grid initiative), and thus the number of potential adopters of the g-Eclipse platform will increase in the same way. However, as g-Eclipse is following a frameworks and exemplary tools approach, other middleware like UNICORE and GT4 can also be integrated by extending g-Eclipse.

With the proposed g-Eclipse project, Eclipse enters the area of Grid applications and e-Science. The consortium that is submitting this proposal will receive funding by the European Union under contract number IST-034327 to implement and establish g-Eclipse.

Scope

The objectives of the g-Eclipse project are:

• Development of a general framework that can be used by Grid users, Grid developers and Grid operators

  The software developed in the g-Eclipse project will consist of the "core Grid plug-ins" for the Eclipse platform. These will enable and standardize the access of Grid infrastructures from within Eclipse, independent of the used middleware. The framework will integrate functionality for the following use cases:

  • Grid Virtual Organization management:

    A functionality that allows the dynamic creation and management of Virtual Organizations and their resources will be developed. This includes authentication and authorization of users on remote Grid resources.

  • Grid job management:

    One of the standard actions on the Grid is the submission of Grid jobs and their monitoring. A functionality will support Grid users and developers to create and manage Grid jobs independently from the middleware.

  • Grid file management:

    The management of distributed data in a Grid environment is based on file catalogues. The file management functionality will allow the management of distributed files using functionalities of the Grid file catalogues.

  • Grid infrastructure monitoring:

    Hardware and other resources of the Grid infrastructure might change over time. Therefore a good virtualization tool for Grid infrastructure resources is needed by Grid users, developers and operators. A functionality will be developed to allow for the visual representation and monitoring of Grid resources.

  • Grid application monitoring:

    The execution of Grid applications is different from the execution of applications on a single cluster. The monitoring of applications on remote, not a-priori known resources, needs tools that allow the transparent monitoring of such applications. A frontend that allows monitoring of Grid applications will be provided, existing functionality of TPTP will be leveraged if possible.

  • Grid benchmarking:

    Grid resources are heterogeneous in terms of their computing and communication capabilities. Therefore different Grid actors must be able to explore the potential of a Grid system and resources through standard benchmarks. A benchmarking functionality that allows the execution and presentation of benchmarks on Grid resources will be provided.

  • Grid application deployment:

    A functionality that supports Grid application developers and Grid application users with the deployment of their applications on the Grid will be developed.

  • Grid visualization tools:

    A Grid visualization framework will offer functionalities that allows the visualization of scientific and numerical calculations.

  • Grid workflow builder:

    In the future Grid applications will consist of many interlinked Grid jobs building a complex workflow. A functionality to support the creation of such workflows by a graphical editor will be provided based on GEF (GMF).

  • Grid command console:

    Many existing Grid systems are using a command line interface (CLI) to interact with Grid resources and most Grid users are familiar with these CLIs. Therefore functionality will be delivered that can be used as Grid command console.

• Exemplary implementation of the integration of the widely deployed EGEE middleware gLite

  The g-Eclipse project will demonstrate the usefulness of the general framework by the exemplary support for the gLite middleware of the EGEE project and its components.

  • Grid Virtual Organization management:

    Using the VOMS (virtual organization management service) functionality of authorization, authentication, and management of virtual organizations will be implemented.

  • Grid job and file management, application deployment and monitoring:

    These will be implemented as a connection to the g-Lite middleware.

  • Grid infrastructure monitoring:

    A virtualization tool for the EGEE infrastructure will be provided.

  • Grid benchmarking:

    Already existing functionality of the GridBench will be reused if possible.

  • Grid visualization tools:

    The Grid visualization kernel GVK will be integrated.

Organization

Initial committers

The initial committers will focus on the definition of the general Grid framework and on the exemplary support for the gLite middleware. The evolving and hardening of the general framework will be followed as well. Our goal is the delivery of a middleware independent API needed to develop and integrate middleware dependent tools.

The initial committers are:

Code Contribution

All partners start the Eclipse specific development with this project. Therefore no initial code contributions will be given by the partners at the start of the g-Eclipse project.

Interested parties

• Forschungszentrum Karlsruhe GmbH, Karlsruhe, Germany
• Poznan Supercomputing and Networking Center, Poznan, Poland
• Institut für Graphische und Parallele Datenverarbeitung der Johannes Kepler Universität Linz, Austria
• Dept. of Computer Science, University of Cyprus, Nicosia, Cyprus
• Innoopract Informationssysteme GmbH, Karlsruhe, Germany
• Advanced Computing and Emerging Technologies Centre, University of Reading, UK

The above mentioned partners have committed to provide at least 8 full time resources for the next two years for the implementation of g-Eclipse.

User community

Grid infrastructures for e-Science have been built over the last few years and there is a clear tendency for a long-term provision of such infrastructures for scientists. As the interest in using these infrastructures in the scientific and the commercial domain is rising, the g-Eclipse project expects many early adopters of the projects results. The project plans to advance the project by cooperating with other Grid infrastructure projects, by participating at dedicated Grid conferences and events and by using the standard eclipse.org mechanisms of supporting an open source community of early adaptors.

Tentative Plan