High Level Design (HLD) Specification


 

Data Collection & Communication Component

 

 

 

Eclipse TPTP Core Platform Project

Revision 0.4.0

 November 16, 2004

 

 

 

 

 

 

 

 

 

 

 

 


 

 

Table of Contents

 

1          Introduction.. 8

1.1        Purpose. 8

1.2        Intended Audience. 8

2          System Architecture. 9

2.1        Basic Structure. 9

2.2        Expanded Architecture. 9

Agent Controller Subcomponents. 13

2.3.1      Configuration Manager. 13

2.3.2      Error Handling Services. 13

2.3.3      Logging Service. 13

2.3.4      Agent Manager. 14

2.3.5      Data Manager. 14

2.3.6      Connection Manager. 14

2.3.7      Message Manager. 14

2.4        Service Agents. 14

2.4.1      Process Controller. 15

2.4.2      Environment Info. 15

2.4.3      File Transfer. 15

2.4.4      Remote File Browsing. 16

2.5        Client Library Components. 16

2.5.1      Connection Management & Secured Connection Management 17

2.5.2      Session Management 17

2.5.3      Message Management 18

2.5.4      Agent Management 18

2.5.5      Data Management 18

2.5.6      Event Listeners & Handlers. 18

2.5.7      Command Handlers. 18

2.5.8      Configuration Management – NEW?. 18

2.5.9      File Transfer Management 19

2.6        External Interfaces of Client Library.. 19

2.6.1      Client Library external interfaces. 19

2.6.2      Client library Java packages. 20

2.6.3      Communication API 20

2.6.4      Agent API 24

2.6.5      Event Listener API 26

2.6.6      Command Handler API 28

2.6.7      Message Building API 29

2.6.8      Data Processor API 30

2.6.9      Example of Client Interaction with Client Library. 31

2.7        HTTP Support.. 33

2.8        Agent-to-Agent Communication.. 34

3          Process Sequences.. 36

3.1        Typical Client/Agent Communication Sequence. 36

3.2        Client connection sequence. 36

3.3        Agent connection sequence. 37

3.4        Agent creation sequence. 37

3.5        Message Flow... 37

3.6        Client view for starting connection.. 39

3.7        Interaction for setting up data collection/connection.. 39

3.8        Finding an agent.. 39

3.9        Client view for sending command.. 40

3.10       Client configuration of an agent/Agent Controller.. 40

3.11       How a client can discover that it needs to connect to more than one Agent Controller (Peer2Peer) 40

4          Design Strategies.. 41

4.1        Message Pipeline. 41

4.1.1      Transport Layer. 41

4.1.2      Payload Normalizer. 42

4.1.3      Command Extractor. 43

4.1.4      Message Flow.. 43

4.1.5      Message Buffer Memory Management 45

4.2        Data Transfer and Transport Layer.. 46

4.3        Agent Manager.. 46

4.3.1      General Vision. 46

4.3.2      Reference Counting. 47

4.3.3      Agent Locking. 48

4.3.4      Bound Agents. 48

4.3.5      Commands. 49

4.4        Process Controller Agent.. 50

4.4.1      Overview.. 50

4.4.2      Commands. 50

4.5        Environment Info Agent.. 50

4.5.1      Overview.. 50

4.5.2      Commands. 51

4.6        File Transfer Agent.. 51

4.6.1      Commands. 51

4.7        Remote File Browsing Agent.. 53

4.8        Command Extractor.. 54

4.8.1      Command extractor XML Structure. 54

4.8.2      Command extractor XML Parser. 55

4.9        Agent Controller configuration.. 55

4.9.1      Agent Controller initialization  55

4.9.2      Agent Controller configuration for PayLoad Normalizer and Command Extractor. 55

4.9.3      Agent initialization. 57

4.9.4      Client-Agent Controller interaction. 57

4.10       Event Processing.. 57

4.11       Security: SSL Support.. 58

4.12       Backward Compatibility.. 60

4.13       Session Management.. 61

4.13.1    Session concepts and usage. 61

4.13.2    Session. 63

4.13.3    Session management rules. 63

4.13.4    An example of “stateless” default session info processing. 63

4.13.5    An example of explicit session info processing. 65

4.14       Dynamic Agent Controller configuration.. 65

Agent Controller runtime configuration (serviceconfig.xml). 65

4.14.2    Agent runtime configuration (pluginconfig.xml). 66

4.15       Error Handling Services. 66

4.15.1    Sources of Error. 66

4.15.2    Error Handling Mechanism.. 66

4.15.3    Synchronous and Asynchronous Errors. 67

4.15.4    Exception Hierarchy. 67

4.15.5    Error Code format 67

4.15.6    Common Base Event Model 68

4.15.7    Error Codes and Scenarios. 70

4.16       Logging Service. 74

5          Design – Client Library.. 75

5.1        Connection Management.. 75

5.1.1      Node. 75

5.1.2      Process. 75

5.1.3      Connection. 76

5.1.4      Communication Channels. 77

5.1.5      Connection Sequence Diagram.. 77

5.2        Message Management.. 78

5.2.1      Existing Commands. 78

5.2.2      New Commands. 79

5.3        Command Handlers. 79

5.3.1      Command Handlers for Events. 80

5.4        Agent Management.. 80

5.4.1      New Agent interface. 82

5.5        Data Manager.. 82

5.6        Data Processor interface. 82

5.7        Data Reception Sequence Diagram... 82

5.8        Event Listeners & Handlers. 83

5.8.1      ConnectionListener interface. 83

5.8.2      ProcessListener interface. 83

5.8.3      ErrorListener interface. 84

5.8.4      AgentListener interface. 84

5.9        Exception Handling.. 84

5.10       Execution API 84

5.11       Security.. 84

5.12       Configuration Management.. 85

5.13       Asynchronous Event Handling.. 85

5.13.1    Synchronous Events. 85

5.13.2    Asynchronous Events. 85

5.13.3    Event Subscription. 85

5.13.4    Event Source. 85

5.13.5    Event Listeners. 86

5.13.6    Event Identification. 86

5.13.7    Event Information. 88

5.13.8    Event Class. 88

5.13.8    EventCommandHandler Interface. 88

6          Interfaces.. 89

6.1        Client-Client Library Interface. 89

6.1.1      Existing Hyades Client - Full Packages List 89

6.1.2      Java Packages that need Conversion to C++.. 89

6.1.3      List of Java Classes (Needs porting to C++). 90

6.2        Socket Library Interface. 95

6.3        HTTP Library Interface. 95

6.4        Transport Layer Interface (Changes TBD) 95

6.4.1      Design Overview.. 95

6.4.2      Functions Provided By Transport Library. 95

6.4.3      Functions Provided By Agent Controller. 97

6.5        Payload Normalizer Interface. 98

6.5.1      Agent Controller configuration for PayLoad Normalizer and Command Extractor. 98

6.5.2      Functions Provided by Agent Controller. 98

6.5.3      Functions Provided by PayLoad Normalizer. 99

6.6        Command Extractor Interface. 99

6.6.1      Functions Provided By Command Extractor. 99

6.6.2      Functions Provided by the Agent Controller to interface with the Command Extractor. 99

6.7        Socket Transport Interface. 100

6.7.1      Socket Message Envelope. 100

6.7.2      Connection Handshake. 101

6.8        Named Pipe Transport Interface. 102

6.8.1      Named Pipe Message Envelope. 102

6.8.2      Connection Handshake. 103

6.9        HTTP Transport Interface. 104

6.9.1      Sample HTTP Request Message Envelope. 104

6.9.2      Sample HTTP Response Message Envelope. 105

6.9.3      HTTP Connection Handshake. 105

6.10       Agent Base Classes and Templates. 106

6.10.1    General Class Definitions. 107

6.10.2    Base Agent Interface – IBaseAgent and BaseAgentImpl Classes. 108

6.10.3    Variable interface – IVariable and VariableImpl classes. 110

6.10.4    Event Interfaces – IEventProvider and EventProviderImpl Classes. 111

6.10.5    Data Provider Interface – IDataProvider and DataProviderImpl classes. 112

6.10.6    Attachable interface – IAttachable and AttachableImpl Classes. 113

6.10.7    Base Collector Template – BaseCollectorImpl Class. 114

6.10.8    Example of an Agent class definition. 116

7          Command Protocol.. 117

7.1        Basic Protocol. 117

7.1.1      Destination and Source IDs. 117

7.1.2      Context 118

7.1.3      Command Element and Interface. 118

7.2        Agent Manager Interface. 118

7.2.1      Commands. 118

7.2.2      Events. 122

7.3        Base Agent Interface. 123

7.3.1      Basic Interface. 125

7.3.2      Attachable Interface. 126

7.3.3      Variable Interface. 127

7.3.4      Web Services Interface. 130

7.3.5      Event Interface. 130

7.4        Collector Interface. 131

7.4.1      Basic Interface. 132

7.4.2      Events. 135

7.4.3      Agent State Machine. 139

7.5        Event Provider Interface. 140

7.5.1      Commands. 140

7.6        Process Controller Agent Interface. 140

7.6.1      Commands. 141

7.6.2      Events. 142

7.7        Environment Info Agent Interface. 143

7.7.1      Commands. 143

7.7.2      Message Buffer Memory Management 144

 

1         Introduction

This is the high-level technical design document for the new Agent Controller and associated development.

1.1      Purpose

This document specifies a high level design, including inter-component interfaces and dependencies sufficient to guide low-level design and implementation that will meet the needs described in the requirements documentation.

1.2      Intended Audience

This document is expected to be used by a technical audience of:

·        Software developers tasked to implement the design.

·        Technical staff involved with testing, maintenance, and support.

1.3      Organization of the Document

 

2         System Architecture

2.1      Basic Structure

At the most basic level, the Eclipse TPTP system consists of a client, an Agent Controller and one or more agents. 

 

The client may be an Eclipse workbench with TPTP plug-ins installed, or it can be a standalone application built using a C++ implementation of the TPTP client library.  The client can be running on the same machine as the Agent Controller or it can be on a different system.  In either case, the client communicates with the Agent Controller using an interprocess communication transport that is provided with TPTP.

 

The agents can include both system agents provided as part of TPTP and third party data collection agents.  The agents always run on the say machine as the Agent Controller.  The agents communicate with the Agent Controller using an interprocess transport that is provided by TPTP.

 

The scope of this project includes development of the Agent Controller, a C++ client library, base classes and templates for C++ agent development and several system agents.  The relationship of these components is shown in Figure 2-1.

 

Figure 2-1 – Basic System Structure

 

The components shown in gray above are within the scope of this project and will be described in greater detail in this document.  The other components represent products that will be developed using TPTP.  They are outside the scope of this document.

 

The actual system structure can be more complex than shown above.  Each client can be attached to multiple Agent Controllers, and each Agent Controller can be in communication with multiple clients.  Further details are provided throughout this document.

2.2      Expanded Architecture

The previous section described a very basic view of the system architecture.  This section will further refine the description above and indicate the components involved.  For the purposes of this discussion, only the native/C++ implementation will be considered.  The existing Eclipse-based Java client will continue to be a primary platform, but its architecture will not change.

 

The Agent Controller will communicate with clients and agents through a pluggable message pipeline.  In addition, common transport services needed by both the client library and the controller’s transport layer will be moved into a shared library.  The required components are shown in Figure 2-2 below.

Figure 2-2 – Component Diagram

The diagram above shows the major components that have been identified for this project.  Some other components, including an enumeration of agents and security-related components, are described later in this document but their position in the architecture should be clear from the above document.  The arrows in this diagram do not indicate program flow.  The arrows are intended to indicate a dependency relationship between components.  The arrow points to the component that defines the interface between the two.

 

The Client component in the figure above represents the client application that is using TPTP.  It does not correspond to a component which needs to be developed for this project.  The Client Library is a native shared library which is designed to be used by the client application.  The interface between the Client and the Client Library, including the hierarchy of classes exposed by the client library is detailed below.

 

The Socket Library component is a shared library that is used by both the Client Library and the Socket Transport Layer component.  This library provides basic socket services.  Similarly, the HTTP Library provides basic services related to HTTP communication and is shared between the Client Library and the HTTP Transport Layer.  The interfaces to both of these libraries are specified below.

 

The Socket Transport Layer, HTTP Transport Layer and Named Pipe Transport Layer are all pluggable modules that are loaded by the Agent Controller to manage communications with other components.  The pluggable message pipeline strategy is described below.  All pluggable transport layers use the same interface with the Agent Controller.  This interface is defined below.  Each transport layer specifies its own message envelope and low-level commands to which it responds, if any.  These details are also described below.

 

Each transport layer may work with one or more payload normalizers.  See the message pipeline section below for a full discussion of the role of the payload normalizer.  These payload normalizers are pluggable modules loaded by the Agent Controller and all payload normalizers interact with the Agent Controller using a standard interface.  This interface is described below.

 

Each message pipeline is associated with a single Command Extractor.  The Command Extractor breaks the command stream into individual commands and passes them to the Agent Controller for processing.  In the current design, all message pipelines share a single Command Extractor.  The interface between the Agent Controller and the Command Extractor is described below.

 

The Agent component in the figure above corresponds to both system agents and data collector agents, as both interact with the rest of the system in the same way.  Agents exchange messages with the Agent Controller through the Named Pipe Transport Layer.  Among the deliverables for the current project is a set of C++ templates and base classes that standardize the communications through the named pipe mechanism.  These classes will be used as the basis for further agent development, including the system agents that will be developed as part of this project.  A description of these classes and templates is provided below.  Agents are not required to use the base classes.  These are provided for convenience.

 

2.3      Agent Controller Subcomponents

The Agent Controller can be further subdivided in logical subcomponents.  These may be broken out into shared libraries, or they may be implemented within the Agent Controller itself. This decision will be made during the low-level design.  Because the Agent Controller will be implemented in C, these will not be implemented as classes, but they will be functionally similar to classes.  See Figure 2-3 for an overview of the subcomponents.  Each of these is described briefly below.

 

Figure 2-3 – Agent Controller Subcomponents

 

2.3.1      Configuration Manager

The Agent Controller configures itself and starts agents as defined in the agent configuration file. The startup of the Agent Controller is a multi-step process; the Agent Controller starts itself by reading a configuration file and provides services for servicing clients and agents. The current implementation of the Agent Controller does not recognize modifications to the startup configuration file at runtime. The Agent Controller has to be restarted for the modifications to the startup file to take effect.

2.3.2      Error Handling Services[kac1] 

The Error Handling Services component will provide services for error management, detection, propagation and logging mechanisms. The error management will be based on the Common Base Event schema to make it flexible and extensible.

2.3.3      Logging Service[kac2] 

The Logging Service component will provide interface for logging an informational message, event or an error.

2.3.4      Agent Manager

The Agent Manager will provide an interface that allows clients and agents to obtain information about the available agents and gain access to instances of agents.  Agents also use this interface to register as agents after establishing a connection with the Agent Controller.  This interface is described in the Protocol section below.  The Agent Manager will manage agent lifecycles and agent locking.  The Agent Manager is described in much more detail in the “Strategies” section of this document.

2.3.5      Data Manager

The client requests for data from the agent that is routed through the Agent Controller. The Agent Controller gets the client requested data from shared memory (the agent puts the data on the shared memory) and puts it on the appropriate transport layer to be received by the client.

2.3.6      Connection Manager

The Connection Manager will maintain data structures describing components that have established connections to the Agent Controller.  When a component requests a connection, the Connection Manager will validate the protocol being used, assign a connection ID and create an entry in its connection table.  The Connection Manager will also provide services to allow other Agent Controller subcomponents to obtain connection information, such as the transport layer associated with a given connection ID.

2.3.7      Message Manager

The Message Manager loads the plug-ins for the message pipeline, provides the interfaces called by these plug-ins and manages the routing of messages through the pipeline.  When a command is sent to a subcomponent within the Agent Controller (for instance, to the Agent Manager), the Message Manager will identify the message and call the appropriate internal function for the processing of the command. (Each internal subcomponent will provide a single entry point for all commands sent to that subcomponent.)  The interfaces to the various plug-ins involved in the message pipeline are described below.

2.4      Service Agents

In addition to redesigning the Agent Controller component, this project involves the development of several system agents which will provide general services to clients, other agents and the Agent Controller.  These system agents are shown in the figure below.

Figure 2-4 – System Agents

2.4.1      Process Controller

Provides ability to create, start, and stop a specified application. It sends notifications concerning process state changes (events) for the application it starts, such as when and how it is terminated. Its purpose is to insulate clients from the OS-specific details of creating/controlling an application.

 

A standard process controller will be implemented as a service agent. A standard process control interface will be published, allowing developers to create custom controllers (e.g., to support binary instrumentation prior to application launch).

2.4.2      Environment Info

The EnvironmentInfo service agent provides basic system information about the platform on which it is running.  Its purpose is to provide clients with identifying information that it might display to a user and/or to use in determining its own analysis of the data. For instance, when collecting performance data it is useful to a user to know which CPU was used when they compare data for an application which was run on multiple systems. It may also be necessary for the client to know CPU info when interpreting data for display.

 

(A standard interface will be published, allowing developers to create proprietary replacements – but this is not required for initial release.  Initial version of this service agent is likely to include Intel proprietary methods for determining CPU names.  The open source common interface would include methods to obtain all the other information.)

2.4.3      File Transfer

This agent provides the service to move data files between the client and Agent Controller systems.

 

In order to move files, the Agent Controller will start up its own separate listening thread. The port number is specified in the “serviceconfig.xml.” The default port number is now set at filePort=”10005.”

 

At the startup of the Agent Controller, the “File” server will be created and listening to the specified port number.

 

The client will send in a command request for the file server port connection mechanism (e.g. socket). Once the client holds the socket connection, it can perform the actual file transfer operations:

·        get a file from the Agent Controller

·        send a file to the Agent Controller

·        delete a file in the Agent Controller system

 

It is noted that the current implementation of this “File” server is using the JVM as the underlying mechanism for processing the file transfer.

2.4.4      Remote File Browsing

This agent provides the service for the client to explore directories and files in the Agent Controller system transparently just like local file system.

 

The client needs the ability to launch applications in the target system. In order to launch correctly, it has to specify information such as file name and its path name.

 

In addition, it also needs to verify whether this is an executable file that can be launched or not (e.g. finding out the file attribute info).

 

As a result, the client requires a programmatic way to query, explore and validate through the directory and file systems in the target system. This agent provides such services.

2.5      Client Library Components

The TPTP client library consists of a set of API that facilitates in providing communication interface between a client application like the Eclipse workbench and the Agent Controller running on the target machine. The TPTP client library can be logically viewed as containing the following listed components also as shown in the fig 2.5.1

 

 

Fig 2.5.1 Components of the Client Library

 

The Client library uses the interfaces provided by the various components of the client library to communicate with the Agent Controller.

2.5.1      Connection Management & Secured Connection Management

The Connection Management and Secured Connection Management components handle the communication with the Agent Controller by establishing the physical connection with the target machine. This connection is called Control Channel and is used for sending control messages from one end to another. This component also includes the representation of the target machine node and processes running on it. This component also starts a thread to listen for the commands coming from the Agent Controller. The Secure Connection Management component provides SSL connection to enable secured communication. This component will handle the details of secured authentication, security key store management, etc. A client can optionally either use regular connection or secured connection.

2.5.2      Session Management

The Session Management component will enable the concept of a user session on the Agent Controller. A client application can create multiple user sessions simultaneously and operate within their contexts.

2.5.3      Message Management

The Message Management component provides interfaces and classes for building the control or event messages that go across the established connection to the Agent Controller or agents. There are several commands defined in this component. Please refer to the next sections for a detailed list. A client application will use the Message Management component to build and issue necessary commands for performing an activity like data collection using the agents.

2.5.4      Agent Management

The Agent Management component consists of interfaces and classes that define the client-side implementation of the agents running on the target machine. These agent classes can be treated as proxies for the actual running agents. The client application will create instances of these agents and use them to communicate with the agents (via Agent Controller) running on the target machine.

2.5.5      Data Management

The Data Manager component provides a data channel for facilitating the data transfer from the agents to the client application. The data channel is dedicated for performing the data transfer. It provides the interface definition of Data Processors. A client application will implement the data processor interface to create data processors. The Data Manager component will hand over the data received over the data channel to the registered data processors for further handling and processing of data.

2.5.6      Event Listeners & Handlers

A client application creates various objects that correspond to the actual objects on the Agent Controller and also the agents themselves. The client application ought to know about the status of these objects from time to time. So, the Event Listeners and Handlers component defines the interfaces for getting information about these objects with the help of Listeners that listen to these objects. The client application can implement these interfaces to provide listeners that can listen to one or more objects, for e.g. like Connection, Process, Agent, etc. Each of these listeners defined are first registered with their respective objects so that when an event command is received over the control channel, they get forwarded to these listeners.

A new design is being proposed (check the design strategy section for more details) in addition to the existing mechanism to use the regular Command Handler interface to receive the event commands and handle them using the defined the command handlers.

2.5.7      Command Handlers

The Command Handler component provides handler interface and handler objects to handle the commands coming from the agents or Agent Controller. The commands could be the responses to the commands issued by the client to the Agent Controller or any other associated components with it. This component defines the interface for a Command Handler. The client application can implement this interface to provide handlers and register them.

2.5.8      Configuration Management – NEW?

The current Hyades client implementation doesn’t have any configuration for the client library as the configuration information is maintained by the Eclipse workbench in the preferencesUI. For the C++ client library implementation, we need to be able to specify the following configuration information (though we could depend on the client application if we decide to)

2.5.9      File Transfer Management

The file transfer management component provides client-side implementation of the file transfer mechanism. This component provide interface to initiate requests to the file server running on the Agent Controller.

2.6      External Interfaces of Client Library

The external interfaces exposed by the client library are described in the following sections. A client application uses this interface to interact with the client library. An example of how to interact with the client library also is illustrated.

NOTE: This section currently shows the interfaces based on the current java implementation of the client.

2.6.1      Client Library external interfaces

The fig 2.6.2 shows a different view of the components and the interfaces that are exposed to the client by the client library. The interface definitions of the external interfaces of the client library are shown in the next section[kac3] .

 

 

Fig 2.6.2 Client Library external interfaces and internal components

 

2.6.2      Client library Java packages

The java packages that provide the external interfaces and their implementation are listed below.[kac4] 

 

2.6.3      Communication API

These interfaces or classes are defined in the org.eclipse.hyades.execution.internal.local.control package of the current Java implementation.[kac5] 

2.6.3.1     NodeFactory

The NodeFactory class provides interface for creating a new node that represents the target machine node. A client application can use this Class to create objects of Node to be able to communicate with them. All methods of the NodeFactory class are static members.

2.6.3.1.1      Create Node

This interface allows a client application to create node that acts like a delegate to the target machine. There more than one overloaded interfaces that take different parameters for creating the code. This interface returns an instance of the Node Class. All these methods return an existing instance of Node if a node already exists with the specified IP address or hostname. If a Principal is passed, the Node is associated with the user or application whatever the case may be.

 

Method Signature

Description

Node createNode(InetAddress address)

Creates a Node with the IP address specified. Throws an UnknownHostException if the target machine doesn’t exist in the network.

Node createNode(InetAddress address, Principal principal)

 

Creates a Node with the IP address and the Principal specified. Principal here is the user or application who is creating this node. If the same user tries to create this Node again, this method throws DuplicateUserException.

Node createNode(String hostname)

Creates a Node with the hostname passed as the parameter. The Principal is used as null in this case. Throws an e

Node createNode(String hostname, Principal principal)

Creates a Node with the specified hostname and principal (user). If the same user tries to create this Node again, this method throws DuplicateUserException.

 

2.6.3.1.2      Get Local Host

This interface returns the instance of Node Class that represents the local host machine. Internally the “locahost” is maintained as the first element of the Node List.

 

Method Signature

Description

Node getLocalHost()

Returns an instance of Node Class that represents the localhost (local machine).

 

2.6.3.1.3      Get Node

Searches through the Node list of the factory and return the Node that matches the combination of IP address, hostname and Principal. All these methods thros UnknownHostException if a

 

Method Signature

Description

Node getNode(InetAddress addr)

Returns an instance of Node Class that matches the InetAddress in the Factory Node list.

Node getNode(InetAddress addr, Principal principal)

Returns an instance of Node Class that matches the InetAddress and Principal in the Factory Node list.

Node getNode(String name, Principal principal)

Returns an instance of Node Class that matches the hostname and Principal in the Factory Node list.

 

2.6.3.2     Node

The Node is a representation of the target machine. This is a Java interface that defines the interaction a client can perform on the node or target machine. Internally the NodeImpl class provides the implementation for the Node interface.

 

Interface Signature

Description

void killProcess(Process process)

Terminates the process specified by the Process parameter. Throws InactiveProcessException if the Process is inactive. Throws NotConnectedException if the Node is connected.

void shutdown(long delay)

Causes the Node to shutdown after the specified number of seconds (delay parameter) is elapsed. Throws NotConnectedException if the Node is connected.

void reboot(long delay)

Causes the Node to reboot after the specified number of seconds (delay parameter) is elapsed. Throws NotConnectedException if the Node is connected.

Enumeration listMonitors()

Returns the list of Monitors currently running in the Node. Throws NotConnectedException if the Node is connected.

Enumeration listProcesses()

Returns the list of active processes currently running in the Node. Throws NotConnectedException if the Node is connected.

SetNVPairCommand[] getPropertyValues(String name, String type)

Returns the Property Name-Value pairs of the Node. Throws NotConnectedException if the Node is connected.

Process getProcess(String processId)

Returns the instance of Process Class with the ProcessID equal to the passed-in ProcessID.

InetAddress getInetAddress()

Returns the InetAddress of the Node.

InetAddress[] getAllInetAddresses()

Returns all the InetAddresses of the Node.

String getName()

Returns the Host Name of the Node.

boolean isConnected()

Returns true/false if an Agent Controller is connected or not.

Connection connect(int port)

Connects to an Agent Controller using the port number passed as paramter. Throws AgentControllerUnavailableException. Also throws SecureConnectionRequiredException, UntrustedAgentControllerException and  LoginFailedException.

Connection getConnection()

Returns the established connection.

User getUser()

Returns the User of this Node.

void setUser(User user)

Sets the User of the Node.

Process waitForProcess(String processIdentity, int timeout)

Waits to retrieve the Process with the identity specified until the timeout (passed as parameter) happens. Throws InactiveProcessException if the Process is inactive. Throws NotConnectedException if the Node is connected.

void setApplication(Application app)

Sets the application that will be using the Node.

Application getApplication()

Returns the application that is using the Node.

void setSecurityParameters(ISecureClientParameters manager)

Sets the Security Parameters

ISecureClientParameters getSecurityParameters()

Returns the Security Parameters.

 

2.6.3.3     ProcessFactory

This methods provided by this class allows a client application to create processes on the target machine.

All the methods in this class are static.

 

Method Signature

Description

Process createProcess(Node node)

Creates a Process and returns a Process object. The Process is associated with the Node passed as parameter. The Node object is set-up as the listener of the Process.

Process createProcess(Node node, String executable)

Creates a Process and returns a Process object. The Process is associated with the Node passed as parameter. Also sets the executable for the Process so that it can be launched at a later time. The Node object is set-up as the listener of the Process.

Process createProcess(Node node, String executable, String parameters)

Creates a Process and returns a Process object. The Process is associated with the Node passed as parameter. Also sets the executable and its parameters (passed as arguments to this method) for the Process so that it can be launched at a later time. The Node object is set-up as the listener of the Process.

 

2.6.3.4     Process

The Process is a local delegate of the actual Process running on the target machine. The Process interface defines the

 

Interface Signature

Description

void launch()

Launches the Process on the Node the Process object is associated with If the Process was already launched, ProcessActiveException is thrown. Also throws NotConnectedException and NoSuchApplicationException.

void kill(long delay)

Kills the Process (if active) after specified number of seconds elapsed. Throws InactiveProcessException if the process is inactive. Throws NotConnectedException if the Node is not connected.

Enumeration listAgents()

Returns instances of all the Agents associated with the Process. A Process can be associated with more than one Agent.

Agent getAgent(String name)

Retrieves the Agent that matches the specified name.

Enumeration getAgentsByType(String type)

Retrieves the associated agents list that belong to the agent type passed-in as parameter.

void addProcessListener(ProcessListener listener)

Adds a Process Listener (passed as parameter) to the Process. The Listener gets a notification if an event is generated by the Process.

void removeProcessListener(ProcessListener listener)

Removes the Process Listener of the Process.

Node getNode()

Returns the Node the Process is associated with. Throws InactiveProcessException if the process is inactive.

void setName(String name)

Sets the Name of the Process.

String getName()

Returns the Name of the Process.

String getUUID()

Returns the UUID of the Process if it has one, null otherwise.

void setExecutable(String exe)

Sets the Executable for the Process that can be launched at a later time. Throws ProcessActiveException if the process is active since the executable can’t be set when the process is active.

String getExecutable()

Return the Executable name the Process is set with.

String getProcessId()

Returns the ProcessID. Throws InactiveProcessException if the process is inactive

boolean isActive()

Returns true/false if the Process is active or inactive.

Console getConsole()

Returns the Console object for this Process

void addEnvironmentVariable(Variable variable)

Adds an environment variable used while launching the executable. Throws ProcessActiveException if the process is active since the environment can’t be changed when the process is active.

Variable getEnvironmentVariable(String name)

Returns the environment variable matching the name passed as parameter.

void removeEnvironmentVariable(Variable variable)

Removes the specified environment variable. Throws ProcessActiveException if the process is active since the environment can’t be changed when the process is active.

Enumeration getEnvironment()

Gets the entire environment for this process.

void setParameters(String parameters)

Sets the parameters for the Process. Throws ProcessActiveException if the process is active since the parameters can’t be changed when the process is active.

String getParameters()

Returns the parameters for the Process.

void setLocation(String location)

Sets the home directory for the Process. Throws ProcessActiveException if the process is active since the location can’t be changed when the process is active.

 

2.6.4      Agent API

This section describes the interfaces and classes that are related to agent management. A client application should use this API to create new agents, configuring them, monitoring applications and collecting and transferring the data. This API is located in org.eclipse.hyades.internal.execution.local.control package.

2.6.4.1     AgentFactory

The AgentFactory Class can be used by the client application to create new agents (instances of Agent interface) and use them to perform various activities on the agents running on the target machine.

 

Method Signature

Description

Agent createAgent(Process process, String name)

Creates an instance of Agent with the Name passed and returns it. The Agent is associated with the Process passed as parameter.

Agent createAgent(Process process, String name, String type)

Creates an instance of Agent with the Name and Type passed and returns it. The Agent is associated with the Process passed as parameter

 

2.6.4.2     Agent

This is an interface that defines the agent contract, the operations a client can perform on an agent. Internally AgentImpl class provides implementation for this interface. The AgentFactory Class is used to create instances of the Agent.

 

Interface Signature

Description

void addAgentListener(AgentListener listener)

Adds the passed in listener as Agent Listener to the agent running on the target machine. Events raised by the Agent will notify the added/registered listener about the Event. A listener has to implement the AgentListener interface to be able to register with the Agent. Currently Active, Inactive and Error events are notified.

void removeAgentListener(AgentListener listener)

Removes the specified Agent Listener that was added previously from the Agent.

Process getProcess()

Returns the Process the Agent associated with.

void setAutoAttach(boolean auto)

Sets Auto Attach mode, where attachment to the Agent happens automatically if the Process is active.

boolean isAutoAttach()

Return true/false if the attach is auto.

void attach()

Attaches to the Agent running on the target machine. Throws InactiveAgentException if the agent is not active. Throws InactiveProcessException if the process is not active.

void detach()

Detaches from the Agent running on the target machine. Throws InactiveAgentException if the agent is not active. Throws InactiveProcessException if the process is not active.

String getType()

Returns the Type of the Agent.

String getName()

Returns the Name of the Agent.

String getUUID()

Returns the UUID of the Agent.

boolean isActive()

Checks and returns if the Agent is Active or Inactive.

boolean isMonitored()

Returns if the Agent is currently being monitored.

boolean isAttached()

Returns if the Agent is currently attached to a Client.

boolean isAttached(boolean remote)

Returns if the Agent is currently attached to a Client.

void startMonitoring(DataProcessor processor)

Starts the Agent Monitoring. Sets-up the Data Channel for the Agent and the data channel is used for transferring the data from the Agent. The Data Processor passed is used for handing over the data received over the data channel. Throws InactiveAgentException if the agent is not active.

void stopMonitoring()

Stops monitoring the Agent, stops the data channel servers. Throws InactiveAgentException if the agent is not active.

void setConfiguration(AgentConfiguration config)

Sets the Agent Configuration of the local Agent instance using the configuration object. This Configuration is published to the Agent on the target machine later.

AgentConfiguration getConfiguration()

Returns the Agent Configuration, instance of AgentConfiguration Class.

void publishConfiguration()

Publishes the Configuration to the Agent on the target machine. Throws InactiveAgentException if the agent is not active.

void invokeCustomCommand(CustomCommand command)

Invokes/Sends a Custom Command on the agent running on the target machine. Throws InactiveAgentException if the agent is not active.

boolean isToProfileFile()

Returns if the Data collected by the agent is being written to a file.

String getProfileFile()

Returns the fully qualified path of profiling file

void setProfileFile(String _profileFile)

Sets the full qualified path of the profiling file.

 

2.6.4.3     AgentConfigurationEntry

This class defines an entry or configuration parameter that is part of the Agent Configuration. Instances of this class are added to the AgentConfiguration object to build the configuration of the agent. Each agent configuration entry is defined with type, name and value.

 

Method Signature

Description

AgentConfigurationEntry()

Constructor

String getName()

Returns the Name the configuration entry.

String getType()

Returns the Type of the configuration entry.

String getValue()

Returns the Value of the configuration entry.

boolean isEnabled()

Returns if this entry is enabled or not.

void setName(String new_name)

Sets the Name of the entry.

void setType(String new_type)

Sets the Type of the entry.

void setValue(String new_value)

Sets the Value of the ewntry.

void setEnabled(boolean enabled)

Enables or disables the entry.

boolean equals(AgentConfigurationEntry rhs)

Compares this entry with another entry passed as parameter. Compares the Type, Name and Value strings of these entries.

 

2.6.4.4     AgentConfiguration

Each instance of an agent has a set of associated properties/attributes. These can be configured by a client. A client application uses the instances of AgentConfigurationEntry and AgentConfiguration to build the agent property configuration and is set for the Agent. The AgentConfiguraiton class internally maintains a list to which the configuration entries are added and/or deleted.

 

Method Signature

Description

AgentConfiguration()

Constructor

void addEntry(AgentConfigurationEntry

entry)

Adds an AgentConfigurationEntry passed to the entry list maintained by this AgentConfiguration instance.

void clear()

Clears the configuration entry list.

String getAgentName()

Returns the Agent Name

AgentConfigurationEntry getEntryAt(int offset)

Returns the Configuration entry or parameter at a specific index.

void setAgentName(String name)

Sets the Agent Name

int size()

Returns the length of the configuration entry list.

boolean removeEntry(AgentConfigurationEntry entry)

Removes the specified AgentConfigurationEntry from the entry list.

 

2.6.5      Event Listener API

This Event Listener API defines the interfaces for various types of Listeners that allows a client application to listen to events coming from objects like Process, Connection, Agent, etc. and provide implementation to handle the events. The client application should create objects that implement these interfaces to be able to register them as listeners of respective event generation objects. The client application optionally may choose to implement all the following interfaces within the same object. This API is located in org.eclipse.hyades.internal.execution.local.control package.

2.6.5.1     ProcessListener

This interface defines the methods that need to be implemented by an object in the client application to be able to get the “process launched” and “process exited” events from the Process.

 

Interface Signature

Description

void processLaunched(Process process)

This method of the object implementing the ProcessListener interface is invoked when the specific process is launched on the target machine. The Process object with the details about the process is also passed.

void processExited(Process process)

This method of the object implementing the ProcessListener interface is invoked when the specific process is launched on the target machine. The Process object with the details about the process is also passed.

 

2.6.5.2     ConnectionListener

This interface defines the methods that need to be implemented by an object in the client application to be able to get the “Connection closed” event from the connection established by a Node.

 

Interface Signature

Description

void connectionClosed(Connection connection)

This method of the object implementing the ConnectionListener interface is invoked when the connection of the Node is closed with the target machine. The Connection object with the details about the connection is also passed.

 

2.6.5.3     ErrorListener

This interface defines the methods that need to be implemented by an object in the client application to be able to receive and handle the errors.

 

Interface Signature

Description

void error(String errorId, String error)

This method of the object implementing the ErrorListener interface is invoked when an error occurs during the interaction with the Agent Controller or agents on the target machine. Information about the error  (Error ID and Error Message) are also passed as parameters.

 

2.6.5.4     AgentListener

This interface defines the methods that need to be implemented by an object in the client application to be able to receive and handle the “Active”, “Inactive” and “Error” events from the Agent running on the target machine.

 

Interface Signature

Description

void agentActive(Agent agent)

This method of the object implementing the AgentListener interface is invoked when an Agent becomes active. The Agent object is passed as parameter.

void agentInactive(Agent agent)

This method of the object implementing the AgentListener interface is invoked when an Agent becomes inactive. The Agent object is passed as parameter.

void error(Agent agent, String errorId, String errorMessage)

This method of the object implementing the AgentListener interface is invoked when an error is generated during any interaction with the Agent on the target machine. The Agent object is passed as parameter along with the Error ID and Error Message.

void handleCommand(Agent agent, CommandElement command)

This method of the object implementing the AgentListener interface is invoked for every other event other than active, inactive or error. This acts like a default event handler.

 

2.6.5.5     AgentPeerListener

This interface defines the methods that need to be implemented by an object in the client application to be able to receive and handle the “Peer Waiting” event from the Agent running on the target machine.

 

Interface Signature

Description

void peerWaiting(Agent agent, Agent peer)

This method of the object implementing the AgentPeerListener interface is invoked if a peer agent requests to be monitored. This message comes from the Agent the client is currently monitoring. Information about both the agents is passed as parameters.

 

2.6.6      Command Handler API

This interface defines the methods that need to be implemented by an object in the client application to be able to register itself as the handler of a command received over the control channel. The command may be a command that is the response of a command sent to the Agent Controller or an event from the target machine. This API is located in org.eclipse.hyades.internal.execution.local.common package.

2.6.6.1     CommandHandler

 

Interface Signature

Description

void incommingCommand(Node node, CommandElement command)

This method of the object implementing the CommandHandler interface is invoked when this object registered previously as a handler for a specific command coming from the Agent Controller or agents.

 

2.6.6.2     AcknowledgementHandler

 

Interface Signature

Description

void incommingAcknowledgement(InetAddress server, long ticket)

This method of the object implementing the AcknowledgementHandler interface is invoked when this object registered previously as a handler for an AcknowledgementMessage.

 

2.6.7      Message Building API

This API is located in org.eclipse.hyades.internal.execution.local.common package.

2.6.7.1     ControlMessage

Any message going over the control channel from one end to another end between client and Agent Controller is an instance of this class. This Class extends from the Message interface. A message consists of a header block and a list of commands along with the command data. Each command is an instance of CommandElement. All the commands in the TPTP infrastructure extend from CommandElement. The ControlMessage contains one or more instances of CommandElement (each instance is a command). The client application can use the methods in this class to build the necessary message.

 

Method Signature

Description

ControlMessage()

Constructor

public void appendCommand(CommandElement command)

Appends the passed in instance of the CommandElement to the control message. A CommandElement is nothing but a Command. Basically this will add the command to message that will be passed over the control channel.

CommandElement getCommand(int offset)

Returns the command at the specified index.

int getCommandCount()

Returns the number of commands present in the control message.

RAString getKey()

Returns the authentication key.

long getLength()

Returns the size of the message. This is same as what getSize() method returns.

int getSize()

Calculates and returns the size that includes the size of each of these parameters – length field,  authentication key, command count field and commands themselves

int readFromBuffer(byte[] buffer, int offset)

Reads the control message from the buffer and builds the command list.

void setKey(RAString key)

Sets the authentication key

int writeToBuffer(byte[] buffer, int offset)

Writes the control message to the buffer.

 

2.6.7.2     AcknowledgementMessage

 

Method Signature

Description

AcknowledgementMessage()

Constructor

int readFromBuffer(byte[] buffer, int length)

Reads the acknowledgement message from the buffer.

int writeToBuffer(byte[] buffer, int offset)

Writes the acknowledgement message to the buffer.

 

2.6.7.3     CustomCommand

This is the RA_CUSTOM_COMMAND command that is sent to the agents. This can be used to send any custom message across. This class extends from the SimpleAgentInfoCommand and is located in org.eclipse.hyades.internal.execution.local.common package.

 

Method Signature

Description

CustomCommand()

Constructor

String getData()

Returns the string data.

byte[] getDataBinary()

Returns the data in bytes.

int getSize()

Returns the size of the command.

int readFromBuffer(byte[] buffer, int offset)

Reads the

void setData(String data)

 

int writeToBuffer(byte[] buffer, int offset)

 

 

2.6.7.4     BinaryCustomCommand

This is the RA_BINARY_CUSTOM_COMMAND command and extends from CustomCommand Class.

 

Method Signature

Description

BinaryCustomCommand()

Binary

 

2.6.8      Data Processor API

This API defines the interfaces related to receiving and handling or processing the data collected by the agents. The client application can receive the data using the data processors and load the data using the data loaders into the pre-defined data models.

2.6.8.1     DataProcessor

This interface extends from the IDataProcessor defined in org.eclipse.hyades.execution.core package. The client application can implement the methods defined in this interface to process the data received over the data channel.

 

Interface Signature

Description

void incommingData(byte[] buffer, int length, java.net.InetAddress peer)

This method of the object implementing the Data Processor interface is invoked when data is received over the data channel. The object can provide implementation to handle the data. The data here is passed as byte array.

void incommingData(char[] buffer, int length, java.net.InetAddress peer)

Overload for the above method. The data is passed as character array.

void invalidDataType(byte[] data, int length, java.net.InetAddress peer)

 

void waitingForData()

This method is invoked if there is no data available on the data channel

 

2.6.9      Example [kac6] of Client Interaction with Client Library

This section shows how a client can interact with the client library using the interface provided. This scenario shows launching an agent and sending a message to it and retrieving the data from the agent.

 

string hostname = “machine1”;

int port = 10000;

//Create a Node

Node node = NodeFactory.createNode(hostname);

// Connect the node to the Agent Controller running on the target machine

node.connect(port);

 

// Create a Process object on this node

Process process = ProcessFactory.createProcess(node);

// Set the location of the process

process.setLocation(“C:\\temp”);

 

//Add a Listener for the process.

// Listener for all types of events – ConnectionListener, ProcessListener, AgentListener

CustomListener listener = new CustomListener();

node.getConnection().addConnectionListener(listener); //listen to connectionClosed event

//Set the listener for the events from Process – process launched and process exited events

process.addProcessListener(listener);

 

// Create an agent with the process associated with it

Agent agent = AgentFactory.createAgent(process, “test”, “test”);//process, agent name and type

//Set the Listener to listen to active/inactive events from agent

agent.addAgentListener(listener);

//Set auto attach so that the agent attaches as soon as the process and agent are active

agent.setAutoAttach(true);

 

// Configure the Agent

AgentConfiguration ac = new AgentConfiguration;

// Create a configuration entry – agent property

AgentConfigurationEntry acEntry= new AgentConfigurationEntry();

acEntry.setName(“agentcolor”);

acEntry.setType(“color”);

acEntry.setValue(“blue”);

//Add it to the agent configuration

ac.addEntry(acEntry);

//Set the agent configuration

agent.setConfiguration(ac);

 

//Set the executable and environment for the process

process.setExecutable("java.exe"); // java runtime

String param = “SampleApplication”; // The java class name

process.setParameters(param);

process.addEnvironmentVariable(new Variable("CLASSPATH",”C:\\temp”));

           

// Now Launch the process

process.launch(); // This launches the process along with the agent, assumption that application contains agent

 

//Issue a custom command to the agent

// create a new message

ControlMessage message=new ControlMessage();

MyCustomCommand myCommand = new MyCustomCommand();//extends from CommandElement

myCommand.setProcessId(process.getProcessId());

myCommand.setAgentName(agent.getName())

message.appendCommand(myCommand);

MyCommandHandler commandHandler = new MyCommandHandler();//implements CommandHandler interface

node.getConnection().sendMessage(message, commandHandler);// The command handler will receive the response to handle

….

 

// Create the data processor to handle the data

DataProcessor AgentDataProcessor = new DataProcessor();

//tell agent to start monitoring

agent.publishConfiguration();

agent.startMonitoring(AgentDataProcessor); // establishes the data channel

….

 

2.7      HTTP Support

When socket connection is not feasible (e.g. through the proxy server or firewall), HTTP as the transport protocol is the preferred way for the communication.

 

Because HTTP cannot keep the connection persistent (like in the case of socket connection), it maintains the “virtual” persistent connection using a mechanism called “connection id.”

 

The client will first send in a request for authentication and be given back a “connection id” where it will have to send it in along with subsequent requests so that the agent controller treats them as parts of the same connection.

 

This HTTP support is implemented as a plug-in. It will maintain the same “Transport Layer” interface as the socket connection layer does.

 

 

The HTTP server is expected to be a lightweight custom implementation running in the same process as the Agent Controller. It is not a general purpose replacement HTTP server, rather it only supports the message exchange needs for client communication with the Agent Controller.

2.8      Agent-to-Agent Communication

This is expected to behave like a client to agent communication. It does not matter to the Agent Controller where commands are coming from or going to.  If an agent wishes to talk with another agent, it asks the Agent Controller’s Connection Manager to get a connection (connection ID) which it then uses to send messages, exactly the same as the client-side does.  Alternatively, agents that know each other can choose to set up any method for communication that they wish and not go through the Agent Controller.

 

Future enhancement: Optimize agent-to-agent communication.

 

 

In this diagram, Agent #2 communicates with Agent #1 in the same way as the TPTP client communicates with Agent #1.

 

In another words, an Agent processes requests from both client and other agents in the same way.

3         Process Sequences

3.1      Typical Client/Agent Communication Sequence

 

 

Important notes:

 

-         Currently, the data channel is created from a request of the Agent Controller. This will be a change of flow. The initial request to create the data channel will be moved back to the client (instead of the server).

 

3.2      Client connection sequence

   Client opens socket

   Client send connect request

   Transport layer calls AC to create connection object

   AC validates protocol compatibility

   AC assigns an ID

   AC returns ID to transport layer

   Transport layer sends reply to client

 

3.3      Agent connection sequence

   Agent opens named pipe

   Agent send connect request

   Transport layer calls AC to create connection object

   AC validates protocol compatibility

   AC assigns an ID

   AC returns ID to transport layer

   Transport layer sends reply to agent

   Agent registers as an agent with AC

 

3.4      Agent creation sequence

To be specified.

3.5      Message Flow

The diagram below shows the general flow of command message from the component sending the command to the component receiving the command.

Figure 3-5 – Message Flow Collaboration

The details of how the command is formed in the source component are beyond the scope of this section.  The diagram above begins with the source component sending (1) the command to one of the Agent Controller’s pluggable transport layers.  The transport layer will strip any message envelope and call (2) a method provided by the Message Manager to pass the message along.  The Message Manager will use its internal tables to determine the message pipeline needed.  If the message pipeline contains one or more payload normalizers, the Message Manager will call (3) a method provided by the Payload Normalizer, which may return a new payload and size.  (NOTE: This introduces a memory management issue.  What happens if the payload normalizer needs to reallocate the message buffer?)  The Message Manager will then call (4) the Command Extractor associated with the given pipeline.  The Command Extractor will make a function call (5) back into the Message Manager for each command contained in the message.  The Command Extractor will determine the destination ID when it is parsing the commands and include this in its call to the Message Manager.  Next, the Message Manager will call (6) the Connection Manager to determine the transport layer associated with the destination ID.  If the transport layer thus determined requires a payload normalizer as part of its outgoing pipeline, the Message Manager will call (7) a method provided by this Payload Normalizer, which may modify the message and its size.  The Message Manager then calls (8) a method provided by the outgoing Transport Layer to direct the message to its destination.  Finally, the Transport Layer will send (9) the message (via the transport) to the destination.

 

Although this is the general path, in practice it will be unusual for all of these components to be required.  The diagram below shows the specific call sequence for a client that is connected to the Agent Controller through a socket connection, sending a command to an agent.

Figure 3-6 – Typical Client-to-Agent Sequence

In this example, neither the Socket Transport Layer nor the Named Pipe Transport Layer require a payload normalizer.  This simplifies the flow considerably.  Notice also that the client sends the message asynchronously and is free to perform other tasks while the message is being processed.  The client may choose to block for a time while waiting for the response to this command.  That is a client implementation-specific detail.  The agent likewise has a dedicated thread for receiving messages, and is free to engage in other activities prior to the arrival of the message.

 

The message flow is different for messages sent to components that are internal to the Agent Controller.  This flow is shown in the figure below, including the response.

Figure 3-7 – Client-to-Agent Manager Sequence

 

In this case, the command is sent directly to the Agent Manager by way of a function call, and the Connection Manager is not involved until the Agent Manager sends a reply and the transport layer for the destination of the reply is needed.

3.6      Client view for starting connection

 

3.7      Interaction for setting up data collection/connection

 

3.8      Finding an agent

 

3.9      Client view for sending command

3.10Client configuration of an agent/Agent Controller

3.11How a client can discover that it needs to connect to more than one Agent Controller (Peer2Peer)

 

4         Design Strategies

4.1      Message Pipeline

A basic architectural strategy is to provide a mechanism whereby individual layers can be dynamically loaded to process messages.  The libraries to be loaded are specified in the Agent Controller’s configuration file.  The configuration may specify multiple message pipelines, and a given component may be used by more than one pipeline.  The pipeline used for any given incoming message is determined by the transport layer on which the message is received.  The pipeline used for outgoing messages is determined by the transport layer which will be used to send the message.

 

The basic types of plug-ins are transport layers, payload normalizers and command extractors.  See Figure 4-1 below.  The Agent Controller can be configured to load multiple instances of each plug-in type.

 

 

Figure 4.1.1 – Plug-in Layers

A message pipeline is composed of a transport layer, zero or more payload normalizers and a command extractor.  Each of these components plays a role in processing incoming messages.  The components which are needed for a message pipeline depend on the format of incoming and outgoing data.  The Agent Controller configuration file describes the pipelines which are needed, and based on this configuration the Agent Controller will load the necessary libraries and connect them.

4.1.1      Transport Layer

The Transport Layer is a pluggable component which is responsible for sending and receiving data from another component in the system by way of some interprocess communication mechanism.  The other component (which may be a client, an agent or another agent controller) establishes a connection with the transport layer (possibly through some kind of pluggable transport layer of its own).

 

The agent controller will load the Transport Layer library, set its configuration (for instance, port number for the Socket Transport Layer) and pass it the function pointer of a method to call when it receives incoming messages.  This may be a function provided either by a Payload Normalizer or a Command Extractor.  Both will provide a function with the same signature, so the Transport Layer does not need to know which it is.

 

The Transport Layer provides a function to receive outgoing messages to be sent to its communication partner.  This function will be called either directly by the Agent Controller or by a Payload Normalizer, depending upon the pipeline configuration.  Again, the Transport Layer has no need to distinguish between these two cases.

 

The following figure shows the ways in which Transport Layers are represented in this discussion.  These two figures are equivalent and represent the same set of interfaces.

 

Figure 4.1.2 – Transport Layers

 

4.1.2      Payload Normalizer

A Payload Normalizer is a pluggable component which performs some transformation or filtering on the messages before passing them along to the next component in the pipeline.

 

The agent controller will load the Payload Normalizer library, and pass it pointers to the functions it should call next for incoming and outgoing messages.  The Payload Normalizer itself will provide separate functions to receive incoming and outgoing messages (“incoming” and “outgoing” being defined by the message flow relative to the Agent Controller).  For incoming messages, the message may have come from a Transport Layer or another Payload Normalizer, and it may be going to either another Payload Normalizer or a Command Extractor.  The Payload Normalizer does not need to know where the message came from or where it is going.

 

The Payload Normalizer will perform some operation on incoming messages, and it performs the inverse operation for outgoing messages.  For instance, if the Payload Normalizer decompresses incoming messages, it will compress outgoing messages.

 

The following figure shows the ways in which Payload Normalizers are represented in this discussion.  These two figures are equivalent and represent the same set of interfaces.

 

 

Figure 4.1.3 – Payload Normalizers

 

4.1.3      Command Extractor

The Command Extractor is a component which is responsible for identifying individual commands within messages and determining the destination ID of the command.

 

Figure 4.1.4 – Command Extractor

The Agent Controller will load the Command Extractor and give it the pointer of a function to call when it has received commands.  This will be a function provided by the Agent Controller.  The Command Extractor will provide a function which is to be called when an incoming message is ready to have its commands extracted.  This function may be called either by a Transport Layer or a Payload Normalizer, depending upon how the message pipeline is configured.  The Command Extractor does not need to know which component is calling this function.

4.1.4      Message Flow

When the Agent Controller starts up, it will read its configuration file and load all the message pipeline components needed according to the configuration.  It will then locate the message processing entry points provided by each component and notify each of the entry points of the next component in the pipeline for incoming or outgoing messages.

 

When messages arrive on any transport layer, the message will be passed through the various components in the message pipeline to be processed.  The message will not be sent to the Agent Controller until it has passed through the entire pipeline.  When the Agent controller needs to send an outgoing message, it will call a single entry point (either on a Payload Normalizer of a Transport Layer, depending upon the pipeline configuration).  The message will then be passed through the pipeline (again using entry points set up at initialization time by the Agent Controller) until it reaches the Transport Layer, at which time it will be sent to the target.  The Agent Controller is not involved in the process after its initial call to send the message.

 

The figure below shows the message flow through two hypothetical message pipelines from a client to an agent and back.  This figure supposes that the system has been configured to use two payload normalizers on the pipeline between the client and the Agent Controller (though the pipeline is based on the Transport Layer, not the object connecting to it).  The system supposed does not require any Payload Normalizers between the agent and the Agent Controller.

Figure 4.1.5 – Message and Reply Flow

 

4.1.5      Message Buffer Memory Management

The Message Buffer Manager provides buffers to the components of the Agent Controller.  It controls the life of the buffers and does some amount of buffer reuse to prevent memory fragmenting.  The Transport Layers and Payload Normalizers are likely going to need to resize their message buffers as they manipulate the content.  They will call on the Message Buffer Manager to allocate, resize, and release buffer space.

4.2      Data Transfer and Transport Layer

How do these play together?

4.3      Agent Manager

The Agent Manager is a subcomponent of the Agent Controller that manages access to agents and maintains agent lifecycles.  The intent of the Agent Manager is to abstract details such as when and how an agent is launched.  However, there are cases where agents are loaded by bootstrap mechanisms attached to other processes.  In other cases agents may be launched independent of the Agent Controller by an external mechanism.  The Agent Manager supports the loading of agents in this way.  However, it is the joint responsibility of the client and the agent to work together to establish particular agent identity (for example, the correlation of agent to process) in those cases.  The Agent Manager will not attempt to limit the life cycle of agents that it did not launch.  The precise way this is handled is described below.

4.3.1      General Vision

Before going into the details of the interface commands, it is probably worth discussing the general vision for agent management that lies behind these commands.

 

One of the requirements for the Agent Manager is to support a mechanism whereby clients can discover agents and agent capabilities without actually instantiating the agents.  In order to support that requirement, the Agent Manager will build an internal data structure containing metadata for the agents with the following basic structure (internal representation will be determined during the low level design):

 

Agents

        org.eclipse.hyades.envinfo

                Interfaces

                        org.eclipse.hyades.agent

                        org.eclipse.hyades.envinfo

                Engine Data

                        Single instance = true

                        Bound = false

                        Max clients = unlimited

                        Command = <command to launch agent>

                        Path = <path to agent>

                        Options = <command line options>

                Client Data

                        <agent specific metadata goes here>

 

The Agent Manager will build this table at start up using the metadata provided by each installed agent.

 

The Agent Manager will use the entries under the “engine data” to determine when and how to launch a new instance of an agent to meet requests.  When a product is installed, it will add its agents to this registry.  [kac7] 

 

In addition, the Agent Manager may (optionally) query the listed agents during the startup sequence to verify that the given information has not changed.  All agents must support an externally accessible function which provides the above metadata.  [kac8] By default, the Agent Manager will attempt to load each agent as a dynamic library to verify the information in this table.  However, this behavior may be turned off by way of the Agent Controller configuration file.

 

The “Agents” section will contain entries for all available agents.  Each agent will have an “Interfaces” subsection in its metadata which lists all of the interfaces it supports, and an “Engine Data” subsection which contains information the Agent Manager needs to manage the lifecycle of the agent. 

 

In addition, the agent may optionally provide a “Client Data” subsection which provides proprietary metadata.  This “Client Data” is intended only to be used to provide static metadata that will allow the client to discover the nature of an agent before it is created.  This should be static information which is always applicable for the agent, not the agent configuration.

4.3.2      Reference Counting

The Agent Manager uses a system of reference counting to manage agent lifecycles.  In general, the Agent Manager does not create an instance of an agent until it is needed by a client or another agent.  When the Agent Manager creates the agent, it gives the client (or agent) that requested use of the agent a reference to the new agent.  If the agent supports multiple clients, each subsequent client requesting access to the agent also gets a reference, and the Agent Manager maintains a reference count for the agent.

 

The Agent Manager supports distinct concepts of having a reference to an agent vs. being attached to an agent.  A client (or another agent) may “get” a reference to an agent that it is not actively using.  Other clients or agents may reference or attach [kac9] to this same agent, but as long as the client keeps its reference, the agent will be running.  When a client (or agent) is ready to actively use an agent, it should “attach” to the agent.  When a client is finished actively using an agent it should “detach” from the agent, and when it is completely finished with the agent, it must “release” its reference to the agent.

 

Whether or not multiple clients can attach to an agent will vary from one agent to the next, but it is always possible for multiple clients to get a reference to the same agent.  When working with an agent that only allows a single client to be attached, a client may wish to attach immediately to claim exclusive use of the agent.

 

The Agent Manager decrements the reference count for an agent when a client (or another agent in the role of client) releases reference to the agent.  When the reference count for an agent reaches zero, the Agent Manager shuts down that agent.

 

In the case where the agent is started by some external mechanism, the Agent Manager counts an implicit reference when the agent connects, and the Agent Manager will never attempt to shut it down.  The case of an agent that is started in response to the client launching a process fits this category.  Because the agent has a reference to itself, the Agent Manager will never attempt to shut down this type of agent, and the agent is responsible for managing its own lifecycle.

 

Finally, there are cases where the Agent Controller itself requests that agents start up (usually system agents) when it starts up (depending on how the Agent Controller is configured).  In this case, the Agent Manager holds a reference to the agent and does not shut down the agent until the Agent Controller is shutdown.

 

The following table summarizes the relationship between agent creation and reference counting:

 

Agent Creation Method

Lifetime Management

Created by Agent Controller per client request

Reference counted, agent terminated when reference count falls to 0.

Created by Agent Controller as a system agent

Agent Controller referenced, agent terminated on shutdown

Launched by external mechanism

Implicit reference, agent owns its lifecycle

Launched as client-created process

Implicit reference, process determines lifecycle

Table 4-1 – Reference Counting Summary

 

The reference count is used only to determine if and when the Agent Manager should attempt to shut down running agents.  If the Agent Manager determines that an agent is no longer available (for instance, it disconnects from the engine or an attempt to communicate with it fails) the engine will delete its connection entries and notify any objects holding reference, regardless of the reference count. 

 

Reference counts are entirely internal to the Agent Manager and are not exposed to clients and agents.  Clients and agents are responsible for requesting a reference to agents that they use, but they should not rely on the reference being counted or used to manage agent lifetimes.

4.3.3      Agent Locking

In some circumstances, it may be desirable to obtain exclusive access to an agent that could otherwise be shared.  A client (or agent) can request exclusive access when attaching to an agent or when obtaining a reference to an agent by specifying the appropriate flag.  The agent will remain exclusively locked until the client detaches from the agent.  If the component which has locked an agent releases its reference to the agent or disconnects from the Agent Controller (including broken connections in error conditions) the lock will automatically be released.

4.3.4      Bound Agents

The simplest case for which the Agent Manager is designed involves the launching of agents which exist as independent processes.  However, TPTP must also support agents which are tightly bound to the process which they are monitoring.  These agents are generally started as a side-effect of starting the process they are meant to monitor, and their lifecycle is tightly couple with the life-cycle of this process.

 

For example, the JVMPI agent for monitoring a Java process is started using a special command line option when invoking the “java” process.  The agent itself is loaded as a library by the java process.

 

The Agent Controller would like to be aware of the existence of such agents.  However, it must be able to distinguish these agents from “unbound” agents.  Clients can query the Agent Manager for information about unbound agents, and the Agent Manager can instantiate unbound agents on behalf of the client.  On the other hand, the Agent Manager does not have sufficient knowledge to properly invoke bound agents.

 

Therefore, in order to use bound agents, a client must have particular knowledge about that agent and must start the associated process explicitly.  The client will be able to discover the presence of bound agents and get their metadata from the Agent Manager.  However, the client will need to use the process controller agent in order to start these agents.  Once the agent has been started and has registered with the Agent Manager, it can be located through the Agent Manager interface.

4.3.5      Commands

The table below summarizes the commands supported by the Agent Manager.  Details about these commands can be found in the “Protocol” section of this document

 

Command

Description

Query available agents

Returns a list of known agents, optionally filtered by supported interfaces

Query running agents

Returns a list of running agents, optionally filtered by supported interfaces

Get agent metadata

Returns the metadata associated with a given agent name

Get agent registry

Returns the complete set of agent registry information

Get agent

Obtains an instance of a particular agent, by name

Get specific agent

Obtains a reference to an agent, by ID

Release agent

Releases a reference to an agent

Attach to agent

Notifies the Agent Controller that the caller would like to begin actively using the specified agent

Detach from agent

Notifies the Agent Controller that the caller is no longer using the specified agent

Register agent

Called by an agent instance to notify the Agent Controller that the agent instance is available for use

Deregister agent

Called by an agent instance to notify the Agent Controller that the agent instance is no longer available for use

Table 4-2 – Agent Manager Interface Summary

4.4      Process Controller Agent

4.4.1      Overview

The Process Controller service is an Agent that is started by the Agent Controller during its startup.  It may be used by the Agent Controller to launch other agents and by clients/agents to launch applications to be monitored. Applications may be shell scripts or executable binaries.  They include console apps requiring user interaction (I/O from/to client side) during execution.[kac10] 

 

There may be more than one process controller service agent running at the same time.

 

The standard process controller always launches a process on same system on which it is running[kac11] .

4.4.2      Commands

The table below summarizes the commands supported by the standard Process Controller agent.  Details about these commands can be found in the “Protocol” section of this document

 

Command

Description

Validate process

Checks if specified executable exists and is runnable.  Allows pre-launch check of exe input.

Start process

Launches specified executable

Stop process

Terminates specified process ID

Pause process

Pause execution of specified process ID

Resume process

Resume execution of specified process ID

Attach process

? only useful for debug, so don’t include in standard?

Detach process

?ditto?

Events

 

Process exited

The result of normal termination, exit code is reported

Process terminated

The result of an abnormal termination, reason code is reported

Table 4-3 – Process Controller Agent Interface Summary

Do we need “pre-create” and “post-create” events?  Seems like the collector agent would provide those before/after calling the process controller, so they don’t belong in the process controller.

4.5      Environment Info Agent

4.5.1      Overview

The Environment Info service is an Agent that provides system information about the platform on which it is running.  It is started by the Agent Controller, making it readily available for servicing requests. There may be more than one environment info service agent running at the same time.

4.5.2      Commands

The table below summarizes the commands supported by the standard EnvironmentInfo agent.  Details about these commands can be found in the “Protocol” section of this document

 

Command

Description

Get System info

Returns system name, IP address, RAM, # CPUs, CPU architecture

Get OS info

Returns operating system name and version info

Get CPU info

Returns CPU identifier/name, speed, #caches, cache size, features

Table 4-4 – EnvironmentInfo Agent Interface Summary

If obtaining CPU information involves proprietary code, custom implementations would be written by product owners.

4.6      File Transfer Agent

Current implementation:

-         It requires JVM

-         It starts a separate File server and listening to a different port number

-         “RA_MANAGE_FILE” is a new command to be handled by RAServer and returns the file location

-         “org.eclipse.hyades.collection.framework” package contains Java classes that performs the actual file transfer.

 

Here is the list of interested modules for the implementation of file transfer in Java:

 

Index

Package

Description

1.

Org.eclipse.hyades.collection.framework

(see the “internal” package under this package)

-         Performs the actual file transfer operations

2.

“RACommS.c” program under “RAServer” project

-         Start File Server using JVM

-         Handle “RA_MANAGE_FILE” command by sending the file location

 

For this release, we will re-implement this mechanism using C/C++ and provide standard system commands to perform file transfer.

4.6.1      Commands

 

The following commands are supported through the File Transfer interface.  Clients and agents use these commands to move files between the client and Agent Controller systems.

 

Command:                              

getFile

Parameters:

sourceFileName

String

Fully-qualified path file name

destFileName

String

Fully-qualified path file name

Response:

Transfer status code

-         successful

-         source file is not found

-         source file is found but unable to transfer

-         unable to create destination file

Response Parameters:

No data

Description:

Clients (or agents) may use this command to copy a file (sourceFileName) from the Agent Controller system to the local system at the specified destination file name (destFileName).

 

 

Command:                              

putFile

Parameters:

sourceFileName

String

Fully-qualified path file name

destFileName

String

Fully-qualified path file name

Response:

Transfer status code

-         successful

-         file not found

-         file is found but unable to transfer

-         unable to create destination file

 

Response Parameters:

No data

Description:

Clients (or agents) may use this command to copy a file (sourceFileName) from the local system to the specified destination location (destFileName) on the Agent Controller system.

 

Command:                              

deleteFile[kac12] 

Parameters:

fileName

String

Fully-qualified path file name

Response:

deletion status code

-         successful

-         file not found

-         file is found but unable to delete

 

Response Parameters:

No data

Description:

Clients (or agents) may use this command to delete a given file (fileName) onn Agent Controller system.

4.7      Remote File Browsing Agent

 

This agent provides functions for exploring files and directories in the Agent Controller system.[kac13] 

 

This feature enables the application to allow the user to browse and select which application to run in the target system as well as setting up files and directories properly before running them.

 

It provides the following set of functions

-         Perform basic file operations (create, remove, rename, etc.)

-         Get basic attributes and info about a file or directory

-         Search for files selectively

 

 

#

Operations

Description


1.

 

CREATE

Create a new file or a new directory

 

2.

 

REMOVE

 

Delete an existing file or directory (if empty).

This should support recursive deletion for directory.

 

 

3.

 

MOVE

 

Rename an existing file and/or move it to a new location

 

 

4.

 

 

COPY

 

Copy the current file into a new duplicate [kac14] file

 

 

5.

 

GET FILE INFO

 

Return basic file info:

-         absolute path

-         canonical path

-         file or directory indicator

-         last modified date/time

-         creation date/time

-         file attributes (e.g. hidden, system, etc.)

-         file size or total directory size

 

 

6.

 

 

BASIC FILTERING

 

 

User should be able to use basic filtering criteria (e.g. “*.*”, “*.txt”, etc.).

 


7.

 

SEARCH BY NAME

 

Search for a given file in a given directory (recursively)

 

 

4.8      Command Extractor

The purpose of the command extractor is to parse a message consisting of a batch of commands into discrete command blocks. The standard message used by the pipelines may contain a sequence of one or more command blocks. The command extractor will forward the commands individually to the Agent Controller.

4.8.1      Command extractor XML Structure

The command extractor can receive the following type of XML structure either as a single command or a batch of commands.

a)      Structure 1: Single command block

<Cmd

   dest="101"

   src="102"

   ctxt="10001">

      <agentMgr:registerAgent

                   xmlns:agentMgr="org.eclipse.hyades.agentMgr">

         <agentID>102</agentID>

         <processID>4067</processID>

         <agentName>org.eclipse.hyades.processControllerAgent</agentName>

      </agentMgr:registerAgent>

</Cmd>

 

4.8.2      Command extractor XML Parser

Agent Controller uses the SAX parser implemented by Xerces for processing configuration files in xml structure. The command extractor parses xml input at a greater frequency, hence a decision has to be made for a Agent Controller specific XML parser versus the Xerces parser. The points in favor of the Xerces parser are: Time –to-deliver, already portable code across platforms and the parser can accommodate design changes to the XML structure like namespace validation, XPath support, character set etc. The points in favor of a custom xml parser: light-weight parser with a very small footprint and speed for the command extraction.

4.9      Agent Controller configuration

The startup of the Agent Controller is a multi-step process; the Agent Controller starts itself reading a configuration file. The startup configuration of the Agent Controller is read from a XML file. The current implementation of the Agent Controller (v3.1.x) uses the SAX parser whose implementation is provided by the Xerces XML parser.

 

                                Figure  – Agent Controller, agent and client interaction

4.9.1      Agent Controller initialization  [kac15] 

The Agent Controller initializes by reading the serviceconfig.xml file. The serviceconfig.xml contains apart from the configuration information, the debug level and the logging information. See Agent Controller for additional information.

4.9.2      Agent Controller configuration for PayLoad Normalizer and Command Extractor 

The Agent Controller configuration file (serviceconfig.xml) will define the message pipeline, the transport layer for the message pipeline, Payload Normalizers and Denormalizers for the message pipeline and the command extractor. See below for the XML Schema for the Agent Controller configuration for Payload Normalizer and Command Extractor. . See Payload Normalizer Interface for the specifications of the PayLoad Normalizer and the Command Extractor.

 

<xsd:schema xmlns:xsd="http://www.w3.org/2001/XMLSchema">

<xsd:element name="MessagePipeline">

<xsd:complexType>

 <xsd:sequence minOccurs="1" maxOccurs="unbounded">

<xsd:element name="MessageConfiguration">

 <xsd:sequence>

<xsd:element name="TransportLayer" type="xsd:string" minOccurs="1" maxOccurs="unbounded"/>

<xsd:element      name="PayLoadNormalizer" type="xsd:string"      minOccurs="0"     maxOccurs="unbounded"/>

<xsd:element      name="CommandExtractor" type="xsd:string" minOccurs="1" maxOccurs="1"/>  

<xsd:element      name="PayLoadDENormalizer" type="xsd:string"      minOccurs="0" maxOccurs="unbounded"/>

 </xsd:sequence>

   </xsd:element>

  </xsd:sequence>

</xsd:complexType>

</xsd:element>

</xsd:schema>

 

Example:

Case i)

<MessagePipeline>

<MessageConfiguration>

<TransportLayer>TL</TransportLayer>

<PayLoadNormalizer>HTTP-PN</PayLoadNormalizer> 

<PayLoadNormalizer>HTTPS-PN</PayLoadNormalizer>

<CommandExtractor>DefCommExtractor</CommandExtractor>

</MessageConfiguration>

</MessagePipeline>

 

Case ii)

<MessagePipeline>

<MessageConfiguration>

<TransportLayer>TL</TransportLayer>

<PayLoadNormalizer>HTTP-PN</PayLoadNormalizer> 

<PayLoadNormalizer>HTTPS-PN</PayLoadNormalizer>

<CommandExtractor>DefCommExtractor</CommandExtractor>

<PayLoadDENormalizer>HTTPS-PN</PayLoadDENormalizer>  

</MessageConfiguration>

<MessageConfiguration>

<TransportLayer>TL-II</TransportLayer>   

<PayLoadNormalizer>SOAP1.0-PN</PayLoadNormalizer>    

<CommandExtractor>DefCommExtractor</CommandExtractor>

<PayLoadDENormalizer>HTTPS-PN</PayLoadDENormalizer>  

</MessageConfiguration>

</MessagePipeline>

 

4.9.3      Agent initialization 

The agent initialization is performed when the Agent Controller starts by reading the agent configuration file. Following the initialization of the agents, the agents request the Agent Controller to register the agent and receive a ConnectionId following a successful registration.

4.9.4      Client-Agent Controller interaction

A client connecting to the Agent Controller requests for a connection, the Agent Controller then performs the protocol validation and returns a connectionId following the creation of a connection object as shown in the interaction diagram.

4.10Event Processing

The diagram below shows the processing flow involved in registering an event handler for events provided by some agent and the processing flow when an event occurs.

This diagram starts with some object in the client application wishing to register to receive events from some agent.  The object creates an EventHandler (which should derive from the CommandHandler class) and passes that handler as a parameter in a call to the Connection object’s registerEventListener class.  The Connection object will generate a context ID corresponding to this event handler and add the event handler to its ContextMapper (via the addContext method).  This is equivalent to the way reply handlers are set up in the sendMessage method.

 

The Connection object then sends an addEventListener command to the agent via the Agent Controller.  This command has its own context ID, which is distinct from the event handler ID.  The event handler ID is passed to the agent as the listenerID parameter of the addEventListener command. 

 

When an event occurs within the agent, the event is sent as a command.  The destination ID for the command is the client ID, and the context ID is the listener ID that was passed in to the addEventListener call (i.e. the event handler ID).  This command is received by the SocketReaderThread object in the client library and passed to the Connection object for processing.  The Connection object uses its ContextMapper to find the event handler based on the context ID in the event command.  This is identical to the way any other command is handled.  When the handler has been found, the handler’s incomingCommand method is called.

 

The event handler may, if it wishes, propagate this event to multiple other objects.  However, this is outside of the scope of the TPTP client library.  The TPTP client library will handle events only as described above.

4.11Security: SSL Support[kac16] 

Secure Socket Layer (SSL) is a protocol to secure data transmission across the wire.

SSL makes use of the public-and-private key encryption system.

 

In order to make an SSL connection, the SSL protocol requires that the Agent Controller should have a digital certificate installed.

 

The digital certificate serves as the credential for authentication.

 

Typically, digital certificates are signed by an independent and trusted party called “Certification Authority” (CA).

 

Basically, this is a re-implementation of the current SSL support in Java using C/C++.

Additional features are planned but not for this release.

 

Here is the list of interested modules for the implementation of the security:

Index

Package

Description

1.

org.eclipse.hyades.security

- Eclipse plugin for Hyades security

- SSL connection

- Import and manage digital certificate

2.

“RACommS.c” program under “RAServer” project

- Start secure server using JVM security

 

3.

“Security.c” program under “RAServer” project

- Perform user authentication

 

4.12Backward Compatibility

Compatibility with old clients and agents will be achieved using a special message pipeline.  The new Agent Controller will include a pluggable transport layer and payload normalizer specifically designed to handle the protocol stream used in releases 3.2 and earlier of Hyades, and a separate transport layer and payload normalizer designed to communicate with agents that were design for use with versions 3.2 and earlier.  See the figure below.

 

Figure 4.9.1 – Backward Compatibility Pipelines

 

Using these pipelines, old protocol streams will be converted into the new format and old commands will be translated into equivalent commands in the new protocol.  In some cases, a legacy command will need to be handled as a legacy command by a special subcomponent of the Agent Controller.  See the table below for command mapping information.

 

Old Command

New Interface and Command

RA_AUTHENTICATE

Need more info

RA_LAUNCH_PROCESS

ProcessController::launchProcess

RA_KILL_PROCESS

ProcessController::killProcess

RA_QUERY_PROCESS_LIST

ProcessController::queryOwnedProcessList

RA_QUERY_AGENT_LIST

AgentManager::queryRunningAgents, Legacy::getAgentProcess

RA_QUERY_AGENT_DETAILS

Served by compatibility layer

RA_REGISTER_AGENT_NOTIFICATION

Need more info

RA_ATTACH_TO_AGENT

AgentManager::attachToAgent

RA_DETACH_FROM_AGENT

AgentManager::detachFromAgent

RA_START_MONITORING_AGENT_LOCAL

DataManager::openDataChannel

RA_STOP_MONITORING_AGENT

DataManager::closeDataChannel

RA_AGENT_QUERY_STATE

AgentMgr???

RA_SET_NAME_VALUE_PAIR

Legacy::setNameValuePair

RA_GET_PROPERTY_LIST

Legacy::getPropertyList

RA_MANAGE_FILE

FileTransfer::<getFile/putFile/deleteFile>

RA_RESOURCE_LOCATION

FileTransfer::???

RA_CUSTOM_COMMAND

Legacy::customCommand

RA_BINARY_CUSTOM_COMMAND

Legacy::binaryCustomCommand

RA_LOCAL_AGENT_ACTIVE

Not really used??

RA_AGENT_SCOPING_INFORATION

AgentManager::registerAgent

RA_AGENT_CONFIGURATION

Handled by compatibility layer?

RA_AGENT_MONITOR_REQUEST

 

RA_CONTROLLER_REQUEST_MONITOR

 

RA_PEER_UNREACHABLE

 

RA_CONTROLLER_MONITOR_PEER

 

 

4.13Session Management[kac17] 

A session is a logical connection between the client and the Agent Controller.

The session management provides the mechanism to maintain state information and group a set of related requests into a single unit-of-work.

 

In addition, the session management can provide

-         the contexts to support multiple users across the same connection

-         the mechanism for persistence support across multiple connections or restart.

 

In this release, we will provide the mechanism for the client and agent to manage the “session info” within their unit of work (across multiple requests and responses).

4.13.1 Session concepts and usage

Figure 1 below shows a basic usage of a session. It typically consists of one client connection, one Agent Controller and a set of related agents. System agents are normally shared and used by multiple sessions. It is up to the users to define and implement their agents such that they can be shared or not (i.e., used exclusively) by each session. Each session will require at least one connection for the data exchange.

 

Figure 1 – Multiple Sessions per [kac18] Agent Controller

 

Figure 2 below shows how the client application can make use of session information to support multiple users over the same connection.

 

Figure 2 – Multiple sessions over the same client-Agent Controller connection

 

4.13.2 Session

For each message header, there will be an optional field for the session id. The indicator will be in the “Flags” byte.

 

Bytes 0-3

Magic number

Bytes 4-7

Flags

16 bytes

MD5 code (optional)

16 bytes

Session ID

 

4.13.3 Session management rules

 

Index

Description

1.

Session ID is created, assigned, and managed by the Agent Controller only

2.

Session ID is an optional field in the message header of every request

3.

By default, session is the connection (session ID is zero or the absence of the session ID itself).

4.

Session ID is returned at the connection time (if requested).

5.

Security (authentication and authorization) is performed at the connection and not at session level.

 

4.13.4 An example of “stateless” default session info processing

Figure 4 below shows how the client can simply submit stateless requests to the Agent Controller without worrying about session information. There is no additional overhead of processing if the client does not make use of session information.

 

Figure 4 – Example of default (stateless) session


 

4.13.5 An example of explicit session info processing

Figure 5 below shows how the client can create and submit requests within a single session. The Agent Controller will manage the association (mapping) between the session id and the unique GUID string that assigns to a specific session.

 

Figure 5 – Example of session creation and management

 

 

4.14Dynamic Agent Controller configuration

The current implementation of the Agent Controller does not recognize modifications to the startup configuration file at runtime (serviceconfiguration.xml ). The Agent Controller has to be restarted for the modifications to the startup file to take effect. The agents get their startup configuration from the Agent Controller, which in turn obtains the agent startup configuration from a file (pluginconfig.xml). Any changes to the agent startup configuration file also require restarting the Agent Controller for agent configuration modifications to take effect.

The Agent Controller will allow certain configuration parameters for the agent and the Agent Controller to be configured at runtime.

4.14.1 Agent Controller runtime configuration (serviceconfig.xml)

The Agent Controller startup configuration file serviceconfig.xml will allow the following configuration parameters to be modified at runtime.

a)      Logging

b)      Debug-Level

c)      List of Agents to be started by the Agent Controller

4.14.2 Agent runtime configuration (pluginconfig.xml)

The Agent startup configuration file pluginconfig.xml will allow the following configuration parameters to be modified at runtime.

a)      Logging

b)      Debug-Level

4.15Error Handling Services

4.15.1 Sources of Error

The various possible areas/components where we need to support error handling are as listed below.

Note: Though the terminology in this section is referring to error handling, the mechanism is applicable for logging also as the CBE schema supports writing both informational and error messages to a log file.

·        Communication between Client and Agent Controller

·        Errors within Agent Controller

·        Communication between Agent Controller and Agent

·        Agent-to-Agent Communication

·        Errors within Agent Process

 

It will be consistent and flexible to handle all such errors using the same error handling mechanism.

4.15.2 Error Handling Mechanism

4.15.3 Synchronous and Asynchronous Errors

The errors could occur due to one of the following ways.

·        Synchronous Error Event – The context for such event happens as a response to a command sent by a component.

·        Asynchronous Error Event – The asynchronous error events are generated due to an error condition that has happened randomly in a component. This error event information will be sent as an Event and the error code and information will be sent along with it.

4.15.4 Exception Hierarchy

The errors generated in any component of TPTP will lead to a language specific exception in case of component that is implemented in C++ or Java. The error code and error information is used in building the exception instance and passed along with the Exception. The Base Exception Hierarchy will consist of the following classes and can be extended to add custom Exceptions. This hierarchy will assist in understanding the exact error domain. The error codes defined in the next sections are the source of these exceptions. The errors generated will be wrapped into one the exception classes along with the CBE block

4.15.4.1 TPTPException

This is the base Exception and all other exceptions should extend from this one. This Exception can be used to represent an error from any of the TPTP components.

The Exception class will have the following data members and respective access methods.

·        Error Code

·        Error Information

4.15.4.2 TPTPAgentControllerException

This Exception class will represent the errors occurring in Agent Controller component.

4.15.4.3 TPTPAgentException

This Exception should be thrown when there is an error in the Agent process itself.

4.15.4.4 TPTPRemoteException

Some of the errors or exceptions generated need to be propagated to the TPTP Client depending on the error situation. In such cases the error code and information block will be sent to the Client across the wire. On the Client machine, this should be translated into TPTPRemoteException.

4.15.5 Error Code format

Whenever an error occurs, the corresponding component will return the following information to the caller of this component.

·        Error Code – This code will be used to uniquely identify the specific error. This is an integer value. Various ranges will be defined to represent the errors raised in different contexts and components.

·        Error Information – This information block will give more custom information about the error raised by a component.

Error Code:                           

Represents the Uniquely identifiable integer value. The error codes  will be defined as static constant in a Class definition or header file.

Data:

 

The proposal is to use the Common Base Event Model based block for sending the error information. This will allow in digesting the error information in flexible manner – error handling, logging, tracing and other purposes.

4.15.6 Common Base Event Model

The Common Base Event Model is a standard model that supports logging, management, problem determination and other types of events by defining an XML based schema specification. The format of the event uses UTF-8 encoding. This specification ensures completeness of data based on 3-tuple structure giving information about

·        The component reporting a particular situation

·        The component affected by the situation

·        The situation itself

 

The schema defines several properties for giving the complete information and data of an event with ability to publish the information about the 3-tuple given above. Please refer to the CBE Schema for detailed information.

4.15.6.1 Data Types

CBE model supports the following XML schema data types ( and their array variation) and signed data types.

·        byte

·        short

·        int

·        long

·        float

·        double

·        string

·        dateTime

·        hexBinary

·        boolean

4.15.6.2 CBE Schema

We will use the following attributes from CBE model for defining the error event information in TPTP components. The Error Source component should add these attributes in the error information block whenever and error is generated. Some of them are common and required and should be part of the information block of the every error generated.

4.15.6.2.1  globalInstanceId

Represents the primary identifier for the event and should be a globally unique id. The length of this Id should be within the range of 128-256 bits. This is an optional property and not mutable.

4.15.6.2.2  creationTime

This is the date and time the error was generated and should be specified as dateTime XML schema data type. This is a required property and is not mutable.

4.15.6.2.3  Severity

This indicates the severity level of the error. The range of values is 0-70. The reserved values are 0-Unknown, 10-Information, 20-Harmless, 30-Warning, 40-Minor, 50-Critical and 60-Fatal. This property is Optional and is not mutable.

4.15.6.2.4  Priority

This defines the importance of the error event. The predefined values are 10-Low, 50-Medium and 70-High and the range of values is 0 to 100. This property is Optional and is mutable.

4.15.6.2.5  ReporterComponentId

This is the Identification of the component that reported the error event or situation on behalf of the affected component. The data type is a complex type defined by ComponentIdentification type. This is a required property and is not mutable. The ComponentIdentification type describes the following information

·        Location

·        LocationType

·        Application

·        ExecutionEnvironment

·        Component

·        Subcomponent

·        ComponentTypeId

·        InstanceId

·        ProcessId

·        ThreadId

4.15.6.2.6  sourceComponentId

This is the identification of the component that was impacted by the event or situation. This also is described by the ComponentIdentification complex data type defined by the CBE XML schema.

4.15.6.2.7  situationType

This describes the type of the situation that caused the error to be generated. This is an extensible string value and must not exceed 512 characters. This is Optional and is not mutable. The reserved key words are – FEATURE, DEPENDENCY, REQUEST, CONFIGURE, CONNECT, CREATE and UNKNOWN.

4.15.6.2.8  Msg

This is the error message that describes the error in a human readable text message string. This property is Optional.

4.15.6.2.9  MsgDataElement

 

4.15.6.2.10            sequenceNumber

This is the error source defined number that allows multiple messages to be sent and processed in a logical sequence, regardless of the order error messages are captured by consumer.

4.15.7 Error Codes and Scenarios

This section lists error codes and the scenarios in which they may be generated. It also mentions about important data that will be included in the error information block. A set of error code ranges will be defined to distinguish the errors generated in various components of TPTP and also to avoid any clash between error codes among them. Each error code is identified with a unique ID (should be defined as static constant variables in a Class or header file) and the value is of type integer.

4.15.7.1 Error Code Ranges (Error Codes are integer values)

Generic Errors              0-1000

Communication Protocol           1001 – 2000

Agent Controller                                               2001 – 3000

Client                                       3001 – 4000

Base Agent                               4001 – 5000

System Agents              5001 – 6000

Custom Agents             >10000

NOTE: We can define other ranges for agents of various types (for e.g. performance, monitoring, logging, testing agents).

4.15.7.2 Generic Errors

The following list of error codes are very generic and can be used in any component of TPTP.

4.15.7.2.1  Success – TPTP_SUCCESS

This indicates the generic success of any action taken. The error information will optionally include any information about the action that is performed successfully.

Error ID

 TPTP_SUCCESS

Value

0

 

4.15.7.2.2  Failure - TPTP_ERROR

This indicates the failure and gives the information about the error situation.

Error ID

 TPTP_ERROR

Value

1

 

4.15.7.2.3  Process Launch Error – TPTP_PROCESS_LAUNCH_ERROR

This error should be reported when there is an error while launching the process.

Error ID

 TPTP_SUCCESS

Value

2

 

4.15.7.2.4  Thread Launch Error – TPTP_THREAD_LAUNCH_ERROR

This error should be reported when there is an error while creating a new thread.

Error ID

 TPTP_PROCESS_LAUNCH_ERROR

Value

3

 

4.15.7.2.5  Communication Error – TPTP_COMM_ERROR

This error should be reported when there is an interruption during the communication between any of Client-Agent Controller, Agent Controller-Agent and Agent-Agent.

Error ID

 TPTP_COMM_ERROR

Value

4

 

4.15.7.2.6  Max Thread Limit Reached TPTP_MAX_THREAD_LIMIT_ERROR

This error should be thrown when the system is not able to create any more threads due to the resource limitation of the system itself or based upon the limit set in the TPTP server configuration file that is used to control the resource allocation.

Error ID

 TPTP_MAX_THREAD_LIMIT_ERROR

Value

5

 

4.15.7.2.7  Max Process Limit Reached - TPTP_MAX_PROCESS_LIMIT_ERROR

This is similar to the Max Thread Limit error but is applicable to the number of processes.

Error ID

 TPTP_MAX_PROCESS_LIMIT_ERROR

Value

6

 

4.15.7.2.8  File Access or Open or Create Error – TPTP_FILE_ACCESS_ERROR

The file access error should be reported when any component is trying to read/open or create a new file. The error information should include the reason for the failure.

Error ID

 TPTP_FILE_ACCESS_ERROR

Value

7

 

4.15.7.2.9  File Write Error – TPTP_FILE_WRITE_ERROR

The file access error should be reported when any component is trying to write to a file. The error information should include the reason for the failure.

Error ID

 TPTP_FILE_WRITE_ERROR

Value

8

 

4.15.7.3 Agent Controller Errors

4.15.7.3.1  Agent Controller Startup Error – AgentController_STARTUP_ERROR

During the Agent Controller startup, it will perform several tasks before the server is ready to take requests from Clients. This error code should be used to report any errors that occur during startup and should give the detailed information about error context and the source of the component. Other specific error codes may be used if they are defined to describe a possible situation created in a component.

4.15.7.3.2  Agent Controller Configuration Error – AgentController_CONFIG_ERROR

This error code should be thrown when Agent Controller is not able to apply the configuration specified in the server configuration file. This could be during startup or run-time server configuration.

4.15.7.3.3  Agent DLL Load Failed Error – AgentController_AGENTDLL_LOAD_ERROR

Agent Controller needs to load the respective Agent DLL before creating an agent of a specific type. This error should be reported to indicate the failure to load the DLL. The reasons for the failure can be an invalid DLL path, invalid DLL file, non-existent, etc.

4.15.7.3.4  Agent Launch Error – AgentController_AGENT_LAUNCH_ERROR

The errors that can occur when Agent Controller launches an agent process should be reported using this error code.

4.15.7.3.5  Agent Controller Shutting Down – AgentController_SHUT_DOWN_ERROR

When Agent Controller is shutting down it will throw this error code to let know the systems participating in the existing connections.

4.15.7.3.6  Invalid Client Version  - AgentController_INVALID_CLIENT_VER_ERROR

The TPTP Client library will be identified with versioning mechanism. When there is a request from a Client, Agent Controller will determine if the version supplied by the client is compatible with Agent Controller before serving the request.

4.15.7.3.7  Invalid Agent Version  - AgentController_INVALID_AGENT_VER_ERROR

This is similar to the Client version validation. Agent Controller will check the agent version and will return an error message representing success or failure.

4.15.7.3.8  System Agent not available  - AgentController_SYSTEM_AGENT_UNAVAILABLE

When Agent Controller is any other component is in need of a system service and that service is not available or down or can’t be started, this error should be reported.

 

4.15.7.4 Communication between Client and Agent Controller

4.15.7.4.1  Connection Failure – AgentController_CONNECTION_ERROR

This error code indicates the failure to create a connection between any of the combinations of Agent Controller-Client, Agent Controller-Agent Controller, Agent Controller-Agent and Agent-Agent.

4.15.7.4.2  Connection Time Out – AgentController_CONNECTION_TIMEOUT

This will be raised when the connection has timed out, which means there is no activity through that connection to Agent Controller and it has been idle for the time period equal to the time-out value. Any communication after the connection is timed out will not be serviced by Agent Controller.

4.15.7.4.3  Session Dropped – AgentController_SESSION_DROPPED_ERROR

A client can optionally create and work under a session after establishing the connection. In this case, any activity from client will be associated with that session. When the session gets dropped for some reason, this error should be reported giving information about the SessionID and the reason for the dropped session if known.

4.15.7.4.4  Agent Connection Failure – AgentController_AGENT_CONN_ERROR

A connection to an agent can be established from a Client or Agent Controller or another Agent. This error should be reported if there is some problem while making the connection.

4.15.7.4.5  Agent Communication Failure – AgentController_AGENT_COMM_ERROR

This error is thrown when the communication is failed between and Agent Controller and Agent.

4.15.7.4.6  Data Channel Creation Error – AgentController_DATA_CHANNEL_CREATE_ERROR

This throws an error code when Agent Controller is not able to create the data channel successfulyl.

4.15.7.4.7  Data Channel Communication Failure – AgentController_DATA_CHANNEL_COMM_ERROR

This error code should be reported when the data channel can’t be used to transfer the data over the wire.

4.15.7.5 Errors in Agent

4.15.7.5.1  Agent Startup Failure - AGENT_STARTUP_FAILURE

This error code should be thrown when there is an error during agent startup.

4.15.7.5.2  Agent Configuration Validation Error – AGENT_CONFIG_INVALID

Every agent has a set of properties like metadata and custom properties. These properties will be set or configured before using the agent.

4.15.7.5.3  Agent Run Failure – AGENT_RUN_FAILURE
4.15.7.5.4  Data Send Failure – AGENT_DATA_SEND_FAILURE
4.15.7.5.5  Invalid Agent Command Error – AGENT_COMMAND_INVALID
4.15.7.5.6  Attach Error – AGENT_ATTACH_PROCESS_ERROR
4.15.7.5.7  Detach Error – AGENT_DETACH_PROCESS_ERROR
4.15.7.5.8  Agent Pause Error – AGENT_PAUSE_ERROR
4.15.7.5.9  Agent Resume Error – AGENT_RESUME_ERROR
4.15.7.5.10            Agent Stop Error – AGENT_STOP_ERROR
4.15.7.5.11            Agent Cancel Error – AGENT_CANCEL_ERROR

 

4.16Logging Service

TBD: Determine whether the current logging agent can be used to log system and error events.

 

Things are under considerations:

-         How do we log during system startup, initialization and shutdown?

-         Asynchronous vs. synchronous logging

-         Should we perform the log in a different process?

-         Time and event correlation mechanism?

-         Client and server loggin?

5         Design – Client Library

The following sections give more design details of the components and their interfaces inside the client library.

5.1      Connection Management

This component provides implementation for local representation of the target machine node and the processes running on them. The Node object provides API to establish a connection with the Agent Controller running on the target machine to be able to communicate with it. The java package org.eclipse.hyades.execution.local.control contains all the classes that are part of this component.

5.1.1      Node

The Node object is a local unique representation of the physical entity in the network. This is specified by the name and IP address of the machine. The NodeImpl class implements the Node interface and provides mechanism to connect to the machine or node (that is being represented by it). NodeImpl uses the ConnectionImpl class to establish the connection. A few of the important methods implemented by NodeImpl class are listed below.

5.1.2      Process

The Process object is a representation of a process running on the target machine. The ProcessImpl class provides the implementation for the Process interface. This class can be used to set the process executable, parameters, etc and launch a process on the target machine. A process can have one or more associated agents and can be registered with one or more process listeners to listen to this process. When a process is launched, the ProcessImpl will setup a local Console thread (window) to get the std input, output and error messages. The org.eclipse.hyades.execution.local.common.Console class implements the Console thread.

The ProcessImpl class implements the following methods.

5.1.3      Connection

The ConnectionImpl provides the physical connection mechanism with the Agent Controller. This will use the socket library or http library interface for establishing the connection using the IP address and port number of the Agent Controller. The ConnectionImpl class implements the Connection interface consisting of the following methods.

 

 

The connect() and disconnect() are used for establishing a connection with Agent Controller and disconnecting. During the connect() call, a thread SocketReaderThread is started to listen to the socket for any incoming commands. The sendMessage() is used for sending the control messages to the Agent Controller. It takes two parameters – the message itself and instance of the command handler. The ConnectionImpl class has a member – instance of ContextMapper class. The ContextMapper keeps a context map of all the commands sent over using this connection. It saves the command contexts and their handler object references so that when a response to this command arrives from the Agent Controller, the saved mapping is used to route the response and hand over the message to the respective handler of the command. The processControlMessage() is called by the SocketReaderThread to send the messages received from the Agent Controller for further processing.

5.1.4      Communication Channels

There are two communication channels available between the client and the Agent Controller serving different purposes as described below.

5.1.4.1     Control Channel

The control channel is used as a communication path for sending commands from client to the Agent Controller and vice versa. This channel is used also to receive the asynchronous events from the Agent Controller or agents. A separate thread called SocketReaderThread is created for reading the commands or events received over the control channel.

5.1.4.2     Data Channel

The data channel is used specifically for transferring the data from Agent Controller to the client application. Agent Controller creates this channel, with a request on demand from the client.

5.1.5      Connection Sequence Diagram

The Fig.5.1.1 shows the simplified sequence diagram of the actions happening in the Connection Management component.

 

 

Fig.5.1.1 Connection Management Sequence Diagram

 

5.2      Message Management

In order to communicate with the Agent Controller or to use any services provided by the Agent Controller, the Agent Controller has defined set of commands that need to be issued. The base class of all the messages is Message. The message sent over the control channel is an instance of ControlMessage class that extends from Message class. The ControlMessage class holds an array of instances of CommandElement class and each instance represents a command. Any command going to the Agent Controller will be embedded into this control message. Currently, all the commands are defined as separate classes in org.eclipse.hyades.internal.execution.local.common package. Each command class defines the command string and any command parameters. The command classes extends from CommandElement class.

5.2.1      Existing Commands

There are several commands defined for communicating with the Agent Controller, agents and other components. These commands will be supported without any change for keeping the ability to communicate with the Agent Controller using the current protocol. These same commands will be ported into the C++ version of the client library also. The following snapshot gives the list of existing commands in the client library in the

 

5.2.2      New Commands

A new set of commands will be added both in C++ and Java implementations to map the new protocol being defined as part of this design.

 

TODO: list of the commands

5.3      Command Handlers

A Command Handler is an object that handles a command received over the control channel from the Agent Controller. The commands could be the events generated by the components on the target machine or the responses to the commands issued by the client. CommandHandler Class defines the interface that should be implemented by the command handlers.

 

The CommandHandler interface definition has one method shown below that must be implemented by the Command Handlers.

 

incommingCommand(Node node, CommandElement command)

 

5.3.1      Command Handlers for Events

For the proposed new design for event handling using Command Handlers a new interface will be defined that extends from the CommandHandler interface.

 

public interface EventCommandHandler implements CommandHandler{

           

            void incommingCommand(Node node, CommandElement command);

 

}

5.4      Agent Management

Every agent running on the target machine is locally represented by an instance of AgentImpl Class. The client application will have to create an instance of AgentImpl class to be able to communicate with the actual agent running on the target machine. This instance will work like a proxy for the actual agent object. The AgentImpl class has implementation for the following methods.

Once an instance of AgentImpl is created, a client application can set the agent configuration using the setConfiguration() and publishConfiguration() methods. The AgentConfiguration class can be used to for preparing the configuration. The attach() and detach() methods are used to attach to the agent and detach from the agent. To be able to listen to events coming from the agent, the client should create instance of AgentListener class and register with the agent using addAgentListener(). The removeAgentListener() is to de-register the agent listener. The startMonitoring() should be used to establish the data channel and start the data collection. During the startMonitoring call, an instance of TCPDataServer (server thread listening to the data channel port) class is created that is used for data transfer later when data is ready. The startMonitoring() method also takes a parameter – instance of the DataProcessor interface. A data processor is an object that handles and processes the data received over data channel. The TCPDataProcessor is started with the data processor object created and passed to the startMonitoring() call by the client. The stopMonitoring() is to stop the TCPDataServer. The invokeCustomCommand() method should be used to send any command other than standard ones.

5.4.1      New Agent Interface

TODO: define changes to the agent interface based on the Base Agent Hierarchy defined on agent controller side.

5.5      Data Manager

The Data Manager component provides services to listen to the data channel port, read the data and hand over the data to the respective data processor. A thread called TCPDataServer is created on the client side to listen to the data channel port for any incoming data. The TCPDataServer after receiving the data, it further passes along to the TCPDataProcessor or any other registered data processor for handling the data received. The Data Processors load the data using the designated Data Loaders and Data Models. The TCPDataServer has the following methods.

 

The startServer() method starts the server listens to the data channel port. This basically starts two threads in the data manager component.

 

 The Data Manager component can read the following types of data.

5.6      Data Processor interface

The Data Processor interface is defined as shown below. A client application should implement these methods to define data processors that can be registered to receive data from the BufferFlusher thread.

5.7      Data Reception Sequence Diagram

 

 

5.8      Event Listeners & Handlers

The client application would be interested in knowing about the status of the several components and tasks on the remote machine. The Event Listeners serve such purpose where in the client registers the listeners with a specific component of interest for specific events. The client library has Event Listener interfaces that can be implemented by the client application to define custom event listeners. Whenever an event (for which a listener registered) is sent by the components on the target machine, they get forwarded to these Event Listener objects and can be handled as required by the client application. The various Event Listener interfaces are listed below.

5.8.1      ConnectionListener interface

This interface can be implemented by an object to listen to any events generated in the context of the Connection between the client and Agent Controller. The Connection closed is the only event specified in this interface. This event is sent when the connection with the Agent Controller is closed.

5.8.2      ProcessListener interface

The ProcessListener interface can be implemented by an object to listen to any events generated in the context of a process on the target machine. This interface is currently defined for two events,

As part of the event a parameter is passed (an instance of org.eclipse.hyades.internal.execution.local.control.Process.Process class) that contains more information about the process.

5.8.3      ErrorListener interface

This interface can be implemented to listen to any errors from the components on the target machine.

NOTE: not used any where

5.8.4      AgentListener interface

The AgentListener interface can be implemented for receiving events from an agent. Currently the following events are supported.

5.9      Exception Handling

The current client library implementation defines several exceptions meant to be thrown in specific scenarios. All of these Exception Classes extend from Exception Class (Java). Please refer to section 6.1 for a detailed list of available Exceptions.

NOTE: May be it makes sense to define an exception hierarchy to manage and catch them in a better fashion. 

Currently the following Exceptions are defined.

 

DaemonConnectException -

DataChannelConnectionException -

DataChannelCreationException -

DataChannelNotInitializedException -

ExecutionComponentStateException -

MessageDeliveryException -

MessageProcessingException -

TimeoutException -

UnknownDaemonException -

InvalidDataChannelAccessException -

RemoteInvocationException -

ConsoleNotStartedException -

InvalidMessageException -

MessageOverFlowException -

5.10Execution API

5.11Security

The org.eclipse.hyades.execution.security package provides the secured client and server (used on the Agent Controller side) implementations to enable the secured communication between client and Agent Controller. The SecureConnectionImpl class extends ConnectionImpl class to provide a reimplementation of connection mechanism in a secured way. A secured SSL Socket is created that ensures proper secured authentication of the user. This component also provides implementation of SecureClient and SecureServer classes that enable the security on the client and server respectively.

5.12Configuration Management

To be specified.

5.13Asynchronous Event Handling

The TPTP Agent Controller can generate two types of events – synchronous events and asynchronous events.

But the design in this section deals with asynchronous event in particular. An event here is an indication that a specific task or action has been completed as a response to a previously issued command.

5.13.1 Synchronous Events

Synchronous events are generated and propagated as a response to a command issued from a client machine to Agent Controller or a specific agent for performing a specific action or accessing information. These events will also be treated as a command but these are sent from the Agent Controller to the client machine. An agent or another Agent Controller can play the role of client machine in this context and query information or issue commands. Agent Controller will treat all of them in a similar manner and will send the same information or event as a response, regardless of the source of the command.

5.13.2 Asynchronous Events

The asynchronous events can happen randomly in time and mainly generated by Agent Controller or agents and are propagated to the Event Listeners of that specific event or event collection. There can be multiple event listeners for the same event or event collection. A definition of event listener is a component that is interested in knowing about a specific event that has happened in another component. A listener can consume the event notifications by simply logging that event or could trigger some other complex process. In Agent Controller design, the listener can be any of Client, another Agent Controller or another Agent. And primary event generation sources are Agent Controller itself and any or all of the deployed agents.

5.13.3 Event Subscription

The event sources and event listeners are sometimes located across the machine boundaries. So, we should restrict the event propagation to avoid unnecessary event traffic over the network by sending every event generated to every listener in the network. This puts a requirement for an event registration or subscription process where in an event listener registers itself with the event source component for a list of events that it wishes to receive notifications. This allows an event source component (before sending an event) to obtain a filtered list of listeners that have shown interest in getting event notifications about a specific event and send the event only to those listeners.

5.13.4 Event Source

The event source will have the implementation to do the following.

So, Agent Controller and agents will provide the above implementation (though they will share the common implementation to help code re-use). In case of agents, the base agent framework will provide implementation and handle the above activities without the need for every agent (that inherits from the base agent framework) doing it separately.

5.13.5 Event Listeners

As mentioned in the previous paragraphs, event listeners should register with an event source showing interest for a specific list of events from that source. Once the listener object registers or subscribes for a set of events, it should start listening for events coming from the registered event sources. So, the event listener implementation should include,

5.13.6 Event Identification

In Agent Controller design we will end up with several event sources with possibly more than one event from each source, as there could be many running agents apart from the Agent Controller itself. So, we should be able to identify every event uniquely so that each event has its own identification. It means that we should tag each event with a unique identification number within a single Agent Controller called Event ID. This Event ID should solely represent that event irrespective of which event source it was generated from.

 

 

5.13.7 Event Information

Apart from Event ID, an event source should send more information about the event so that the event listener can fully understand the event and handle the event in a better manner. The event information should broadly include the following parameters.

The event information will be defined XML format for flexible consumption by various event listeners. Basically event source build the event as an XML message with all the required event information before sending the event.

5.13.8 Event Class

TODO: To be defined

5.13.9 EventCommandHandler Interface

TODO: To be defined

6         Interfaces

Note: The questions below are intended to guide the eventual content.  The questions themselves are not part of the design.

6.1      Client-Client Library Interface

Which parts of existing class hierarchy will be supported in C++?

Will any additional classes be added?

What additional classes are needed?

What are the methods for those classes?

6.1.1      Existing Hyades Client - Full Packages List

6.1.2      Java Packages that need Conversion to C++

Following is the initial list of packages being considered for convertion from Java to C++ based on product need. (The security packages need not be converted if there is no plan for security in the first phase.)

6.1.3      List of Java Classes (Needs porting to C++)

6.1.3.1     org.eclipse.hyades.collection.framework.channel – used on the Agent Controller side

 

Java Class or Interfaces Names

Class Information

 

DataChannelHelper

Helper Class for creating and connecting Data Channel

 

DataChannelImpl

Implementation of a Data Channel between two processes as shared memory buffer. Provides connect, destroy, create, disconnect, flush etc operations for the data channel

Uses native implementation in hcclsm.dll and the dll is loaded by DataChannel class

 

 

 

 

6.1.3.2     org.eclipse.hyades.execution.core

 

Java Class Names or Interfaces Names

Class Information

 

IControlMessage

Interface for passing control events between client and remote machine

 

IDataChannel

Data Channel interface

 

IDataProcessor

Data Processor interface. Processes the data coming over the data channel from the remote machine

 

IEclipseExecutionComponentFactory

 

 

IExecutableObject

 

 

IExecutionComponent

Base class for all the components in the runtime

 

IExecutionComponentFactory

 

 

IExecutionComponentStateChangeListerner

 

 

IExecutionEnvironment

A component with knowledge of environment that the component and it’s children reside

 

IExecutor

Launch and Kill an executable object

 

INode

Physical entity (e.g. machine) on the network

 

IOrderedProperty

Property with ordered values

 

IProcessConsole

Interface for the console handler

 

IRemoteHyadesComponent

Interface exposed by remote agents

 

ISession

Abstraction of a connection with a remote node

 

DaemonConnectException

 

 

DataChannelCreationException

 

 

DataChannelConnectionException

 

 

DataChannelNotInitializedException

 

 

ExecutionComponentStateException

 

 

InvalidDataChannelAccessException

 

 

MessageDeliveryException

 

 

MessageProcessingException

 

 

TimeOutException

 

 

UnknownDaemonException

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

6.1.3.3     org.eclipse.hyades.execution.core.impl

 

Java Class Names or Interfaces Names

Class Information

 

IJavaTaskExecutableObject

 

 

IProcessExecutableObject

 

 

ControlMessageImpl

 

 

ExecutionComponentImpl

 

 

ExecutionEnvironmentImpl

 

 

JavaExecutionEnvironment

 

 

JavaProcessExecutableObjectImpl

 

 

JavaTaskExecutableObject

 

 

JavaTaskExecutorImpl

 

 

JavaTaskRemoteHyadesComponentImpl

 

 

OrderedPropertyImpl

 

 

ProcessExecutableObjectImpl

 

 

ProcessExecutorImpl

 

 

 

 

 

 

 

 

 

6.1.3.4     org.eclipse.hyades.execution.local

 

Java Class Names or Interfaces Names

Class Information

 

ClassReleationship

 

 

ConfigurationElementRelationship

 

 

EclipseExecutionComponentFactoryImpl

 

 

ExecutableObjectStub

Remote invocation semantics

 

ExecutionComponentFactoryImpl

Execution Component Factory

 

ExecutionComponentStub

Stub for the Execution Component

 

ExecutionEnvironmentStub

 

 

ExecutorStub

 

 

JavaProcessExecutableObjectStub

 

 

JavaProcessExecutorStub

 

 

JavaTaskExecutableObjectStub

 

 

JavaTaskExecutorStub

 

 

JavaTaskRemoteHyadesComponentStub

 

 

NodeImpl

Implementation for Inode interface. Provides connection mechanism for the node

 

RemoteObjectStub

 

 

SessionImpl

Implementation of a session.

 

SessionStub

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

6.1.3.5     org.eclipse.hyades.internal.collection.framework – used on the Agent Controller side

 

Java Class Names or Interfaces Names

Class Information

 

ClientHandlerImpl

 

 

ConnectionHandlerImpl

 

 

SecureServer

 

 

 

 

 

 

 

 

 

6.1.3.6     org.eclipse.hyades.internal.config.generator – used on the Agent Controller side

 

Java Class Names or Interfaces Names

Class Information

 

Agent

 

 

AgentControllerConfiguration

 

 

AgentControllerConfigurationFactory

 

 

AgentControllerEnvironment

 

 

Allow

 

 

Application

 

 

Deny

 

 

FileError

 

 

Group

 

 

GroupDefinition

 

 

Hosts

 

 

Member

 

 

Option

 

 

Parameter

 

 

Security

 

 

Service

 

 

SetConfig

 

 

SetConfigCommon

 

 

UserDefinition

 

 

Variable

 

 

 

 

 

 

6.1.3.7     org.eclipse.hyades.internal.execution.local.common

The package contains classes that define the commands that are sent to the remote machine.

 

Java Class Names or Interfaces Names

Class Information

 

AcknowledgementHandler

 

 

CommandHandler

 

 

Constants

 

 

DataProcessor

 

 

DataServerListener

 

 

Options

 

 

AcknowledgementMessage

 

 

ActiveAgentListCommand

 

 

AgentActiveCommand

 

 

AgentDetailsCommand

 

 

AgentInactiveCommand

 

 

AttachToAgentCommand

 

 

AuthenticateCommand

 

 

AuthenticationFailedCommand

 

 

AuthenticationSuccessfulCommand

 

 

CommandElement

 

 

Console

 

 

ControlMessage

 

 

ControlServerDaemon

 

 

CustomCommand

 

 

DetachFromAgentCommand

 

 

DetailedAgentInfoCommand

 

 

DetailedProcessCommand

 

 

ErrorCommand

 

 

KillProcessCommand

 

 

LaunchProcessCommand

 

 

Message

 

 

MonitorPeerRequestCommand

 

 

ProcessExitedCommand

 

 

ProcessLaunchedCommand

 

 

QueryAgentDetailsCommand

 

 

QueryAgentListCommand

 

 

QueryProcessListCommand

 

 

RABinaryArray

 

 

RAInetAddress

 

 

RAString

 

 

RegisterAgentInterestCommand

 

 

RegisteredProcessListCommand

 

 

ServerSecurityInfoCommand

 

 

SetNVPairCommand

 

 

SimpleAgentInfoCommand

 

 

SimpleProcessCommand

 

 

StartMonitoringRemoteAgentCommand

 

 

StopMonitorCommand

 

 

TCPDataServer

 

 

ConsoleNotStartedException

 

 

InvalidMessageException

 

 

MessageOverflowException

 

 

 

 

 

 

 

 

 

6.1.3.8     org.eclipse.hyades.internal.execution.local.control

This package contains the core classes that represent the different entities like Node, Process, Application, Agent, etc.

 

Java Class Names or Interfaces Names

Class Information

 

Agent

 

 

AgentListener

 

 

AgentPeerListener

 

 

Application

 

 

CommandHandler

 

 

Connection

 

 

ConnectionListener

 

 

ErrorListener

 

 

Node

 

 

Process

 

 

ProcessListener

 

 

AgentConfiguration

 

 

AgentConfigurationEntry

 

 

AgentFactory

 

 

AgentImpl

 

 

ConnectionImpl

 

 

ContextMapper

 

 

NodeFactory

 

 

NodeImpl

 

 

ProcessFactory

 

 

ProcessImpl

 

 

SecureConnectionImpl

 

 

Variable

 

 

AgentActiveException

 

 

AgentControllerUnavailableException

 

 

InactiveAgentException

 

 

InactiveProcessException

 

 

NoSuchApplicationException

 

 

NotConnectedException

 

 

NotImplementedException

 

 

ProcessActriveException

 

 

 

 

 

 

6.1.3.9     org.eclipse.hyades.execution.security  – partly used on the Agent Controller side

 

Java Class Names or Interfaces Names

Class Information

 

IClientHandler

 

 

IConnectionHandler

 

 

IKeystoreManager

 

 

ISecureClientParameters

 

 

ISecureServerParameters

 

 

KeystoreHelper

 

 

KeystoreManagerFactory

 

 

SecureClient

 

 

SecureServer

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

6.1.3.10 org.eclipse.hyades.internal.execution.security

 

Java Class Names or Interfaces Names

Class Information

 

AuthenticationListener

 

 

User

 

 

UserFactory

 

 

DuplicateUserException

 

 

LoginFailedException

 

 

SecureConnectionRequiredException

 

 

UntrustedAgentControllerException

 

 

 

 

 

 

 

 

 

 

 

 

6.2      Socket Library Interface

What common services should be provided by the socket library?

Define the API for those services.

6.3      HTTP Library Interface

What common services should be provided by the HTTP library?

Define the API for those services.

6.4      Transport Layer Interface (Changes TBD)[awk19] 

6.4.1      Design Overview

Pluggable transport layers are loaded by the Agent Controller as external libraries.  The Agent Controller supports an arbitrary number of such transport layers, which are specified in the Agent Controller configuration file.  When the Agent Controller is started, it reads its configuration file and loads the library for each transport layer.

 

After loading the transport layer library, the Agent Controller calls a function to initialize the transport layer, passing in a transport layer ID and a table of functions which the transport layer uses to call back into the Agent Controller.

 

All connections with this transport layer are negotiated within the transport layer library.  The transport layer calls a function provided by the Agent Controller to create an entry in the Agent Controller connection table, but it also maintains its own local list of connections.

 

When a message arrives from a client, the transport layer processes the message envelope and, if the message is not a low level message intended for the transport layer itself (e.g. a connection message), it calls a function provided by the Agent Controller, passing in the message payload (but not the message envelope).

6.4.2      Functions Provided By Transport Library

The transport layer provides the following functions to be called by the Agent Controller:

 

startTransportListener

 

tpt_uint startTransportListener(  transport_layer_data_t*   transportData );

 

The Agent Controller will call this function to initialize the transport library.  The transport_layer_data structure is defined as follows:

 

typedef tpt_uint32 (*addConnectionEntry_ptr_t)(tpt_uint32, tpt_uint32*);
typedef tpt_uint32 (*removeConnectionEntry_ptr_t)(tpt_uint32);
typedef tpt_uint32 (*processMessage_ptr_t)(tpt_uint32, tpt_string);

typedef tpt_uint32 (*logError_ptr_t)(tpt_string);

 

typedef struct {
   tpt_uint32_t                transportID;
   tpt_string_t                configurationData;
   addConnectionEntry_ptr_t    addConnectionEntry;
   removeConnectionEntry_ptr_t removeConnectionEntry;
   processMessage_ptr_t        processMessage;
   logError_ptr_t              logError;
} transport_layer_data_t;

 

The Agent Controller assigns each transport library a locally unique transportID, which the Agent Controller will use to identify this transport layer in its internal connection table.  The transport library should begin actively listening for incoming connections.

 

The configurationData parameter will be an XML fragment which gives transport- specific configuration information.  For example, for a socket layer, this may give a port number.  The Agent Controller will read this block from its configuration file but will make no attempt to interpret it.

 

The remaining members of the structure are function pointers that will allow the transport library to call back to the Agent Controller.  The transport layer should make copies of these function pointers rather than storing the pointer to the incoming structure pointer.  The details of each of these functions are given below.

 

stopTransportListener

 

void stopTransportListener();

 

The Agent Controller will call this function during shutdown to tell the transport layer to terminate any existing connections and stop accepting new connections.  After this function is called, the transport layer should be entirely inactive unless a subsequent call to startTransportListener is made.

 

terminateConnection

 

void terminateConnection(tpt_uint32  connectionID);

 

The Agent Controller will call this function to tell the transport layer to terminate a specific connection.  This will occur only when the Agent Controller itself needs to initiate the process of terminating the connection.  The transport layer should send the low level DISCONNECT message and then end the connection.  The transport layer should call the Agent Controller removeConnectionEntry function when it has completed processing this command.

 

sendMessage

 

tpt_uint32 sendMessage( tpt_uint32  connectionID,

                        tpt_uint32  size,

                        tpt_string  commandBlock )

 

The Agent Controller will call this function to send a command block to a specific target which previously connected using this transport layer.  The transport layer should wrap the command block in a message envelope and send the message to the specified target.

6.4.3      Functions Provided By Agent Controller

The Agent Controller provides the following functions to be called by the transport layer:

 

addConnectionEntry

 

tpt_uint32 addConnectionEntry(   tpt_uint32  transportID,

                                 tpt_uint32* connectionIDPtr );

 

When a client connects to the transport layer, the transport layer should call this function to allow the Agent Controller to add the new connection to its internal connection table.  The Agent Controller will assign a locally unique connection ID, which the transport layer should both send to the client as part of the CONNECTTION_COMPLETE message and store for future reference (see the sendMessage function above).

 

removeConnection

 

tpt_uint32 removeConnection( tpt_uint32 connectionID );

 

When a connection is ended, whether intentionally (for instance, by a call to the terminateConnection function) or as a result of an error, the transport layer should call this function to allow the Agent Controller to remove the connection from its internal connection table.  If the connection has ended as a result of an error condition, the transport layer should also call the logError function to record the error.

 

 

processMessage

 

tpt_uint32 processMessage( tpt_uint32  size,

                           tpt_string  payload );

 

The transport layer calls this function when a message arrives from a client that is not a low level message intended for the transport layer itself.  The transport layer processes and removes the message envelope and passes the payload to this function for processing by the Agent Controller.

 

logError

 

tpt_uint32 logError( tpt_string errorMsg );

 

The transport layer should call this function to record any error condition that may be of interest to the user.  (TBD: This function needs more substance.)

6.5      Payload Normalizer Interface

Agent Controller on receiving a message from the transport layer will check for the need for normalizing the message. A message pipeline can have more than one Payload Normalizer. If the message is determined to be a candidate for normalizing then the Payload Normalizer will normalize the message (either filter or reformat the message) as expected by the command extractor.  If no Payload Normalizer is specified for the message pipeline, then the message will be passed directly to the command extractor. The sequence of events for the Payload Normalizer interface, the Command Extractor interface and the Agent Controller is shown in the figure below.

Pending: Payload Normalizer use case

                                            fig:6.5 – Agent Controller, Payload Normalizer, Command Extractor interaction

 

6.5.1      Agent Controller configuration for PayLoad Normalizer and Command Extractor

Please see Agent Controller configuration for the specification of the message pipeline.

6.5.2      Functions Provided by Agent Controller

The Agent Controller provides the following functions to be called by the Payload Normalizer:

 

getMessageConfig

 

char* getMessageConfig( tpt_uint32  transportID)

 

When the Agent Controller receives a message from the Transport Layer the AC retrieves the configuration information for the instance of transport layer in context by reading the message configuration information as defined in the serviceconfig.xml file.

 

getRetMessageConfig

 

char* addConnectionEntry(  tpt_uint32  transportID)

 

The RetMessageConfig retrieves the payload de-normalizer information defined in the serviceconfig.xml to format the message to be returned to the TL.

6.5.3      Functions Provided by PayLoad Normalizer

The Payload Normalizer provides the following interface to communicate with the AC

 

normalizeMessage

 

char* normalizeMessage( tpt_uint32  PayLoadNormalizerID, char* Message)

 

For each of the payload normalizers defined for the transport layer in the message configuration the messages will be normalized in order.

 

deNormalizeMessage

 

char* deNormalizeMessage( tpt_uint32 PayLoadDENormalizerID, char* Message)

 

For each of the payload de-normalizers defined for the transport layer in the message configuration the messages will be de-normalized in order.

6.6      Command Extractor Interface

The purpose of the command extractor is to parse a message consisting of a batch of commands into discrete command blocks. The standard message used by the pipelines may contain a sequence of one or more command blocks. The command extractor will forward the commands individually to the Agent Controller.

6.6.1      Functions Provided By Command Extractor

The Command Extractor provides the following functions to communication with the Agent Controller

 

parseMessage

 

char[] parseMessage(             char* Message)

 

For every message received that may be a batch of messages or unary messages the message is parsed and returned as single messages.

6.6.2      Functions Provided by the Agent Controller to interface with the Command Extractor

The Agent Controller provides the following functions to be called by the command extractor

 

getCommandExtractor

 

char* getCommandExtractor( tpt_uint32  transportID)

 

Retrieves the command extractor to be used for the instance of TL in context for parsing and un-batching commands.

 

getMessageConfig

 

char* getMessageConfig( tpt_uint32  transportID)

 

When the Agent Controller receives a message from the Transport Layer the AC retrieves the configuration information for the instance of transport layer in context by reading the message configuration information as defined in the serviceconfig.xml file.

6.7      Socket Transport Interface

6.7.1      Socket Message Envelope

The socket and SSL transport layers both expect the following message format:

 

Bytes 0-3

Magic number

Bytes 4-7

Flags

Bytes 8-23

MD5 code (optional)

Bytes 24-27

Payload size

All remaining bytes

Payload

 

This message envelope will be used for both incoming and outgoing messages.

 

The “magic number” is used to identify the data as conforming to the expected protocol stream.  If an error occurs while the data is being read, the transport layer will be able to look for this number as an identifier of the beginning of a new message envelope and thus achieve some recovery from an error condition.

 

The “flags” entry is used to identify messages intended for the transport layer, such as the connection handshake described below.  This field also indicates whether or not the message contains an MD5 code.

 

If the MD5 flag is set in the flags field, the 16-bytes immediately following the flags field will specify an MD5 hash code which will be used to verify the integrity of the incoming message.  (See http://www.faqs.org/rfcs/rfc1321.html for a description of MD5.)  If the MD5 flag is not set in the flags field, these 16 bytes will be ignored.

 

The payload size value indicates the number of bytes that comprise the message payload.  For special messages, such as the connection handshake, this is processed by the transport layer, but for the majority of messages, the entire payload is forwarded to the next layer for processing.

 

6.7.2      Connection Handshake

When a client or agent connects to a transport layer, a basic handshake takes place to establish and verify the connection.  Until this handshake has taken place, no other commands will be accepted.

 

The handshake is performed by the connecting component sending a special message with a flag set to indicate the command intended, and payload data set as necessary, and the engine sending a response accepting or rejecting the connection.

 

Flag Value:

CONNECT (bit 0 of byte 5 [first flag byte])

Payload Parameters:

No data

Description:

This command is sent to initiate a handshake from any object that is connecting to the engine via a socket connection.  Initially, only clients are expected to connect in this way, but the protocol is capable of handling connections from other object types.

 

Flag Value:

DISCONNECT (Bit 1 of Byte 5 [first flag byte])

Payload Parameters:

Bytes 0-3

Reason code

Description:

This command is sent when an object is ready to disconnect from the Agent Controller.  It may also be sent by the Agent Controller if it has some reason to terminate the connection (e.g. it is being shutdown).  The Agent Controller will notify any objects that are associated with the object being disconnected and free any associated resources.  If the Agent Controller initiated the disconnection, the associated objects will be notified and released immediately after the message is delivered.  Otherwise, this will happen immediately after the message is received.

 

The reason code indicates why the connection is being terminated.  This field is meant for Agent Controller-initiated disconnections.  The socket transport layer will ignore these fields when it is the recipient of the message.  The reason code is meant only as a means of providing program level details.  If anything needs to be communicated to the user, it will be done by means of the error logging facility (TBD).

 

In addition, the following are sent in response to the above commands.

 

Command:

CONNECTTION_COMPLETE (Bit 2 of Byte 5 [first flag byte])

Parameters:

Bytes 0-3

Connection ID

Description:

This command will be sent in reply to CONNECT if the connection is accepted.  The connection ID is a value that the Agent Controller to which a connection has been made will use to identify the object that is receiving the reply.  The object receiving this reply should use this value in the sourceID field in the command header for any commands it sends after receiving this message.

 

Command:

CONNECTTION_REFUSED (Bit 3 of Byte 5 [first flag byte])

Parameters:

Bytes 0-3

Reason code

Description:

This command will be sent in reply to CONNECT if the transport layer refuses the connection for any reason.  The reason code is a value that the Agent Controller provides to give some indication of why the connection was refused.  It will be a value defined in TPTP header files.  If anything needs to be communicated to the user, it should be done by means of the error logging facility (TBD).

6.8      Named Pipe Transport Interface

The Named Pipe Transport Layer uses nearly the same message envelope and low level commands as the Socket Transport Layer.  The message envelope does not contain an MD5 code and therefore the corresponding flag is not supported.  The named pipe transport layer also expects additional payload data for the connect command.

 

6.8.1      Named Pipe Message Envelope

The named pipe transport layer expects the following message format:

 

Bytes 0-3

Magic number

Bytes 4-7

Flags

Bytes 8-11

Payload size

All remaining bytes

Payload

 

This message envelope will be used for both incoming and outgoing messages.

 

The “magic number” is used to identify the data as conforming to the expected protocol stream.  If an error occurs while the data is being read, the transport layer will be able to look for this number as an identifier of the beginning of a new message envelope and thus achieve some recovery from an error condition.

 

The “flags” entry is used to identify messages intended for the transport layer, such as the connection handshake described below.

 

The payload size value indicates the number of bytes that comprise the message payload.  For special messages, such as the connection handshake, this is processed by the transport layer, but for the majority of messages, the entire payload is forwarded to the next layer for processing.

6.8.2      Connection Handshake

When a client or agent connects to a transport layer, a basic handshake takes place to establish and verify the connection.  Until this handshake has taken place, no other commands will be accepted.

 

The handshake is performed by the connecting component sending a special message with a flag set to indicate the command intended, and payload data set as necessary, and the engine sending a response accepting or rejecting the connection.

 

Flag Value:

CONNECT (bit 0 of byte 5 [first flag byte])

Payload Data:

Bytes 0-3

Pipe name string length

Bytes 4-(4+str length)

Return pipe name

Description:

This command is sent to initiate a handshake from any object that is connecting to the engine via a named pipe connection.  Only agents are expected to connect in this way, but the specification does not prevent other objects from doing so should it prove useful.

 

Flag Value:

DISCONNECT (Bit 1 of Byte 5 [first flag byte])

Payload Parameters:

Bytes 0-3

Reason code

Description:

This command is sent when an object is ready to disconnect from the Agent Controller.  It may also be sent by the Agent Controller if it has some reason to terminate the connection (e.g. it is being shutdown).  The Agent Controller will notify any objects that are associated with the object being disconnected and free any associated resources.  If the Agent Controller initiated the disconnection, the associated objects will be notified and released immediately after the message is delivered.  Otherwise, this will happen immediately after the message is received.

 

The reason code indicates why the connection is being terminated.  This field is meant for Agent Controller-initiated disconnections.  The socket transport layer will ignore these fields when it is the recipient of the message.  The reason code is meant only as a means of providing program level details.  If anything needs to be communicated to the user, it will be done by means of the error logging facility (TBD).

 

In addition, the following are sent in response to the above commands.

 

Command:

CONNECTTION_COMPLETE (Bit 2 of Byte 5 [first flag byte])

Parameters:

Bytes 0-3

Connection ID

Description:

This command will be sent in reply to CONNECT if the connection is accepted.  The connection ID is a value that the Agent Controller to which a connection has been made will use to identify the object that is receiving the reply.  The object receiving this reply should use this value in the sourceID field in the command header for any commands it sends after receiving this message.

 

Command:

CONNECTTION_REFUSED (Bit 3 of Byte 5 [first flag byte])

Parameters:

Bytes 0-3

Reason code

Description:

This command will be sent in reply to CONNECT if the transport layer refuses the connection for any reason.  The reason code is a value that the Agent Controller provides to give some indication of why the connection was refused.  It will be a value defined in TPTP header files.  If anything needs to be communicated to the user, it should be done by means of the error logging facility (TBD).

6.9      HTTP Transport Interface

The main differences between HTTP and socket support are

-         HTTP can support proxy server

-         HTTP request is stateless

-         HTTP cannot keep the socket persistent across different calls.

 

As a result, HTTP requests will be required to connect for the 1st time and be assigned a “session id.” The same session id must be supplied in subsequent calls to identify the same single source of requests. In another words, this session id is equivalent to the socket connection between the requester and the Agent Controller.

 

The content of the request and response will be the same as in the socket interface except that this is in XML format. Basically, this is a simplified version of SOAP where we don’t have to carry known information such as encoding, parameter types, source and destination of the services. Please see following sections for examples of a typical HTTP request and response.

6.9.1      Sample HTTP Request Message Envelope

POST  HTTP/1.0

Host: 127.0.0.1

Content-Type: text/xml; charset=utf-8

Content-Length: 501

<Envelope>

<Header>

                        <Magic> 1234 </Magic>

                        <Flags>  255  <Flags>

                        <MD5>  123456789ABCDEF </MD5>

</Header>

<Payload>

            <Cmd>

                        <!—command specific data -->

            </Cmd>

</Payload>

</Envelope>

 

 

Notes:

-         See the description for each field in the “Socket Message Envelope” section.

6.9.2      Sample HTTP Response Message Envelope

HTTP/1.0 200 OK

 

Content-Type: text/xml; charset=utf-8

Content-Length: 501

<Envelope>

<Header>

                        <Magic> 1234 </Magic>

                        <Flags>  255  <Flags>

                        <MD5>  123456789ABCDEF </MD5>

</Header>

<Payload>

            <Resp>

                        <!— response specific data -->

            </Resp>

</Payload>

</Envelope>

6.9.3      HTTP Connection Handshake

The connection must be established in order for the Agent Controller to keep track of, verify and manage its users.

 

For HTTP, this call is required so that the Agent Controller can assign a unique id to establish a “session” with this user.

 

CONNECT request

 

<Payload>

            <Cmd>

                        <Connect/>

            </Cmd>

</Payload>

 

CONNECT response

 

<Payload>

            <Resp>

                        <ReturnCode> 0 </ReturnCode>

                        <SessionID> 12345678ABCDEF </SessionID>

            </Resp>

</Payload>

6.10Agent Base Classes and Templates[kac20] 

The preliminary UML diagram below shows the TPTP agent class hierarchy. There are several interfaces and their implementation classes are defined. The IBaseAgent and IbaseCollector interfaces are the basic definitions for agents and collectors. The IBaseCollector extends the IBaseAgent interface as every collector also is an agent. The interfaces Iattachable, IeventProvider, Ivariable, IdataProvider and their implementations capture some common concepts that are useful. An agent can extend these classes to consume this behavior depending on the need and behavior of the agent itself. The TPTPBaseAgentImpl class is the top most class in the hierarchy and defines very basic operations required to be in the  base agent. Every TPTP agent will inherit the operations defined by this class. Extending from this base agent are the other next level classes that define the operations specifically for e.g. agents that perform data collection, agents that monitor or manage, agents that control, etc. . Currently only the BaseCollectorImpl class is defined to be the base class for all agents that are collector. The BaseCollectorImpl class extends from BaseAgentImpl class and also implements the interfaces defined in the IbaseCollector. The hierarchy can be extended to cover the broader set of agents based on their type, characteristics and behavior. The agent developers can create the actual agents by extending from one of the base implementation classes in the hierarchy and either use the provided default implementation or re-implementing the methods.

 

NOTE: The class hierarchy shown in the diagram currently doesn’t have details about the methods and their scope. This diagram will be updated in the later versions of the document.

 

 

There[kac21]  are several benefits for having such interface hierarchy.

·        Agents can have a uniform interface

·        Default implementation could be provided for the base agent interface

·        Agents can be categorized to give it an identity based on the agent type.

·        Code re-use is possible across the agents in the hierarchy

·        Agent developers can extend this interface

Some of the things the base agent framework will handle are,

The following sections define the agent template classes in C++. Similar interfaces and classes will be defined in Java for use by agents implemented using Java.

6.10.1 General Class Definitions

 

6.10.1.1 Class Variable[kac22] 

Data Member/Method Names

Description

int id

Variable identification

string* name

Variable Name

string*  description

Variable Description

int type

Variable Type – int, long. String, etc.

int read_access

Specifies the read access conditions

int write_access

Specifies the write access conditions

int event_based

Event based, single-state or value based

string* xsdTypeNameSpace

XSD information

string* xsdTypeNCName

XSD information

void* value

Variable Value

 

6.10.1.2 Class VariableGroup

Data Member/Method Names

Description

int id

Variable identification

string* name

Variable Name

string*  description

Variable Description

VariableGroup* subGroups

Sub groups within this Variable Group

Variable[] variables

Variables in this Variable Group

 

6.10.1.3 CmdBlock – Data Structure

Defined in the Agent Controller protocol interface

6.10.1.4 Event Data Structure - Event

 

Data Member

Description

Char* id

Event identification

char* name

Event Name

char*  description

Event Description

Void* eventData

Event Data

 

6.10.2 Base Agent Interface – IBaseAgent and BaseAgentImpl Classes

The Base Agent interface and class define the interfaces and provides default implementations respectively.

6.10.2.1 IBaseAgent interface

Class IBaseAgent

{

public:

int processCommand(CommandBlock* cmd) = 0;

int terminate()=0;

int checkCompatibility(char* agentName)=0;

 

}

6.10.2.2 BaseAgentImpl Class

Class BaseAgentImpl:IBaseAgent

{

public:

int processCommand(CommandBlock* cmd);

int terminate();

int checkCompatibility(char* agentName);

 

private:

int register();

int initialize();

}

6.10.2.2.1  Process [kac23] Command[kac24] 

This method parses and processes all the commands meant for this agent or agent process. Respective specific methods of the agent class will be called based upon the Command ID value.

Every Agent Class in the Agent interface hierarchy can have its own processCommand() method but will execute processCommand() method of the parent class first. The BaseAgentImpl class provides the implementation for processing the basic commands that are part of the BaseAgentImpl class.

Signature

int processCommand(CommandBlock* cmd)

 

6.10.2.2.2  Register with Agent Controller[kac25] 

Provides implementation for registering this agent with the Agent Controller establishing the required communication path between the Agent Controller and the agent used for any future communication[kac26] . The implementation for this method is provided by the BaseAgentImpl class itself. This is a private method of the class and is meant to be called during agent initialization.

Signature

int register()

 

6.10.2.2.3  Initialize

This interface initializes the agent to a state, which can allow the agent to start the collection. The agent can implement this method to perform any required initialization like resource initialization though not all agents will need such activity.

Signature

int initialize()

 

6.10.2.2.4  Check Compatibility (with another agent)

Signature

int checkCompatibility(char* agentName)

Description:

This command checks the compatibility of the agent with another agent in terms of running both of these agents simultaneously. The command takes the second agent name as the parameter.

 

NOTE: Need to evaluate if we have to extend this for more than two agents. This can be part of the agent configuration instead, which will be read by Agent Controller and apply the conditions while serving the request to run the agents simultaneously.

6.10.2.2.5  Terminate or Shutdown

Signature

int terminate()

Description:

This interface allows the client/Agent Controller to shut down or terminate the agent. After this action, the agent instance will be deleted and will not be available for service. The response will be either success or failure. If it is error, will also return an error code as response data.

6.10.3 Variable interface – IVariable and VariableImpl classes

This interface and implementation class define the behavior useful for grouping , retrieving and setting the values of the agent properties.

6.10.3.1 IVariable interface

Class IVariable

{

public:

VariableGroup[] getAgentVariableGroup()=0;

VariableGroup[] getAgentVariableGroup(char* groupID)= 0;

int setAgentVariables(Variable[] variables)=0;

Variable* getAgentVariable(int variableID)=0;

int setAgentVariable(Variable* variable)=0;

}

6.10.3.2 VariableImpl Class

 

Class VariableImpl: public IVariable

{

public:

VariableGroup[] getAgentVariableGroup();

VariableGroup[] getAgentVariableGroup(char* groupID);

int setAgentVariables(Variable[] variables);

Variable* getAgentVariable(int variableID);

int setAgentVariable(Variable* variable);

}

6.10.3.2.1  Get Agent Variables

This interface returns the list of agent properties and their values from all the variable groups.

Signature

VariableGroup[] getAgentVariableGroup()

 

6.10.3.2.2  Get Agent Variables

This interface returns the list of agent properties and their values within a specific variable group.

Signature

VariableGroup[] getAgentVariableGroup(char* groupID)

 

6.10.3.2.3  Set Agent Variables

This interface allows for setting the values of the agent properties.

Signature

int setAgentVariables(Variable[] variables)

 

6.10.3.2.4  Get Agent Variables

This command returns the value of a single property (requested) of the agent. It takes the property name as a parameter.

Signature

Variable* getAgentVariable(int variableID)

 

6.10.3.2.5  Set Agent Variable

This sets the value of a single property of the agent. It takes the Property Name and Value as parameters.

Signature

int setAgentVariable(Variable* variable)

 

6.10.4 Event Interfaces – IEventProvider and EventProviderImpl Classes

The Event interface defines the methods for handling the event listener registration and de-registration mechanism and also sends the events for the registered listeners.

6.10.4.1 IEventProvider interface

Class IEventProvider

{

public:

int addEventListener( char* listenerID, char* interfaceID)=0;

int removeEventListener(char* listenerID, char* interfaceID)= 0;

int sendEvent(Event* event)=0;

}

6.10.4.2 EventProviderImpl Class

Class EventProviderImpl: public IEventProvider

{

public:

int addEventListener( char* listenerID, char* interfaceID);

int removeEventListener(char* listenerID, char* interfaceID);

int sendEvent(Event* event);

}

 

6.10.4.2.1  Add Agent Event Listener

This interface registers an interested listener to a specific event. The agent instance maintains a table of events (that can be generated by the agent) and the interested event listeners. When an event is triggered, this information is sent only to the interested listeners by routing via Agent Controller. The parameter list contains the event listener id and interface ID for which the listener would like to register for notifications.

Signature

 

int addEventListener( char* listenerID, char* interfaceID)

 

6.10.4.2.2  Remove Agent Listener

This interface removes a registered event listener indicating that the listener is no longer wishes to receive a notification when event is triggered. The parameter list contains the event listener id and interface ID for which the listener would like to de-register for notifications.

Signature

 

int removeEventListener(char* listenerID, char* interfaceID)

 

6.10.4.2.3  Send Event

Signature

int sendEvent(Event* event)

Description:

Provides the implementation for sending any events from the agent to the subscribed listeners through agent manager.

6.10.5 Data Provider Interface – IDataProvider and DataProviderImpl classes

NOTE: To be defined in detail

6.10.5.1 IDataProvider interface

 

6.10.5.2 DataProviderImpl classes

6.10.5.3 Set the Data File Name and Path (Is this required?)[kac27] 

Sets the file path and name into which the data collected by the agent will be recorded. This is applicable only when the agent collection writes data into a file.

6.10.5.4 Get the Data File

Returns the data file name and path.

6.10.6 Attachable interface – IAttachable and AttachableImpl Classes

This interface defines the

6.10.6.1 IAttachable interface

Class IAttachable

{

public:

int attach(char* processID)=0;

int detach(char* processID)= 0;

int isAttached(char* processID)=0;

}

6.10.6.2 AttachableImpl Class

Class AttachableImpl : public IAttachable

{

public:

int attach(char* processID);

int detach(char* processID);

int isAttached(char* processID);

}

6.10.6.2.1  Attach to Process

Signature

int attach(char* processID)

Description:

This command informs an agent to a process with Process ID passed as a parameter. The command returns success or failure as response.

6.10.6.2.2  Detach from Process

Signature

int detach(char* processID)

Description:

This command informs an agent to detach itself from the process with Process ID (passed as a parameter). The command returns success or failure as response.

NOTE: Do we need to specify the Process ID as parameter? Could be useful if we have multi-process scenario (in which case the parameter will be a Process ID list).

6.10.6.2.3  Is Attached

Signature

int isAttached(char* processID)

Description:

This command responds by indicating if the agent is attached to a process or not. If the agent is still attached to a process, the response data will contain the Process ID.

6.10.7 Base Collector Template – BaseCollectorImpl Class

This section describes the base interface definition of the agents that specifically carry out the performance or other types data collection of an application under observation. This interface has to be implemented by every collector though it can extend the interface by adding other methods.

6.10.7.1 IBaseCollector interface

Class IBaseCollector : public IBaseAgent

{

public:

int run()=0;

int pause()= 0;

int resume()=0;

int cancel()=0;

int stop()=0;

int snapshot()=0;

int getData()=0;

int getAgentState()=0;

int startMonitoring(char* ipAddress, int portNo)=0;

}

 

6.10.7.2 BaseCollectorImpl Class

Class BaseCollectorImpl : public BaseAgentImpl, IBaseCollector

{

public:

int run();

int pause();

int resume();

int cancel();

int stop();

int snapshot();

int getData();

int getAgentState();

int startMonitoring(char* ipAddress, int portNo);

}

6.10.7.2.1  Start Monitoring

Signature

int startMonitoring(char* ipAddress, int portNo)

Description:

This interface will create keep the data channel ready to be able to transfer the data collected during Run operation. The command data includes the data stream information that should be used to transfer any data collected by the agent. The response will be either success or failure. If it is error, will also return an error code as response data.

 

6.10.7.2.2  Run

Signature

int run()

Description:

This interface runs the agent and starts the data collection and will continue to collect the data until Stop or Pause command is issued. The response will be either success or failure. If it is error, will also return an error code as response data.

NOTE: The Run command doesn’t take any parameters as it is just starting the data collection. We can add few parameters if we want to run the collection in different modes like – run for a certain span of time, Start the collection at a specific time, collect the data in alternating run and stop mode.

6.10.7.2.3  Pause

Signature

int pause()

Description:

This interface can be used to move the status of the agent that is running into a pause mode. During the pause mode the agent is active but is in idle state without performing any collection. The response will be either success or failure. If it is error, will also return an error code as response data.

6.10.7.2.4  Resume

Signature

int resume()

Description:

This interface can be used to change the agent state back to running mode from a previously paused state. When this operation is performed, the agent will resume the collection it has started before. One usage case for the Pause/Resume sequence could be to perform a periodic data collection. The response will be either success or failure. If it is error, will also return an error code as response data.

6.10.7.2.5  Cancel

Signature

int cancel()

Description:

When this interface is called, the agent will cancel any currently running data collection. This moves the agent to Configured state so that another collection can be configured and run. After this action, the agent will be still active and can provide services again. The response will be either success or failure. If it is error, will also return an error code as response data.

6.10.7.2.6  Stop

Signature

int stop()

Description:

This will stop the currently running data collection and the collected data can be transferred to the client. The response will be either success or failure. If it is error, will also return an error code as response data.

6.10.7.2.7  Snapshot

Signature

int snapshot()

Description:

This interface allows a client to request the snapshot of the data collected until the point of request. The agent will initiate a data transfer after it receives this command. The data transfer could be the delta of the data collected since the last data transfer. The response will be either success or failure. If it is error, will also return an error code as response data.

Note: This command may not be meant for the agents that transfer real-time data (send data regularly as they collect) since they have been sending the data anyway.

6.10.7.2.8  Get Data (Transfer Data)

Signature

int getData()

Description:

This command initiates the data transfer using the data channel created (when Start Monitoring command is issued).

6.10.7.2.9  Get Agent State

Signature

int getAgentState()

Description:

Returns the state of the agent.

6.10.8 Example of an Agent class definition

NOTE: To be defined.

7         Command Protocol

7.1      Basic Protocol

All commands emerging from the command extractor must be represented as XML fragments, using the following format:

 

<xsd:complexType name="CmdType">

   <xsd:sequence>

         <xsd:any minOccurs="1" maxOccurs="1">

   </xsd:sequence>

   <xsd:attribute name="dest" type="xsd:int">     

   <xsd:attribute name="src"  type="xsd:int">     

   <xsd:attribute name="ctxt" type="xsd:int">     

</xsdComplexType>

 

For example:

 

<Cmd

   dest="101"

   src="102"

   ctxt="10001">

      <agentMgr:registerAgent

                   xmlns:agentMgr="org.eclipse.hyades.agentMgr">

         <agentID>102</agentID>

         <processID>4067</processID>

         <agentName>org.eclipse.hyades.processControllerAgent</agentName>

      </agentMgr:registerAgent>

</Cmd>

 

7.1.1      Destination and Source IDs

The destination ID is not intended to be globally unique, but is only unique within a single instance of the Agent Controller.  The same is true of the source ID.  These IDs are the numbers which are used to identify the objects on the Agent Controller receiving the message.

 

The destination ID specifies the object that is the intended recipient of the command.  This may be a static ID that identifies the Agent Controller itself (TPTP_ENGINE_ID) or it may be the ID that the engine assigned to a client or agent when that object connected to the engine (see CONNECT_COMPLETE in the Socket Transport Layer Interface above).

 

The source ID specifies the object that is sending the command.  When the destination object replies to a command, it will move the source ID to the destination ID field.

 

The current design is intended only to support routing of commands to objects which are directly connected to the Agent Controller processing the command.

7.1.2      Context

The context is an arbitrary number assigned by the sender of a command.  Clients may use this context value to correlate reply messages to the originating message.  The usage of this element is unchanged from the previous protocol.  The Agent Controller ignores this value.

7.1.3      Command Element and Interface

The XML “Command” block is meant to enclose exactly one command element.  The element name indicates the command being sent, and the XML namespace is used to specify the interface to which the command belongs.

 

The interface and command name together uniquely identify the command that is being sent.  The interface tells the object receiving the command what the command name means.  If the interface is not supported by the object, the command will be rejected.

 

Interfaces are identified in a globally unique way using interface names, which follow the Java package naming conventions (for example, org.eclipse.hyades.agent.manager).

7.2      Agent Manager Interface

This section defines the commands that can be sent to the Agent Manager

the event interfaces that the Agent Manager will support.

7.2.1      Commands

The following commands are supported through the Agent Manager interface.  Clients and agents use these commands to locate agents, get references to those agents and attach to those agents.  More specific interaction with the agents will be managed through interfaces provided by the agents themselves.

 

Command:                              

queryAvailableAgents

Parameters:

interfaces

String list, optional

The interfaces used to filter the request

Response:

availableAgents

Response Parameters:

agentNames

String list

Names of agents meeting the criteria

Description:

Clients (or agents) may use this command to obtain a list of agents known by the Agent Manager regardless of whether or not an instance of the agent is running.  The client may specify zero or more interface IDs.  If interface IDs are specified, the list returned will be only agents that support all of the specified IDs.  If no interface IDs are specified, the list returned will be all available agents.  The Agent Manager does not create instances of these agents at this time.  It also does not distinguish single-instance, single-client agents that are already in use (and thus are only potentially available).  The idea is that clients can use this method to allow the user to select an agent for future use, although the agents listed can generally be used immediately.

 

Command:

queryRunningAgents

Parameters:

interfaces

String list, optional

The interfaces used to filter the request

Response:

runningAgents

Response Parameters:

agentIDs

Integer list

IDs of running agents

Description:

Clients (or agents) may use this command to request a list of agents that are currently active and running.  The list may be filtered by interface IDs as in the previous command.  However, unlike the previous command, this command returns a list of identifiers of specific instances of agents.  Also, if an instance of an agent cannot be attached to, it will not be returned in this list.  This command is expected to be used to access agents for immediate use.

 

Command:

getAgentMetadata

Parameters:

agentName

String

Name of the agent whose metadata is being requested

Response:

agentMetadata

Response Parameters:

metadata

String

XML fragment giving the metadata

Description:

This command allows clients to obtain agent-specific metadata.  The data returned isis is the “Client Data” subsection in the agent registry (see “General Vision” above) as an XML fragment.  If the agent has no such metadata, the response will return zero as the string length, but a response will still be issued.

 

Command:

getAgentRegistry

Parameters:

None

Response:

agentRegistry

Response Parameters:

interfaces

String

XML fragment giving all data

Description:

This command allows clients to obtain the full list of agents and their metadata.  The data is returned as an XML fragment.

 

Command:                              

getAgent

Parameters:

agentName

String

Name of the requested agent

flags

Integer, optional

Flags, as described below

Response:

agentReference

Response Parameters:

agentID

Integer

ID of the requested agent

Alternate Response:

agentUnavailable

Alternate Response Parameters:

reason

Integer

Code indicating the reason for failure

Description:

Clients (or agents) should use this command to obtain a reference to an agent.  Unless the client requests otherwise (via the flags field), the Agent Manager may fulfill this request by returning a reference to an already running or by spawning a new instance of the agent.  Alternatively, the client may use the flags field to specifically request a new instance.  It is possible that the requested agent will not be available when requested.  In that case, a different response command will indicate the reason.

 

Possible flag values are: (TBD)

 

Command:                              

getSpecificAgent

Parameters:

agentID

Integer

The ID of the agent to be referenced

Response:

agentReference

Response Parameters:

agentID

Integer

The ID of the agent whose reference is being returned

Alternate Response:

agentUnavailable

Alternate Response Parameters:

reason

Integer

Code indicating the reason for failure

Description:

This command allows a client or agent to get a reference to a specific instance of an agent.  This command assumes that the component issuing the command has found out about an instance of an agent by some other means.  For example, a client may create two agents and then issue a proprietary command to them asking them to work together.  In this case, one agent may request a reference to the other agent and the client can release its reference.

 

Command:                              

releaseAgent

Parameters:

agentName

String

Name of the agent being released

Description:

A client (or agent) uses this command to release its reference to an agent that it is no longer interested in working with.  The client should not attempt to use the agent after it has made this call as the agent may be shut down.  No response will be sent to this command.  The client should assume that the release has taken place immediately.

 

Command:

attachToAgent

Parameters:

agentID

Integer

ID of the agent to which to attach

flags

Integer, optional

Flags, as described below

Response:

agentAttachedSuccessful

Response Parameters:

No data

Alternate Response:

agentAttachDenied

Alternate Response Parameters:

reason

Integer

Code indicating the reason for failure

Description:

This command allows a client (or agent) to request active access to an agent to which it has previously obtained a reference.  Depending on the agent implementation or the value of the flags field, this may grant the client exclusive access to the agent until the client detaches from the agent.  If another client (or agent) has already obtained exclusive access to the agent, this request may be denied.  It is possible for a client to be denied attachment to an agent to which it has a reference, even if it has previously attached and subsequently detached.  Clients must attach to an agent before they will be able to send commands to that agent.

 

Possible flag values are: (TBD)

 

Command:                              

detachFromAgent

Parameters:

agentID

Integer

ID of the agent from which to detach

Description:

This command allows a client (or agent) to end its active attachment to an agent.  No response will be sent to this command.

 

Command:                              

registerAgent

Parameters:

agentID

Integer

ID of the agent (obtained during connection)

processID

Integer

ID of the process containing the agent

agentName

String

Name of the agent being registered

Response:

agentRegistered

Response Parameters:

No data

Alternate Response:

agentRegistrationRefused

Alternate Response Parameters:

reason

Integer

Code indicating the reason for failure

Description:

Agents issue this command to notify the Agent Manager that they are available.  The Agent Manager will use the Process ID and agent name to recognize agents that the Agent Manager itself has launched.

 

Command:

deregisterAgent

Parameters:

agentID

Integer

ID of the agent being deregistered

Response:

agentDeregistered

Response Parameters:

No data

Description:

Agents issue this command to notify the Agent Manager that they are no longer available.  An agent may make itself temporarily unavailable using this command and then later re-register.  The Agent Manager will not remove the agent from its connection list (as established by Layer Zero commands) because of this command.

7.2.2      Events

The following event interface is supported by the Agent Manager.

 

HII_AGENT_MGR_AGENT_EVENTS

This interface allows clients and agents to receive notification of events related to changes in the availability of agents and changes in agent states.

 

Event:                                     

agentRegistered

Parameters:

agentID

Integer

ID of the agent that was registered

Description:

This event is sent when an agent registers with the Agent Manager.

 

Event:                                     

agentDeregistered

Parameters:

agentID

Integer

ID of the agent that was deregistered

Description:

This event is sent when an agent deregisters with the Agent Manager.

 

Event:                                     

agentAvailable

Parameters:

agentID

Integer

ID of the agent that has become available

Description:

This event is sent when a client detaches from an agent, making the agent available to be attached to by other clients.

 

Event:

agentUnavailable

Parameters:

agentID

Integer

ID of the agent has become unavailable

Description:

This event is sent when a client attaches to an agent and the agent cannot accept attachments from other clients.

 

7.3      Base Agent Interface

The Agent Controller or another agent requiring services from any agent will invoke these interfaces. Some of these activities will trigger the events that will be communicated to the source to indicate the status of an issued command. The Base Agent Interface defines the basic set of commands that are required to communicate with any agent.

 

NOTE: All the commands for an agent need to be issued specifically for that agent by including Agent ID in the command. This Agent ID will be part of the “Command Header” and is not part of the “Command Data”.

Common Definitions:

 

<xsd:complexType name="Variable">

                <xsd:element name="ID" type="xsd:string"/>

                <xsd:element name="Name" type="xsd:string"/>

                <xsd:element name="Description" type="xsd:string"/>

                <xsd:element name="Type" type="xsd:string"/>

                <xsd:element name="Value" type="xsd:string"/>

</xsd:complexType>

               

<xsd:complexType name="VariableGroup">

                <xsd:element name="ID" type="xsd:string"/>

                <xsd:element name="Name" type="xsd:string"/>

                <xsd:element name="Description" type="xsd:string"/>

                <xsd:element name="Variable" type="Variable" minOccurs="1"/>                            

</xsd:complexType>

               

<xsd:simpleType name="Answer">

                <xsd:restriction base="xsd:string">                          

                                <xsd:enumeration value="Yes"/>

                                <xsd:enumeration value="No"/>

                </xsd:restriction>

</xsd:simpleType>

               

<xsd:simpleType name="Success">

                <xsd:restriction base="xsd:string">                          

                                <xsd:enumeration value="Yes"/>

                                <xsd:enumeration value="No"/>

                </xsd:restriction>

</xsd:simpleType>

               

               

<xsd:simpleType name="ErrorCodeType">

                <xsd:element name="ErrorCode" type="xsd:string"/>

<xsd:element name=”ErrorInfo” type=”xsd:string”/><!—should put CBE block in here ŕ

</xsd:simpleType>

 

 

7.3.1      Basic Interface

7.3.1.1     Get Agent Name

Command:

GetAgentName

Parameters:

            None

Response:

AgentName

Response Parameters:

AgentName

string

Name of the Agent

Description:

This command returns the agent name specified in the agent configuration file. Agent Name is a string representation of the agent - for e.g. “org.eclipse.hyades.jvmpiagent”.

7.3.1.2     Check Compatibility (with another agent)

Command:

CheckComaptibility

Parameters:

AgentName

string

Name of the Agent

Response:

AgentCompatible

Response Parameters:

Compatible

Answer

Returns if Yes or No

ErrorCode

ErrorCodeType

Error Code Details

Description:

This command checks the compatibility of the agent with another agent in terms of running both of these agents simultaneously. The command takes the second agent name as the parameter.

 

NOTE: Need to evaluate if we have to extend this for more than two agents. This can be part of the agent configuration instead, which will be read by Agent Controller and apply the conditions while serving the request to run the agents simultaneously.

7.3.1.3     Terminate or Shutdown

Command:

Terminate

Parameters:

            None

Response:

AgentTerminated

Response Parameters:

Result

Success

Success or Failure

ErrorCode

ErrorCodeType

Error Code Details

Description:

This command checks the compatibility of the agent with another agent in terms of running both of these agents simultaneously. The command takes the second agent name as the parameter.

 

This interface allows the client/Agent Controller to shut down or terminate the agent. After this action, the agent instance will be deleted and will not be available for service. The response will be either success or failure. If it is error, will also return an error code as response data.

7.3.2      Attachable Interface

This interface defines the commands related to attaching/detaching an agent to an external process or application.

NOTE: We need to re-define the commands in a multi-process attachment scenario after understanding it better. In this case the commands can take Process ID list as parameter.

7.3.2.1     Attach to Process

Command:

     AttachToProcess

Parameters:

ProcessID

string

Process identification

Response:

Attached

Response Parameters:

Result

Success

Success or Failure

ErrorCode

ErrorCodeType

Error Code Details

Description:

This command informs an agent to a process with Process ID passed as a parameter. The command returns success or failure as response.

7.3.2.2     Detach from Process

Command:

     DetachFromProcess

Parameters:

ProcessID

string

Process identification

Response:

Detached

Response Parameters:

Result

Success

Success or Failure

ErrorCode

ErrorCodeType

Error Code Details

Description:

This command informs an agent to detach itself from the process with Process ID (passed as a parameter). The command returns success or failure as response.

NOTE: Do we need to specify the Process ID as parameter? Could be useful if we have multi-process scenario (in which case the parameter will be a Process ID list).

7.3.2.3     Is Attached

Command:

     IsAttached

Parameters:

            None

Response:

Attached

Response Parameters:

ProcessID

string

ProcessID to which agent attached

Result

Success

Success or Failure

ErrorCode

ErrorCodeType

Error Code Details

Description:

This command responds by indicating if the agent is attached to a process or not. If the agent is still attached to a process, the response data will contain the Process ID.

7.3.3      Variable Interface

The Variable Interface defines a very generic command protocol to configure the agent properties. This interface can be used to set or get the properties of the agent.

The table below gives the common data type definitions that are used in the variable interface – Variable and Variable Group. A Variable data type represents the property of an agent. Each Variable is uniquely identified within an agent using an ID and it has other fields like name (the agent property name), description, type (data type of the variable) and value (value of the agent property). The possible values for the type ca  be – INT, FLOAT, DECIMAL, A VariableGroup represents a group or set of Variables. The VariableGroup is a representation of a collection of agent properties. A Variable Group can contain one or more other Sub Variable Groups. This can be used to set or get the values of more than one agent properties.

Common Definitions:

<xsd:complexType name="Variable">

                <xsd:element name="ID" type="xsd:string"/>

                <xsd:element name="Name" type="xsd:string"/>

                <xsd:element name="Description" type="xsd:string"/>

                <xsd:element name="Type" type="xsd:string"/>

                <xsd:element name="Value" type="xsd:string"/>

</xsd:complexType>

               

<xsd:complexType name="VariableGroup">

                <xsd:element name="ID"  type="xsd:string"/>

                <xsd:element name="Name"  type="xsd:string"/>

                <xsd:element name="Description"  type="xsd:string"/>

                <xsd:element name="Variable"  type="Variable" minOccurs="1"/>

<xsd:element name="SubGroup"  type="VariableGroup" minOccurs="0"/>                          

</xsd:complexType>

 

7.3.3.1     Get Variables Group

Command:

     GetVariableGroup

Parameters:

GroupID

string

Variable Group ID

Response:

VariableGroupList

Response Parameters:

AgentVariableGroup

VariableGroup

Variable Group list with contained variables

Result

Success

Success or Failure

ErrorCode

ErrorCodeType

Error Code Details

Description:

This interface returns the list of all the agent property groups with the agent property name value pairs within each group or within in the sub groups (if any) that are present in the group. This command takes an optional parameter “GroupID” and when it is passed, the command response will include only the agent properties that are part of the variable group with the specified GroupID.

7.3.3.2     Get Variables Values

Command:

     GetVariableValues

Parameters:

VariableID

string

ID of the Variable the values of which are being retrieved

Response:

VariableValues

Response Parameters:

AgentVariable

Variable

Variable with values

Result

Success

Success or Failure

ErrorCode

ErrorCodeType

Error Code Details

Description:

This interface returns the Variables list along with the values of the variables requested in the command. The command takes the list of Variables IDs for which the values need to be retrieved. The response to the command will include the Variable type blocks apart from the success or failure information. This command should be used to retrieve the values of one or more agent properties.

7.3.3.3     Set Variables Values

Command:

     SetVariableValue

Parameters:

AgentVariable

Variable

Variable with the new values

Response:

setVariableResponse

Response Parameters:

AgentVariable

Variable

Variable with new set values

Result

Success

Success or Failure

ErrorCode

ErrorCodeType

Error Code Details

Description:

This interface sets the values of the variables passed with the command. The command takes the list of Variables for which the values need to be set. The response to the command will include the Variable type blocks (with the latest variable values after the command is executed) apart from the success or failure information. This command should be used to set the values of one or more agent properties. The agent implementation of this interface can validate the variable values for correctness and throw an exception if there is any mismatch in terms of data type, valid range of variable values, etc.

7.3.3.4     Reset Configuration

Command:

     ResetsetVariables

Parameters:

            None

Response:

ResetVariablesResponse

Response Parameters:

Result

Success

Success or Failure

ErrorCode

ErrorCodeType

Error Code Details

Description:

 

This interface can be used to reset the agent configuration to bring it back to the initialized state. The response will be either success or failure. If it is error, will also return an error code as response data.

7.3.4      Web Services Interface

This interface defines the protocol for using an agent in web services environment (specifically in choreography framework).

NOTE: Currently this interface has only one command but in future we can add more commands once we understand the usage model.

7.3.4.1     Get WSDL Definition

Command:

     GetWSDLDefinition

Parameters:

            None

Response:

WSDLDefinition

Response Parameters:

WSDL

String

The XML string representing the WSDL definition of the agent

Result

Success

Success or Failure

ErrorCode

ErrorCodeType

Error Code Details

Description:

This command returns the agent definition in WSDL format. The definition describes the agent giving its name, configuration, interfaces and their definition, etc. The format of the returned definition could be in text message and is in WSDL format (XML format).

7.3.5      Event Interface

This interface consists of command protocol for registering and de-registering the Event Listeners with a specific agent. This will allow a client to listen for a specific set of events it is interested in. The event registration process will avoid sending unnecessary events (triggered by agents) across the network even if there are no even handlers. The agent will maintain the mapping information about the events and their listeners.

7.3.5.1     Add Agent Event Listener

Command:

     AddEventListener

Parameters:

ClientID

string

Client identification

HandlerID

integer

Handle to the Event Handler

InterfaceID

string

Event ID list

Response:

ListenerAdded

Response Parameters:

Result

Success

Success or Failure

ErrorCode

ErrorCodeType

Error Code Details

Description:

This interface registers an interested listener to a specific event. The command takes the parameters, List of Event IDs – Event IDs to be registered for, Server ID – this is the ID of the Agent Controller to which the agent is attached to, Listener ID – this is the ID of Client or another Agent, Listener Context ID – the actual object on the Client/Agent that will wait for and process the event. The agent instance maintains a table of events (that can be generated by the agent) and the interested event listeners registered. When an event is triggered, this information is sent only to the interested listeners by routing via Agent Controller. The response will be either success or failure. If it is error, will also return an error code as response data.

7.3.5.1.1      Remove Agent Event Listener

Command:

     RemoveEventListener

Parameters:

ClientID

string

Client identification

HandlerID

integer

Handle to the Event Handler

InterfaceID

string

Event ID list

Response:

ListenerRemoved

Response Parameters:

Result

Success

Success or Failure

ErrorCode

ErrorCodeType

Error Code Details

Description:

This interface removes a registered event listener indicating that the listener no longer wishes to receive a notification when event is triggered. The command data for this command is same as that for “Add Agent Event Listener”. The response will be either success or failure. If it is error, will also return an error code as response data.

7.4      Collector Interface

This section describes the base interface definition of the agents that specifically carry out the performance or other types data collection of an application under observation. This interface has to be implemented by every collector though it can extend the interface by adding other methods.

7.4.1      Basic Interface

7.4.1.1     Start Monitoring

Command:

     StartMonitoring

Parameters:

IPAddress

String

Client identification

PortNumber

integer

Port Number Client is listening at

Response:

StartMonitoringResponse

Response Parameters:

Result

Success

Success or Failure

ErrorCode

ErrorCodeType

Error Code Details

Description:

This interface will create keep the data channel ready to be able to transfer the data collected during Run operation. The command data includes the data stream information that should be used to transfer any data collected by the agent. The response will be either success or failure. If it is error, will also return an error code as response data.

7.4.1.2     Run

Command:

     Run

Parameters:

            None

Response:

RunResponse

Response Parameters:

Result

Success

Success or Failure

ErrorCode

ErrorCodeType

Error Code Details

Description:

This interface runs the agent and starts the data collection and will continue to collect the data until Stop or Pause command is issued. The response will be either success or failure. If it is error, will also return an error code as response data.

NOTE: The Run command doesn’t take any parameters as it is just starting the data collection. We can add few parameters if we want to run the collection in different modes like – run for a certain span of time, Start the collection at a specific time, collect the data in alternating run and stop mode.

7.4.1.3     Pause

Command:

     Pause

Parameters:

            None

Response:

PauseResponse

Response Parameters:

Result

Success

Success or Failure

ErrorCode

ErrorCodeType

Error Code Details

Description:

This interface can be used to move the status of the agent that is running into a pause mode. During the pause mode the agent is active but is in idle state without performing any collection. The response will be either success or failure. If it is error, will also return an error code as response data.

7.4.1.4     Resume

Command:

     Resume

Parameters:

            None

Response:

ResumeResponse

Response Parameters:

Result

Success

Success or Failure

ErrorCode

ErrorCodeType

Error Code Details

Description:

This interface can be used to change the agent state back to running mode from a previously paused state. When this operation is performed, the agent will resume the collection it has started before. One usage case for the Pause/Resume sequence could be to perform a periodic data collection. The response will be either success or failure. If it is error, will also return an error code as response data.

7.4.1.5     Cancel

Command:

     Cancel

Parameters:

            None

Response:

CancelResponse

Response Parameters:

Result

Success

Success or Failure

ErrorCode

ErrorCodeType

Error Code Details

Description:

When this interface is called, the agent will cancel any currently running data collection. This moves the agent to Configured state so that another collection can be configured and run. After this action, the agent will be still active and can provide services again. The response will be either success or failure. If it is error, will also return an error code as response data.

7.4.1.6     Stop

Command:

     Stop

Parameters:

            None

Response:

StopResponse

Response Parameters:

Result

Success

Success or Failure

ErrorCode

ErrorCodeType

Error Code Details

Description:

This will stop the currently running data collection and the collected data can be transferred to the client. The response will be either success or failure. If it is error, will also return an error code as response data.

7.4.1.7     Snapshot

Command:

     Snapshot

Parameters:

            None

Response:

SnapshotResponse

Response Parameters:

Result

Success

Success or Failure

ErrorCode

ErrorCodeType

Error Code Details

Description:

This interface allows a client to request the snapshot of the data collected until the point of request. The agent will initiate a data transfer after it receives this command. The data transfer could be the delta of the data collected since the last data transfer. The response will be either success or failure. If it is error, will also return an error code as response data.

Note: This command may not be meant for the agents that transfer real-time data (send data regularly as they collect) since they have been sending the data anyway.

7.4.1.8     Get Data (Transfer Data)

Command:

     GetData

Parameters:

            None

Response:

GetDataResponse

Response Parameters:

Result

Success

Success or Failure

ErrorCode

ErrorCodeType

Error Code Details

Description:

This command initiates the data transfer using the data channel created (when Start Monitoring command is issued).

NOTE: This command is added with an assumption that the client will initiate the data transfer when it wishes to get the data collected. In case of agents that transfer data in frequently as data is collected (real-time transfer), this command doesn’t make sense. So, this is meant for agents that save the data locally.

7.4.1.9     Get Agent State

Command:

     GetAgentState

Parameters:

            None

Response:

AgentState

Response Parameters:

State

string

Agent State Name

Result

Success

Success or Failure

ErrorCode

ErrorCodeType

Error Code Details

Description:

Returns the state of the agent. The agent could be in one of the states illustrated in the agent state machine in section 1.1.1.5.

7.4.2      Events

As a result of a command from a client, an agent will perform a specific activity and if that activity is asynchronous and occupies certain time span, the client can’t wait until the completion of the command. And the client will not get the response within the same synchronous call. So, the agent should necessarily send an asynchronous notification to the client after successful completion of the command. This notification is called “Event”. Such an event notification will include the event id and other information related to the event. Not all the clients may be interested in listening to the events generated by the agent. The clients will create listeners and indicate the interest by registering (please check the “Event Interface” section) with agents for a specific event(s). And the listeners created by the client will wait for the event notification from agent.

 

This section lists the events that will be generated by the collector agents during their state transition.

Note: The Event Commands for each of the events are defined with separate Command IDs. It is possible (if we decide to do so) to use the same Command ID for all the events and distinguish the events with event information in the command data.

 

<xsd:complexType name="Event">

                <xsd:element name="ID" type="xsd:string"/>

                <xsd:element name="Name" type="xsd:string"/>

                <xsd:element name="Description" type="xsd:string"/>

                <xsd:element name="Time" type="xsd:string"/>

                <xsd:element name="SourceID" type="xsd:string"/>

                <xsd:element name="ListenerID" type="xsd:string"/>

                <xsd:element name="ListenerContextID" type="xsd:string"/>

                <xsd:element name="InterfaceID" type="xsd:string"/>

                <xsd:element name="Info" type="xsd:hexBinary"/>

</xsd:complexType>

 

 

7.4.2.1     Initialized

Triggered after an agent is initialized.

Command:                            

CID_COLLECTOR_EVENT_INIT

Data:

Bytes 0-3

Server ID

Bytes 4-7

Listener ID

Bytes 8-11

Listener Context ID

Bytes 12-15

Event Interface ID

Remaining Bytes

Event Information

 

<xsd:complexType name="Initialized" type=”Event”>

 

 

7.4.2.2     Configured

This event is triggered after the agent configuration is completed.

Command:                            

CID_COLLECTOR_EVENT_CONFIGURED

Data:

Bytes 0-3

Server ID

Bytes 4-7

Listener ID

Bytes 8-11

Listener Context ID

Bytes 12-15

Event Interface ID

Remaining Bytes

Event Information

 

<xsd:complexType name="Configured" type=”Event”>

 

7.4.2.3     Started

This event is triggered after the successful completion of the agent Start action.

Command:                            

CID_COLLECTOR_EVENT_STARTED

Data:

Bytes 0-3

Server ID

Bytes 4-7

Listener ID

Bytes 8-11

Listener Context ID

Bytes 12-15

Event Interface ID

Remaining Bytes

Event Information

 

<xsd:complexType name="Started" type=”Event”>

 

7.4.2.4     Paused

This event is triggered when a Pause command is issued and the agent completes a successful pause of the data collection.

Command:                            

CID_COLLECTOR_EVENT_PAUSED

Data:

Bytes 0-3

Server ID

Bytes 4-7

Listener ID

Bytes 8-11

Listener Context ID

Bytes 12-15

Event Interface ID

Remaining Bytes

Event Information

 

<xsd:complexType name="Paused" type=”Event”>

 

7.4.2.5     Resumed

This event is triggered when a Resume command is issued and the agent restarts the collection.

Command:                            

CID_COLLECTOR_EVENT_RESUMED

Data:

Bytes 0-3

Server ID

Bytes 4-7

Listener ID

Bytes 8-11

Listener Context ID

Bytes 12-15

Event Interface ID

Remaining Bytes

Event Information

 

<xsd:complexType name="Resumed" type=”Event”>

 

7.4.2.6     Data Ready

This event indicates that the data collection is done and the data is created and ready to be transferred upon a client’s request.

Command:                            

CID_COLLECTOR_EVENT_DATA_READY

Data:

Bytes 0-3

Server ID

Bytes 4-7

Listener ID

Bytes 8-11

Listener Context ID

Bytes 12-15

Event Interface ID

Remaining Bytes

Event Information

 

<xsd:complexType name="DataReady" type=”Event”>

 

7.4.2.7     Cancelled

This event is triggered when a Cancel command is issued and the data collection is cancelled.

Command:                            

CID_COLLECTOR_EVENT_CANCELLED

Data:

Bytes 0-3

Server ID

Bytes 4-7

Listener ID

Bytes 8-11

Listener Context ID

Bytes 12-15

Event Interface ID

Remaining Bytes

Event Information

 

<xsd:complexType name="Cancelled" type=”Event”>

 

7.4.2.8     Completed

Command:                            

CID_COLLECTOR_EVENT_COMPLETED

Data:

Bytes 0-3

Server ID

Bytes 4-7

Listener ID

Bytes 8-11

Listener Context ID

Bytes 12-15

Event Interface ID

Remaining Bytes

Event Information

 

<xsd:complexType name="Completed" type=”Event”>

 

7.4.2.9     Failed

Command:                            

CID_COLLECTOR_EVENT_FAILED

Data:

Bytes 0-3

Server ID

Bytes 4-7

Listener ID

Bytes 8-11

Listener Context ID

Bytes 12-15

Event Interface ID

Remaining Bytes

Event Information

 

<xsd:complexType name="Failed" type=”Event”>

 

7.4.3      Agent State Machine

The diagram illustrates the collector agent state transition during its life cycle. The state transition occurs due to a command from a client, Agent Controller or another agent. Every collector agent will maintain it’s own state and will control the state transitions. At any particular state, it can accept only certain set of commands as indicated by the state machine. And there are only few valid state transitions that can occur from a particular state. The collector agent state machine diagram shows the valid states and all the possible transitions that can occur. An error will be returned if the client sends a command that is not allowed and invalid.

 

 

7.5      Event Provider Interface

7.5.1      Commands

Command:                              

addEventListener

Parameters:

interfaceID

String

ID of the event interface the caller would like to listen to

listenerID

Integer

A caller-defined ID representing the listener context

Response:

listenerAccepted

Response Parameters:

No data

Alternate Response:

listenerRejected

Alternate Response Parameters:

Reason

Integer

Code indicating the reason for failure

Description:

This command allows clients or agents to register to receive Agent Manager events.  See below for details of the available events.

7.6      Process Controller Agent Interface

Below are the commands that can be sent to the standard Process Controller agent and the

event interfaces it supports.

7.6.1      Commands

The following commands are supported through the Process Controller interface.  Clients, agents, and the Agent Controller use these commands to start, stop, pause, and resume processes. 

 

Command:                              

validateProcessToLaunch

Parameters:

applicationName

String

Name of the application to check for

workingDir

String

Name of directory to check for, may be null.

Response:

processValidationResults

Response Parameters:

validApp

Integer

0 if valid, else error code

validWorkingDir

Integer

0 if valid, else error code

Description:

Use this command to check if an application exists and is executable, and, optionally, if the working directory to be used by that application exists and is writeable.  This is expected to be used by clients wanting to confirm application-to-launch information coming in through a UI, prior to actually launching the application.

 

Command:

startProcess

Parameters:

applicationName

String

Name of application to launch

commandLineArgs

String list

Command line argument list for the application

workingDir

String

Working directory for the application

EnvironmentVars

String list

List of environment variable/values for the application

Response:

startProcessSuccessful

Response Parameters:

processID

Integer

Process ID value returned by the exec call

Alternate Response:

startProcessFailed

Alternate Response Parameters:

reason

Integer

Code indicating the reason for failure

 

Command:

stopProcess

Parameters:

processID

Integer

Process ID of application to stop

Response:

stopProcessSuccessful

Response Parameters:

      No data

Alternate Response:

stopProcessFailed

Alternate Response Parameters:

reason

Integer

Code indicating the reason for failure

 

Command:

pauseProcess

Parameters:

processID

Integer

Process ID of application to pause

Response:

pauseProcessSuccessful

Response Parameters:

      No data

Alternate Response:

pauseProcessFailed

Alternate Response Parameters:

reason

Integer

Code indicating the reason for failure

 

Command:

resumeProcess

Parameters:

processID

Integer

Process ID of application to resume

Response:

resumeProcessSuccessful

Response Parameters:

      No data

Alternate Response:

resumeProcessFailed

Alternate Response Parameters:

reason

Integer

Code indicating the reason for failure

 

7.6.2      Events

The following event interface is supported by the Agent Manager.

 

HII_PROCESS_CONTROLLER_AGENT_EVENTS

This interface allows clients and agents to receive notification of events related to changes in the state of a process that was launched by the process controller agent.

 

Event:                                     

processExited

Parameters:

processID

Integer

ID of the process that exited

exitCode

Integer

Value returned by the apps call to exit()

Description:

This event is sent when a process launched by the process controller exits normally.

 

Event:                                     

processTerminated

Parameters:

processID

Integer

ID of a process that terminated abnormally

reason

Integer

Code indicating the reason for failure

Description:

This event is sent when a process launched by the process controller is terminated.  This could be the result of an execution fault in the program or because of an external force (i.e., someone killed the process manually).  The reason code indicates the cause.

7.7      Environment Info Agent Interface

 

7.7.1      Commands

The following commands are supported through the Environment Info interface.  Clients, agents, and the Agent Controller use these commands to get information about the system on which the Environment Info agent is running. 

 

Command:                              

getSystemInfo

Parameters:

      None.

Response:

systemInfo

Response Parameters:

systemName

String

System name

ipAddress

String

IP address

numCPU

Integer

Number of CPUs

cpuArchitecture

String list

List of CPU architecture types

systemRAM

Integer

Size of RAM

Description:

Returns the system’s name, IP address, number of CPUs, the architecture of each CPU, and the amount of RAM.

 

Command:

getOperatingSystemInfo

Parameters:

      None.

Response:

operatingSystemInfo

Response Parameters:

osName

String

Operating system name

osVersion

String

OS version info

Description:

Returns the operating system name and version information.

 

Command:

getCPUInfo

Parameters:

      None.

Response:

cpuInfo

Response Parameters:

cpuName

String

CPU name

cpuSpeed

String

CPU speed

numCaches

Integer

Number of caches

cacheSizes

Integer list

List of sizes – one for each cache

features

Integer

Feature bits - uninterpreted

Alternate Response:

cpuInfoNotAvailable

Alternate Response Parameters:

reason

Integer

Code indicating the reason for failure

Description:

One response is sent for each CPU.

Returns CPU identifier/name, speed, #caches, cache size, features.

The error response would indicate that the agent did not know how to determine info about a CPU.

7.7.2      Message Buffer Memory Management

TBD


 [kac1]Vishnu - update

 [kac2]Vishnu-update

 [kac3]Explain what the blocks are relative to the interfaces.

 [kac4]Need to understand why internal interfaces are being used rather than external interfaces.

1)       Are they used because client developers did not find the external

2)       Did the external interface not provide sufficient interfaces for client side needs

 

Need to get this answered by Richard/Joe.

 [kac5]This is an internal package which clients are accessing as external.

 [kac6]Review what they do now vs. what we need to be doing instead.

 [kac7]If an agent is launched from external mechanism and registers itself with the AC – does its info go into the permanent registry or is it only known during the current run of the AC?

 

 [kac8]Is this to allow for an agent being updated and then perhaps have it’s ifaces out of sync with the registry?  If so, having the metadata in an xml file paired with the agent would resolve this – unless an install error occurred.

 

 [kac9]Define the difference in what you can do.  Can you send cmds to an agent that you hold a reference to, but are not attached to?

 [kac10]how is this supported in current RAC?

 [kac11]Do we provide Thread level control?  Perhaps not in the standard process controller, but a custom one?

 

 [kac12]Is this to allow clients to remove temp files they have created on the remote system? 

 [kac13]What is the need for this kind of file manipulation?  I believe the request was to have a way for a UI to offer a list of files to select from when data collection is on a remote system.

 [kac14]A local copy?

 [kac15]From prev mtg -  Agent Controller Start up

a) Read Config { Server, Agent Discovery/Metadata} (Read - XML Parser)

b) Msg Pipeline established

c) Starting transport layer plug-ins

(Starting listening socket)

d) Start System Agents

e)Init Logging (lib or service) (Logging service)

 [kac16]Need to describe how this fits into the architecture.

Does the host request an SSL type connection?  Is it in a config file?

AC uses this to establish which connections?  Cmd channel, data channel, agents?

 [kac17]Need to describe current form of session management so that we can determine if any more is needed.  That is, do we need to make any changes as Hoang is proposing in this section.

 [kac18]Whoever has the original  - Need to update the picture terminology.

 [awk19]The Transport Layer, Payload Normalizer and Command Extractor interfaces need to be updated to reflect the direct message flow described above.

 [kac20] For Entire Document:

·         Add hyper links for section references

 [kac21]NEED To Replae Hyades with TPP in above figure – I couldn’t figure out how.

 [kac22]Is choreography group driving this?  What is their need? Are they trying to go around the Agent Controller such that a client side impl could not cover it.

 

Hoang says this has been discussed with the Hyades group and agreed upon.

 

Need example of how this would be used.

Change to “ [kac23]Handle”

 [kac24]Impl by base agent

 [kac25]Provided by base impl

 [kac26]Explain who calls this

 [kac27]Does not belong in base class for sure.  Might be done in a collector specific version of the base class – but seems like it should be done as a customer cmd