B2J Distributed Engine Architecture

Author: Antony Miguel, Last Updated 30th March 2006

---

The distributed engine transport layer is designed to be deadlock free.
The layer will be described in terms of two nodes (A and B) communicating point to point and the connection between them.
Each connection between two nodes can be logically split into 4 parts:
1. An up/down stream for outgoing (A-B-A) transactions
2. An up/down stream for incoming (B-A-B) transactions
3. A stream for A-B asynchronous messages
4. A stream for B-A asynchronous messages
NOTE: The asynchronous message streams do not have buffer planes because in the case of the engine the asynchronous messages are never rerouted, they are point to point messages
NOTE: The transaction streams do not have ACK based finite buffering because the nature of the transaction ensures the same thing (transaction response == ACK)
NOTE: The two multiplexers (x3 and x2) could potentially be mixed into a single multiplxer (x4) but the ack-finite-buffer streams are implemented as a generic feature over a single bidirectional stream and therefore do their own multiplexing
The nodes' point to point connection (e.g. a TCP/IP socket) is first multiplexed into 3 separate bi-directional streams.
One stream is used for outgoing transactions, another is for incoming transactions, and the last is for asynchronous messaging.

---

The engine contains a pluggable point to point transport layer which provides the point to point connection in the multiplexed transport layer. Alternative transports can be added via the use of an Eclipse extension point. The engine's dependancy management facilities ensure that the transport implementation is then passed to the rest of the engine without further installation (where possible).
As standard, the engine comes with a TCP/IP implementation of the point to point Session Transport extension point.

---

When sending a stream of messages asynchronously, it may be possible to lock the underlying transport layer, depending on that layer’s buffering strategy.
To ensure that there are a finite (and known) number of messages in the underlying layer at any one time, the outgoing stream sends N messages. The other end of the stream, reading the messages, then sends acknowledgements every N/2 (or some other factor) messages. The outgoing stream will then receive this acknowledgement and allow a further N/2 messages into the stream, thus guaranteeing that there will be no more than N/2 messages in the underlying transport layer at any time.

---

The transaction streams in the engine are used for the majority of communication and transactions can be rerouted.
The transaction streams do not need ACK based finite buffering because the nature of the transaction provides this – if a send occurs then another send cannot occur until a receive for the previous send has occurred, thus guaranteeing at most 1 message in the underlying transport at any time. This feature prevents deadlock of the underlying layer.
The transaction streams do, however, need buffer planes. Buffer planes mean that A can make a transaction request on B, and then B can make a transaction request on A ad infinitum without deadlock occuring.
This means that however the application layer chains its transactions and whatever the dependencies are, deadlock will not occur.

---

The Asynchronous streams in the engine are used only for point to point messaging. The messages sent across these streams are never rerouted.
These streams need ACK based finite buffering for the reasons above.
Because these streams never reroute messages, they do not need buffer planes.