Streaming subscription protocol specification

The history capabilities of the Ditto protocol consists of 3 commands and 4 events that together implement the reactive-streams protocol over any duplex transport layer. For each streaming subscription request, Ditto acts as the reactive-streams publisher of pages of historical events, and the client acts as the subscriber. By reactive-streams means, the client controls how fast pages are delivered to it and may cancel a request before all results are sent.

While connections do not expose or require a duplex transport layer, the streaming subscription protocol is available for them as well: Send commands from client to Ditto via any connection source. For each command, 0 or more events from Ditto to client are published to the reply-target of the source.

For reactive-streams on the JVM, a publisher-subscriber pair is identified by a Subscription object according to reference equality. Similarly, the streaming subscription protocol commands and events of one request are identified by a subscription ID.

Each streaming subscription protocol command or event corresponds to a reactive-streams signal and are bound by the same rules in the reactive-streams specification.

Interaction pattern

For one request, the commands from client to Ditto should follow this protocol:

subscribe request* cancel?

The client should send one “subscribeForPersistedEvents” command, followed by multiple “request” commands and an optional “cancel” command.

In response to a “subscribeForPersistedEvents” command and after each “request” command, Ditto will send 0 or more events to the client according to the following protocol:

created next* (complete | failed)?

A “created” event bearing the subscription ID is always sent. 0 or more “next” events are sent according to the amount of results requested by the client. A “complete” or “failed” event comes at the end unless the client sends a “cancel” command before the results are exhausted.

There is no special event in response to a “cancel” command. The client may continue to receive buffered “next”, “complete” or “failed” events after sending a “cancel” command.

In addition to the rules of reactive-streams, Ditto guarantees that no “complete” or “failed” event will arrive before the client expresses its readiness by a first “request” command. The reason is to facilitate concurrency at the client side. Without the extra guarantee, a multi-threaded client would have to process a “complete” or “failed” event in parallel of the preceding “created” event. It would put the burden of sequentialization at the client side and complicate the programming there.

Commands from Client to Ditto

Subscribe for persisted events

Sent a “subscribeForPersistedEvents” command to Ditto to start receiving persisted events as results. Ditto will always respond with a “created” event.

The options where to start/stop historical persisted events from can be specified in the value field of a “subscribeForPersistedEvents” command. If no options are provided at all, the complete available history for the specified entity is streamed as a result.

Request

After obtaining a subscription ID from a “created” event, use “request” commands to tell Ditto how many results you are prepared to receive. Ditto will send “next” events until all requested results are fulfilled, the results are exhausted, or an error occurred.

Cancel

After obtaining a subscription ID from a “created” event, use a “cancel” command to stop Ditto from sending more items of the results. Pages in-flight may yet arrive, but the client will eventually stop receiving events of the same subscription ID.

Events from Ditto to Client

Created

To any “subscribeForPersistedEvents” command, Ditto will always respond with a “created” event.

Next

Each “next” event contains one item of the results. Ditto will not send more “next” events for a given subscription ID than the total number of items requested by previous “request” commands.

Complete

A streaming subscription ends with a “complete” or a “failed” event from Ditto, or with a “cancel” command from the client. Ditto sends a “complete” event when all items of the results are delivered to the client.

Failed

A streaming subscription ends with a “complete” or a “failed” event from Ditto, or with an “cancel” command from the client. Ditto sends a “failed” event when an internal error occurred, or when the client breaches the reactive-streams specification. It is not possible to “request” more items of the streaming subscription results after a “failed” event.