Hi Marc,
thanks for starting this discussion. Here are some thoughts from my side:
- Regarding ordering of messages, that’s always a difficult question in a distributed system. We can always “version” commands sent from one application, so that device can check if everything is in order. But as many applications can send commands to the device, there’s no way to sync them all using protocol only. So, in my point of view the application need to worry about this on a higher level and make sure that device is in the correct state, even after the command is executed successfully.
- For the scale of everything, there’s always a limit a single broker/router can take. So, as with telemetry, we should try to tackle scalability beyond that limit with EnMasse. Using a router mesh and partition queues over multiple brokers behind it. In terms of number of queues, we can also think of using a single queue to address multiple devices (consuming with selectors), but that can be tricky in terms of security.
- In the brokered case, we need at least a reply address (queue?), per device for commands (that’s the common patter I have seen for this). So that app issues a command by sending a message to a queue, and subscribing to another address (usually queue in the past), waiting for a reply with a timeout.