Skip to main content
[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index] [List Home]
Re: [sparkplug-wg] Sparkplug B Sequence Requirement and Issues
Hi Leland,

Per Bryce’s note below, some brokers (Hive, AWS IoT Core, others) can deliver messages out of order. Our MQTT Engine module in Ignition will let you set a time “window” in which it will wait and recover out-of-order messages. 

I agree on #3 below. We have implementations where 1000’s of metrics can be published in a single DDATA message (which is also efficient on the wire).

-Arlen


On Apr 19, 2023, at 3:11 PM, Bryce Nakatani <bnakatani@xxxxxxxxxx> wrote:

Hello Leland,

I take it that your 1 to 50 device actors operate under a single edge node ID.

By nature of the additional processing required, longer payloads take longer to serialize and publish than smaller payloads.

1. I've seen this. Consider that message reordering may occur at the MQTT broker.
2. Using an asynchronous implementation could be responsible for this. If you're not careful about when/how the sequence ID is assigned and how publications are published, a sequence ID misordering could occur.
3. I don't see limiting metric quantity as a solution. This might lead to a backlog and might lead to problems with timestamps and how to control congestion.
4. MQTT Engine 4.0.15 recovers out-of-order publications. It's part of the 3.0 specification (see section 5.9).

If you want to monitor the traffic, you could start with the Sparkplug Listener example in the tahu package and modify it to track the sequence order.

Bryce

On Wed, Apr 19, 2023 at 12:21 PM Leland Falde <leland.falde@xxxxxxxxxxxxxxx> wrote:

Good day!

 

I am having trouble guaranteeing that a message sent with the proper sequence number arrives at the primary application (Ignition) in the same sequence the message was sent.

 

I will be adding some logging to help confirm this…but here is my suspicion.

 

‘The 80’ of messages contain 1 – 3 changed metrics.

Our system aggregates a production event at event end into a single report with 100+ changed metrics.

Does a large payload take longer in transit to get from publisher to primary application?

Why is there a sequence requirement in the first place? Would be nice if the sequence could be enforced per device instead of at the node level. Having many device actors exacerbates this issue?

 

We can have anywhere from 1 to 50 device actors generating asynchronous events. Because of this issue, I’ve implemented an async FIFO queue that can be throttled.

 

  1. Has anyone experienced this issue?
  2. Would anyone willing to share their similar experience and any best practices?
  3. Is there a metric count that I should limit my messages to?
    1. I am leery to break up the message into smaller metric chunks due to unknown issues downstream with handling historian data writes. 
  4. Is there any way to guarantee that the messages sent with the proper sequence number is received and/or handled in the order they are sent?

 

com.cirruslink.common.sparkplug.util.exception.SequenceNumberException: For Group= ________________ - Message Sequence number ERROR: expected=61 but received=62
at com.cirruslink.mqtt.engine.gateway.sparkplug.SparkplugPayloadHandler.handleSeqNumberCheck(SparkplugPayloadHandler.java:401)
at com.cirruslink.mqtt.engine.gateway.sparkplug.SparkplugBPayloadHandler.onDeviceMessage(SparkplugBPayloadHandler.java:1467)
at com.cirruslink.mqtt.engine.gateway.sparkplug.SparkplugBPayloadHandler.handleDeviceData(SparkplugBPayloadHandler.java:1078)
at com.cirruslink.mqtt.engine.gateway.sparkplug.SparkplugPayloadHandler.handlePayload(SparkplugPayloadHandler.java:137)
at com.cirruslink.mqtt.engine.gateway.EngineCallback.lambda$messageArrived$1(EngineCallback.java:229)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
at java.base/java.lang.Thread.run(Unknown Source)

 

Leland D. Falde

Senior Software Developer, Insight Centerpoint™

 

Miller Electric Mfg. LLC  An ITW Welding Company

500 East Biddle Street, Jackson, MI 49201    Teams: call via Teams   Mobile: 517.745.4442

MillerWelds.com  │  Facebook  │  Twitter  │  YouTube

 

Disclaimer - This email and any files transmitted with it are confidential and may contain privileged or copyrighted information. You must not present this message to another party without first gaining permission from the sender. If you are not the intended recipient, you must not copy, print, distribute or use this email or the information contained in it for any purpose other than to notify us.

If you received this email in error, please notify the sender immediately and delete this email from your system. We do not guarantee that this email is free from viruses or any other defects although due care has been taken to minimize the risk.

_______________________________________________
sparkplug-wg mailing list
sparkplug-wg@xxxxxxxxxxx
To unsubscribe from this list, visit https://www.eclipse.org/mailman/listinfo/sparkplug-wg
_______________________________________________
sparkplug-wg mailing list
sparkplug-wg@xxxxxxxxxxx
To unsubscribe from this list, visit https://www.eclipse.org/mailman/listinfo/sparkplug-wg

Back to the top