Skip to main content

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index] [List Home]
Re: [hono-dev] Changes in settlement of telemetry and dispatch router config?

see below


On 20.09.2017 15:29, Pellmann Marc (INST/ECS4) wrote:

Hi *,

 

so let me make my questions/proposals more conrete (see mails from Kai und me):

 

Do we want provide full end-to-end flow control, to be able to reject messages from devices at the adapters if consumer could not process any more?

 

 

To archive this we need to change at API level:

 

            Allow (or demand?) AT MOST ONCE for telemetry.

Telemetry API is currently defined using AT_MOST_ONCE semantics (i.e. pre-settled). What changes do you propose? Make it require the client to send unsettled messages (i.e. AT_LEAST_ONCE)?

 

 

To have this in our example setup we need also:

 

Change the Dispatch Router configuration to balanced for telemetry.

 

 

My proposel is, that we need to be able to do full flow-control with our default setup (which means Dispatch Router and Artemis as ‘AMQP Network’). So we should allow AT_MOST_ONCE and change config to balanced. We should also document this dependencies.

by "allow AT_MOST_ONCE" you mean that by default Telemetry uses unsettled (AT_LEAST_ONCE) messages but a client MAY also send pre-settled messages?

 

So the user has the freedom to use it or not.

 

 

@Dejan, @Paolo – what do you think? – Did I missed something in relation to the Dispatch Router or AMQP?

 

 

Mit freundlichen Grüßen / Best regards 

Marc Pellmann

(INST/ECS4) 
Bosch Software Innovations GmbH | Ullsteinstr. 128 | 12109 Berlin | GERMANY | www.bosch-si.com 

Sitz: Berlin, Registergericht: Amtsgericht Charlottenburg; HRB 148411 B
Aufsichtsratsvorsitzender: Dr.-Ing. Thorsten Lücke; Geschäftsführung: Dr.-Ing. Rainer Kallenbach, Michael Hahn



From: hono-dev-bounces@xxxxxxxxxxx [mailto:hono-dev-bounces@xxxxxxxxxxx] On Behalf Of Hudalla Kai (INST/ECS4)
Sent: Donnerstag, 14. September 2017 14:58
To: hono-dev@xxxxxxxxxxx
Subject: Re: [hono-dev] Changes in settlement of telemetry and dispatch router config?

 

See below

 

On 14.09.2017 14:06, Pellmann Marc (INST/ECS4) wrote:

Hi *,

 

as investigated in https://github.com/eclipse/hono/issues/337, we have some problems with our actual use of settlement as well as the dispatch router configuration in regards of full flow control.

 

First of all a configuration of multicast together with link-routing is not an intended use-case of the dispatch router [1]. So we should not configure it this way in our example – flow control is not useable, because the router will always settle the messages. I propose to change this to balanced. I think this is also a more realistic use case, that the consumer has multiple instances, that should be balanced.

Not sure about the "link-routing" case, but it seems that sending unsettled messages to a multicast address will be refused by default in Dispatch Router 1.0.0 (though it will be configurable to restore the behavior of previous versions).

 

The second finding is, that it might be a better approach to use unsettled also for telemetry, to be able to use flow control/back pressure. So we will be able to handle overload as far at the devices side as possible – at least in the adapter (dependent on the device protocol). In this case we would change telemetry to also use at least once (like events - but we don’t need to wait). From the numbers this seams not a big difference in throughput.

WDYT?

I think it depends on what problem we think we have and want to fix.
 
We are currently sending telemetry messages to multicast addresses pre-settled. Doing so, we are seeing many (telemetry) messages being dropped when we run our JMeter tests, i.e. we often receive less messages than we have sent. Hono Messaging doesn't seem to drop the messages (we can tell because of the metrics) so we assume that the Dispatch Router drops them. The problem here seems to be that the Dispatch Router starts to discard messages aggressively, if the upstream senders send more messages than the downstream consumers can handle (i.e. if any of the downstream consumers is not providing enough credits). The number of credits the Dispatch Router hands out to Hono Messaging seems to not be linked to the credits the DR receives from the downstream consumers. Note that this is true for both multicast as well as balanced target addresses.

So, if we want to make sure that the Protocol Adapters only get as many credits as the DR can forward to its downstream consumers, we need to send telemetry messages unsettled, because only this way the DR is able to provide end-to-end flow control. However, if we do so, we also need to make sure that the telemetry addresses are balanced (not multicast), as Marc pointed out.

 

1: https://issues.jboss.org/browse/ENTMQIC-1932?focusedCommentId=13395229&page=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-13395229

 

Marc Pellmann

(INST/ECS4)
Bosch Software Innovations GmbH | Schöneberger Ufer 89-91 | 10785 Berlin | GERMANY
| www.bosch-si.com | marc.pellmann@xxxxxxxxxxxx

Sitz: Berlin, Registergericht: Amtsgericht Charlottenburg; HRB 148411 B
Aufsichtsratsvorsitzender: Dr.-Ing. Thorsten Lücke; Geschäftsführung: Dr.-Ing. Rainer Kallenbach, Michael Hahn





_______________________________________________
hono-dev mailing list
hono-dev@xxxxxxxxxxx
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://dev.eclipse.org/mailman/listinfo/hono-dev



-- 
Mit freundlichen Grüßen / Best regards
 
Kai Hudalla
Chief Software Architect
 
Bosch Software Innovations GmbH
Schöneberger Ufer 89-91
10785 Berlin
GERMANY
www.bosch-si.com
 
Registered Office: Berlin, Registration Court: Amtsgericht Charlottenburg; HRB 148411 B
Chairman of the Supervisory Board: Dr.-Ing. Thorsten Lücke; Managing Directors: Dr.-Ing. Rainer Kallenbach, Michael Hahn
 


_______________________________________________
hono-dev mailing list
hono-dev@xxxxxxxxxxx
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://dev.eclipse.org/mailman/listinfo/hono-dev

-- 
Mit freundlichen Grüßen / Best regards

Kai Hudalla
Chief Software Architect

Bosch Software Innovations GmbH
Schöneberger Ufer 89-91
10785 Berlin
GERMANY
www.bosch-si.com

Registered Office: Berlin, Registration Court: Amtsgericht Charlottenburg; HRB 148411 B
Chairman of the Supervisory Board: Dr.-Ing. Thorsten Lücke; Managing Directors: Dr.-Ing. Rainer Kallenbach, Michael Hahn


Back to the top