Re: [mosquitto-dev] Advice needed (long, sorry)

[Manuel Domínguez Dorado <manolodd@xxxxxxxxx>]

Hi all,

I'm still on my challenge to achieve 100K concurrent connections. I'm learning lots of things :-)

This time we used a set of 5 VM each one of them bombing the Mosquitto broker, creating idle connections (no susbcriptions, no publishings).

With this configuration we achieved "only" 31K concurrent connections. We decided to change the server certificate, that was using 4096 bits, and create a new one using 1024 that should consume less resources. After this change,

this time we achieved up to 40K connections. it was still possible to create new connections to the broker, but connections already established started to be disconnected because no keepalive was received. We changed keepalive value to a higher value and this time we obtained 52K concurrent connections. After this number, the vistualization server we were using to bomb the broker become exhausted, reason why the number of concurrent connections is not greater.

After this experiment, Roger, we have some suggestions:

- Is there any way of making OpenSSL lib to use more than a single processor even being Mosquitto single threaded? It could be interesting as in our case the broker's server was almost unused except for the processor being used by Mosquitto (something similar to use "openssl speed -multi 4")

- Keepalive mesages are not being sent properly when this number of establishes connections is maintained. I suppose this is for the same reason that the processor used by Mosquitto is exhausted and cannot process this amount of keepalive messages on time).

In general, It would be desirable to implement muti-threading in Mosquitto, because it is a bit extrange to have a 8 core server almost idle and a Mosquitto running on it completely overloaded. Today the trend in new processors is not to have a greater frequency but a greater number of cores, so it seems logic to make use of them. Is there a plan to develop a multi-threaded Mosquitto? The possibility of doing TLS offloading over the "free" processors would be great, too.

Ok, I'll continue posting here our advances.

Kindest regards.

2015-03-04 8:55 GMT+01:00 Manuel Domínguez Dorado <manolodd@xxxxxxxxx>:

Hi, 

yes we started doing client tests using Java. It has no effect on the broker side, but on the client side, the test die at ~2K concurrent connections due to 

java.lang.OutOfMemoryError (1/10 of connections achieved from nodejs).  But it is a matter of memory resources. In real conditions neither the client nor the backend will support this great number of MQTT connections. Only the broker and the haproxy will support them.

We will repeat the test tuning the memory used by nodejs and using several VM clones to start connecting to the broker.

Ciao

2015-03-04 7:47 GMT+01:00 Stefano Costa <stefano.costa@xxxxxxxxxxx>:

Il 03/03/2015 23:18, Manuel Domínguez Dorado ha scritto:

Hi,

- Desktop PC (4 cores, 8 GB RAM) runing a single nodejs test that
establishes TLS connections to the Mosquitto broker at 10 new
connections/sec.

Again, did you try with anything different from nodejs at client side? Python etc? (should not make it any different anyway)

--

---
Manuel Domínguez Dorado
ingeniero ARROBA ManoloDominguez PUNTO com
http://www.ManoloDominguez.com

--

---
Manuel Domínguez Dorado
ingeniero ARROBA ManoloDominguez PUNTO com
http://www.ManoloDominguez.com

Attachment: BrokerLoad.png
Description: PNG image

Attachment: VirtualizationServerClientsBombing.png
Description: PNG image

Attachment: BrokerActiveConnections.png
Description: PNG image

Attachment: BrokerConnectionsCount.png
Description: PNG image