Hi guys,
I would like to request some advice from you. I've been reading a lot about MQTT and all technologies around it in the last year.
We (three people) are trying to create an small company based on a product we're developing. It is a mobile app (20%) and a backend (80%) that will be conected using MQTT.
We have some constraints: budget and time (always the same) so we have decided to put it in the market progressively:
1st) Deploying our infrastructure on premise to test the platform with a very reduced number of users (~200), mainly friends and 2nd grade friends :-)
2nd) After that, we have a rack with three medium-size proliant servers we want to use. We expect to use them to host our infrastructure until:
a) The number of users reach a number that makes the bussines sustainable from an economic point of view (~100K). No matter if it is profitable at this point; we only expect at least that it is not a cost.
or b) Our infrastructure cannot scale more and we have to migrate it to AWS althoug the number of users does not garantees the business sustainability (this would be bad).
3rd) Once reached the number of users to make the platform profitable, we will migrate it to AWS and became a "serious" platform.
All of us have a job so this is a part-time "hobby". This is an attempt to have our company in about 2 years.
During 1st) and 2nd) all costs will be supported by our familiar budget as the platform will be free for all users. Once in 3rd) we expect to start charging a fee; this is the moment when we will be really a company.
For that reasons, we want to "assure" that our infrastructure can support an increasing number of users until 100K before we start charging a fee.
Our base is composed by:
- Android app + MQTT paho.
- Mosquitto 1.3.5 (1.4 next week) + mosquitto-auth-plug
- Haproxy load balancer between clients and Mosquitto.
- Haproxy load balancer between Mosquitto and the backend.
- A Java backend highly threaded.
- A MySQL cluster in master-master configuration.
- HirakiCP as connection pool to the MySQL cluster.
- Haproxy between the backend and the MySQL cluster.
- All running on Ubuntu Server 14.04.1 hosts that have been tuned to improve performance and scalability (the number of connections allowed, the parameters for TCP, number of sockets...).
- The messages we are sending through Mosquitto are API messages (client-backend, backend-client and client-client). We need TLS+QoS2+Persistence.
We are designing API messages very compact and taking advantages of
topics to avoid transferring unneeded data (less bandwith, less ram,
less processing). Also, we are designing the application to minimize the
number of messages that need to be exchanged and the sending rate, Applications I/O to disk is being progressively monimized, too.
Really, everything is redundant; everything except the broker. We
have tried some bridge configuration but they don't scale so it is not
an option, but a temporary patch. After all, we still need a solution to make Mosquitto scale. Notice that 100K users does not mean
that all of them will be connected concurrently.
