Rest API and Multi-node Hazelcast cluster - Created Room not propagated to other nodes
Description
This is a bug in the REST API's create MUC Room service, when Hazelcast clustering is enabled, and there is more than one Openfire/Hazelcast node in the cluster.
Hello, I've started performance testing various aspects of Openfire 4.2.3, with a three node hazelcast cluster (plugin 2.3.0), and the REST API. We're using JMeter, some custom user-simulation bots (implemented with Smack), and custom applications using Strophe.js.
I find that when we do rapid sequences of Create User, Create Room, Connect via XMPP, Join room via XMPP, often we get failures on trying to join the room. Doing approximately, 200 of those sequences, we sometimes see failure rates as high as 66% in JMeter.
The failure seems to come in when a MUCRoom is created via REST. I can immediately be joined via XMPP Stanzas on the node where the REST API call was received. However, if a user tries to join the MUC Room from one of the other two nodes in the cluster, it will fail. Upon an attempt to join the room, it gives a 404, item-not-found error. Interestingly, I have found that when the newly created room is then updated via the Openfire web admin, that then causes the room to be available on the other nodes.
In the case of creating a room, with multi-node clustering enabled, sending a RoomAvailableEvent before the RoomUpdatedEvent seems to resolve this issue.
Below is an excerpt and then a snippet which appears to fix the issue.
This is a bug in the REST API's create MUC Room service, when Hazelcast clustering is enabled, and there is more than one Openfire/Hazelcast node in the cluster.
Hello, I've started performance testing various aspects of Openfire 4.2.3, with a three node hazelcast cluster (plugin 2.3.0), and the REST API. We're using JMeter, some custom user-simulation bots (implemented with Smack), and custom applications using Strophe.js.
I find that when we do rapid sequences of Create User, Create Room, Connect via XMPP, Join room via XMPP, often we get failures on trying to join the room. Doing approximately, 200 of those sequences, we sometimes see failure rates as high as 66% in JMeter.
The failure seems to come in when a MUCRoom is created via REST. I can immediately be joined via XMPP Stanzas on the node where the REST API call was received. However, if a user tries to join the MUC Room from one of the other two nodes in the cluster, it will fail. Upon an attempt to join the room, it gives a 404, item-not-found error. Interestingly, I have found that when the newly created room is then updated via the Openfire web admin, that then causes the room to be available on the other nodes.
In the case of creating a room, with multi-node clustering enabled, sending a RoomAvailableEvent before the RoomUpdatedEvent seems to resolve this issue.
Below is an excerpt and then a snippet which appears to fix the issue.
https://github.com/igniterealtime/Openfire/blob/5cc28a0b781b75cca75b1b0728bfe8a7408ff723/src/plugins/restAPI/src/java/org/jivesoftware/openfire/plugin/rest/controller/MUCRoomController.java#L277
createRoom - addition of RoomAvailableEvent appears to fix this problem