In the centralized controlled environment, such as OpenFlow, we need to be prepared for fault tolerance on the controller. This is generally the first thing we need to consider in our deployment. In production network, we need to plan for controller failure or any communication failure between controller and the switch. Prior to OpenFlow 1.3, the orchestration of controllers needed to be done via a third party software or between controllers themselves for negotiating the primary controller. Starting in OpenFlow 1.3, this can be done via the OFPT_ROLE messages between the controller and the switch.
In OpenFlow 1.3, the switch can connect to multiple controllers and the controller's role can be equal, master, or slave. In equal role, it is just as if the switch has two masters, each having full access to the switch. In a slave role, the controller has only read access to the switch, does not receive asynchronous messages, and is denied any flow-mod, packet-out, or other state change messages. In a master role, the controller has full access to the switch; the switch will only have one master controller. In the case of two controllers trying to become the master, the tie-breaker will be on the 64-bit generation ID that is included in the OFPT_ROLE_REQUEST messages; the larger value wins.
To have equal role controllers that a switch connects to, is pretty tricky for coordination. The controllers must be in synchronization with each other to avoid any duplicate flows. So I would recommend having master-slave relationship in your deployment. The number of controllers in a cluster differs from network to network, but it is safe to say that you should have at least one master and one slave. Let's look at an example.