Swarm Intelligence is the collective behaviour of many individuals toward a certain task, and each operates autonomously and in a decentralised manner. This concept was inspired by the biological behaviours of some animals, such as ants and bees, where it ensures the process of solving problems more efficiently through cooperation and division of labour, which is being modified and infused into swarm robotic technology.

As the world is heading towards a future where automated robotic agents will be carrying out complex tasks without human intervention, research on improving the collective intelligent behaviours of robots has gained significant importance. Experimenting with a swarm of physical robots is challenging since swarm intelligence experiments demand a large number of robotic agents, typically prohibitively expensive, requiring special hardware and a well-controlled environment.

As an alternative, computer-based simulations with virtual agents have been predominantly used in the literature. Simulations are easier to set up, less expensive, typically faster, and more convenient to use than physical swarms. The two main advantages of virtual robot simulators are the flexibility to add new features and efficiency. However, the persistent issue with the pure virtual approach is that the simulators cannot always represent the intricacies in real-world experiments, how they react to complex physics, noisy sensor data, control loop delays, etc.

This research aims to introduce a mixed reality framework by merging the two realities and modelling swarm behaviours in such a way that both real and virtual robotic agents, as well as the environmental elements, can co-exist and interact with each other. In this approach, all real agents and environmental elements in the system are simultaneously represented in virtual reality, while the simulated environment may contain various additional elements that are purely virtual. Thus, this paves the way for creating safer and lower-risk environments for extensive testing of swarm robotics behaviours and an unrestricted means of achieving agent and environment expansions that are physically impossible or expensive in reality.

As the outcome of this study, we introduced a modular and customisable framework with a set of protocols and procedures. It can be followed to conduct swarm behavioural experiments with both real and virtual robots simultaneously in the same environment with different physical and virtually defined environments using inter-reality sensing and communication.

Keywords: Swarm Robotics, Mixed Reality, Robot Simulation Framework