Real-time driver state detection A vision system for controlling an industrial robot with hand movements Integration of a camera and virtual reality with a robotic arm Application of computer vision in the recognition and interpretation of sign letters Mapping of characteristic points of the human face using machine learning and Blender software Six-legged robotic insect Two-way communication based on the MQTT protocol within the virtual and real environment

Two way communication based on the mqtt protocol within the virtual and real environment

Student: Luka Kolar

In this masters thesis, two-way communication based on the MQTT protocol is implemented to connect events from both the real and virtual environments. Using tools provided by the Unreal Engine 5 program, a virtual version of the Laboratory for Manufacturing and Assembly Systems Planning is set up and devices within the space equipped with sensors for monitoring changes, primarily lighting and a refrigerator, are modeled. Furthermore, using the MQTT plugin and the Blueprint programming system within Unreal Engine 5, the logic for device animation in the virtual environment and the exchange of MQTT messages between the devices and their virtual counterparts is defined. This results in a fully functional digital twin model of the laboratory, allowing monitoring and control of the real environment's state. Additionally, the existing Internet of Things solution is integrated with the Home Assistant platform, where the communication of devices in the real environment for remote control is defined.

Repository: https://repozitorij.fsb.unizg.hr/islandora/object/fsb%3A10948