Real-time driver state detection A vision system for controlling an industrial robot with hand movements Integration of a camera and virtual reality with a robotic arm Application of computer vision in the recognition and interpretation of sign letters Mapping of characteristic points of the human face using machine learning and Blender software Six-legged robotic insect Two-way communication based on the MQTT protocol within the virtual and real environment

A vision system for controlling an industrial robot with hand movements

Student: Matija Zidarić

Powerful cameras and modern artificial intelligence models allow for interesting synthesis and different applications. The focus of this thesis is the development of a vision system that recognizes hand gestures and controls the industrial robot in real time. In the Python programming language, a MediaPipe hands model is used to recognize and track the hand being captured by the Intel RealSense D435 camera, withRoboDK as the intermediary for communication with robots. Due to system noise, digital filtering of coordinates was carried out. Robot grips have been made by 3D printing for the purpose of experimental evaluation. The application was tested on UR5 and Fanuc LR Mate 200ic industrial robots.

Repository: https://repozitorij.fsb.unizg.hr/islandora/object/fsb%3A10907