Pengendalian Lengan Robot 6-DOF yang Efisien Bandwidth melalui Pengenalan Gerakan Tangan Berbasis Koordinat

Didik Sukoco; Catur Rakhmad Handoko; Joessianto Eko Poetro

doi:10.56211/blendsains.v4i4.1650

Authors

Didik Sukoco Politeknik Perkapalan Negeri Surabaya
Catur Rakhmad Handoko Politeknik Perkapalan Negeri Surabaya
Joessianto Eko Poetro Politeknik Perkapalan Negeri Surabaya

DOI:

https://doi.org/10.56211/blendsains.v4i4.1650

Keywords:

Komunikasi Berbandwidth Rendah; Lengan Robot 6-DOF; MediaPipe; Pengenalan Gerakan Tangan; Teleoperasi; Visualisasi Skeletal

Abstract

Sistem teleoperasi konvensional sering mengalami latensi tinggi dan kebutuhan bandwidth yang besar akibat ketergantungan yang tinggi pada streaming video real-time. Artikel ini mengusulkan kerangka kerja teleoperasi berbandwidth rendah untuk pengendalian lengan robot 6-DOF menggunakan pengenalan gerakan tangan berbasis koordinat. Sistem ini menggunakan kerangka kerja MediaPipe untuk mendeteksi 21 titik landmark tangan, yang kemudian dipetakan ke parameter kontrol esensial: posisi Cartesian (X, Y, Z), rotasi, jari-jari, dan panjang lengan. Berbeda dengan metode tradisional, protokol kami hanya mentransmisikan paket koordinat berukuran 24 byte, sehingga secara signifikan mengurangi beban jaringan. Untuk menjaga kesadaran situasional operator, visualisasi kerangka tulang real-time diimplementasikan sebagai umpan balik utama, bukan video berbitrate tinggi. Hasil eksperimen simulasi menunjukkan bahwa sistem mencapai latensi end-to-end di bawah 50 ms dan akurasi posisi 98,4%, sambil mengonsumsi bandwidth jaringan hanya 6,05 kbps. Ini mewakili pengurangan bandwidth lebih dari 95% dibandingkan sistem berbasis video standar. Temuan ini menunjukkan bahwa paradigma berbasis koordinat yang diusulkan sangat efektif untuk menerapkan aplikasi robotik di lingkungan dengan konektivitas tidak stabil atau terbatas, seperti bedah jarak jauh dan tanggap bencana.

Downloads

Download data is not yet available.

References

[1] P. Farajiparvar and A. Pandya, “A Brief Survey of Telerobotic Time Delay Mitigation,” Front. Robot. AI, 2020.

[2] R. Das, “A review on tele-manipulators for remote diagnostic procedures and surgery,” CSI Trans. ICT, 2023.

[3] B. M. Rianu, A. Kitagawa, W. Dirgantara, and Subairi, “Hand Gesture Recognition-Based Control System Using Computer Vision for Transporter Robot,” in Proceedings of the IEEE International Conference on Robotics, Automation and Mechatronics (ICRAMET), Nov. 2023.

[4] C. Thormann, M. Matour, and A. W. Winkler, “Gesture Control of a Lightweight Industrial Robot Supported by Augmented Reality,” in Proceedings of the IEEE International Conference on Methods and Models in Automation and Robotics (MMAR), Aug. 2023.

[5] H. Kawasaki, “Decentralized adaptive coordinated control of multiple robot arms,” Automatica, 2006.

[6] Y. Nakamura and H. Hanafusa, “Inverse kinematic solutions with singularity robustness for robot manipulator control,” J. Dyn. Syst. Meas. Control, 1986.

[7] S. Kucuk and Z. Bingul, “The inverse kinematics solutions of industrial robot manipulators,” presented at the Conference on Mechatronics, 2004.

[8] J. Chen, “Development of a user experience enhanced teleoperation approach,” presented at the International conference on collaboration technologies and systems, 2016.

[9] M. Laghi, “Unifying bilateral teleoperation and tele-impedance for enhanced user experience,” Int. J. Robot. Res., 2020.

[10] J. Du and others, “Sensory manipulation as a countermeasure to robot teleoperation delays,” Sci. Rep., vol. 14, Feb. 2024.

[11] D. Jin and others, “Mitigating Latency Effects on Subjective Experience in Robot Teleoperation Using a VR-Enabled Virtual Spring,” in Proceedings of the IEEE International Symposium on Mixed and Augmented Reality (ISMAR), Oct. 2024.

[12] C. C. Nguyen, “Advanced user interfaces for teleoperated surgical robotic systems,” Adv. Sens. Energy Mater., 2023.

[13] G. Rudd, “Intuitive gesture-based control system with collision avoidance for robotic manipulators,” Ind. Robot Int. J. Robot. Res. Appl., 2020.

[14] J. S. Han, “A study on real-time hand gesture recognition technology by machine learning-based mediapipe,” J. Syst. Manag. Sci., 2022.

[15] Y. Zhou, “A novel finger and hand pose estimation technique for real-time hand gesture recognition,” Pattern Recognit., 2016.

[16] Q. Gao and others, “An Efficient RGB-D Hand Gesture Detection Framework for Dexterous Robot Hand-Arm Teleoperation System,” IEEE Trans. Hum.-Mach. Syst., Feb. 2023.

[17] Y. Qin and others, “AnyTeleop: A General Vision-Based Dexterous Robot Arm-Hand Teleoperation System,” in Proceedings of the Robotics: Science and Systems (RSS), Jul. 2023.

[18] V. D. Cong, “Control the Robot Arm through Vision-Based Human Hand Tracking,” FME Trans., 2024.

[19] M. Cognolato, “Improving robotic hand prosthesis control with eye tracking and computer vision,” Front. Artif. Intell., 2022.

[20] W. Rhee and others, “Unconstrained lightweight control interface for robot-assisted minimally invasive surgery using MediaPipe,” Virtual Real., May 2024.

[21] B. M. Santos, G. Gonçalves, and V. H. Pinto, “Shadow Dexterous Hand Control via MediaPipe and BioIK Integration,” in Proceedings of the IEEE International Conference on Autonomous Robot Systems and Competitions (ICARSC), Apr. 2025.

[22] H. H. Li and C. C. Hsieh, “Dynamic Hand Gesture Recognition Using MediaPipe and Transformer,” presented at the Engineering

Proceedings, 2025.

[23] M. Harris and A. S. Agoes, “Applying hand gesture recognition for user guide application using MediaPipe,” presented at the 2nd International Seminar of Science and Applied Technology, 2021.

[24] G. Sung and others, “On-device Real-time Hand Gesture Recognition,” ArXiv Prepr. ArXiv211100038, Oct. 2021.

[25] H. Wu and others, “An interactive gesture control system for collaborative manipulator based on Leap Motion Controller,” Adv. Mech. Eng., May 2024.