Rancang Bangun Aplikasi Penerjemah Bahasa Isyarat Secara Real-Time Menggunakan Algoritma Convolutional Neural Network (CNN) Berbasis Android

The development of Artificial Intelligence (AI), particularly in the field of computer vision, has created opportunities to develop communication media for individuals with hearing and speech impairments. One of the challenges that still exists is the limited public understanding of Indonesian Sign Language (BISINDO), which hinders effective communication between deaf individuals and the general public. This study aims to design and develop a real-time Android-based Indonesian Sign Language (BISINDO) translation application using the Convolutional Neural Network (CNN) algorithm. The system development method employed in this study is the Prototype model, while gesture classification is performed using a CNN model implemented with TensorFlow Lite, MediaPipe Hand Landmarker, and CameraX on Android devices. The research stages include BISINDO alphabet gesture dataset collection, data preprocessing and augmentation, CNN model training, TensorFlow Lite model conversion, Android application implementation, and system evaluation using Black Box Testing and User Acceptance Testing (UAT). The results of the Black Box Testing indicate that all main application features, including gesture detection, text-to-gesture translation, Text-to-Speech (TTS) conversion, the BISINDO dictionary, translation history, and user interface functions, operated according to the expected functional requirements. The experimental results demonstrate that the CNN model achieved an accuracy of 97%, precision of 97%, recall of 97%, and an F1-score of 97% in recognizing BISINDO alphabet gestures. Furthermore, the User Acceptance Test (UAT), involving 10 respondents consisting of five deaf users and five general users, obtained a satisfaction score of 84.64%, which falls into the Very Good category. These findings indicate that the developed application functions properly, is easy to use, is capable of translating BISINDO gestures into text in real time, and is well accepted by users as a communication support tool.
Keywords: Indonesian Sign Language (BISINDO), Convolutional Neural Network (CNN), Computer Vision, Android, TensorFlow Lite.