Aplikasi pengenalan karakter BISINDO menggunakan convolutional neural network dengan single shot multibox detector

Yefta, Christopher (2019) Aplikasi pengenalan karakter BISINDO menggunakan convolutional neural network dengan single shot multibox detector. Bachelor thesis, Universitas Pelita Harapan.

[img] Text (Title)
Title.pdf
Available under License Creative Commons Attribution Non-commercial Share Alike.

Download (936kB)
[img] Text (Abstract)
Abstract.pdf
Available under License Creative Commons Attribution Non-commercial Share Alike.

Download (213kB)
[img] Text (ToC)
ToC.pdf
Available under License Creative Commons Attribution Non-commercial Share Alike.

Download (268kB)
[img] Text (Chapter1)
Chapter1.pdf
Available under License Creative Commons Attribution Non-commercial Share Alike.

Download (234kB)
[img] Text (Chapter2)
Chapter2.pdf
Restricted to Registered users only
Available under License Creative Commons Attribution Non-commercial Share Alike.

Download (667kB) | Request a copy
[img] Text (Chapter3)
Chapter3.pdf
Restricted to Registered users only
Available under License Creative Commons Attribution Non-commercial Share Alike.

Download (427kB) | Request a copy
[img] Text (Chapter4)
Chapter4.pdf
Restricted to Registered users only
Available under License Creative Commons Attribution Non-commercial Share Alike.

Download (538kB) | Request a copy
[img] Text (Chapter5)
Chapter5.pdf
Restricted to Registered users only
Available under License Creative Commons Attribution Non-commercial Share Alike.

Download (215kB) | Request a copy
[img] Text (Bibliography)
Bibliography.pdf
Available under License Creative Commons Attribution Non-commercial Share Alike.

Download (212kB)
[img] Text (Appendices)
Appendices.pdf
Restricted to Repository staff only
Available under License Creative Commons Attribution Non-commercial Share Alike.

Download (785kB) | Request a copy

Abstract

BISINDO Character Recognition is a desktop-based application developed to recognize the character of Indonesian Sign Language also known as Bahasa Isyarat Indonesia (BISINDO). The Indonesian Sign Language that can be recognized by the application includes static and dynamic hand gestures that convey alphabets and numeric values. Recognition of hand gestures is done in real-time through a webcam and translation is accomplished through the Convolutional Neural Network (CNN) model. The process of hand gesture recognition by the BISINDO Character Recognition application begins with the phase of detecting the hand by utilizing the Single Shot MultiBox Detector (SSD) model. Then image processing is performed on the image of the detected hand which includes the hand segmentation stage and image enhancement. The image will be input for CNN models that have been trained previously. The dataset used in the training process includes 72 one-minute videos that were extracted into 169.013 images of left and right hand images for a total of 36 BISINDO alphabetical and numerical gesture categories. The study was conducted by measuring the performance of the model in terms of the recognition of hand gestures. The final project testing was carried out on 36 BISINDO character categories with 30 videos each with a frame rate of 25fps per category. The test results show that the application can properly recognize BISINDO hand gestures with a black background and sufficient lighting. The application can recognize 92.2% or 83 from 90 BISINDO dynamic hand gesture videos and 94.8% or 939 from 990 BISINDO static hand gesture videos, so that the average approval generated was 94.6% for all BISINDO hand gesture videos./ Aplikasi Pengenal Karakter BISINDO merupakan aplikasi berbasis desktop yang dibangun untuk mengenali karakter Bahasa Isyarat Indonesia (BISINDO). Bahasa Isyarat Indonesia yang dapat dikenali oleh aplikasi tersebut adalah gerakan tangan statis dan dinamis yang bersifat alphabetikal dan numerikal. Pengenalan karakter BISINDO dilakukan secara real-time melalui webcam dan akan diterjemahkan dengan menggunakan model Convolutional Neural Networks (CNN). Proses pengenalan gerakan tangan oleh aplikasi Pengenal Karakter BISINDO dimulai dengan tahap pendeteksian tangan dengan menggunakan model Single Shot MultiBox Detector (SSD). Pengolahan citra kemudian dilakukan pada gambar tangan yang terdeteksi, yang mencakup tahap segmentasi tangan dan image enhancement. Gambar tersebut akan menjadi input bagi model CNN yang telah dilakukan pelatihan sebelumnya. Dataset yang digunakan dalam proses pelatihan berjumlahkan 72 video berdurasi 1 menit yang diekstrak menjadi 169.013 gambar tangan kanan dan kiri untuk 36 kategori gestur alphabetikal dan numerikal BISINDO. Penelitian dilakukan dengan mengukur performa model dalam pengenalan gerakan tangan. Pengujian tugas akhir ini dilakukan terhadap 36 kategori karakter BISINDO dengan masing-masing 30 video dengan frame rate 25fps per kategori. Hasil pengujian menunjukkan bahwa aplikasi dapat dengan baik melakukan pengenalan gerakan tangan BISINDO dengan warna latar hitam dan pencahayaan yang cukup. Aplikasi tersebut dapat mengenali 92.2% atau 83 dari 90 video gerakan tangan dinamis BISINDO dan 94.8% atau 939 dari 990 video gerakan tangan statis BISINDO, sehingga rata-rata akurasi yang dihasilkan adalah 94.6% untuk seluruh video gerakan tangan BISINDO.

Item Type: Thesis (Bachelor)
Creators:
CreatorsNIMEmail
Yefta, ChristopherNIM00000026157christopheryefta@gmail.com
Contributors:
ContributionContributorsNIDN/NIDKEmail
Thesis advisorYugopuspito, PujiantoNIDN0324086701UNSPECIFIED
Thesis advisorSutrisno, SutrisnoNIDN0331126201UNSPECIFIED
Additional Information: SK 82-16 YEF a
Uncontrolled Keywords: deteksi tangan; Bahasa Isyarat Indonesia (BISINDO); Convolutional Neural Network (CNN); Single Shot MultiBox Detector (SSD)
Subjects: Q Science > QA Mathematics > QA75 Electronic computers. Computer science
Divisions: University Subject > Current > Faculty/School - UPH Karawaci > School of Information Science and Technology > Informatics
Current > Faculty/School - UPH Karawaci > School of Information Science and Technology > Informatics
Depositing User: Users 2765 not found.
Date Deposited: 18 Nov 2019 00:34
Last Modified: 10 Jan 2022 06:55
URI: http://repository.uph.edu/id/eprint/5605

Actions (login required)

View Item View Item