Enhancing facial expression recognition: leveraging mobilnetv3 for periocular analysis

Ramadhan, Sinar Bayu (2024) Enhancing facial expression recognition: leveraging mobilnetv3 for periocular analysis. Masters thesis, Universitas Pelita Harapan.

[thumbnail of Title] Text (Title)
1.[FINAL] File Tesis - Sinar Bayu Ramadhan-Ti_SINAR BAYU RAMADHAN.pdf
Available under License Creative Commons Attribution Non-commercial Share Alike.

Download (150kB)
[thumbnail of Abstract] Text (Abstract)
2.[FINAL] File Tesis - Sinar Bayu Ramadhan-Ab_SINAR BAYU RAMADHAN.pdf
Available under License Creative Commons Attribution Non-commercial Share Alike.

Download (370kB)
[thumbnail of ToC] Text (ToC)
3.[FINAL] File Tesis - Sinar Bayu Ramadhan-To_SINAR BAYU RAMADHAN.pdf
Available under License Creative Commons Attribution Non-commercial Share Alike.

Download (507kB)
[thumbnail of Chapter 1] Text (Chapter 1)
4.[FINAL] File Tesis - Sinar Bayu Ramadhan-Ch_SINAR BAYU RAMADHAN.pdf
Available under License Creative Commons Attribution Non-commercial Share Alike.

Download (1MB)
[thumbnail of Chapter 2] Text (Chapter 2)
5.[FINAL] File Tesis - Sinar Bayu Ramadhan-Ch_SINAR BAYU RAMADHAN.pdf
Restricted to Registered users only
Available under License Creative Commons Attribution Non-commercial Share Alike.

Download (1MB)
[thumbnail of Chapter 3] Text (Chapter 3)
6.[FINAL] File Tesis - Sinar Bayu Ramadhan-Ch_SINAR BAYU RAMADHAN.pdf
Restricted to Registered users only
Available under License Creative Commons Attribution Non-commercial Share Alike.

Download (1MB)
[thumbnail of Chapter 4] Text (Chapter 4)
7.[FINAL] File Tesis - Sinar Bayu Ramadhan-Ch_SINAR BAYU RAMADHAN.pdf
Restricted to Registered users only
Available under License Creative Commons Attribution Non-commercial Share Alike.

Download (3MB)
[thumbnail of Chapter 5] Text (Chapter 5)
8.[FINAL] File Tesis - Sinar Bayu Ramadhan-Ch_SINAR BAYU RAMADHAN.pdf
Restricted to Registered users only
Available under License Creative Commons Attribution Non-commercial Share Alike.

Download (290kB)
[thumbnail of Bibliography] Text (Bibliography)
10.[FINAL] File Tesis - Sinar Bayu Ramadhan-B_SINAR BAYU RAMADHAN.pdf
Available under License Creative Commons Attribution Non-commercial Share Alike.

Download (825kB)
[thumbnail of Appendices] Text (Appendices)
11.[FINAL] File Tesis - Sinar Bayu Ramadhan_A_SINAR BAYU RAMADHAN.pdf
Restricted to Repository staff only
Available under License Creative Commons Attribution Non-commercial Share Alike.

Download (2MB)

Abstract

With the increasing use of online meeting and virtual reality (VR) with all its limitations, there is a growing need for new methods to understand user behavior and emotions. Traditional methods for analyzing user emotions often rely on facial expressions, typically focusing on the entire face. On the other hand, recent research has begun to highlight the significance of the periocular area as a valuable source of information for this purpose. This study explores the effectiveness of utilizing the periocular area for facial expression classification through neural network architectures, including variants of MobileNet and EfficientNet. Training was conducted using the TFEID dataset, and evaluation was performed using the Chinese Face dataset installed on mobile devices. This research employed a transfer learning approach to assess the performance of architectures on Android platforms. Metrics such as accuracy, precision, recall, and F1-score were used to evaluate the effectiveness of each model, revealing the potential of periocular features in enhancing facial expression classification. The findings of this study demonstrate the robustness of MobileNet architectures in this domain, contributing to the field of expression recognition systems. This research not only enhances understanding of user expressions in virtual environments but also offers practical solutions for real-world applications, paving the way for more effective interaction models in online and VR settings. / Seiring meningkatnya penggunaan ruang pertemuan daring dan virtual reality (VR) dengan segala keterbatasannya, memunculkan adanya kebutuhan baru akan metode yang dapat digunakan untuk memahami perilaku dan emosi pengguna. Metode tradisional untuk menganalisis emosi pengguna sering kali bergantung pada ekspresi wajah, yang biasanya berfokus pada seluruh wajah. Di sisi lain, penelitian baru-baru ini mulai menyoroti pentingnya area periokular sebagai sumber informasi penting untuk kebutuhan tersebut. Penelitian ini mengeksplorasi keefektifan pemanfaatan area periokular untuk klasifikasi ekspresi wajah melalui arsitektur jaringan saraf, antara lain varian MobileNet dan EfficientNet. Pelatihan dilakukan dengan memanfaatkan dataset TFEID, dan evaluasi dilakukan dengan menggunakan dataset Chinese Face yang dipasang pada perangkat seluler. Penelitian ini menggunakan pendekatan transfer learning untuk menilai kinerja arsitektur pada platform Android. Metrik seperti akurasi, presisi, recall, dan F1-score digunakan untuk mengevaluasi keefektifan setiap model, yang mengungkap potensi fitur periokular dalam meningkatkan klasifikasi ekspresi wajah. Temuan dalam penelitian ini menunjukkan ketangguhan arsitektur MobileNet dalam domain ini, memberikan kontribusi dalam bidang sistem pengenalan ekspresi. Penelitian ini tidak hanya meningkatkan pemahaman tentang ekspresi pengguna di lingkungan virtual, tetapi juga menawarkan solusi praktis untuk aplikasi dunia nyata, membuka jalan bagi model interaksi yang lebih efektif dalam pengaturan ruang daring dan VR.
Item Type: Thesis (Masters)
Creators:
Creators
NIM
Email
ORCID
Ramadhan, Sinar Bayu
NIM01679220015
UNSPECIFIED
UNSPECIFIED
Contributors:
Contribution
Contributors
NIDN/NIDK
Email
Thesis advisor
Hareva, David Habsara
NIDN0316037206
UNSPECIFIED
Uncontrolled Keywords: Periokular ; MobileNet ; EfficientNetV2 ; TFEID ; Chinese Face ; Pengenalan Ekspresi
Subjects: T Technology > TK Electrical engineering. Electronics Nuclear engineering > TK7885-7895 Computer engineering. Computer hardware
Divisions: University Subject > Current > Faculty/School - UPH Karawaci > School of Information Science and Technology > Master of Informatics
Current > Faculty/School - UPH Karawaci > School of Information Science and Technology > Master of Informatics
Depositing User: Phillips Iman Heri Wahyudi
Date Deposited: 22 Feb 2025 06:53
Last Modified: 22 Feb 2025 06:53
URI: http://repository.uph.edu/id/eprint/67182

Actions (login required)

View Item
View Item