{"id":23296,"date":"2024-12-15T15:23:40","date_gmt":"2024-12-15T08:23:40","guid":{"rendered":"https:\/\/stei.itb.ac.id\/?page_id=23296"},"modified":"2024-12-16T08:33:01","modified_gmt":"2024-12-16T01:33:01","slug":"happiness-index-measurement-based-on-facenet-based-video-analytics","status":"publish","type":"page","link":"https:\/\/stei.itb.ac.id\/en\/prima\/happiness-index-measurement-based-on-facenet-based-video-analytics\/","title":{"rendered":"Happiness Index Measurement Based on Facenet-based Video Analytics"},"content":{"rendered":"<div class=\"wpb-content-wrapper\"><div class=\"fullwidth\" ><div class=\"vc_row wpb_row vc_row-fluid kepala vc_custom_1734236300248 vc_row-has-fill\"><div class=\"wpb_column vc_column_container vc_col-sm-12\"><div class=\"vc_column-inner\"><div class=\"wpb_wrapper\"><div class=\"container\" ><div class=\"vc_row wpb_row vc_inner vc_row-fluid\"><div class=\"wpb_column vc_column_container vc_col-sm-12\"><div class=\"vc_column-inner\"><div class=\"wpb_wrapper\"><div class=\"vc_btn3-container vc_btn3-inline vc_do_btn\" ><button class=\"vc_general vc_btn3 vc_btn3-size-lg vc_btn3-shape-rounded vc_btn3-style-modern vc_btn3-icon-left vc_btn3-color-white\" onclick=\"history.back()\"><i class=\"vc_btn3-icon fas fa-home\"><\/i> Kembali ke Beranda<\/button><\/div><\/div><\/div><\/div><\/div><\/div><\/div><\/div><\/div><\/div><\/div><div class=\"fullwidth\" ><div class=\"vc_row wpb_row vc_row-fluid vc_custom_1734170418007\"><div class=\"wpb_column vc_column_container vc_col-sm-4\"><div class=\"vc_column-inner\"><div class=\"wpb_wrapper\">\n\t<div class=\"wpb_text_column wpb_content_element\" >\n\t\t<div class=\"wpb_wrapper\">\n\t\t\t<p><strong>Dr. Fadhil Hidayat, S. Kom., M. T.<\/strong><br \/>\nSTEI ITB<\/p>\n\n\t\t<\/div>\n\t<\/div>\n<\/div><\/div><\/div><div class=\"wpb_column vc_column_container vc_col-sm-4\"><div class=\"vc_column-inner\"><div class=\"wpb_wrapper\">\n\t<div class=\"wpb_text_column wpb_content_element\" >\n\t\t<div class=\"wpb_wrapper\">\n\t\t\t<p><strong>Prof. Dr. Ir. Suhono Harso Supangkat, M. Eng.<\/strong><br \/>\nSTEI ITB<\/p>\n\n\t\t<\/div>\n\t<\/div>\n<\/div><\/div><\/div><div class=\"wpb_column vc_column_container vc_col-sm-4\"><div class=\"vc_column-inner\"><div class=\"wpb_wrapper\">\n\t<div class=\"wpb_text_column wpb_content_element\" >\n\t\t<div class=\"wpb_wrapper\">\n\t\t\t<p><strong>Ir. Budiman Dabarsyah, M.S.EE.<\/strong><br \/>\nSTEI ITB<\/p>\n\n\t\t<\/div>\n\t<\/div>\n<\/div><\/div><\/div><\/div><\/div><div class=\"fullwidth\" ><div class=\"vc_row wpb_row vc_row-fluid vc_custom_1734168902943\"><div class=\"wpb_column vc_column_container vc_col-sm-12\"><div class=\"vc_column-inner\"><div class=\"wpb_wrapper\">\n\t<div class=\"wpb_text_column wpb_content_element\" >\n\t\t<div class=\"wpb_wrapper\">\n\t\t\t<p><strong>Abstract<\/strong><br \/>\nFacial expression recognition, a vital aspect of human communication, is increasingly pivotal in the era of digital advancement. This research presents an innovative approach to facial expression recognition using the Xception model and advanced smoothing algorithms. The model is trained using transfer learning to enhance its accuracy in identifying emotions like happiness, sadness, anger, surprise, disgust, and fear. The integration of smoothing algorithms further improves the system&#8217;s reliability by reducing noise and fluctuations in the detected emotions. The system has shown promising results with an accuracy of 71% and future work aims to improve accuracy and explore its application in customer satisfaction analysis. It is hoped that this system will be able to aiming to improve event experiences and customer engagement in various sectors, such as education, technology, health, etc..<\/p>\n<p><strong>Keyword:<\/strong> Emotion Recognition, Xception, Transfer Learning, Smoothing Algorithms.<\/p>\n<p><strong>Introduction<\/strong><br \/>\nEmotion recognition technology, which interprets human emotions through digital interfaces, is becoming essential in sectors like retail, healthcare, and entertainment to enhance customer engagement and satisfaction. By analyzing facial expressions in real time, businesses can gain valuable insights and tailor their interactions, surpassing traditional feedback methods like surveys. This study explores an emotion recognition system using video analysis to assess customer satisfaction at events, providing immediate feedback and enabling timely adjustments. This technology has the potential to revolutionize customer engagement across various industries.<\/p>\n\n\t\t<\/div>\n\t<\/div>\n<\/div><\/div><\/div><\/div><\/div><div class=\"fullwidth\" ><div class=\"vc_row wpb_row vc_row-fluid vc_custom_1734169162466 vc_row-o-content-bottom vc_row-flex\"><div class=\"wpb_column vc_column_container vc_col-sm-12\"><div class=\"vc_column-inner\"><div class=\"wpb_wrapper\">\n\t<div  class=\"wpb_single_image wpb_content_element vc_align_center wpb_content_element\">\n\t\t\n\t\t<figure class=\"wpb_wrapper vc_figure\">\n\t\t\t<div class=\"vc_single_image-wrapper   vc_box_border_grey\"><img loading=\"lazy\" decoding=\"async\" width=\"496\" height=\"473\" src=\"https:\/\/stei.itb.ac.id\/wp-content\/uploads\/Research-Method.jpg\" class=\"vc_single_image-img attachment-full\" alt=\"\" title=\"Research Method\" \/><\/div><figcaption class=\"vc_figure-caption\">Picture 1. Research Method<\/figcaption>\n\t\t<\/figure>\n\t<\/div>\n<\/div><\/div><\/div><\/div><\/div><div class=\"fullwidth\" ><div class=\"vc_row wpb_row vc_row-fluid vc_custom_1734168902943\"><div class=\"wpb_column vc_column_container vc_col-sm-12\"><div class=\"vc_column-inner\"><div class=\"wpb_wrapper\">\n\t<div class=\"wpb_text_column wpb_content_element\" >\n\t\t<div class=\"wpb_wrapper\">\n\t\t\t<p><strong>Research Method<\/strong><br \/>\nFigure 1 explains the stages of the method used to build the system. The training model uses the FER-2013 dataset. The Xception model, integrated with FaceNet and DLIB, which is well-known for its accuracy and feature extraction capabilities, is used with transfer learning to utilize the pre-trained weights for emotion detection. The model is then used for inference implementation using real-time video input. The system will recognize a person&#8217;s emotions and divide them into three emotion classes.<\/p>\n<p><strong>Discussion &amp; Result<\/strong><br \/>\nThe developed system successfully recognizes facial emotions and divides them into three classes, namely positive, neutral, and negative as shown in Figure 2. The emotion that meets the Positive criteria is happy. The Negative criteria consist of the disgust, sad, fear and angry classes, while the Neutral ones consist of neutral and surprised. The confusion matrix is \u200b\u200bused to evaluate the model on the system as shown in Table 1. The model has a fair good accuracy of 71.1%. This is in line with other evaluation components, including precision of 72.1%, recall of 71.1%, and F1-score of 71.4%. The detection results are then displayed in the dashboard as in Figure 3.<\/p>\n\n\t\t<\/div>\n\t<\/div>\n<\/div><\/div><\/div><\/div><\/div><div class=\"fullwidth\" ><div class=\"vc_row wpb_row vc_row-fluid vc_custom_1734169162466 vc_row-o-content-bottom vc_row-flex\"><div class=\"wpb_column vc_column_container vc_col-sm-12\"><div class=\"vc_column-inner\"><div class=\"wpb_wrapper\">\n\t<div  class=\"wpb_single_image wpb_content_element vc_align_center wpb_content_element\">\n\t\t\n\t\t<figure class=\"wpb_wrapper vc_figure\">\n\t\t\t<div class=\"vc_single_image-wrapper   vc_box_border_grey\"><img loading=\"lazy\" decoding=\"async\" width=\"1280\" height=\"409\" src=\"https:\/\/stei.itb.ac.id\/wp-content\/uploads\/Emotion-Recognition-Result-positive-neutral-sad.jpg\" class=\"vc_single_image-img attachment-full\" alt=\"\" title=\"Emotion Recognition Result (positive, neutral, sad)\" \/><\/div><figcaption class=\"vc_figure-caption\">Picture 2. Emotion Recognition Result (positive, neutral, sad)<\/figcaption>\n\t\t<\/figure>\n\t<\/div>\n<\/div><\/div><\/div><\/div><\/div><div class=\"fullwidth\" ><div class=\"vc_row wpb_row vc_row-fluid vc_custom_1734168902943\"><div class=\"wpb_column vc_column_container vc_col-sm-12\"><div class=\"vc_column-inner\"><div class=\"wpb_wrapper\">\n\t<div class=\"wpb_text_column wpb_content_element\" >\n\t\t<div class=\"wpb_wrapper\">\n\t\t\t<p>A challenge arises when inferring facial expressions due to flickering changes in expression classes. This study implements a smoothing algorithm to make recognition more stable. There are 3 candidate algorithms to be used, namely Kalman Filter, Moving Average, and Median Filter as shown in Figure 4. The results of the graph show changes in recognition classes compared to frames. It can be seen that Moving Average and Median Filter are smoother and more suitable for use than Kalman Filter which is very volatile in recognition class shifts.<\/p>\n\n\t\t<\/div>\n\t<\/div>\n<\/div><\/div><\/div><\/div><\/div><div class=\"fullwidth\" ><div class=\"vc_row wpb_row vc_row-fluid vc_custom_1734169162466 vc_row-o-content-bottom vc_row-flex\"><div class=\"wpb_column vc_column_container vc_col-sm-4\"><div class=\"vc_column-inner\"><div class=\"wpb_wrapper\">\n\t<div  class=\"wpb_single_image wpb_content_element vc_align_center wpb_content_element\">\n\t\t\n\t\t<figure class=\"wpb_wrapper vc_figure\">\n\t\t\t<div class=\"vc_single_image-wrapper   vc_box_border_grey\"><img loading=\"lazy\" decoding=\"async\" width=\"479\" height=\"290\" src=\"https:\/\/stei.itb.ac.id\/wp-content\/uploads\/System-Dashboard.jpg\" class=\"vc_single_image-img attachment-full\" alt=\"\" title=\"System Dashboard\" \/><\/div><figcaption class=\"vc_figure-caption\">Picture 3. System Dashboard<\/figcaption>\n\t\t<\/figure>\n\t<\/div>\n<\/div><\/div><\/div><div class=\"wpb_column vc_column_container vc_col-sm-4\"><div class=\"vc_column-inner\"><div class=\"wpb_wrapper\">\n\t<div  class=\"wpb_single_image wpb_content_element vc_align_center wpb_content_element\">\n\t\t\n\t\t<figure class=\"wpb_wrapper vc_figure\">\n\t\t\t<div class=\"vc_single_image-wrapper   vc_box_border_grey\"><img loading=\"lazy\" decoding=\"async\" width=\"495\" height=\"244\" src=\"https:\/\/stei.itb.ac.id\/wp-content\/uploads\/Evaluation-Matrix.jpg\" class=\"vc_single_image-img attachment-full\" alt=\"\" title=\"Evaluation Matrix\" \/><\/div><figcaption class=\"vc_figure-caption\">Table 1. Evaluation Matrix<\/figcaption>\n\t\t<\/figure>\n\t<\/div>\n<\/div><\/div><\/div><div class=\"wpb_column vc_column_container vc_col-sm-4\"><div class=\"vc_column-inner\"><div class=\"wpb_wrapper\">\n\t<div  class=\"wpb_single_image wpb_content_element vc_align_center wpb_content_element\">\n\t\t\n\t\t<figure class=\"wpb_wrapper vc_figure\">\n\t\t\t<div class=\"vc_single_image-wrapper   vc_box_border_grey\"><img loading=\"lazy\" decoding=\"async\" width=\"527\" height=\"397\" src=\"https:\/\/stei.itb.ac.id\/wp-content\/uploads\/Smoothing-algorithm-comparison.jpg\" class=\"vc_single_image-img attachment-full\" alt=\"\" title=\"Smoothing algorithm comparison\" \/><\/div><figcaption class=\"vc_figure-caption\">Picture 4. Smoothing algorithm comparison (Kalman filter, Moving Average filter, Median filter)<\/figcaption>\n\t\t<\/figure>\n\t<\/div>\n<\/div><\/div><\/div><\/div><\/div><div class=\"fullwidth\" ><div class=\"vc_row wpb_row vc_row-fluid vc_custom_1734168902943\"><div class=\"wpb_column vc_column_container vc_col-sm-12\"><div class=\"vc_column-inner\"><div class=\"wpb_wrapper\">\n\t<div class=\"wpb_text_column wpb_content_element\" >\n\t\t<div class=\"wpb_wrapper\">\n\t\t\t<p><strong>Conclusion<\/strong><br \/>\nThis paper presents a video analytics-based emotion recognition system that combines the feature extraction capabilities of the Xception model with an effective smoothing algorithm to address the flickering challenge of class changes in expression recognition. The model has a fair good accuracy of 71.1%. This is in line with other evaluation components, including precision of 72.1%, recall of 71.1%, and F1-score of 71.4%. Future work will explore the integration of multimodal data sources, such as audio and text, to improve the accuracy and richness of emotion analysis. It is hoped that this system will be able to aiming to improve event experiences and customer engagement in various sectors, such as education, technology, health, etc.<\/p>\n<p><strong>Publication<\/strong><br \/>\n\u2022 Face Recognition for Automatic Border Control: A Systematic Literature Review Journal (https:\/\/doi.org\/10.1109\/ACCESS.2024.3373 264)<br \/>\n\u2022 Comparative Analysis of FaceNet and ArcFace in Minimizing False Positives for Enhanced Access Control Security (https:\/\/doi.org\/10.1109\/ICISS62896.2024.1 0750931)<br \/>\n\u2022 Face Morph Detection: A Systematic Review (https:\/\/doi.org\/10.1109\/ICISS55894.2022.9 915233)<\/p>\n\n\t\t<\/div>\n\t<\/div>\n<\/div><\/div><\/div><\/div><\/div>\n<\/div>","protected":false},"excerpt":{"rendered":"Kembali ke Beranda Dr. Fadhil Hidayat, S. Kom., M. T. STEI-ITB Prof. Dr. Ir. Suhono Harso Supangkat, M. Eng. STEI-ITB Ir. Budiman Dabarsyah, M.S.EE. STEI-ITB Abstract Facial expression recognition, a [...]","protected":false},"author":1,"featured_media":0,"parent":22933,"menu_order":0,"comment_status":"closed","ping_status":"closed","template":"","meta":{"footnotes":""},"class_list":["post-23296","page","type-page","status-publish","hentry"],"_links":{"self":[{"href":"https:\/\/stei.itb.ac.id\/en\/wp-json\/wp\/v2\/pages\/23296","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/stei.itb.ac.id\/en\/wp-json\/wp\/v2\/pages"}],"about":[{"href":"https:\/\/stei.itb.ac.id\/en\/wp-json\/wp\/v2\/types\/page"}],"author":[{"embeddable":true,"href":"https:\/\/stei.itb.ac.id\/en\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/stei.itb.ac.id\/en\/wp-json\/wp\/v2\/comments?post=23296"}],"version-history":[{"count":2,"href":"https:\/\/stei.itb.ac.id\/en\/wp-json\/wp\/v2\/pages\/23296\/revisions"}],"predecessor-version":[{"id":23586,"href":"https:\/\/stei.itb.ac.id\/en\/wp-json\/wp\/v2\/pages\/23296\/revisions\/23586"}],"up":[{"embeddable":true,"href":"https:\/\/stei.itb.ac.id\/en\/wp-json\/wp\/v2\/pages\/22933"}],"wp:attachment":[{"href":"https:\/\/stei.itb.ac.id\/en\/wp-json\/wp\/v2\/media?parent=23296"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}