{"id":23308,"date":"2024-12-15T15:39:56","date_gmt":"2024-12-15T08:39:56","guid":{"rendered":"https:\/\/stei.itb.ac.id\/?page_id=23308"},"modified":"2024-12-16T08:35:53","modified_gmt":"2024-12-16T01:35:53","slug":"generative-language-model-in-intelligent-tutoring-system-for-english-learning-eksperimen-pemanfaatan-generative-language-model-dalam-intelligent-tutoring-system-untuk-pembelajaran-bahasa-inggris","status":"publish","type":"page","link":"https:\/\/stei.itb.ac.id\/en\/prima\/generative-language-model-in-intelligent-tutoring-system-for-english-learning-eksperimen-pemanfaatan-generative-language-model-dalam-intelligent-tutoring-system-untuk-pembelajaran-bahasa-inggris\/","title":{"rendered":"Generative Language Model in Intelligent Tutoring System for English Learning (Experiment on Utilizing Generative Language Model in Intelligent Tutoring System for English Learning)"},"content":{"rendered":"<div class=\"wpb-content-wrapper\"><div class=\"fullwidth\" ><div class=\"vc_row wpb_row vc_row-fluid kepala vc_custom_1734236300248 vc_row-has-fill\"><div class=\"wpb_column vc_column_container vc_col-sm-12\"><div class=\"vc_column-inner\"><div class=\"wpb_wrapper\"><div class=\"container\" ><div class=\"vc_row wpb_row vc_inner vc_row-fluid\"><div class=\"wpb_column vc_column_container vc_col-sm-12\"><div class=\"vc_column-inner\"><div class=\"wpb_wrapper\"><div class=\"vc_btn3-container vc_btn3-inline vc_do_btn\" ><button class=\"vc_general vc_btn3 vc_btn3-size-lg vc_btn3-shape-rounded vc_btn3-style-modern vc_btn3-icon-left vc_btn3-color-white\" onclick=\"history.back()\"><i class=\"vc_btn3-icon fas fa-home\"><\/i> Back to Home<\/button><\/div><\/div><\/div><\/div><\/div><\/div><\/div><\/div><\/div><\/div><\/div><div class=\"fullwidth\" ><div class=\"vc_row wpb_row vc_row-fluid vc_custom_1734170418007\"><div class=\"wpb_column vc_column_container vc_col-sm-3\"><div class=\"vc_column-inner\"><div class=\"wpb_wrapper\">\n\t<div class=\"wpb_text_column wpb_content_element\" >\n\t\t<div class=\"wpb_wrapper\">\n\t\t\t<p><strong>Ayu Purwarianti<\/strong><br \/>\nSTEI ITB<\/p>\n\n\t\t<\/div>\n\t<\/div>\n<\/div><\/div><\/div><div class=\"wpb_column vc_column_container vc_col-sm-3\"><div class=\"vc_column-inner\"><div class=\"wpb_wrapper\"><\/div><\/div><\/div><div class=\"wpb_column vc_column_container vc_col-sm-3\"><div class=\"vc_column-inner\"><div class=\"wpb_wrapper\"><\/div><\/div><\/div><div class=\"wpb_column vc_column_container vc_col-sm-3\"><div class=\"vc_column-inner\"><div class=\"wpb_wrapper\"><\/div><\/div><\/div><\/div><\/div><div class=\"fullwidth\" ><div class=\"vc_row wpb_row vc_row-fluid vc_custom_1734168902943\"><div class=\"wpb_column vc_column_container vc_col-sm-12\"><div class=\"vc_column-inner\"><div class=\"wpb_wrapper\">\n\t<div class=\"wpb_text_column wpb_content_element\" >\n\t\t<div class=\"wpb_wrapper\">\n\t\t\t<p><strong>Abstract<\/strong><br \/>\nIn global era such as now, English language capability is an important skill for existing and future workforce in Indonesia. This phenomena drives the availability of various commercial English language learning tools, which is not affordable for lower-middle Indonesia economic groups. In NLP (natural language processing), people start using GLM (generative language model) to learn language, especially for English language.<\/p>\n<p>Our research combines the usage of GLM and common message application available on handphone to make the English language learning affordable for most people in Indonesia. We collected English learning materials and exams as the resource for our Intelligent Tutoring System for English learning. We\u2019ve built and deployed the modules where user can access it by Telegram message application. We also conducted user survey to analyze people opinion on our English learning module.<\/p>\n<p>Survey results indicate the following quantitative values: SUS score of 90, SEQ score of 6.89, a helpfulness rating of 4.39 out of 5, a friendliness rating of 4.61, and a conciseness score of 4.00. Although these numbers show that the chatbot greatly meets user needs, there is a room for improvement in content presentation, relevance of example materials, time needed to generate response in conversation, and language usage in some sections.<br \/>\n<strong>Keyword<\/strong>: English language learning, Intelligent Tutoring System, Generative Language Model.<\/p>\n<p><strong>Introduction<\/strong><br \/>\nEnglish proficiency is essential for communication, active participation in the global environment, and enhancing understanding in various fields, which are often available in English-language literature. However, according to data from Education First in 2022, Indonesia ranked 81st out of 111 countries surveyed on English proficiency, scoring 469 out of a total score of 800. This indicates that Indonesia still has a low level of English proficiency.<\/p>\n<p>With the rapid advancement of technology, many applications can be utilized to address this issue. One such application is the Intelligent Tutoring System (ITS). An Intelligent Tutoring System is a computer system designed to provide customized instruction or feedback to learners without requiring a teacher&#8217;s intervention (Psotka and Mutter, 1988).<\/p>\n<p>The ITS application for English language learning utilizes GLM to build a chatbot that can serve as an assistant and tutor for learners. The chatbot can function as a provider of materials, exercises, or a daily conversation partner to help learners improve their English skills.<\/p>\n<p>This research is conducted to analyze, develop, and experiment with GLM models in ITS for English language learning.<\/p>\n<p><strong>Research Method<\/strong><br \/>\nBelow is the description of each phase in our researches:<br \/>\n1. Application flow design<br \/>\nWe designed the English language learning application flow to be implemented in our module on Telegram, such as shown in Picture 1<\/p>\n<p>2. Data Preparation<br \/>\nWe collected learning materials from 55 websites on English grammar and vocabulary topics. The quality of the collected English learning article data was evaluated through article categorization, material categorization, and data selection. The final result consisted of 95 articles grouped into five levels of difficulty: beginner, elementary, pre- intermediate, intermediate, and upper- intermediate.<\/p>\n\n\t\t<\/div>\n\t<\/div>\n<\/div><\/div><\/div><\/div><\/div><div class=\"fullwidth\" ><div class=\"vc_row wpb_row vc_row-fluid vc_custom_1734169162466 vc_row-o-content-bottom vc_row-flex\"><div class=\"wpb_column vc_column_container vc_col-sm-12\"><div class=\"vc_column-inner\"><div class=\"wpb_wrapper\">\n\t<div  class=\"wpb_single_image wpb_content_element vc_align_center wpb_content_element\">\n\t\t\n\t\t<figure class=\"wpb_wrapper vc_figure\">\n\t\t\t<div class=\"vc_single_image-wrapper   vc_box_border_grey\"><img loading=\"lazy\" decoding=\"async\" width=\"210\" height=\"253\" src=\"https:\/\/stei.itb.ac.id\/wp-content\/uploads\/43a.png\" class=\"vc_single_image-img attachment-full\" alt=\"\" title=\"43a\" \/><\/div><figcaption class=\"vc_figure-caption\">Picture 1. User Flowchart of ITS<\/figcaption>\n\t\t<\/figure>\n\t<\/div>\n\n\t<div  class=\"wpb_single_image wpb_content_element vc_align_center wpb_content_element\">\n\t\t\n\t\t<figure class=\"wpb_wrapper vc_figure\">\n\t\t\t<div class=\"vc_single_image-wrapper   vc_box_border_grey\"><img loading=\"lazy\" decoding=\"async\" width=\"915\" height=\"691\" src=\"https:\/\/stei.itb.ac.id\/wp-content\/uploads\/43b.jpg\" class=\"vc_single_image-img attachment-full\" alt=\"\" title=\"43b\" \/><\/div><figcaption class=\"vc_figure-caption\">Picture 2. Chatbot Greeting and Menus in Telegram<\/figcaption>\n\t\t<\/figure>\n\t<\/div>\n<\/div><\/div><\/div><\/div><\/div><div class=\"fullwidth\" ><div class=\"vc_row wpb_row vc_row-fluid vc_custom_1734168902943\"><div class=\"wpb_column vc_column_container vc_col-sm-12\"><div class=\"vc_column-inner\"><div class=\"wpb_wrapper\">\n\t<div class=\"wpb_text_column wpb_content_element\" >\n\t\t<div class=\"wpb_wrapper\">\n\t\t\t<p>3. Model Development &amp; Evaluation<br \/>\nExperiments were conducted to select an open-source GLM-based model for ITS. Among four GLM model candidates, LLama 3 (7B) was chosen as the main model due to its balance between performance and computational efficiency, fine-tuning capabilities, and ability to provide relevant responses in language learning. The chatbot was implemented on Telegram platform using Flask framework and a PostgreSQL database. The chatbot was then evaluated through user survey using usability testing to assess its alignment with user needs.<\/p>\n<p><strong>Discussion and Result<\/strong><br \/>\nThe implemented chatbot features three main functions: learning materials, solving practice questions, and AI conversation. During usability testing, these menus were used in 9 testing scenarios conducted by 2 respondents. The respondents then provided comments and ratings on the usability (user satisfaction) and conciseness of the chatbot.<\/p>\n<p>Based on the usability testing results, the following quantitative scores were obtained: a SUS score of 90, an SEQ score of 6.89, a helpfulness rating of 4.39 out of 5.00, and a friendliness rating of 4.61 out of 5.00. These scores indicate that the chatbot&#8217;s design meets user needs; however, there is feedback for improving the chatbot&#8217;s implementation. Suggestions include refining the presentation of content by the chatbot, adjusting example sentences in the material to match the difficulty level, reducing response time in AI conversations, and modifying the language that remains stiff in the material and practice menus.<\/p>\n<p>Meanwhile, the chatbot&#8217;s conciseness was rated at 4.00 out of 5.00. The chatbot was deemed capable of providing clear answers to user questions in a concise manner. However, the character limit on Telegram messages resulted in some parts of lengthy responses being cut off, which affected the overall conciseness.<\/p>\n<p><strong>Conclusion<\/strong><br \/>\nIn this study, an ITS module was designed and developed utilizing GLM to assist English language learning in Indonesia. The ITS module was implemented as a chatbot accessible through the Telegram platform, leveraging the LLama 3 (7B) model, the Flask framework with Redis, and a PostgreSQL database. The chatbot is implemented with five menus, including three main features for English language learning: a feature for studying materials, a feature for completing exercises, and a feature for engaging in conversations with AI.<\/p>\n<p>The chatbot was tested using usability testing methods to measure its usability against user needs and to gather comments or suggestions for future development. The testing used quantitative metrics, resulting in an average SUS score of 90, an average SEQ score of 6.89, an average helpfulness score of 4.39 out of 5.00, an average friendliness score of 4.61 out of 5.00, and an average conciseness score of 4.00 out of 5.00. Based on these figures, the chatbot is deemed to meet user needs and provide clear, user-friendly responses. However, these figures are only initial values, and there is still room for improvement. These improvements include presenting content in a more engaging format, tailoring sentence examples in the material to the appropriate level, reducing the necessary time to generate responses in conversation, and using less rigid language in the chatbot&#8217;s material and exercise menus.<\/p>\n\n\t\t<\/div>\n\t<\/div>\n<\/div><\/div><\/div><\/div><\/div><div class=\"fullwidth\" ><div class=\"vc_row wpb_row vc_row-fluid vc_custom_1734169162466 vc_row-o-content-bottom vc_row-flex\"><div class=\"wpb_column vc_column_container vc_col-sm-12\"><div class=\"vc_column-inner\"><div class=\"wpb_wrapper\"><\/div><\/div><\/div><\/div><\/div><div class=\"fullwidth\" ><div class=\"vc_row wpb_row vc_row-fluid vc_custom_1734168902943\"><div class=\"wpb_column vc_column_container vc_col-sm-12\"><div class=\"vc_column-inner\"><div class=\"wpb_wrapper\"><\/div><\/div><\/div><\/div><\/div>\n<\/div>","protected":false},"excerpt":{"rendered":"Kembali ke Beranda Ayu Purwarianti STEI-ITB Abstract In global era such as now, English language capability is an important skill for existing and future workforce in Indonesia. This phenomena drives [...]","protected":false},"author":1,"featured_media":0,"parent":22933,"menu_order":0,"comment_status":"closed","ping_status":"closed","template":"","meta":{"footnotes":""},"class_list":["post-23308","page","type-page","status-publish","hentry"],"_links":{"self":[{"href":"https:\/\/stei.itb.ac.id\/en\/wp-json\/wp\/v2\/pages\/23308","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/stei.itb.ac.id\/en\/wp-json\/wp\/v2\/pages"}],"about":[{"href":"https:\/\/stei.itb.ac.id\/en\/wp-json\/wp\/v2\/types\/page"}],"author":[{"embeddable":true,"href":"https:\/\/stei.itb.ac.id\/en\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/stei.itb.ac.id\/en\/wp-json\/wp\/v2\/comments?post=23308"}],"version-history":[{"count":3,"href":"https:\/\/stei.itb.ac.id\/en\/wp-json\/wp\/v2\/pages\/23308\/revisions"}],"predecessor-version":[{"id":23589,"href":"https:\/\/stei.itb.ac.id\/en\/wp-json\/wp\/v2\/pages\/23308\/revisions\/23589"}],"up":[{"embeddable":true,"href":"https:\/\/stei.itb.ac.id\/en\/wp-json\/wp\/v2\/pages\/22933"}],"wp:attachment":[{"href":"https:\/\/stei.itb.ac.id\/en\/wp-json\/wp\/v2\/media?parent=23308"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}