Volume 29 Number 1, 2025 Feasibility of administering chatbot-based speaking assessment for low-level EFL students

Feasibility of administering chatbot-based speaking assessment for low-level EFL students

Repository record updated on:
Nov. 15, 2025, 1:02 p.m.
Database record updated on:
Nov. 17, 2025, 8:22 p.m.
Last Harvested on:
Nov. 17, 2025, 8:22 p.m.
access-status:
open.access
bitstream:
[['https://scholarspace.manoa.hawaii.edu/bitstreams/22b4ae78-ebc1-4298-94f2-5eabfffa60d4/download', '29_01_10125-73660-v2.pdf']]
bitstream_txt:
[['https://scholarspace.manoa.hawaii.edu/bitstreams/dd4e47df-aada-451b-ba82-6c8d4121fa1f/download', 'full_text']]
collection:
Volume 29 Number 1, 2025
contributor.author:
Nguyen, Thuy Thi Kim, Jungtae
date.accessioned:
2025-11-15T00:24:58Z
date.available:
2025-11-15T00:24:58Z
date.copyright:
2025
date.issued:
2025-11-17
description.abstract:
This study examines the feasibility of using a chatbot-based English speaking test (C-EST) in assessing low-level students’ speaking skills. The study included three phases. First, the C-EST was compared with a face-to-face interview test (F-EST) using the ACTFL OPI, focusing on interrater reliability, holistic and analytic test scores, and participants’ perceived number of questions, topic familiarity, and difficulty level. Second, the study investigated participants’ perceived effectiveness of chatbot-based tasks and their interest, motivation, and comfort in taking the C-EST. Last, the study constructed arguments for the C-EST based on test design considerations and the results from the above analyses. Thirty-nine low-level university students completed both tests and questionnaires. Follow-up interviews were conducted with seven participants. Three raters rated both tests. The findings revealed that two tests were comparable regarding interrater reliability, holistic and analytical scores, and participants’ perceptions. Furthermore, participants’ perceptions of the effectiveness of chatbot tasks were uniformly favorable, and their levels of interest, motivation, and comfort were similarly high. The study supported the positive potential of chatbots in speaking assessment for low-level learners and revealed limitations to be resolved. Future suggestions for advancing and implementing chatbots in EFL classrooms are discussed.
endingpage:
26
format.extent:
26
identifier.citation:
Nguyen, T. T., & Kim, J. (2025). Feasibility of administering chatbot-based speaking assessment for low-level EFL students. Language Learning & Technology, 29(1), 1–26. https://doi.org/10.64152/10125/73660
identifier.doi:
https://doi.org/10.64152/10125/73660
identifier.issn:
1094-3501
identifier.uri:
https://hdl.handle.net/10125/73660
language:
eng
number:
1
publicationname:
Language Learning & Technology
publisher:
University of Hawaii National Foreign Language Resource Center Center for Language & Technology
rights.license:
Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License
rights.uri:
https://creativecommons.org/licenses/by-nc-nd/4.0/
site_url:
/item/10125-73660/
startingpage:
1
subject:
chatbots, chatbot-based speaking assessment, chatbot-based tasks, low-level EFL students
title:
Feasibility of administering chatbot-based speaking assessment for low-level EFL students
type:
Article Text
volume:
29