Abstract

Empirical data on automated writing evaluation (AWE) has been accumulating over the past several years, but previous research has concentrated on the impact of using machine feedback alone, and only a little is known about how learners view the usage of AI-assisted tools when combined with teacher feedback. More empirical evidence from instances in which machine and teacher feedback is used in combination is necessary to determine optimal approaches to incorporating AWE tools in classroom instruction. A study was conducted to assess Japanese college students’ perceptions of three types of feedback when they experienced them in a certain order. Participants in two groups received Grammarly feedback on their first drafts, the teacher’s indirect feedback on their second drafts, and the teacher’s direct feedback on their third drafts. The questionnaire administered to the first group examined participant perceptions of the respective type of feedback, whereas the questionnaire given to the second group verified the strengths and weaknesses of each type of feedback and investigated the type of feedback participants believed was most effective in improving their writing in English. The content analysis approach was used for textual analysis of the responses to open-ended questions, and a chi-square test was performed for scale responses. The quantitative and qualitative analysis of questionnaire data revealed that, though machine feedback was favorably received, participants appreciated human direct and indirect feedback more highly for its appropriateness and reliability. Moreover, the largest number of participants rated the teacher’s direct feedback as the most beneficial to their writing progress. The findings imply that machine feedback should not be used alone in writing instruction but as a supplement to teacher feedback. In addition, the participants’ response suggests that using Grammarly feedback in a “form then content” sequence might be one feasible pedagogical approach in the classroom.

Suggested citation