by Lu Jian Hao
Generative Artificial Intelligence (GAI) tools like ChatGPT afford opportunities for teachers to enhance productivity through applications like drafting lesson plans and student testimonials. However, similar usage of ChatGPT by students to complete their assignments for them can be problematic as it trades off learning for productivity.
Student Use of ChatGPT for Productivity
Many students in my upper secondary Geography classes have been using ChatGPT to write their essay assignments. The more blatant cases involved submitting unedited ChatGPT output, while more conscientious students took the effort to perform cosmetic edits to avoid the suspicion that they did not submit original work. This raised a few issues:
i. As they did not produce the work themselves, students might associate feedback for improvement with ChatGPT’s shortcomings rather than learning gaps they should narrow or close to produce better essays. Over time, this could devalue feedback and discourage its uptake by students.
ii. If students are not acting on given feedback to improve, they would unlikely be able to independently craft good quality essays on demand (e.g. in a test situation), which begs the question of whether they have learnt, according to Sadler’s (2007) three conditions for learning.
iii. Persistent use of GAI to substitute their personal involvement and effort in learning tasks could cause students to develop a negative habit of seeking shortcuts just to fulfil deliverables, which further impedes learning.
Beyond my own classes, I have heard reports of students submitting GAI-generated responses for weighted alternative assessments. Though my colleagues were upset over students’ lack of effort, their suggestion to use AI detection software to determine if plagiarism took place is neither viable nor sustainable. This is because the nature of GAI-generated output – probability-based, not unoriginal, and not attributable to a person or organization – has disrupted the very notion of plagiarism.
To arrest and address the situation painted above, an educational approach would seem more feasible than one that aims to ‘catch students in the act’ of what teachers would consider inappropriate GAI usage. As such, I have chosen to modify my essay task to encourage the use of ChatGPT for self-assessment and feedback generation, which I believe would help my students shift away from the use of GAI for mere productivity enhancement and towards learning instead.
Redirecting ChatGPT Usage via Task Modification
I used to assign Geography essay tasks via Google Classroom with instructions and a simple writing guide (see Figure 1 below). The platform allows me to embed a grading rubric that ‘feeds up’ the success criteria to students while serving as a quick form of feedback as the corresponding level descriptor would be highlighted upon mark entry (see Figure 2 below). In addition, personalized feedback may be assigned to students via email, which makes it easy for both teacher and student to monitor if feedback has been read or acted upon through the “mark as resolved” check () button (see Figure 3 below).
I have since modified the essay task to include instructions for students to use ChatGPT to self-assess and generate feedback (supported by a prompt sequence I provided – see Annex) before submitting a revised draft for teacher grading after acting on the given feedback (see Figure 4 below). The intent is to provide an easier way for students to obtain feedback on their essays, as well as an immediate platform for them to act on the feedback to improve their essays.
Based on survey responses I gathered, my students generally found ChatGPT useful for assessing their Geography essays (see Figure 5 above), highlighting the speed and convenience of getting their essays graded compared to the usual process of teacher marking. Response towards the usefulness of ChatGPT for feedback generation was slightly more positive (see Figure 6 above). Many students appreciated the detailed and easy-to-understand feedback that offered practical suggestions (with further prompting) for closing gaps identified in their essays. Some even reported noticeable improvements in their essays (i.e. higher marks) after acting on the given feedback. Overall, many students expressed confidence in using ChatGPT to self-assess and generate feedback for improvement without teacher involvement (see Figure 7 below).
However, there were also students who found the feedback generated by ChatGPT difficult to understand, though they added that they were able to prompt ChatGPT further for a simplified version of the feedback, as well as samples of how improvements may be made to their essays in view of the given feedback. Hence, 28% of the students surveyed did not feel confident about using ChatGPT for self-assessment and feedback generation, and the poll was rather split between students who felt they could develop independence in learning through this approach, and those who felt otherwise (see Figure 8 above).
In fact, a handful of students indicated preference for teacher marking despite the benefits of using ChatGPT. They generally perceived that teachers are more up-to-date with the syllabus, are trained to know what the examiners would look out for, and can hence provide more accurate feedback – in contrast, ChatGPT output could suffer from hallucinations. Some students also preferred interacting with a teacher to work through feedback, as they would like to be coached and prompted via verbal exchanges.
Teacher Reception towards Use of ChatGPT for Learning
My experience with getting students to leverage the affordances of ChatGPT for self-assessment and feedback generation was mainly positive. ChatGPT has been very accurate thus far in identifying the level of attainment for my students’ essays. It has also been grading rather accurately with a ±1 mark difference compared to my assessment. In general, ChatGPT provides good and accurate feedback. As seen in Figure 9 below, it suggested for my student to offer a more balanced evaluation in her concluding paragraph, which I agreed with. While she tried to act on this feedback for her second draft, the attempt fell short and resulted in a comment from teacher grading that focused again on her evaluation
However, I had concerns about the potential confusion ChatGPT could cause through inaccurate grading and providing inappropriate feedback, which some students encountered. On rare occasions, I do find myself disagreeing with ChatGPT’s grading and feedback. As seen in Figure 10 above, the difference between ChatGPT and teacher grading was 3 marks – quite drastic for a task worth only 9 marks. Moreover, ChatGPT did not address the descriptive nature of my student’s essay, but instead commended her for presenting well-rounded arguments. As a result, much of the feedback generated did not direct my student to work on her missing explanations, which resulted in some confusion between ChatGPT and teacher grading and feedback that I had to process with her.
Overall Impact & Reflection
As yet another good invention with good intentions, GAI is a useful tool that holds powerful promises when used in meaningful and appropriate ways, as well as threats when done otherwise. Left unguided, my students were lured by the productivity enhancement they could achieve by getting ChatGPT to complete their assignments with little effort on their part. However, this approach to GAI usage short-circuits the learning process as a trade-off to getting work done. In the long run, this would not be beneficial for the development of my students.
By explicitly instructing my students to use ChatGPT for self-assessment and feedback generation in the process of writing their Geography essays, I have managed to shift many of them away from inappropriately using the GAI tool to fulfil their assignments and towards enhancing their learning instead. This approach has been well-received by my students, who have since demonstrated greater propensity towards acting on the ChatGPT-generated feedback in their updated drafts, resulting in a general improvement in the quality of their essays.
My approach to leveraging ChatGPT for student self-assessment and feedback generation illustrates the promise of technological affordances in helping students to learn. Through actively engaging students in the feedback literacy behaviours (Dawson et al., 2024) of seeking, making sense of and using feedback information to improve their work, the approach elevates students’ role and participation in assessment (Tay, 2018) and develops them into competent assessors of their own work, assisted by GAI.
With the aim of ‘sustainable assessment’ (Boud, 2000) in mind, I have taught my students to modify the prompts for use with other Geography essay questions. I have also developed a GPT4-chatbot that I availed to my students that further automates the assessment and feedback generation process. By making this approach of using ChatGPT transferable, I aim to liberate students from relying on the teacher for grading and feedback, and to promote greater independence and self-regulation in learning.
However, further work is required to ensure that the modified Geography essay task leveraging ChatGPT for self-assessment and feedback generation yields optimal learning benefits for my students. I have summarized some suggestions in Table 1 below, alongside the pre- and post-modification task designs for comparison.
Concluding Remarks
Moving forward, further imagination and experimentation would be required to figure out how the use of GAI can strengthen the feedback-action loop in support of students’ learning, both for tasks that they would face in their schooling years, as well as to develop the self-regulatory capacity to continue doing so for future tasks they would face post-education. With the promises GAI holds for education, opportunities abound in ensuring that assessment is done well in the service of our students’ learning, growth and development.
References
Boud, D. (2000). Sustainable Assessment: Rethinking assessment for the learning society. Studies in Continuing Education, 22(2), 151–167. https://doi.org/10.1080/713695728
Dawson, P., Yan, Z., Lipnevich, A., Tai, J., Boud, D. & Mahoney, P. (2024). Measuring what learners do in feedback: the feedback literacy behaviour scale. Assessment & Evaluation in Higher Education, 49(3), 348-362. https://doi.org/10.1080/02602938.2023.2240983
Sadler, D.R. (2007). Perils in the meticulous specification of goals and assessment criteria. Assessment in Education: Principles, Policy & Practice. 14(3), 387-392. https://doi.org/10.1080/09695940701592097
Tay, H.Y. (2018). The role of heads of departments in assessments. In K.H.K. Tan, M.A.
Heng and Ratnam-Lim C. (Eds.), Curriculum Leadership by Middle Leaders: Theory, Design and Practice (pp. 73-87). Routledge. https://doi.org/10.4324/9781315760889
Annex
ChatGPT prompts for essay grading and feedback
|
Step 1: Paste the following prompt into ChatGPT You are a secondary school teacher in Singapore, teaching Geography. You assigned the following essay question to your students: ‘Tourism brings more benefits than harm to tourist destination regions.’ How far do you agree with the statement? Support your answer with reasons and examples. Please ask your student to submit the essay response, and grade it with a corresponding level and specific numerical score (mark) according to the following rubric:
Step 2: Copy and paste your essay answer into ChatGPT Step 3: Paste the following prompt into ChatGPT Please also provide 3 points of feedback to help your student improve the essay response by at least 1 mark. |