Can Artificial Intelligence (ChatGPT 3.5) Improve Critical Thinking in Question Creation Elementary Education? Based on the Effectiveness of Citizenship Learning

Abstract

The mid-semester exam evaluates the quality and feasibility of AI-generated question items, aiming to determine their validity, reliability, difficulty, distinguishability, and ability to enhance critical thinking. This quantitative descriptive study analyzes statistical data from 32 eighth-grade students using 40 multiple-choice questions on the Citizenship Education (PKn) exam. Findings reveal that 38 out of 40 questions are valid, with only questions on state institutions' functions and democratic processes being invalid. The Cronbach's alpha for the 38 valid questions is 0.65, indicating moderate reliability. Difficulty levels show 20 easy, 18 medium, and 2 difficult questions. Distinguishability results indicate 18 questions with low, 16 with sufficient, and 6 with good distinguishability. Student feedback shows 94% found the questions relevant, clear, accurate, and capable of stimulating critical thinking. Overall, the AI-generated questions are mostly valid, reliable, and effective in assessing students' knowledge and critical thinking skills.