Cosmos : teaching web programming with AI

Abstract

Artificial Intelligence (AI) has become an integral part of education, particularly through the emergence of generative AI, which includes Large Language Models (LLMs) like ChatGPT. While these technologies offer promising opportunities for personalized learning and problem-solving, concerns regarding accuracy and student engagement persist. This research explores students preferences for generative conversational agents, comparing specialized agents to general-purpose ones like ChatGPT. Through Design Science Research (DSR), a specialized conversational agent named Cosmos is developed to assist students in a web programming course. The study investigates whether students prefer specialised agents over ChatGPT and examines their preferences regarding specialised conversational agents. Additionally, the research evaluates the quality of the answers provided by Cosmos, shedding light on its potential to guide students towards solutions while promoting active learning. Our results indicate that students generally prefer specialized conversational agents like Cosmos over ChatGPT, especially when tackling course-related tasks. Cosmos exhibits reliability and an appealing user interface, earning praise from students for its simplicity and design. Recommendations for enhancement, such as improving visibility, navigation, and functionality, have been identified to inform future advancements in educational technology. Additionally, the findings suggest that the quality of Cosmos's responses, as assessed by the course instructor, is generally high, although some adjustments are necessary based on the ratings and subsequent discussions. This research seeks to bridge the gap between theoretical insights and practical applications in AI education tools, providing valuable insights into student preferences and the quality of answers of specialized conversational agents like Cosmos compared to general-purpose models such as ChatGPT.