Papers

This paper investigates supervised fine-tuning of large language models (LLMs) to improve their pedagogical alignment in computing education, addressing concerns that LLMs may hinder learning outcomes. The project utilised a proprietary dataset of 2,500 high quality question/answer pairs from programming course forums, and explores two research questions: the suitability of university course forums in contributing to fine-tuning datasets, and how supervised fine-tuning can improve LLMs’ alignment with educational principles such as constructivism. Initial findings suggest benefits in pedagogical alignment of LLMs, with deeper evaluations required.

This paper introduces DCC Sidekick, a web-based conversational AI tool that enhances an existing LLM-powered C/C++ compiler by generating educational programming error explanations. The tool seamlessly combines code display, compile- and run-time error messages, and stack frame read-outs alongside an AI interface, leveraging compiler error context for improved explanations. We analyse usage data from a large Australian CS1 course, where 959 students engaged in 11,222 DCC Sidekick sessions, resulting in 17,982 error explanations over seven weeks. Notably, over 50% of interactions occurred outside business hours, underscoring the tool's value as an always-available resource. Our findings reveal strong adoption of AI-assisted debugging tools, demonstrating their scalability in supporting extensive CS1 courses. We provide implementation insights and recommendations for educators seeking to incorporate AI tools with appropriate pedagogical safeguards.

This paper presents the Debugging C Compiler (DCC), a system that composes a suite of compilers with static and dynamic analysis tools to support introductory C programming students. Using C in our introductory computing courses exposes students to low-level mechanics of the operating system...

In the challenging field of introductory programming, high enrolments and failure rates drive us to explore tools and systems to enhance student outcomes, especially automated tools that scale to large cohorts. This paper presents and evaluates the dcc --help tool, an integration of a Large Language Model (LLM) into the Debugging C Compiler (DCC) to generate unique, novice-focused explanations tailored to each error. dcc --help prompts an LLM with contextual information of compile- and run-time error occurrences, including the source code, error location and standard compiler error message. The LLM is instructed to generate novice-focused, actionable error explanations and guidance, designed to help students understand and resolve problems without providing solutions. dcc --help was deployed to our CS1 and CS2 courses, with 2,565 students using the tool over 64,000 times in ten weeks. We analysed a subset of these error/explanation pairs to evaluate their properties, including conceptual correctness, relevancy, and overall quality. We found that the LLM-generated explanations were conceptually accurate in 90% of compile-time and 75% of run-time cases, but often disregarded the instruction not to provide solutions in code. Our findings, observations and reflections following deployment indicate that dcc --help provides novel opportunities for scaffolding students' introduction to programming.

Students in introductory computing courses often lack the experi- ence and knowledge to help them effectively identify and resolve errors in their code. Programming Error Messages (PEMs) can be a valuable source of information and guidance for novices, as these messages are often the first indication of an issue during debugging. However, in many cases, such as with standard C compiler imple- mentations, PEMs are largely not suitable for novices. Confusing, often misleading, and filled with terse language and jargon, these messages instead act as an additional source of difficulty and stress, particularly for novices. In this paper, we present DCC Sidekick, which integrates the Debugging C Compiler (DCC) with a Large Language Model (LLM) in a web-based dashboard to produce contextual, accurate guidance conducive to student learning. This dashboard is directly accessible from the output of the compiler, and provides a bird’s-eye-view of program source, compiler output, and a conversational AI interface to help unravel cryptic error messages. We aim to deploy DCC Sidekick to a C-based CS1 cohort at a large higher education insti- tution to investigate how novice students utilise the conversational explanation interface during debugging activities. In this work, we present our experience designing and building DCC Sidekick.