About Expression Tutor
Research
These research papers feature studies on expressions in students code and the use of the "Expression as Tree" notional machine:
Expressions in Java: Essential, Prevalent, Neglected?
Luca Chiodini, Igor Moreno Santos, Matthias Hauswirth
SPLASH-E 2022
Abstract:
Expressions are the building blocks of formal languages such as lambda calculus as well as of programming languages that are closely modeled after it. Although expressions are also an important part of programs in languages like Java, that are not primarily functional, teaching practices typically don't focus as much on expressions.
We conduct both a theoretical analysis of the Java language, as well as an empirical analysis of the use of expressions in Java programs by novices, to understand the role expressions play in writing programs. We then proceed by systematically analyzing teaching materials for Java to characterize how they present expressions.
Our findings show that expressions are an essential construct in Java, that they are prevalent in student code, but that current textbooks do not introduce expressions as the central, general, and compositional concept they are.
Read full paperUsing Notional Machines to Automatically Assess Students' Comprehension of Their Own Code
Bevilacqua Joey, Luca Chiodini, Igor Moreno Santos, Matthias Hauswirth
SIGCSE 2024
Abstract:
Code comprehension has been shown to be challenging and important for a positive learning outcome. Students don't always understand the code they write. This has been exacerbated by the advent of large language models that automatically generate code that may or may not be correct. Now students don't just have to understand their own code, but they have to be able to critically analyze automatically generated code as well.
To help students with code comprehension, instructors often use notional machines. Notional machines are used not only by instructors to explain code, but also in activities or exam questions given to students. Traditionally, these questions involve code that was not written by students. However, asking questions to students about their own code (Questions on Learners' Code, QLCs) has been shown to strengthen their code comprehension.
This poster presents an approach to combine notional machines and QLCs to automatically generate personalized questions about learners' code based on notional machines. Our aim is to understand whether notional machine-based QLCs are effective. We conducted a pilot study with 67 students to test our approach, and we plan to conduct a comprehensive empirical evaluation to study its effectiveness.
Read full paperAssessing the Understanding of Expressions: A Qualitative Study of Notional-Machine-Based Exam Questions
Bevilacqua Joey, Luca Chiodini, Igor Moreno Santos, Matthias Hauswirth
Koi Calling 2024
Abstract:
Background and Context. Notional machines are widespread in computing education. While they often are used to explain programming concepts, prior work on visual program simulation demonstrated their use for assessment. This paper presents a qualitative study of the use of a notional machine identified in prior literature — "expression as tree" - as an instrument to assess the understanding of expressions.
Objectives. The study answers the following research questions: What are the mistakes that students make when answering questions based on the "expression as tree" notional machine? What underlying reasons plausibly explain these mistakes? Are there plausible relationships between the mistakes and programming language misconceptions?
Method. We collect a corpus of 542 hand-drawn expression tree diagrams from 12 exams in 6 university programming courses at two different levels over the course of 4 years. We devise and use a coding approach tailored to the qualitative analysis of those diagrams. Our qualitative data analysis approach is unique due to the specific form of the data - hand-drawn diagrams - that admits a wide variety of mistakes, due to the theoretically well-defined programming language constructs that underpin the visualizations, and due to the fact that for each diagram there exists a unique solution that can serve as a reference.
Findings. Our results show that even a single question based on the notional machine is able to reveal a rich variety of mistakes related to the structure, typing, and evaluation of expressions. We identify and categorize 48 mistakes and describe outstanding ones in detail, supported by example diagram snippets. We find mistakes that are plausibly connected to known misconceptions, and others that suggest the existence of new ones.
Implications. Our findings shed light on novel pedagogical strategies to teach programming and provide insights instructors can use to transform their practices. The use of the "expression as tree" notional machine as an assessment instrument can provide valuable insights into students' understanding of several key aspects of programming.
Read full paper
The backstory
Who?
Expression Tutor was developed at the Lugano Computing Education research lab (LuCE) at the Software Institute of the Università della Svizzera italiana (USI) in Lugano, Switzerland. The development was partially funded by the Swiss National Science Foundation project 200021_184689.
Contributors (in alphabetical order):
Alen Sugimoto
Feedback generation and misconception detection
Bachelor student in the Faculty of Informatics at the Università della Svizzera italiana (USI) - Internship
Andrea Gallidabino
Management and contributions throughout
Postdoctoral fellow at the Software Institute, Università della Svizzera italiana (USI)
Armend Azizi
Classroom extension
Master student at the Software Institute, Università della Svizzera italiana (USI) - Master thesis
Cristian Buratti
Custom activity designer and runtime
Master student at the Software Institute, Università della Svizzera italiana (USI) - Master UROP internship
Igor Moreno Santos
Programming language theory
PhD student at the Software Institute, Università della Svizzera italiana (USI)
Joey Bevilacqua
Automatic activity generation and analysis backend
Master student at the Software Institute, Università della Svizzera italiana (USI) - Bachelor thesis and internship
Luca Chiodini
Management and contributions throughout
PhD student at the Software Institute, Università della Svizzera italiana (USI)
Matthias Hauswirth
Management and contributions throughout
Professor at the Software Institute, Università della Svizzera italiana (USI)
Mejrima Smajilbasic
Automated worked examples
Bachelor student in the Faculty of Informatics at the Università della Svizzera italiana (USI) - Bachelor thesis
Nicola Brunner
Text-UI for visually impaired students
Bachelor student in the Faculty of Informatics at the Università della Svizzera italiana (USI) - Bachelor thesis
Renato Iannace
Java bytecode editor
Bachelor student in the Faculty of Informatics at the Università della Svizzera italiana (USI) - Bachelor thesis
Tiziano Marsetti
Expression tree component
Bachelor student in the Faculty of Informatics at the Università della Svizzera italiana (USI) - Bachelor thesis
Why?
A large fraction—often the majority—of source code consists of expressions. When teaching programming in various languages, we noticed that students often did not fully understand the expressions they created. To help us teach expressions, and to make the relevant concepts more visible, we developed the “expression as tree” notional machine and its web-based realization in the form of Expression Tutor.
How?
Expression Tutor is a React web application built on top of Next.js. Its backend consists of different services implemented using a multitude of technologies.