Automatic In-depth Correction of Exercises

Abstract

Several work has been done in the last years to increase the effectiveness of teaching aids through the usage of recent technologies: Internet and the Web. In a previous project [1] we have shown how the production, manipulation and sharing of Multiple Choice Quizes (MCQs) is a very effective didactic tool. In this paper we present a novel technique for the automatic correction of exams in the area of technical sciences. Our technique is based on a data structure which is more complex than classic Multiple Choice Quizzes (MCQs) and then we obtain a better understanding of the student's level of preparation. Our idea is based on the fact that it is very easy (for the student) to fill web forms that represent the state of a given data-structure. Example of data-structures easily representable on a web form are: - undirected and directed graphs, - finite state automata, - truth tables, PLAs, Boolean functions, - stacks, trees, lists .... We formulate questions in such a way that the student should reply by solving the exercise and then by showing the state of the used data structures at a given point of the solution process and/or at the end. During the correction we analyse the data-structure's properties and we compare them with both correct and incorrect examples given by the teacher. The above mentioned tests produce a pattern that is matched against a set of solution and common error patterns, this way we obtain a finer understanding of student's mistakes. Notice that an exercise with a long-winded solution is prone to propagation of errors. Moreover, in some exercise we give to the student the freedom to choose among diverse solution paths. This makes harder the correction process. To avoid such non-determinism or error's propagation the teacher should either: - split the exercise in smaller pieces - or be very specific in the exercise definition - or submit to the system the full set of correct replies. In some cases producing the full set of correct replies could be a harder work than just doing the correction by hand. To avoid this drawback we specialize (by subclassing) the software for the analysis of data structures to automatically recognize larger sets of solutions. This technique can be integrated in a web-based framework to deliver self-evaluation tools to students through the automatic creation of WWW pages and forms. E.g. by simply attaching to each kind of mistake a special explanation, the teacher can deliver (through the web) to each student a personalized lesson on the specific mistake. Moreover, a finer understanding of the student preparation can be used to guide him/her through the teaching material on web. Exams in several fields of sciences (e.g. "Digital Electronics", "Algorithms and Data Structures") can be automatically corrected this way in a manner very similar to the way the teacher is used to. Our technique can be used also to deliver in-class exams through the use of especially designed paper forms to be read with Optical Mark Readers (OMR). Web pages are easier to design and use than OMR forms, and thus on-line tests can contain exercises on more complex data structures than with OMR forms. Both kinds of forms are designed as much as possible similar to the formalism used during normal paperwork. We want that the stress derived for encoding the student's replies is negligible or else the number of errors would be higher.

Future work

References