Goals:
Instead of following the traditional ways to assess an MTH module such as quizzes and exams, we hope to take some innovations to achieve the following goals:
increase the variety of questions and answers; let students take the initiative to demonstrate what they have learned and what they are interested in; extend present knowledge to possible research areas for SURF, FYP, postgraduate studies, etc.
These goals are never easy for mathematical subjects, which usually emphasize accurate solutions. However, Operational Research (OR) is an interdisciplinary subject that combines mathematics, management science, and industrial engineering. It explores a wide range of practical applications. Moreover, the third-year students seem to be the ideal group as they have gained plenty of experience in learning and teaching at XJTLU and are extremely keen on exploring further opportunities. Overall, we are confident that the MTH203: Introduction to Operational Research could be considered a model for implementing assessment redesign. Initially, there are three assessment components: coursework 1 (15%), coursework 2 (15%), and the final exam (70%), all of which are short answer questions.
Coursework 1: Involve open questions to encourage variety and innovation
Certainly, there is a solid theoretical foundation in the field of Operational Research, but in MTH203, we intend to focus on the connection between math knowledge points and real applications. As a result, our questions could be constructed in one of two ways: either compile the question background to implement the given technical points (e.g., constitute a story that leads to a Markov Decision Process with the given timeline, states, and matrices); or develop/interpret answers from different perspectives (e.g., the "efficiency" of an operation system could be measured by completed jobs per unit time, waiting time per job, or utilization rate of the servers, depending on the situation). In coursework 1, we have made it as a take-home assignment with more than half of the short answer questions open-ended. In other MTH modules, students do not typically receive this type of question. They are given more time and are permitted to make references, but they produce significantly different results. From the submissions, we find that the training was effective because many of the answers fully demonstrated independence and critical thinking. Additionally, the open-ended question setting has reduced our worry about problems with academic dishonesty.
Coursework 2: Make it a festival
After having individual-based coursework 1, we feel it is necessary to set the second one as a group project which assesses different learning outcomes. Particularly, students would learn to scope their projects with self-created backgrounds and appropriate OR techniques and communicate the results with a general audience. The former is implemented by letting students determine their team and topic following one of the few pathways (which lead to different skills, such as algorithm design, modeling, computer programming, etc.), and the latter is accomplished by creating a fun, relaxing and open atmosphere that emphasizes mind-sharing rather than examining. "OR Festival" was therefore created in 2021. After trying various forms (a one-day event with online or onsite ppt presentations but hard to completely open to the public), we have reached an ideal version: The OR Festival 2023 was held in the lobby of the MB building displaying 45 posters (i.e., 45 groups) for an entire afternoon, where students presented their projects with incredibly creative ideas and answered questions not only from module examiners, but also from peers and other staff members. We served refreshments and offered awards (based on votes from students and examiners) for reference letters or FYP supervision. Colleagues from Learning Mall and university marketing and communications were also invited and our outreach appears to have been successful: the event has been well reported to the School of Mathematics and Physics, and many students have been interviewed and invited to participate in future projects. We are proud that not just the MTH203 students but everyone who joined has deemed this a feasible and innovative practice.
Feedback, Issues & Thoughts
To guide our assessment design and implementation, students’ opinions have been collected throughout the teaching period on many details (e.g., timing of coursework, grouping, peer review, presentation preparation, etc.), to motivate their participation. So we have finally received very positive feedback from almost everyone: the formats of the assessments have been very different from other MTH modules, students get good opportunities to demonstrate/practice various types of skills, and Operational Research has been largely promoted as a field for future interests.
There are also issues worth mentioning, some may not even have solutions but we would like to share thoughts:
The "prep-mark ratio" refers to the amount of time students spend preparing for assessments in comparison to the grades they obtain. This problem was raised by students in their feedback. In coursework 1 and 2, we tend to give students more difficult questions and longer preparation periods (each lasting one week) to fully support their efforts. However, receiving only 15 marks overall after consistently working hard can make some students feel "unworthy," especially if they are required to complete multiple assessment tasks for various modules during a certain period. It is unnecessary to blame the coursework setting, but moving forward, we'll support students in learning better time management skills and being less mark-focused.
Clearly, a high degree of student-centered assessment results in a high marking burden, which would be a serious issue for modules with enormous sizes. The marking of open-ended questions and self-designed projects has required a great deal of work from the module examiners because MTH203 has been taught in simultaneous sessions with each having about 150 students during the previous three years. In the future, it is believed that further approaches including reducing class size, introducing smart classroom settings, and implementing an intelligent computer marking system will be able to help alleviate the issue.
