The Essence of Good Assessment
For a long time, I have been writing about topics related to MBA and life. I think that has been the case for almost the whole of the last two years since I started writing. This week, I want to do something different. I want to engage my PhD students (interesting that I am now on the other side of the table).
Some of you may be aware I teach a course of teaching pedagogy along with Dr. Urvashi Rathod. For all practical purposes, she’s the driving force of the course, and I am trying to learn as much as I am teaching through this course. Dr Rathod has been lovely and has sat through all my sessions just as I sat through hers (technically, two faculty are teaching the course simultaneously - This is indeed a rare occurrence).
In one of the classes last week, we engaged with assessments. It’s a tricky topic to discuss, then, but there were plenty of ideas that came up during the classroom discussions, and I felt that I should document them here for a broader audience to
Remember the Kobayashi Maru? Yes, the test from Star Trek? If you’re not familiar with the idea. Please watch this video down below.
Did not understand what was going on? No problem. Let me set the context here.
Kirk is a young student at the Star Fleet Academy, and he wants to become a captain. This is one simulated test that leads up to his captaincy. According to the test, the USS Kobayashi Maru is a spaceship currently in crisis deep in space. Kirk and his team have to engage in a simulated exercise to rescue the crew of the estranged ship. The programming of the test is such that everyone who attempts it loses. The idea is to understand if a to-be captain can face defeat (and how he/she gracefully embraces it). Kirk is brilliant and figures out a way to rig the test (that happens before this scene), thereby winning (to everyone’s surprise). Interested in knowing more? Watch the movie. But the core idea here is the idea of assessment.
Remember the management adage, what gets measured gets done? In the case of the Kobayashi Maru test, everyone was aware of what was being measured—the captain’s ability to handle an unwinnable situation. While every captain was prepared physically and mentally to see the destruction of his starship and crew, Kirk decided that he would show the world how the test was flawed in the first place. While Kirk does not state this explicitly, we can understand this as Kirk’s revolt against the philosophy behind the test. In effect, Kirk argues that some things cannot be tested, regardless of how immersive the simulation is or how pure the intentions behind the test are.
Despite having this extremely appealing argument in front of us, we go on to test generation after generation with tests that are more and more complicated to ensure that they are selected for ‘something greater’. Sometime back, I read somewhere that there is a highly competitive test in China: Gaokao, with millions of Chinese kids writing it year after year to get selected in the most prestigious universities in the country. The NEET, CAT and the JEE in India are similar tests. They are taken by lakhs of students every year, all in the pursuit of landing in one of the dozens of AIIMS, IIMs and IITs dotted across our country. The marks students score in their tenth and twelfth board exams continue to influence their lives well into their post-graduation. Therefore, looking into some of the ideas behind the assessment may be a good idea.
The Essence of Good Assessment
This is something that came out of the classroom discussion. And I for one, am happy that the students were all thinking along these lines.
All students agreed that a well-crafted assessment should serve as a bridge between current knowledge and future learning. It was also discussed that the assessments are not just for students but also for the teacher who learns a lot through developing the assessment. They also pointe out to several characteristics that they felt were key to a good assessment. Let me list them down below:
Fairness
At its core, fairness in assessment means that every student is provided with an equal opportunity to demonstrate their understanding and abilities. This requires careful consideration of the assessment's design to ensure it does not inadvertently favour a particular group of students over others. For instance, incorporating various question types can cater to different learning styles and abilities, thus levelling the playing field.
Accuracy
An accurate assessment accurately reflects the extent of a student's comprehension and skills. It should be aligned with the course's learning objectives and be capable of distinguishing between different levels of student achievement. This precision not only aids in grading but also helps identify areas where students may need additional support or enrichment.
Very often, the assessments do not meet the entirety of their goal since only the marks that a student scores are communicated and not the feedback on the answers. A good habit young faculty could develop is setting up a session to discuss the answers after the exam.
Enjoyability
Though often overlooked, the enjoyment factor of an assessment can significantly impact student engagement and motivation. An enjoyable assessment can stimulate curiosity and encourage a deeper exploration of the subject matter. Techniques such as gamification, real-world problem-solving scenarios, or integrating elements of storytelling can make assessments more engaging and less daunting.
This is perhaps why we get to hear a lot of great stories of interesting assessments that students were subjected to. Students know that the faculty is putting in effort and the effort motivates them to learn.
Enabling Learning
Beyond measuring knowledge, a good assessment should also contribute to the learning process. This involves providing timely and constructive feedback that students can use to improve their understanding and skills. Feedback should be specific, actionable, and oriented towards growth, helping students see where they can improve and how they might improve.
When students are young, there are regular parent-teacher meetings and interactions that happen to hone the skills of the student. But then, as we grow older, such PTA meetings trickle down and the learning process gets restricted to just the student.
By focusing on these principles, educators can design assessments that evaluate student performance and enrich the learning experience.
The ultimate aim is to cultivate a learning environment where assessments foster improvement and curiosity rather than simply acting as benchmarks for grading. This approach enhances students' academic and professional journeys and equips them with the complexities and challenges of the real world. It parallels the Kobayashi Maru scenario from Star Trek, preparing cadets for the unpredictability of space exploration by teaching resilience and adaptability in the face of unwinnable situations.
However, it's crucial to acknowledge that designing good and comprehensive assessments demands significant time and effort from educators. This scenario presents a unique case where 'time' becomes a dependent variable—a more comprehensive evaluation, while beneficial, poses implementation challenges, especially considering the amount of time faculty members spend on research and administration. In this context, we should think about AI and other recent technological innovations.
While assessments, in their current form, might seem like a necessary evil or, at worst, a means of enforcing conformity, they remain the most effective tool we have to ensure a fair allocation of opportunities. Nonetheless, the imperfections of current assessment methodologies underscore the need for continuous evolution.
By embracing technological advancements and innovative educational practices (such as AI, simulation games, etc), we can strive towards assessments that are fair, comprehensive, adaptable, and reflective of diverse learning styles and intelligences. In doing so, we keep the door open for developing more equitable ways to measure and foster growth, moving closer to an educational paradigm where assessments serve as catalysts for learning and personal development.
