Redesign questions

At times the majority of the assessment questions are in fact tested at a higher level and are not vulnerable in light of advancements of AI. There might, however, be some questions that are more easily answerable by AI software directly. In deciding whether to redesign the question consider: 

• Does the assessment test lower level cognitive skills? 
• Is it based on students demonstrating common knowledge where answers can be easily googleable? 
• What is the purpose of the question? 

At times questions that test understanding that can be easily googleable serve a specific purpose. They can be easing the student in and give them confidence to answer more difficult questions that follow. If that is the case, AI presents a very similar level of ‘threat’ to Google. Plagiarism rules should be made clear to the students and you might consider rethinking the weighting of that question. 

If the above is not the purpose of the question, then you can redesign your question to make it less susceptible to AI. Some strategies include: 

• Include diagrams – AI doesn’t do well with reading images  
• Use multi step questions – break the question down in such a way that answering one part will lead to answering another 
• Prepare questions whose answer requires making decisions based on a diverse set of data. For example: "Solely based on all the [examinable material] of this module so far and nothing else, what would X Y Z?". The goal here is to limit the answer to a specific subset of knowledge too varied for AI to consume and/or isolate, such as specific/in-house PDF/PowerPoint 4 Currently, OpenAI do not provide corporate or institutional licences for their subscription ChatGPT4 product. 7 slides/tutorials/lecture notes, Panopto videos, EdStem posts, etc., so that, if an answer contains something outside of these constraints, it will raise alarms. (Imperial College guidance).
• Convert generic questions to scenario-based questions – you can learn more about this approach from UCL’s video. One thing to remember is to make sure that the scenario does not have a clear factual answer as this would be more easily answerable through AI.  
• Include assertion reasoning questions or binary statements .
• Design maths questions that require a specific method or formula or Require real-world knowledge or context.

UCL produced a very useful video outlining the value of and specific approaches to the last two strategies.