Embedding AI into assessment

The aim of the module is for students to explore applications of AI in finance, health and other markets and to build new business models, products or technical concepts. The final product of the module is a pitch of a new AI venture that they wish to launch in either a corporate setting or as an independent start-up.  

 The module is purely based on coursework which comprises of the following: 

• The final report of 10-20 slides of a written submission and the final presentation on the group output (that resembles a Dragon’s Den style verbal pitch 
• Written individual assignment elaborated where students are asked to use AI to critically evaluate its output about the originality of their business idea.  
• Peer evaluation and group project contribution that are graded 
• Quizzes to ensure that students are doing the required work and engaging in class.  
• Class participation  

One of the coursework assessment, the individual written assignment, was redesigned in response to the rise of large language model systems like ChatGPT. Before the popularisation of Chat GPT the task read: 

Following your problem identified in Written Assignment 1, please write a short essay of 200-500 words on the topic of “what is unique about your solution”. As before, you may potentially (but are not required to) describe the same unique attributes that differentiate your solution from others that might exist, but you should provide your own individual perspective on the answers. 

This task was changed once ChatGPT became widely used to include a deliberate use of an AI tool (students weren’t exclusively restricted to using ChatGPT but ended up choosing it as their preferred tool): 

Following your problem identified in Written Assignment 1, you will again be collaborating with your LLM-AI. You will be submitting 2 items in this written assignment: 

1. Instruct your LLM-AI of choice to generate a 250-word explanation of what is unique about your AI solution. Your written assignment is to critique this output (250 words +/-). What was good about the result? What did not work well? What would you do differently? 

1. Submit the venture description output that the LLM-AI itself generated. Please identify which LLM-AI you used. 

Group component is the fundamental learning design for this type of module. Most of the learning that students will gain from particularly an entrepreneurship and business plan focused class is peer learning. Therefore figuring out how to operate as part of a team where you have to take a complex task and subdivide it into smaller tasks, navigate domains of status and control, authority, expertise, ability to work with others is what's necessary to succeed in running a start-up.   

This is an elective module that interacts seamlessly with another elective module - Entrepreneurial Journey. Students who take the AI ventures module can pick up their work on their business plans in Entrepreneurial Journey and extend it. They'll get a more rigorous and in depth look at the topic and also more rigorous and in depth set of feedback at end of term.  

Prior to undertaking the assessment the students were shown how use a LLM. This was done through composing AI poetry in class, i.e. the students used AI to compose short poems about AI ventures that were then read in class. The reason for it was making sure that every student had an account set up to enter prompts and generate an output. Through this, students learnt how to use the tool in a non-threatening format. 

Students at the start are asked to sign an honour code of agreement where they agree to contribute to group work. Outside of that there is no preparatory work done to ensure students are well prepared to work in a group. This is because of the nature of the students who tend to have some working experience before going into the MBA and the nature of the programme that is group work heavy. 

Groups are assigned according to interests. A Google sheet is created with students’ ideas for businesses and others interested in a similar idea can sign up and form a team together. This helps to ensure students are driven by their interests and are passionate about the business, which assigning students to groups would not help to achieve. 

Group contribution are peer marked. This way the group self regulates without the need to implement top-down interventions. 

There is no formal preparation for peer feedback and peer marking, again due to the nature of the programme and the students.  

Each component described in assessment overview has weighting attached to it and is formally marked. The distribution is as follows: 

Feedback on the individual written assignment was provided in the form of a sentence or two. The students tended to fall into patterns of response so it was easier to generate feedback that can apply to more than one student. The individual answers were different, but they bucketed nicely into handful of categories - did the student understand that ChatGPT was repetitive and superficial? Did they just spot that it was superficial and not repetitive? Did they just spot that it was repetitive and not superficial? Did they fail to extend the analysis of the output into something they would use in their final submission? Even though there was only one marker involved in marking a substantial amount of submissions, this approach to feedback allowed to lower the feedback burden yet for students to still get useful information they can apply in a timely manner.

This assessment could easily be implemented both in an Online class setting or in person.