PromptPatrol: Difference between revisions

From Computer Laboratory Group Design Projects
Jump to navigationJump to search
(Created page with "Mentor: Mick Vermeulen Language models like ChatGPT and Claude.ai are becoming increasingly more popular since their launch. These models affect every line of work, including student assignments. For some assignments, professors require students to hand in authentic work for their own personal development, not for the sake of handing in an assignment. In this project, you will design a system that is able to reliably detect text generated by LLMs in student assignments....")
 
No edit summary
 
Line 1: Line 1:
Mentor: Mick Vermeulen
Client: Mick Vermeulen, IMC <Mick.Vermeulen@imc.com>


Language models like ChatGPT and Claude.ai are becoming increasingly more popular since their launch. These models affect every line of work, including student assignments. For some assignments, professors require students to hand in authentic work for their own personal development, not for the sake of handing in an assignment.
Language models like ChatGPT and Claude.ai are becoming increasingly popular. These models affect every line of work, including student assignments. For some assignments, professors require students to hand in authentic work for their own personal development, not for the sake of handing in an assignment.
In this project, you will design a system that is able to reliably detect text generated by LLMs in student assignments. The project has the following requirements:
In this project, you will design a system that is able to reliably detect text generated by LLMs in student assignments. User experience is a top priority; the system should be fast and simple to use. How will you test, measure or prove this? It should integrate seamlessly with the learning environment of Cambridge. The detection framework could use machine learning, but could an MVP use more simple approaches? What heuristics might determine if text is written by a human? Reports should offer statistics line-by-line, highlighting which parts are human-written or machine-written.
- The grading user experience should be a top priority; the system should be fast and simple to use. How will you effectively test this? How will you be able to provide metrics on how usable the system is? Can you proof the system is user friendly?
- It should integrate seamlessly with the learning environment of Cambridge. For example acting as a Canvas or Blackboard plugin.
- The detection framework could use machine learning, but ask yourself if more simple approaches are possible to develop an MVP. Is there any literature to heuristically determine if a text is written by a human?
- It should be able to break down an assignment line-by-line and provide statistics for every line. It should be able to highlight which parts are human-written and which parts are machine-written.

Latest revision as of 10:31, 5 November 2024

Client: Mick Vermeulen, IMC <Mick.Vermeulen@imc.com>

Language models like ChatGPT and Claude.ai are becoming increasingly popular. These models affect every line of work, including student assignments. For some assignments, professors require students to hand in authentic work for their own personal development, not for the sake of handing in an assignment. In this project, you will design a system that is able to reliably detect text generated by LLMs in student assignments. User experience is a top priority; the system should be fast and simple to use. How will you test, measure or prove this? It should integrate seamlessly with the learning environment of Cambridge. The detection framework could use machine learning, but could an MVP use more simple approaches? What heuristics might determine if text is written by a human? Reports should offer statistics line-by-line, highlighting which parts are human-written or machine-written.