The Use of AI for Accessible Education
Many times AI has been put on a pedestal as the future of x y & z, however, many seem to agree that education is a sector in particular which will see stark changes in both admin, teaching styles, personalisation and more. I had the pleasure of speaking to three individuals working in the field, including, Vinod Bakthavachalam, Senior Data Scientist at Coursera, Kian Katanforoosh, Lecturer at Stanford University & Sergey Karayev, Co-Founder and CTO of Gradescope.
We began by having Sergey of Gradescope walk us through his product, which has been recently acquired by turnitin. The concept, it seemed was formed from the simple and widespread issue of both lack of consistency, lack of insight through time constraint and delayed feedback on academic work. Sergey focussed then on solutions for these frustrations. Sergey found that scanning the papers onto an online interface when paired with a rubric can allow for accurate marking in seconds across several papers. This also allows for adjustment from the marker (figure one). Changing the points based system of the rubric, then retroactively applies it to the paper markings. This rubric, Sergey suggests also works as feedback as the student is able to see exactly why they are losing points on each question.
Following this, Sergey decided to create an AI assisted grouping technique utilising Machine Learning. The core idea being that if they can analyse exactly what it is the student has written, they can put all the answers that say the same thing into groups which is then marked together using the aforementioned method. The criteria for AI-Assisted grouping is shown below:
Of course, the question arises, what is it that instructors are actually grading? multiple choice or written answers? Sergey suggested that through working with Berkeley professors, they were able to build an interface with 25k annotations of binary, multiple choice, short writing, medium writing, long writing and drawing types. What is the right user interface though for AI Assistance for short answers? You can’t simply group them as they are too high-dimensional and are not mutually exclusive. Another option would be to use answers searching, however, searching keywords is not AI… It was then decided that answer sorting was the right course, using answer sequences to deter patterns in the data.
Sergey then went on to discuss the main sources of complexity found in his work which included handwriting, variation of answers and creating the rubric itself. Some rubrics are just correct/not correct whereas others can be more complex.
Handwriting in general can be a sticking point, not only in education but in various scenarios, evidenced by Sergey in his reference to a study from the UAE citing poor and ilegible handwriting as the cause of 7k deaths per year…. In STEM, however, there is increasing complexity due to the diagram, math and science notation, code and arrows/cross-outs etc. In regard to complexity, Sergey has tested on the Microsoft API with a best performance of 17% which led to their in-house creation of a convnet + transformer which is better and still improving. This system is being tested on both handwriting-like synthetic examples but also whole images using attention to line-by-line to follow. Sergey noted that once handwriting complexity is sorted, there still lies the issue of answer complexity – citing the standard NLP issue of there being a thousand ways to say the same thing.
The crucial point of his work, Sergey suggested is two fold. The decrease in need for marking time allows for not only greater time with students, but also greater allowance for detailed feedback and insight into learning.
Next up we heard from Kian Katanforoosh, a technology entrepreneur and lecturer at Stanford University, where he teaches Deep Learning in the Computer Science department with Prof. Andrew Ng.
