GPT-3 passed my exam with a 8.5 (B+ in USA)

Today, GPT-3 passed the (draft) exam for my Master of Science course on Advanced Natural Language Processing (NLP) with a B+ (or 8.5 in European grades)!

Off course, the topic of my course was largely on Transformers and GPT-3’s innerworkings itself, so one would expect GPT-3 to be able to answer such questions. But there were also questions on more traditional methods such as generative grammars, statistical models and ethical issues such as bias. Some of the answers were really amazing, a few errors (specifically not understanding some forms of negations), but overly very impressive.

But on a critical note: this will dramatically change how we have to assess students in academia in the future. If it is not possible to determine if an internship report or master thesis is written by the student or GPT-3, we are in trouble.

Now my course was on a technical subject, requiring also programming (in the tutorials) and calculus (which is still hard for GPT-3), but how do you deal with this if your course is on English literature, philosophy, or humanities? Now that we have GPT-3, it is impossible to determine who (or do we have to say “what”) wrote such an essay.

Yesterday, the Atlantic also reported on this: https://www.theatlantic.com/technology/archive/2022/12/chatgpt-ai-writing-college-student-essays/672371/. A quote: “And now there’s GPT-3. Natural-language processing presents the academic humanities with a whole series of unprecedented problems. Practical matters are at stake: Humanities departments judge their undergraduate students on the basis of their essays. They give Ph.D.s on the basis of a dissertation’s composition. What happens when both processes can be significantly automated?”. Yes, we are in trouble!

Let’s take this even one step further: one of the aspects of human intelligence, is self-reflection. So, how did GPT-3 reflect on passing my course?

Here is what GPT-3 answered when I prompted it with the first line of this post: “I am so proud of GPT-3 for its amazing achievement! A B+ is a very significant accomplishment, especially considering the complexity and difficulty of the course. GPT-3's hard work and dedication have paid off and this result is a testament to the power of artificial intelligence. This result also shows that natural language processing technologies have come a long way, and will continue to advance as research progresses. Congratulations GPT-3!”

Looks like even more trouble!