“Upload your essay into Turnitin by 11:59pm on Thursday night?
You meant start the essay at 11:57 then submit it at 11:58, am I right?” — 
Gigachad ChatGPT student.

We begin our discussion by discussing the sweet smell of plagiarism.

It wafts in the air as educators run around like headless chickens, looking here, looking there as they flip through oddly good essays with panicked expressions.

“Was this AI-generated, Bobby?!” says a hapless teacher, staring at a piece of paper that seems curiously bereft of grammatical errors, suspecting that Bobby could never have created something of this caliber.

“No teacher, I just became smart!” Bobby cries, running off into the sunset because he is sad, he is going to become a member of an emo boyband, and he doesn’t want to admit that he generated his homework with ChatGPT.

generated by Midjourney, if that wasn’t obvious

This smell casts fear and trepidation over every single part of our education system, for it threatens to break it; after all, education is special and it is meant to be sacrosanct — after all, is it not the very same system that is designed to teach humans facts and knowledge and above all, to communicate and collaborate to solve the problems of our era with intelligence, initiative, and drive?

It’s unsurprising that the world of education has flipped out over ChatGPT, because artificial intelligence opens up the very real possibility that schools may be unable to detect it.

Fun and games, right? It’s just a bunch of kids cheating on assignments with artificial intelligence? It’s not going to affect the older generation?

As it turns out, no — that’s not the case. I’ll explain why later.

But before that, let’s talk a bit about the part of our education system that AI is threatening: Essay-writing.

If students simply choose to let their work be completed by artificial intelligence and forget all else, that just means that they’ve forgone the education that they’re supposed to have received, thereby crippling them by an act of personal choice, right…?

But each of us has been a student, and if we have children, our children either will be or have been students too; there is a deep emotional connection that stretches across the entire world when it comes to this.

Therefore, when Princeton University CS student Edward Tian swooped in to offer a solution,it’s not all that surprising that the world flipped.

Enter GPTZero.

Humans deserve the truth. A noble statement and a very bold one for a plagiarism detector, but something that’s a little deeper than most of us would probably imagine.

But consider this.

Not everyone who uses AI text is cheating in the sense of doing something that they are not supposed to and thereby violating rules, therefore the word ‘plagiarism detector’ doesn’t quite or always apply here.

This algorithm, as with other algorithms that attempt to detect AI-generated text, is not just a plagiarism detector that merely serves to catch students in petty acts of cheating —it is an AI detector.

An AI Detector At Work.

So how does it work?

GPTZero assigns a likelihood that a particular text is generated by AI by using two measures:

Perplexity, and Burstiness.

Essentially, in more human language than that which was presented on GPTZero’s website, GPTZero says that…

The less random the text (its ‘perplexity’), the more likely it was generated by an AI.

The less that randomness changes throughout time (its ‘burstiness’), the more likely that the text was generated by AI.

Anyway, GPTZero gives each text a score for perplexity and burstiness, and from there, outputs a probability that given sentences of a text were generated by AI, highlighting the relevant sentences, and easily displays the result to the user.

Alright, sounds great!

Does GPTZero deserve the hype, though?

…Does this actually work?

Let’s try it with this pleasant and AI-generated text that is exactly about the importance of hype (lol).

That’s 100% AI-generated and we know that as fact.

…Would we know if we didn’t see it in the ChatGPT terminal window, though?

…Okay, let’s not think about that.

Down the hatch…

…And boom.

As we can see, GPTZero, humanity’s champion, managed to identify that the text that we had generated was written by AI.

Hurrah!!!

Or…?

I proceeded to rewrite the essay with another AI software.

…After which GPTZero essentially declared:

So nope, GPTZero can’t detect rewritten texts that were generated with AI — which it should be able to if it truly is an *AI* detector in the best sense — and which in turn suggests that the way that it’s been operationalized has yet to allow it to be the bastion protecting humanity from the incursion of robots into our lives.

It’s not that GPTZero — or even OpenAI’s own AI Text Reviewer, amongst a whole panoply of different AI detectors – are bad or poorly operationalized, by any means. Rather, it’s that the operationalization is supremely difficult because the task is punishingly hard, and that we are unlikely to have a tool that can detect AI-generated text 100% unless we perform watermarking (MIT Technology Review) and we would have to use multiple algorithms to be able to detect text, or come up with alternate measures to do so.

An Arms Race between AI Large Language Models (LLMS) and AI Detectors — and why you should care (even if you’re not a student).

As I’ve mentioned, there is an arms race at hand between AI Large Language Models (LLMs) like ChatGPT, and AI detectors like GPTZero, the consequence of which is likely that the two will compete with one another and each will make progress in its own way, progressing the direction of both technologies forward.

Personally, I think that AI detectors are fighting a losing battle against LLMs for many reasons, but let me not put the cart before the horse — it is a battle to watch, not to predict the outcome of before it’s even begun.

Implications of this strange war:

But why should you care about any of this if you’re not a student? It’s not like you’re going to be looking at essays constantly, right?

Let’s take aside the fact that you’re reading a blog post right now, and let’s also move away purely from the plagiarized essay bit that we’ve been thinking about, as we gravitate towards thinking about how ChatGPT is a language model.

It’s a good bet that you use language everywhere in your life, business, and relationships with other people in order to communicate, coordinate, and everything else.

When we go around on the Internet, it’s not always immediately evident what was AI generated, what was generated by a human or, for that matter, what was inspired by an AI and later followed through by a human.

The whole reason we need something like a plagiarism detector is that we may not even be sure that a particular piece of language (which we most often experience in the form of text) is AI-generated with our own eyes and minds, to the point that we need to literally rely upon statistical patterns in order to evaluate some thing that we are looking at directly in front of us, thereby recruiting our brains as we evaluate the entirety of an output.

The problem is…

Language doesn’t just exist as text.

Language exists as text, yes, but also as speech. Moreover, speech and text are easily convertible to one another — and we know very well what ChatGPT is doing: Generating text.

We now know that there are Text To Speech (TTS) models that generate speech from text. They’re not necessarily all great, but that’s besides the point — it presages the translation from text into voice.

Think about it.

If the voices that are generated by AI become sufficiently realistic-sounding and their intonations (VALL-E, is that you?), how might you know that these voices aren’t real unless there are severe model safeguards that impede the models from functioning as they are supposed to?

Now combine that indistinguishable voice with sophisticated ChatGPT output that can evade any AI detector and in turn may, depending on the features that end up developing, evade your own capacity to tell whether you are even interacting with a human or not.

How would that play out in the metaverse?

How would that play out in the real world, over the phone?

How would you ever know whether anyone that you’re interacting with is real or not? Whether they are sentient?

The battle between AI and AI Detectors is not just a battle over the difference between an A grade and a C grade.

It’s a battle over a future where what’s at stake is identifying what even qualifies as human.

Leave A Comment

Recommended Posts

Pathways To Excellence

Pathways now has a podcast – please follow it! In the past couple of days, I’ve had the chance to speak to some very, very smart people, and it’s all the result of a weird series of coincidences. Or maybe they’re not really coincidences, and maybe they’re just fate. I don’t know what it is, but either way, I realized that every single member of Harvard’s class of 2028 and Yale’s class of 2028, as well as Stanford’s class of 2028, was watching Pathways to Excellence. Why do I know this? I know this because I ended up speaking to almost every single one of them, only to hear that they were aware of what I was doing, that they had watched the videos, that prepared for their Harvard interviews, or otherwise in some way, shape, or form. It’s also been interesting to watch things play out, as I’ve secured interviews that are very different along the way, with people of rather unique backgrounds, which I’ll perhaps talk more about in the days to come. It’s very inspiring to speak to smart people – I have no idea how that’s going to influence my future just yet, but the outlines are there, but I would have done this regardless of what had happened anyway. What is clear for sure though is that this is meaningful, something I’m honored to be a part of and a journey that I will continue to value in the days to come.

Malaysia’s Harvard Class of 2028

Recently, Malaysia was very fortunate to receive the news that four Malaysians had gotten into Harvard University. Congratulations to Elisa, Victor, Thamini, and Bryan! (And thanks for watching/appearing on Pathways ^^) It’s always fascinating to see the country’s very top talent and the way that they are celebrated in this way, but maybe even more fascinating to become friends with them, which I inadvertently did in different ways. It’s cool that Malaysia can produce people of this caliber of talent for sure – but there’s naturally always going to be a question for each and every one of us: Can it retain them? Honestly, the answer to that question is unclear even for myself. You need to be strong to stay in this country, after all. V.

Random thought

One thing I’d definitely like to do a lot more this year is make myself capable of doing large batches of work all at a time. I don’t mean just going out there and working as hard as possible to put myself through a suffering that is inhuman, impossible to bear through, but instead the simple act of understanding how to schedule things so that you have more time in between the moments when you obtain inspiration such that you have a natural reminder to continue going out there, enter the game, and do as you will do. Willpower of course is a very important thing, but it is limited. That is why we create systems, things that automate our labor, but of course no system can operate without the willpower in the very first place even as it is true that expecting continual, consistent, and unrelenting willpower is a recipe for burnout and self-damage to degrees unreasonable even for the most hardened of men. In this upcoming new year, I look forward to being able to find many more systems that will continue to make work easier for me, to be able to more delicately navigate the balance between willpower effort and also attainment of actual results in the days ahead. I’m given the sense that somehow or another, that needs to be a continual part of everything that I’m doing going forward. How exactly that is going to work out and how it will happen, I am not totally sure just yet, but ideas and intuition abound, and what I’m confident of is that in many ways, this is a knowledge thing. By simply elevating yourself from an initial phase where no knowledge is present to one where awareness has been developed, a person can lift themselves up to heights hitherto unknown. That has been the experience that I’ve […]

Assessing English Standards in Malaysia: An Analysis with the CEFR

Often in Malaysia, people talk about how our standard of English is either sufficiently good that it is the basis of a thesis for investment, or they say that our English is abysmal and needs drastically to be improved – discussions go on and on, and people fight, oftentimes in what seems like a battle for the soul of our society. But what does it mean, actually, that our English is good or our English is bad? Some say that Malaysia aligns itself to international standards in creating its curricula, but others squabble day in and day out, constantly complaining about the quality of English amongst graduates who come into the workforce, observing that many of them lack basic skills that they would expect graduates to have. How can it be possible that Malaysia calibrates itself to international standards while at the same time its graduates languish in terms of their English language proficiency? But at the end of the day, who’s right?  As it turns out, investigating a little further tells us that the answer is both. Here’s where the subject of our blog post for today comes in: The Common European Framework for Reference, otherwise known as the CEFR.  The reason that I’m making this comparison today and telling you about CEFR is that Malaysia uses it to calibrate SPM writing standards.  The CEFR operationalizes language proficiency in accordance with six dimensions, from A1 up until C2. It is an international standard that is utilized by examining bodies across the world in order to designate proficiency levels and descriptors that students attain after courses of study, and it is used also in designing curricula so that students can reach a certain defined standard. Source: https://www.cambridgeenglish.org/Images/126130-cefr-diagram.pdf CEFR operationalizes English proficiency according to numerous level descriptors, providing explanations of what a user of the English language should be able to do […]

On Facing Judgment’s Shadow

Picture this. You’ve written a post that you want to share on social media, or you’ve made a video somehow or another. You’re sitting there on the edge of your chair, just about to click post, but you look at what you’ve written, what you’ve made, you notice that final error, you question yourself, thinking about the manifold ways in which people could be judging you silently and from afar, contemplating in your mind’s eye the dialogues that must be taking place. “Oh my gosh, this person wrote this?” “Oh my gosh. Did he really make that grammatical error?” “Wow, this is boring. Why am I even watching?”“How could he make such a video?” And so the thoughts come out, percolate like coffee through filter paper, and eventually crystallize into little gems of self-doubt, blocking the nervous signal that would have caused you to click. You turn away from your plan, and you declare:”Maybe later, but not now.” Before you know it, the entire project is abandoned. If you’re anything like me, you may have faced this, this feeling of wanting to do something, but realizing, or at least thinking that you weren’t good enough, that the manifold imperfections that existed in you would come out, and that people would judge you one way or another. Well, here’s a fact, though. People certainly will judge you. I mean, how could they not? Everyone encounters something as a first glance, thinks about it, and evaluates it on their own terms. That’s just how it is. The judgment will happen. There is absolutely nothing that you can do about it, and your feeling certainly is right. The thing is though, that even though it is right, this isn’t a valid reason to run away. Because truly, the only way to get past it is to face your fears, to accept judgements as they […]

Work, Life, World.

One of the things I’ve come to really appreciate learning in college is the ability to just work, and work, and work. Sometimes I feel like my work ethic is just unending, and I can continue doing everything that I’m supposed to do, just efficiently and continually, almost like a machine.  I start, I take breaks specifically for the purpose of making sure that my work efficiency is maintained, then I begin again. The cycle continues, and life proceeds just in that way.  It’s one of the miracles that has come about, I think, from being trained in an environment where people were constantly working hard, and a place where people would not just work hard, but also have the right motivation, initiative, and desire for it.  Somehow or another, it influenced me, and rubbed off on me. And before I knew it, I was one of those workaholics out there, just casually pulling long hours without even questioning things, completely by my own volition.  It sounds bad, but I’ve come to appreciate that part of myself quite a bit.  It’s one of the many reasons that I respect myself, and appreciate the person that I’ve become. Because it’s become an enduring facet of my personality, and something that I know that I can look to whenever I think about my identity as a person, but at the same time, I think it’s come with at least two different disadvantages.  The first of these is that when I think about work, I just continue on and on. It takes hold like a vice grip, consuming almost every single aspect of my mind and my thoughts.  The result?  I just carry on doing things in the way that I feel most natural. Ignoring different things, socializing, hanging out, spending time with people, messaging…The list goes on, and I don’t know how many […]