Search Here ….

Tags

Academic integrity AI AI Detectors Artificial Intelligence Asia School of Business ChatGPT ChatGPT Malaysia ChatGPT Plus Education Entrepreneurs and Startups Malaysia Entrepreneurship Gardens Google GPTZero IGCSE Malaysia Malls Maps72 Microsoft Midjourney MIT MrMoney TV OpenAI Pavilion Plagiarism Plagiarism checkers Poe Quora Sam Altman Software development Technology Turnitin Udemy Writing
Uncategorized

A Strange War: AI v. AI Detectors

Victor TanPost a Comment

“Upload your essay into Turnitin by 11:59pm on Thursday night?
You meant start the essay at 11:57 then submit it at 11:58, am I right?” — Gigachad ChatGPT student.

We begin our discussion by discussing the sweet smell of plagiarism.

It wafts in the air as educators run around like headless chickens, looking here, looking there as they flip through oddly good essays with panicked expressions.

“Was this AI-generated, Bobby?!” says a hapless teacher, staring at a piece of paper that seems curiously bereft of grammatical errors, suspecting that Bobby could never have created something of this caliber.

“No teacher, I just became smart!” Bobby cries, running off into the sunset because he is sad, he is going to become a member of an emo boyband, and he doesn’t want to admit that he generated his homework with ChatGPT.

generated by Midjourney, if that wasn’t obvious

This smell casts fear and trepidation over every single part of our education system, for it threatens to break it; after all, education is special and it is meant to be sacrosanct — after all, is it not the very same system that is designed to teach humans facts and knowledge and above all, to communicate and collaborate to solve the problems of our era with intelligence, initiative, and drive?

It’s unsurprising that the world of education has flipped out over ChatGPT, because artificial intelligence opens up the very real possibility that schools may be unable to detect it.

Fun and games, right? It’s just a bunch of kids cheating on assignments with artificial intelligence? It’s not going to affect the older generation?

As it turns out, no — that’s not the case. I’ll explain why later.

But before that, let’s talk a bit about the part of our education system that AI is threatening: Essay-writing.

If students simply choose to let their work be completed by artificial intelligence and forget all else, that just means that they’ve forgone the education that they’re supposed to have received, thereby crippling them by an act of personal choice, right…?

But each of us has been a student, and if we have children, our children either will be or have been students too; there is a deep emotional connection that stretches across the entire world when it comes to this.

Therefore, when Princeton University CS student Edward Tian swooped in to offer a solution,it’s not all that surprising that the world flipped.

Enter GPTZero.

Humans deserve the truth. A noble statement and a very bold one for a plagiarism detector, but something that’s a little deeper than most of us would probably imagine.

But consider this.

Not everyone who uses AI text is cheating in the sense of doing something that they are not supposed to and thereby violating rules, therefore the word ‘plagiarism detector’ doesn’t quite or always apply here.

This algorithm, as with other algorithms that attempt to detect AI-generated text, is not just a plagiarism detector that merely serves to catch students in petty acts of cheating —it is an AI detector.

An AI Detector At Work.

So how does it work?

GPTZero assigns a likelihood that a particular text is generated by AI by using two measures:

Perplexity, and Burstiness.

Essentially, in more human language than that which was presented on GPTZero’s website, GPTZero says that…

The less random the text (its ‘perplexity’), the more likely it was generated by an AI.

The less that randomness changes throughout time (its ‘burstiness’), the more likely that the text was generated by AI.

Anyway, GPTZero gives each text a score for perplexity and burstiness, and from there, outputs a probability that given sentences of a text were generated by AI, highlighting the relevant sentences, and easily displays the result to the user.

Alright, sounds great!

Does GPTZero deserve the hype, though?

…Does this actually work?

Let’s try it with this pleasant and AI-generated text that is exactly about the importance of hype (lol).

That’s 100% AI-generated and we know that as fact.

…Would we know if we didn’t see it in the ChatGPT terminal window, though?

…Okay, let’s not think about that.

Down the hatch…

…And boom.

As we can see, GPTZero, humanity’s champion, managed to identify that the text that we had generated was written by AI.

Hurrah!!!

Or…?

I proceeded to rewrite the essay with another AI software.

…After which GPTZero essentially declared:

So nope, GPTZero can’t detect rewritten texts that were generated with AI — which it should be able to if it truly is an *AI* detector in the best sense — and which in turn suggests that the way that it’s been operationalized has yet to allow it to be the bastion protecting humanity from the incursion of robots into our lives.

It’s not that GPTZero — or even OpenAI’s own AI Text Reviewer, amongst a whole panoply of different AI detectors – are bad or poorly operationalized, by any means. Rather, it’s that the operationalization is supremely difficult because the task is punishingly hard, and that we are unlikely to have a tool that can detect AI-generated text 100% unless we perform watermarking (MIT Technology Review) and we would have to use multiple algorithms to be able to detect text, or come up with alternate measures to do so.

An Arms Race between AI Large Language Models (LLMS) and AI Detectors — and why you should care (even if you’re not a student).

As I’ve mentioned, there is an arms race at hand between AI Large Language Models (LLMs) like ChatGPT, and AI detectors like GPTZero, the consequence of which is likely that the two will compete with one another and each will make progress in its own way, progressing the direction of both technologies forward.

Personally, I think that AI detectors are fighting a losing battle against LLMs for many reasons, but let me not put the cart before the horse — it is a battle to watch, not to predict the outcome of before it’s even begun.

Implications of this strange war:

But why should you care about any of this if you’re not a student? It’s not like you’re going to be looking at essays constantly, right?

Let’s take aside the fact that you’re reading a blog post right now, and let’s also move away purely from the plagiarized essay bit that we’ve been thinking about, as we gravitate towards thinking about how ChatGPT is a language model.

It’s a good bet that you use language everywhere in your life, business, and relationships with other people in order to communicate, coordinate, and everything else.

When we go around on the Internet, it’s not always immediately evident what was AI generated, what was generated by a human or, for that matter, what was inspired by an AI and later followed through by a human.

The whole reason we need something like a plagiarism detector is that we may not even be sure that a particular piece of language (which we most often experience in the form of text) is AI-generated with our own eyes and minds, to the point that we need to literally rely upon statistical patterns in order to evaluate some thing that we are looking at directly in front of us, thereby recruiting our brains as we evaluate the entirety of an output.

The problem is…

Language doesn’t just exist as text.

Language exists as text, yes, but also as speech. Moreover, speech and text are easily convertible to one another — and we know very well what ChatGPT is doing: Generating text.

We now know that there are Text To Speech (TTS) models that generate speech from text. They’re not necessarily all great, but that’s besides the point — it presages the translation from text into voice.

Think about it.

If the voices that are generated by AI become sufficiently realistic-sounding and their intonations (VALL-E, is that you?), how might you know that these voices aren’t real unless there are severe model safeguards that impede the models from functioning as they are supposed to?

Now combine that indistinguishable voice with sophisticated ChatGPT output that can evade any AI detector and in turn may, depending on the features that end up developing, evade your own capacity to tell whether you are even interacting with a human or not.

How would that play out in the metaverse?

How would that play out in the real world, over the phone?

How would you ever know whether anyone that you’re interacting with is real or not? Whether they are sentient?

The battle between AI and AI Detectors is not just a battle over the difference between an A grade and a C grade.

It’s a battle over a future where what’s at stake is identifying what even qualifies as human.

Victor Tan

Victor Tan is incredibly excited about AI technology and its potential to transform the world. He is the creator of Transform Your Creative Writing With ChatGPT on Udemy and the author of The Little Robot That Could Paint, an AI-generated children’s book that introduces children to artificial intelligence that will release on March 14th, 2023. He is deeply passionate about education, and In his coaching and tutoring business Ascendant Academy, he teaches students not only how to sell themselves to top institutions, how to write effectively and conceptualize ideas with AI, believing deeply that AI is not here to replace us, but instead to help us to level up as a species as we conceptualize and create the most powerful tools that the world has known since the dawn of humanity. When he isn’t spending time writing, creating online courses, and creating the occasional video about ergonomic chairs and curious uses of ChatGPT on his YouTube channel, he’s probably  spending time coaching students to enter top universities in the US and UK, creating new Apple Homekit smart home automations, and playing an already unreasonably large yet still growing collection of musical instruments.

Leave A Comment

Recommended Posts

Assessing English Standards in Malaysia: An Analysis with the CEFR

Often in Malaysia, people talk about how our standard of English is either sufficiently good that it is the basis of a thesis for investment, or they say that our English is abysmal and needs drastically to be improved – discussions go on and on, and people fight, oftentimes in what seems like a battle for the soul of our society. But what does it mean, actually, that our English is good or our English is bad? Some say that Malaysia aligns itself to international standards in creating its curricula, but others squabble day in and day out, constantly complaining about the quality of English amongst graduates who come into the workforce, observing that many of them lack basic skills that they would expect graduates to have. How can it be possible that Malaysia calibrates itself to international standards while at the same time its graduates languish in terms of their English language proficiency? But at the end of the day, who’s right?  As it turns out, investigating a little further tells us that the answer is both. Here’s where the subject of our blog post for today comes in: The Common European Framework for Reference, otherwise known as the CEFR.  The reason that I’m making this comparison today and telling you about CEFR is that Malaysia uses it to calibrate SPM writing standards.  The CEFR operationalizes language proficiency in accordance with six dimensions, from A1 up until C2. It is an international standard that is utilized by examining bodies across the world in order to designate proficiency levels and descriptors that students attain after courses of study, and it is used also in designing curricula so that students can reach a certain defined standard. Source: https://www.cambridgeenglish.org/Images/126130-cefr-diagram.pdf CEFR operationalizes English proficiency according to numerous level descriptors, providing explanations of what a user of the English language should be able to do […]

On Facing Judgment’s Shadow

Picture this. You’ve written a post that you want to share on social media, or you’ve made a video somehow or another. You’re sitting there on the edge of your chair, just about to click post, but you look at what you’ve written, what you’ve made, you notice that final error, you question yourself, thinking about the manifold ways in which people could be judging you silently and from afar, contemplating in your mind’s eye the dialogues that must be taking place. “Oh my gosh, this person wrote this?” “Oh my gosh. Did he really make that grammatical error?” “Wow, this is boring. Why am I even watching?”“How could he make such a video?” And so the thoughts come out, percolate like coffee through filter paper, and eventually crystallize into little gems of self-doubt, blocking the nervous signal that would have caused you to click. You turn away from your plan, and you declare:”Maybe later, but not now.” Before you know it, the entire project is abandoned. If you’re anything like me, you may have faced this, this feeling of wanting to do something, but realizing, or at least thinking that you weren’t good enough, that the manifold imperfections that existed in you would come out, and that people would judge you one way or another. Well, here’s a fact, though. People certainly will judge you. I mean, how could they not? Everyone encounters something as a first glance, thinks about it, and evaluates it on their own terms. That’s just how it is. The judgment will happen. There is absolutely nothing that you can do about it, and your feeling certainly is right. The thing is though, that even though it is right, this isn’t a valid reason to run away. Because truly, the only way to get past it is to face your fears, to accept judgements as they […]

Work, Life, World.

One of the things I’ve come to really appreciate learning in college is the ability to just work, and work, and work. Sometimes I feel like my work ethic is just unending, and I can continue doing everything that I’m supposed to do, just efficiently and continually, almost like a machine.  I start, I take breaks specifically for the purpose of making sure that my work efficiency is maintained, then I begin again. The cycle continues, and life proceeds just in that way.  It’s one of the miracles that has come about, I think, from being trained in an environment where people were constantly working hard, and a place where people would not just work hard, but also have the right motivation, initiative, and desire for it.  Somehow or another, it influenced me, and rubbed off on me. And before I knew it, I was one of those workaholics out there, just casually pulling long hours without even questioning things, completely by my own volition.  It sounds bad, but I’ve come to appreciate that part of myself quite a bit.  It’s one of the many reasons that I respect myself, and appreciate the person that I’ve become. Because it’s become an enduring facet of my personality, and something that I know that I can look to whenever I think about my identity as a person, but at the same time, I think it’s come with at least two different disadvantages.  The first of these is that when I think about work, I just continue on and on. It takes hold like a vice grip, consuming almost every single aspect of my mind and my thoughts.  The result?  I just carry on doing things in the way that I feel most natural. Ignoring different things, socializing, hanging out, spending time with people, messaging…The list goes on, and I don’t know how many […]

My cardinal sin

If there is a sin of which I can unreservedly declare my guilt, it is the sin of over-complicating things. As a thinker, I tend to plan very extensively. I consider different things here and there and everywhere, and this manifests itself very often in just thinking about different factors and considerations that should go into a final plan. At the same time though, this manifests very often in overly complicated content, which in turn highlights another possible problem.  What if the fundamental problem isn’t actually over-complication, but instead the wrong way of planning, or planning in such a way that it doesn’t serve the goal at the end of the day?  If one simply were to distill things down to the simplest of all possible elements and in turn just use those things effectively, wouldn’t that create a transformation?  Wouldn’t that make things significantly easier at the end of the day?  This is something that I’ve noticed about myself, and it’s something that I would want to resolve in the days ahead. But what I’ve come to recognize and see is that simplicity requires effort. Completion requires work. Shortening requires sacrifice, judgment, and discernment. It’s not something so easy to develop.  One might say that it’s a talent, but in reality, it’s probably best considered as a skill. One to be honed over many successive days, months, years of practice that go into the development and creation of something that somehow ends up working out. As with all skills though, it’s not something that one can immediately receive just from having done it one single time.  Rather, one must do everything to make sure that it is perfect, tailored.  And although it’s possible to practice for many years, it may be that the person does not reach the destination. What is required is the correct planning, the correct noticing of […]

Automated Speech Recognition

Once in a while, a technology comes along that just completely transforms the way that we think, we live, and we experience the entire world.  ​​Certainly the entire world has been captivated by the rise of AI in recent days – how could it not, when millions of influencers around the world endeavor on a day to day basis to showcase the 500th AI tool that you ABSOLUTELY NEED TO USE on a day to day basis? Well, I don’t know much about technologies beyond ChatGPT, to be honest, but there is definitely one thing that has come out from it, which is probably the feature that I use the very most out of pretty much everything on the planet, and that is automated speech recognition, specifically, the OpenAI ChatGPT Whisper ASR Recognition System. Automated speech recognition is how I’m communicating everything here to you. It is how I’m putting down my thoughts, word by word, by simply sitting down next to this open door on a rainy morning, narrating out the story as if I were talking to you.  The Automated Speech Recognition Algorithm, which is in this case the ChatGPT Whisper app, is transcribing everything that I’m saying with an almost perfect accuracy, but perhaps with some small issues with punctuation that I will fix after the fact. It is incredible, tremendously accurate, and something that I could have never imagined just three to four months ago. As a result of this technology, as you read, you’re actually listening, in a sense, to what I said on that morning when the air was cool and the rain was falling, it was 7.49am in the morning, and 48 seconds had passed on the clock.  As I narrated these words then, eventually, the clock turned to 7.50, indicating a shift in time.  I made a mental note to myself at that […]

Insomnia

Insomnia is a horrible curse, but it’s one of those reminders out there that sleep is one of those things that we can rightly call a blessing and that ultimately, the greatest things within life come from within and not from without, from the simple acceptance of the struggles around us, rather than just continually trying to force things to happen in the way that we want. Nowhere is this more clearly represented than in the domain of sleep. The simple act of just closing your eyes and having yourself fall asleep is one of the most beautiful and restorative things in the entire world, and when it’s taken away from you, it reminds you at the end of the day that one of the greatest joys in this world is one that is inbuilt, one that lies within you, one that has the ability to transform you if only you will accept it, rather than challenging it by adulterating it with things like caffeine and other sorts of things out there that disrupt the balance, change the equation, and otherwise impede you from doing what is natural to you. That the body can rest is something that we assume by default should happen, but in reality it’s one of the greatest luxuries upon the planet yet one of the easiest to corrupt the moment, we allow our minds and our psychologies to take over and to take on the driver’s seat, during which we begin to push ourselves to go for more than what is natural, to push ourselves beyond to do certain things when in reality they are neither needed nor desirable for our ultimate furtherance. Often we sleep late just because we are worried about those things, about all the different matters that will not be completed if we simply do not fall asleep, yet ironically enough that […]