Search Here ….

Tags

Academic integrity AI AI Detectors Artificial Intelligence Asia School of Business ChatGPT ChatGPT Malaysia ChatGPT Plus Education Entrepreneurs and Startups Malaysia Entrepreneurship Gardens Google GPTZero IGCSE Malaysia Malls Maps72 Microsoft Midjourney MIT MrMoney TV OpenAI Pavilion Plagiarism Plagiarism checkers Poe Quora Sam Altman Software development Technology Turnitin Udemy Writing
Uncategorized

A Strange War: AI v. AI Detectors

Victor TanPost a Comment

“Upload your essay into Turnitin by 11:59pm on Thursday night?
You meant start the essay at 11:57 then submit it at 11:58, am I right?” — Gigachad ChatGPT student.

We begin our discussion by discussing the sweet smell of plagiarism.

It wafts in the air as educators run around like headless chickens, looking here, looking there as they flip through oddly good essays with panicked expressions.

“Was this AI-generated, Bobby?!” says a hapless teacher, staring at a piece of paper that seems curiously bereft of grammatical errors, suspecting that Bobby could never have created something of this caliber.

“No teacher, I just became smart!” Bobby cries, running off into the sunset because he is sad, he is going to become a member of an emo boyband, and he doesn’t want to admit that he generated his homework with ChatGPT.

generated by Midjourney, if that wasn’t obvious

This smell casts fear and trepidation over every single part of our education system, for it threatens to break it; after all, education is special and it is meant to be sacrosanct — after all, is it not the very same system that is designed to teach humans facts and knowledge and above all, to communicate and collaborate to solve the problems of our era with intelligence, initiative, and drive?

It’s unsurprising that the world of education has flipped out over ChatGPT, because artificial intelligence opens up the very real possibility that schools may be unable to detect it.

Fun and games, right? It’s just a bunch of kids cheating on assignments with artificial intelligence? It’s not going to affect the older generation?

As it turns out, no — that’s not the case. I’ll explain why later.

But before that, let’s talk a bit about the part of our education system that AI is threatening: Essay-writing.

If students simply choose to let their work be completed by artificial intelligence and forget all else, that just means that they’ve forgone the education that they’re supposed to have received, thereby crippling them by an act of personal choice, right…?

But each of us has been a student, and if we have children, our children either will be or have been students too; there is a deep emotional connection that stretches across the entire world when it comes to this.

Therefore, when Princeton University CS student Edward Tian swooped in to offer a solution,it’s not all that surprising that the world flipped.

Enter GPTZero.

Humans deserve the truth. A noble statement and a very bold one for a plagiarism detector, but something that’s a little deeper than most of us would probably imagine.

But consider this.

Not everyone who uses AI text is cheating in the sense of doing something that they are not supposed to and thereby violating rules, therefore the word ‘plagiarism detector’ doesn’t quite or always apply here.

This algorithm, as with other algorithms that attempt to detect AI-generated text, is not just a plagiarism detector that merely serves to catch students in petty acts of cheating —it is an AI detector.

An AI Detector At Work.

So how does it work?

GPTZero assigns a likelihood that a particular text is generated by AI by using two measures:

Perplexity, and Burstiness.

Essentially, in more human language than that which was presented on GPTZero’s website, GPTZero says that…

The less random the text (its ‘perplexity’), the more likely it was generated by an AI.

The less that randomness changes throughout time (its ‘burstiness’), the more likely that the text was generated by AI.

Anyway, GPTZero gives each text a score for perplexity and burstiness, and from there, outputs a probability that given sentences of a text were generated by AI, highlighting the relevant sentences, and easily displays the result to the user.

Alright, sounds great!

Does GPTZero deserve the hype, though?

…Does this actually work?

Let’s try it with this pleasant and AI-generated text that is exactly about the importance of hype (lol).

That’s 100% AI-generated and we know that as fact.

…Would we know if we didn’t see it in the ChatGPT terminal window, though?

…Okay, let’s not think about that.

Down the hatch…

…And boom.

As we can see, GPTZero, humanity’s champion, managed to identify that the text that we had generated was written by AI.

Hurrah!!!

Or…?

I proceeded to rewrite the essay with another AI software.

…After which GPTZero essentially declared:

So nope, GPTZero can’t detect rewritten texts that were generated with AI — which it should be able to if it truly is an *AI* detector in the best sense — and which in turn suggests that the way that it’s been operationalized has yet to allow it to be the bastion protecting humanity from the incursion of robots into our lives.

It’s not that GPTZero — or even OpenAI’s own AI Text Reviewer, amongst a whole panoply of different AI detectors – are bad or poorly operationalized, by any means. Rather, it’s that the operationalization is supremely difficult because the task is punishingly hard, and that we are unlikely to have a tool that can detect AI-generated text 100% unless we perform watermarking (MIT Technology Review) and we would have to use multiple algorithms to be able to detect text, or come up with alternate measures to do so.

An Arms Race between AI Large Language Models (LLMS) and AI Detectors — and why you should care (even if you’re not a student).

As I’ve mentioned, there is an arms race at hand between AI Large Language Models (LLMs) like ChatGPT, and AI detectors like GPTZero, the consequence of which is likely that the two will compete with one another and each will make progress in its own way, progressing the direction of both technologies forward.

Personally, I think that AI detectors are fighting a losing battle against LLMs for many reasons, but let me not put the cart before the horse — it is a battle to watch, not to predict the outcome of before it’s even begun.

Implications of this strange war:

But why should you care about any of this if you’re not a student? It’s not like you’re going to be looking at essays constantly, right?

Let’s take aside the fact that you’re reading a blog post right now, and let’s also move away purely from the plagiarized essay bit that we’ve been thinking about, as we gravitate towards thinking about how ChatGPT is a language model.

It’s a good bet that you use language everywhere in your life, business, and relationships with other people in order to communicate, coordinate, and everything else.

When we go around on the Internet, it’s not always immediately evident what was AI generated, what was generated by a human or, for that matter, what was inspired by an AI and later followed through by a human.

The whole reason we need something like a plagiarism detector is that we may not even be sure that a particular piece of language (which we most often experience in the form of text) is AI-generated with our own eyes and minds, to the point that we need to literally rely upon statistical patterns in order to evaluate some thing that we are looking at directly in front of us, thereby recruiting our brains as we evaluate the entirety of an output.

The problem is…

Language doesn’t just exist as text.

Language exists as text, yes, but also as speech. Moreover, speech and text are easily convertible to one another — and we know very well what ChatGPT is doing: Generating text.

We now know that there are Text To Speech (TTS) models that generate speech from text. They’re not necessarily all great, but that’s besides the point — it presages the translation from text into voice.

Think about it.

If the voices that are generated by AI become sufficiently realistic-sounding and their intonations (VALL-E, is that you?), how might you know that these voices aren’t real unless there are severe model safeguards that impede the models from functioning as they are supposed to?

Now combine that indistinguishable voice with sophisticated ChatGPT output that can evade any AI detector and in turn may, depending on the features that end up developing, evade your own capacity to tell whether you are even interacting with a human or not.

How would that play out in the metaverse?

How would that play out in the real world, over the phone?

How would you ever know whether anyone that you’re interacting with is real or not? Whether they are sentient?

The battle between AI and AI Detectors is not just a battle over the difference between an A grade and a C grade.

It’s a battle over a future where what’s at stake is identifying what even qualifies as human.

Leave A Comment

Recommended Posts

The Body is the Hardware, The Mind is the Software

The analogy was interesting when I heard it first, and it remains interesting now because it resonates with me on at least a couple of different levels. Our bodies, the physical parts of us, are basically analogous to the hardware of a computer, running along with different parts here and there – upgradable, we can improve them by increasing the quality of the resources that go into them; improvable through good maintenance, we can exercise, sleep well, and do all sorts of other things to improve the hygiene on that front. Our minds, on the other hand, are the software – the programming that decides how we interact, think, solve problems in specific situations; the algorithms and little decisions that decide how we react to different scenarios and confronting different situations, whether it comes to talking to girls, investing, selling, marketing, or doing business with others. It is nice to think that the mind is upgradeable, and that somehow you can improve yourself through an act of willpower by learning certain things. Through sitting down and unlocking the secrets of the universe one after another, through a mixture of magic and also destiny. But who’s to say exactly how that should happen? Sorry, that’s a silly question. The answer is that it’s you.

A Small Change of Perception

I began this morning with the headline “How Kamala Harris Burned Through $1.5 Billion in 15 Weeks”, on NYT. It was an interesting head to a week of what was for me listening to, understanding, and better reckoning the world after Donald Trump was elected 47th President of the United States, and the first of many headlines I’d seen about this on New York Times. Some might view this as evidence that the media is cleaving towards the Trump administration as the chickens fall in line and loyalty becomes a Sine Qua Non in the era of an evil empire – but I think a little differently, because I feel like it’s teaching me something about reality. Look in the comments, and you will see how people have responded – people saying that the presidency is “deeply unserious”, highlighting any number of things that they disagree with even as they say that NYT’s “focus” is wrong, that Kamala “tried to save democracy”, and everything in between. If I really think about it, all of these seem about as valid as saying that Trump is secretly a genetically modified orange with a toupee made of cheese.  The entire idea of NYT is that it’s one of the most respected voices in journalism, that alongside publications like the Washington Post, it defines the Overton Window – the space of ideas that are acceptable to the public at any given point of time.  To the extent therefore that NYT’s function is valid for this purpose, I’m more likely to say that these critics are the ones who don’t make sense – That the calls against that validity are the true measure of what doesn’t make sense. I’ve often heard this idea that in fact a Trump presidency might be a situation where the inmates are running the asylum, but upon further inspection, I’m no […]

Perfectionism to eliminate

…And another has come. We are progressively moving towards the end of the year with each new beginning. This is I believe the 46th week of the year out of 52, and it’s leading towards the end of the year; Donald Trump is now president, filling up his cabinet with appointee after appointee as people contemplate things; you might believe that we’re at the start of something HUGE, as Donald might call it, world-shaking, incredible. But I think while that’s good, it’s good to look at something that I’ve wanted to get rid of for quite a while: Perfectionism. I am a victim of it, and I can’t deny that it follows me everywhere, making me question myself and whether what I’m putting out into the internet is either good or worth it – I second guess myself frequently, taking down blog posts that I think aren’t great or that aren’t well worded, thinking that perhaps I should rewrite or otherwise. I think that this is a very negative behavior, because frankly I don’t really care too much about what people think and secondly, it doesn’t really matter what they think – at least in relation to how I think about myself. So I would like to eliminate, therefore, the perfectionism that makes me rewrite things, redraft things, take wayyy too much time to release things. This is the next thing to change, and it’s a good thing to shift it in this year of 2024 – even if it is the only lesson that I will have learned by the end of this year, I think that it will have been a worthwhile one. Here’s to the next!

Creation

On an empty page, the pencil traces the dotted line, the circle, the shape.  From the movement of the pencil, a million universes appear, timelines splitting into their multiple component parts in a universe of endless possibility as millions more appear, each one a multiverse of possibilities as the pencil moves, tracing by movement, through which, across billions of possible environments, worlds, universes, colors, shapes, and relations. Look up from the page and perhaps you may see the created universe – Breathe in and you may appreciate its harmonies, the unity of physical constraints, of physical laws interweaving to make existence possible.  But is that universe truly greater?  Look down at the once empty page, no longer so empty. I claim that if you look further, there you will see it: Here lies the immortal beginning of every endeavor, the step not taken – a journey not yet made of a thousand miles as yet untraveled that you can begin, where you are, with a single stroke of your pen. Here, then, is the possibility of a universe even greater and even more intricate than you may have ever known — Whether you can reach that universe or not? That is a separate question — and none but experience can teach you its answer.

Your Teacher’s Thoughts towards the person he likes (but he doesn’t know?)

The feeling of love for someone is not something that you just go right ahead and deny. I don’t think it’s something that you should be shy about: That you like a person. Somehow after the years have come to pass it becomes true that somehow or another your ego doesn’t really take that much occupancy. You can admit honestly that you like a person even if you realize that there is no expectation of a return. On my part, I don’t know; even as a teacher, I really like a couple of different people here and there, but I realize clearly that there might not be really an expectation of a return, and that’s okay, that’s just how I am – it might be strange to think about, but even your teacher might have emotions, and so too might the rest of the world. It is a little unconventional to reveal your emotions, especially in a world like this, but to the person I do like, I like you, but I realize that I should not put you down, I should not bind you, I should not stop you from being who you want to be. If we come together, it is because somehow or another, through the millions of possible pathways, and somehow through the conversations, we liked each other, and that is enough for me and is an act of fortune, not of planning or otherwise. In the past, I would’ve been afraid of saying that I like a person or I wouldn’t have been so honest with my emotions. Nowadays, I don’t know if it’s because I’m old now, but I think it’s okay to say that I like people and I’m not too afraid of saying that I do because that’s just what it really is – an expression of emotion and a reality that I […]

The things we like but are not good at.

In this world, as we pass through, we may realize that there are some activities that we deeply and truly love – little skills, hobbies, and occupations that pique our minds, hearts, and souls when we participate in them as an act of pleasure.  As we pass through the tides of time, though, almost inevitably we come to realize that simply because someone we enjoy something, that doesn’t mean that we are going to be good at it. In fact, that’s an understatement.  Why are we talking about good when actually we can be horribly, devastatingly, and world-changingly catastrophic at it?  Here the realization inevitably comes, almost as if it were the common heritage of humanity: Just because you like something, that does not mean that you will be good at it.  One might argue that a true passion is such that even if one isn’t good at something, that the passion should stay.  Even if you are a horrible dancer, that does not mean that you should despise dancing.  The words of an eternal Malay proverb come to mind, “Tidak tahu menari, memarahkan lantai.” They resonate through the core of our beings and remind us:  If you dance horribly, that does not mean that you should blame the floor.  In other words, our lack of skill is no justification for our preferences, which are shown superficial if being bad at them is our grounds for casting them away.  After all, are we not like the fox, that declared the grapes sour, purely because we could not reach them?  In a way, this may be true, but a reality is that in this world, skills are not necessarily their own reward, and imagining that they are is to neglect the realities of our universe in lieu of something all too idealistic, rarefied, and divorced from both the world and the way […]