Did AI Write This Blog Post?
By Rachel Ware
In middle school, I had a project in English class where I had to write a story in the style of Edgar Allan Poe. We read several of his short stories and poems in class and had to come up with something original from our understanding of how he wrote. I remember loving the assignment because I had enjoyed the style of writing, and I wrote a dark story about someone getting lost in the maze they had created in their own backyard. When I found the story again years later, I realized that I hadn’t quite hit the level of vocabulary Poe used, but it was not bad for a sixth-grader. At that time, a computer might have been at my level or worse. But today I think a computer - with the right AI - could definitely out-Poe me.
Today I am also a bit out of practice. As a computer science student I write a lot, but not much in the way of dark fiction. But that is not the only reason a computer could more faithfully match an author’s style. There is an area of Linguistics and Artificial Intelligence (AI) research and usage called NLP (Natural Language Processing). This deals with speech and text, so think of Google trying to guess your response to an email or your voice assistant understanding what you wanted to add to your grocery list. There are many uses for this kind of technology anywhere language is used. And there is good motivation to want to improve it, as anyone stuck asking a website chatbot questions knows. However, this is also difficult as language is complex and ever-evolving. Linguists have worked for a long time to track rules of language and change over time, and now trying to use computers to understand language provides new challenges and opportunities.
There have been many AI-based approaches to NLP, but in 2017 a way was shared called the Transformer. In short, this AI was a step up in both quality and speed from other AI models for translating between languages. Its approach has shown the ability to be used across several types of NLP tasks and inspired the type of model used in 2020’s gpt-3 (the third generation of the Generative Pre-trained Transformer) (link). Gpt-3 is interesting because not only is it good at generating human-like text, it was trained on the internet with the most data of any of this type of model so far. More data analyzed generally makes a better AI, but you also need a model that can synthesize that data with quality and relative efficiency as it takes a long time to analyze billions of sentences.
I have been learning about transformers and AI for NLP recently myself, and while I could not explain to you simply how it works (this video does a good job of explaining the original transformer: https://youtu.be/TQQlZhbC5ps), you can have some fun with what it can do. I will stress again that NLP is used across so many tasks, and this is just one example of how it can be used in generating text. This demo will complete the text you enter: https://app.inferkit.com/demo, and if you try it, you might find some different things. Maybe you put in a couple of words and the generated text seemed very different than what you wrote, or maybe it fits very well. Or maybe you didn’t put any text and it just created something on its own. In my experience, it can generate text that is funny or dark, makes a lot of sense, or seems random. But what stands out is that it usually sounds pretty natural. The words go in an order that makes sense and someone could have written it, even if the subject of the text is unusual.
So what do we do with this? The website above is a demo within a contained environment, so you can create something small without worrying about its impact. But what if I wrote this blog post with it? (I did put a few sentences into the demo since I figured the Transformer could probably describe itself better than me, but it didn’t quite hit the points I was looking for). Maybe it could help you start your own short story, and AI is certainly used in programs by companies to better anywhere they want to use natural language without having a person do all the work. There are a few downsides, some technical, like the fact that it can only take in up to a certain length of text at a time, doesn’t constantly learn (that’s the “pre-trained” part of its name), and still makes mistakes that are noticeable on close inspection. There are also the issues of bias that exist throughout AI. This remains an issue, and considering the training of the model is from data on the internet, existing biases are learned by the model, and it can generate hateful language that has to be tuned out by people adjusting the model (link). So while it is an incredible thing, there are still important steps that have to be done before it can be used in the real world at scale, both technical and ethical. It remains important that for every technological leap, there is a diverse group of discerning people considering what it does well and where it falls flat.
Sources:
