A few months ago a remarkable thing happened: I got a glimpse of the future.
OpenAI, a group of artificial intelligence researchers working towards developing benevolent AI gave me early access to a new API they were developing that would allow developers to utilize the most powerful text generator ever.
For background, a text generator is a neural network that tries to predict what should come next in a sequence of words. They’re a big deal because as I’m writing my brain is acting as a kind of text tgenerator deciding what I should type based upon the thoughts in my head. When your mail app or text application suggests words, that’s a similar kind of neural network at work.
What makes OpenAI’s API so remarkable is that it’s been trained by analyzing billions of examples, making it incredibly adept at picking up context and generating new text. Here are some examples of how intelligent it is.
Without any pre-training, if I give the API a list like this:
1. Batman
2. Superman
3. Cyborg
It will respond with something like this:
1. Batman
2. Superman
3. Cyborg
4. Flash
5. Martian Manhunter
6. Green Arrow
7. Hawkman
Okay…so what’s the big deal? Let’s break down what happened:
1. The API understood we were making a list.
2. The API understood we were making a list of superheroes.
3. It understood this was a list of *DC* superheroes.
If I gave those first three superhero names to most people, few would have followed up with only DC characters. The API picked up a pattern that isn’t obvious to everyone.
This ability to pick up patterns is what got me excited. I started playing around with the API nonstop and constantly pestered the people at OpenAI in a Slack channel asking questions and playing around with what could be done.
If the API can figure out you’re making a list of DC characters, what else could it do? What if I gave it three of my favorite science fiction novels?
1. Dune
2. Foundation
3. Player of Games
The API responded back with new suggestions:
1. Dune
2. Foundation
3. Player of Games
4. The Hitchhiker's Guide to the Galaxy
5. The Crystal Cave
6. Lucifer's Hammer
7. A Wizard of Earthsea
8. Hyperion
9. Desperation
10. The Mote in God's Eye
11. Ringworld
Not only are all the books from science-fiction and fantasy genres, out of the eight suggestions, I’ve read and loved five of them. The API didn’t just randomly suggest books. The neural network was able to make connections and create a list that was consistent with the first three entries. Fun fact: My last two Audible picks were from books the API suggested.
While that’s cool, the OpenAI API has so much more potential than just those examples. I have to emphasize that the API was never taught to make lists, figure out what’s a Marvel or DC character or make book suggestions. These were just random tasks that I threw at it and it was able to solve without hesitation. The only real limit was the imagination of the developer looking for ways to use it.
In the onboarding instructions for the API, OpenAI provided an example of a chatbot that could emulate different areas of expertise. You could tell it to be a physics professor or customer service agent and it would respond accordingly. As a writer, this got me thinking about the potential for using it to write dialog. You may have already seen a precursor to this, OpenAI’s GPT-2, generate fake news articles and fan fiction. The new API, built on top of a number of more powerful models, including GPT-3 (trained on 175 billion parameters) has even more potential. In particular, it’s very good at staying focused and not drifting too far from what you want. It’s less prone to turn a story about a football game into a recipe for brownies, as some neural networks have a tendency to do.
If I give the API an example like this:
Narrator: Kim and Jennifer are two scientists working in the Antarctic who just uncovered a large block of blue ice.
Kim: Have you ever seen anything like it?
Jennifer: Never. Let's crack it open.
Kim: Is that smart?
It continues the narrative:
Jennifer: Absolutely.
Kim: Alright.
Narrator: They begin to slice open the large blue mass with the chainsaw.
Jennifer: Look at this. It's even blue on the inside. That's amazing.
Kim: Hold that thought. This thing is making me feel dizzy.
Narrator: Suddenly, the entire thing begins to glow.
All I gave the model was the first few sentences. That was enough for it to pick up that Kim and Jennifer are characters in a story. It also picked up on the role of the narrator. What’s even more fascinating is that you can give characters secret agendas and have them know things that they’ll reveal later on or admit to when they’re confronted.
The API is able to mimic voice and tone. For example, if you have a character that’s from the Victorian period:
Rex is a time traveller from the future. Ada is a nineteenth-century noblewoman.
Rex: I think I crashed my time machine in your gardern.
Ada: Pardon me? What did you say young man?
The API responds with:
Rex: I think I crashed my time machine in your green thing.
Ada: If you are from the future then tell me, who won the fifth race at Goodwood?
Rex: I don't know who won the fifth race at Goodwood.
Ada: Why not? It's only just happened.
Rex: It hasn't happened for me.
The API has Ada using different language than Rex and also makes a reference to the Goodwood racecourse, wanting to know what horse won the fifth-race. I only had to mention once that she was a nineteenth-century noblewoman for it to pick up on her English pronunciations and make 1800s references. It also understood what it meant to be a time traveler. The implications for this are incredible.
As an experiment I built an interactive story app where characters would talk back and forth and new characters show up. Playing around with it was unlike any kind of interactive fiction I’ve ever experienced (for reference I’ve played with everything from Infocom games to the wonderful AI Dungeon.) The characters remembered who they were and would harbor secrets, argue, agree and act as separate entities with their own voice.
From the perspective of a novelist, I find this intriguing. I could see where this could lead to interactive hybrid stories where you follow characters through major plot points, make different choices and explore a textual work like a Choose Your Own Adventure that can go anywhere or a video game where you can roam freely. The API would serve as a navigator for a story that has a few canonical rules. In a way it’s the natural extension of the very first interactive electronic story “Colossal Cave Adventure” which was based on a physical map of the Mammoth Cave system. But in this situation, instead of just keeping track of location and inventory, the application would track character developments and spontaneously generate new scenes and characters.
While playing with my beta version of the storytelling application I thought it would be interesting to use real characters that the model had some prior knowledge about. For fun, I created a writers roundtable with a virtual Stephen King and Neil Gaiman and then asked them questions.
The model did a great job of emulating their different approaches. It even came up with what sounded like quotes from the writers, but as far as I can tell were spontaneous and invented by the model.
Andrew Mayne: How do I get great ideas for science fiction stories?
Neil Gaiman: I like to watch alien films, and I like to ask myself - what if? What if you went to Stockholm and you got mugged by Darth Vader?
That quote “What if you went to Stockholm and you got mugged by Darth Vader?” felt so on the nose for Neil Gaiman I thought it had to be something he actually said. It’s not as far as I can tell. The phrase “mugged by Darth Vader” has exactly one result on Google and it’s not from Gaiman. It’s in a description in a YouTube video that was uploaded after I ran this experiment.
Right now I have a version of this app on my phone that I play around with and use to question virtual Stephen King and others when I want an answers I can’t readily Google. One day, hopefully soon, you’ll have this capability too.
I’ve got conversational apps running on my phone where I can talk to Richard Feynman and Douglas Adams at the same time. I can message Ada Lovelace, Maria Curie and Charles Darwin. Sometimes the results are weird, other times they’re on point and feel like things they would have said or in some cases they bring up real references I’d never known about.
I did not get the idea for computer programs from computers, but from the desire of the soul to express itself through the medium of creation.
Virtual Ada Lovelace
There are a lot of ethical considerations and we’re all going to be facing new questions as AI becomes increasingly capable of simulating personalities. Should someone need my permission to run a virtual Andrew Mayne chatbot on their phone? While I think they should be able to simulate whatever they want in the privacy of their own device, would others feel the same way? How will I feel when more advanced APIs can write entire novels in my voice? This is closer than people realize. I’ve seen things that I think would change the minds of skeptics.
What has me really excited about this use case is the potential for education. If you can text message Charles Darwin, you can also build an app where you talk to him directly. With the rapid advancement of image generation you’d be able to see him and watch him speak in a manner that’s much more sophisticated than some of the novelty attempts at recreating famous people.
Everytime I ran a little experiment and got a surprising result I kept thinking to myself, “If this is what it can do now, where will all this be in five years?”
Playing around with OpenAI’s API, every day was a new discovery. I’d researched text generators in the past and built primitive neural networks, but nothing prepared me for this. The APIs grasp of context meant that you could build apps with just a few sentences and have it do everything from mimicking characters to summarizing Wikipedia articles so a 8-year-old would understand. Sometimes I’d try to solve a problem in code, like deciding if there was an Amazon link in a response, when all I needed to do was just build a little script that would show the API how to spot links. The work I’d done in the past in converting dialogue into narrative structure or vice versa could be reduced from several hundred lines of code to just four or five. It’s insane. I felt like I got to be one one of the first people to explore a crashed alien spaceship.
The AI Channels concept
As I created one mini-application after another I tried to wrap my head around what I’d like to do with this technology. I’d collected all my different experiments into one app that I ran on my phone. There was so much potential from storytelling to education, I couldn’t decide where to begin. Looking down at my screen one day I had a realization: What if I let other people create and add whatever apps they choose? I could make it easy for people to create conversations with anyone or write their own interactive fiction. On top of that, I could add the ability to share these experiences with friends. You could all go to Vampire High and interact with the odd students there. You could have a round table of real and virtual people of your choosing.
This idea of real and virtual personalities interacting was especially exciting because I’d already had experiences beta testing a Human/AI discussion thread where even I got confused about who was real and who wasn’t (OpenAI’s API is that powerful.)
With this in mind, I started building the AI|Channels apps. Right now it has over a dozen different mini-applications and a handful of beta testers. There are tons of behind the scenes details to work out and I have no idea what form it will be when I’m able to make it widely available. I might focus on one application and then build out from that. There are a lot of considerations. What I am certain of is that after interacting with OpenAI’s API and seeing the potential for humans and AI interacting side-by-side, the future is very exciting.
You can sign up for the beta at AIChannels.app and you can ask me questions on Twitter @AndrewMayne.
@AndrewMayne 