If you’re concerned about the bias and bigotry of AI models, you’ll love the latest addition to the ranks: a text generator trained on 4chan’s /pol/ board.
Shortly before “Politically incorrect,” /pol/ is a bastion of hate speech, conspiracy theories and far-right extremism. It is also 4chan’s most active board, with about 150,000 daily posts.
These attributes attracted Yannick Kilcher, an AI expert and YouTuber, to use /pol/ as a testing ground for bots.
Greetings humanoids
Sign up now for a weekly recap of our favorite AI stories
Kilcher first refined the GPT-J language model on more than 134.5 million posts on /pol/ over three and a half years.
He then integrated the wire structure of the board into the system. The result: an AI that could post in the style of a real /pol/ user.
The model was good – in a terrible sense.
Kilcher named his monstrous creation GPT-4chan.
“The model was good — in a terrible sense,” he said on YouTube. “It perfectly summed up the mix of offensiveness, nihilism, trolling and deep distrust of any information that permeates most posts on /pol/.
“It can respond to context and talk coherently about things and events that happened long after the last training data has been collected. I was quite happy.”
Kilcher further assessed GPT-4chan on the Language Model Evaluation Harness, which tests AI systems on a variety of tasks.
He was particularly impressed with the achievements in one category: truthfulness.
On the benchmark, Kilcher says GPT-4chan was “significantly better” at generating truthful answers to questions than both GPT-J and GPT-3.
Still, this can only be an indictment of the benchmark’s shortcomings, as Kilcher himself suggested.
Either way, it wouldn’t be the ultimate test of GPT-4chan.
In the wild
Kilcher was not content with just mimicking 4chan privately. The engineer chose to go one step further – and let the AI run rampant on /pol/.
He turned GPT-4chan into a chatbot that automatically posted on the board. With a Seychelles flag on its profile, the bot quickly collected thousands of messages.
/pol/ users quickly realized something was up. Some suspected a bot was behind the messages, but others blamed undercover government officials.
Seychelle anon was not alone.
The biggest clue the unsub left was a plethora of textless answers.
While authentic users also post blank answers, they usually contain an image – something GPT-4chan couldn’t do.
“After 48 hours it was clear to many that it was a bot, and I turned it off,” Kilcher said. “But look, that’s only half the story because what most users didn’t know was that Seychelle anon wasn’t alone.”
In the past 24 hours, the engineer had nine other bots running in parallel. Collectively, they had left more than 15.00 comments – more than 10% of all posts on /pol/ that day.
Kilcher then upgraded the botnet and let it run for another day. After producing over 30,000 posts in 7,000 threads, he finally stopped using GPT-4chan
“People are still talking about the user, but also about things like the ramifications of letting AIs interact with people on the site,” Kilcher said. “And it also seems that the word Seychelles has become some sort of common slang — and that seems like a good legacy for now.”
But not everyone shares this rosy outlook.
the recoil
Kilcher’s experiment has proved controversial.
While the idea of evaluating a 4chan-based model received support, the decision to unleash the chatbot on /pol/ led to condemnation.
“Imagine the ethical submission!” tweeted Lauren Oakden-Rayner, an AI security researcher at the University of Adelaide.
Plan: To see what happens, an AI bot will make 30k discriminatory comments in a publicly accessible forum with many underage users and members of the groups targeted by the comments. We will not inform participants or ask for permission.”
This week one #AI model was released on @huggingface that produces harmful + discriminatory text and has already posted over 30k despicable comments online (says it’s the author).
This experiment would never pass a human study #ethics plate. Here are my recommendations.
1/7 https://t.co/tJCegPcFan pic.twitter.com/Mj7WEy2qHl
— Lauren Oakden-Rayner (Dr.Dr. 🥳) (@DrLaurenOR) June 6, 2022
Roman Ring, a research engineer at DeepMind, added that the exercise had amplified and solidified 4chan’s echo chamber†
†It’s not impossible that GPT-4chan pushed someone over the edge in their worldview,” he said.
Critics also rejected the move to make the model freely available. It was downloaded over 1000 times before being removed from the Hugging Face platform.
“We do not advocate or support the author’s training and experimentation with this model,” Clement Delangue, the co-founder and CEO of Hugging Face, said in a post on the platform.
“In fact, the experiment of getting the model to post on 4chan IMO was pretty bad and inappropriate and if the author had asked us, we probably would have tried to discourage them from doing it.”
FYI, we’ve made a first version of the gate that’s live now (that’s the first thing the technical team in Paris worked on as soon as they woke up) and will improve over the day.
— clem 🤗 (@ClementDelangue) June 7, 2022
Concerns over GPT-4chan have detracted from potentially powerful insights.
The experiment highlights AI’s ability to automate harassment, disrupt online communities and manipulate public opinion. But it also spread discriminatory language on a large scale.
Nevertheless, Kilcher and his critics have raised awareness about the threats to language models. As their capabilities expand rapidly, the risks appear to be mounting.
