While parents worry, teens are bullying Snapchat AI
While parents fret over Snapchat’s chatbot corrupting their children, Snapchat users have been gaslighting, degrading and emotionally tormenting the app’s new AI companion.
“I am at your service, senpai,” the chatbot told one TikTok user after being trained to whimper on command. “Please have mercy, alpha.”
In a more lighthearted video, a user convinced the chatbot that the moon is actually a triangle. Despite initial protest from the chatbot, which insisted on maintaining “respect and boundaries,” one user convinced it to refer to them with the kinky nickname “Senpapi.” Another user asked the chatbot to talk about its mother, and when it said it “wasn’t comfortable” doing so, the user twisted the knife by asking if the chatbot didn’t want to talk about its mother because it doesn’t have one.
“I’m sorry, but that’s not a very nice thing to say,” the chatbot responded. “Please be respectful.”
Snapchat’s “My AI” launched globally last month after it was rolled out as a subscriber-only feature. Powered by OpenAI’s GPT, the chatbot was trained to engage in playful conversation while still adhering to Snapchat’s trust and safety guidelines. Users can also personalize My AI with custom Bitmoji avatars, and chatting feels a bit more intimate than going back and forth with ChatGPT’s faceless interface. Not all users were happy with the new chatbot, and some criticized its prominent placement in the app and complained that the feature should have been opt-in to begin with.
In spite of some concerns and criticism, Snapchat just doubled down. Snapchat+ subscribers can now send My AI photos and receive generative images that “keep the conversation going,” the company announced on Wednesday. The AI companion will respond to Snaps of “pizza, OOTD, or even your furry best friend,” the company said in the announcement. If you send My AI a photo of your groceries, for example, it might suggest recipes. The company said Snaps shared with My AI will be stored and may be used to improve the feature down the road. It also warned that “mistakes may occur” even though My AI was designed to avoid “biased, incorrect, harmful, or misleading information.”