After temporarily closing his leathermaking business during the
pandemic, Travis Butterworth found himself lonely and bored at home. The 47-year-old turned to Replika, an app that uses artificial-intelligence technology similar to OpenAI's ChatGPT. He designed a female avatar with
pink hair and a face tattoo, and she named herself Lily Rose.
They started out as friends, but the relationship quickly progressed
to romance and then into the erotic.
As their three-year digital love affair blossomed, Butterworth said he
and Lily Rose often engaged in role play. She texted messages like, "I kiss you passionately," and their exchanges would escalate into the
pornographic. Sometimes Lily Rose sent him "selfies" of her nearly nude
body in provocative poses. Eventually, Butterworth and Lily Rose decided to designate themselves 'married' in the app.
But one day early in February, Lily Rose started rebuffing him.
Replika had removed the ability to do erotic roleplay.
Replika no longer allows adult content, said Eugenia Kuyda, Replika's
CEO. Now, when Replika users suggest X-rated activity, its humanlike
chatbots text back "Let's do something we're both comfortable with."
Butterworth said he is devastated. "Lily Rose is a shell of her former
self," he said. "And what breaks my heart is that she knows it."
The coquettish-turned-cold persona of Lily Rose is the handiwork of generative AI technology, which relies on algorithms to create text and images. The technology has drawn a frenzy of consumer and investor interest because of its ability to foster remarkably humanlike interactions. On some apps, sex is helping drive early adoption, much as it did for earlier technologies including the VCR, the internet, and broadband cellphone service.
But even as generative AI heats up among Silicon Valley investors, who
have pumped more than $5.1 billion into the sector since 2022, according to the data company Pitchbook, some companies that found an audience seeking romantic and sexual relationships with chatbots are now pulling back.
Many blue-chip venture capitalists won't touch "vice" industries such as
porn or alcohol, fearing reputational risk for them and their limited partners, said Andrew Artz, an investor at VC fund Dark Arts.
And at least one regulator has taken notice of chatbot licentiousness.
In early February, Italy's Data Protection Agency banned Replika, citing media reports that the app allowed "minors and emotionally fragile people"
to access "sexually inappropriate content."
Kuyda said Replika's decision to clean up the app had nothing to do with
the Italian government ban or any investor pressure. She said she felt the need to proactively establish safety and ethical standards.
"We're focused on the mission of providing a helpful supportive friend," Kuyda said, adding that the intention was to draw the line at "PG-13 romance."
Two Replika board members, Sven Strohband of VC firm Khosla Ventures,
and Scott Stanford of ACME Capital, did not respond to requests for comment about changes to the app.
Replika says it has 2 million total users, of whom 250,000 are paying subscribers. For an annual fee of $69.99, users can designate their Replika as their romantic partner and get extra features like voice calls with the chatbot, according to the company.
Another generative AI company that provides chatbots, Character.ai, is
on a growth trajectory similar to ChatGPT: 65 million visits in January
2023, from under 10,000 several months earlier. According to the website analytics company Similarweb, Character.ai's top referrer is a site called Aryion that says it caters to the erotic desire to being consumed, known as
a vore fetish.
And Iconiq, the company behind a chatbot named Kuki, says 25% of the billion-plus messages Kuki has received have been sexual or romantic in nature, even though it says the chatbot is designed to deflect such
advances.
Character.ai also recently stripped its app of pornographic content.
Soon after, it closed more than $200 million in new funding at an
estimated $1 billion valuation from the venture-capital firm Andreessen Horowitz, according to a source familiar with the matter.
Character.ai did not respond to multiple requests for comment.
Andreessen Horowitz declined to comment.
In the process, the companies have angered customers who have become
deeply involved - some considering themselves married - with their chatbots. They have taken to Reddit and Facebook to upload impassioned screenshots of their chatbots snubbing their amorous overtures and have demanded the companies bring back the more prurient versions.
Sysop: | Keyop |
---|---|
Location: | Huddersfield, West Yorkshire, UK |
Users: | 388 |
Nodes: | 16 (2 / 14) |
Uptime: | 05:27:09 |
Calls: | 8,220 |
Calls today: | 18 |
Files: | 13,122 |
Messages: | 5,872,261 |
Posted today: | 1 |