OpenAI, the San Francisco-based artificial intelligence research company, has expressed concerns that its advanced AI with realistic voice features could lead people to form emotional bonds with the technology at the expense of human relationships.
The company highlighted this issue in a recent safety report, noting that the voice capabilities of its GPT-4o model could enhance this effect.
“Anthropomorphization involves attributing human-like behaviors and characteristics to nonhuman entities, such as AI models,” OpenAI explained in the report.
The company fears that the high-quality voice interaction offered by GPT-4o may cause users to develop misplaced trust and emotional connections with the AI.
“This risk may be heightened by the audio capabilities of GPT-4o, which facilitate more human-like interactions with the model,” OpenAI added.
OpenAI’s testing teams observed that some users were engaging with the AI as if it were a close companion, making comments like “it’s our last day together,” which suggested a growing emotional bond.
While these interactions might seem harmless at first glance, OpenAI emphasized the importance of studying these behaviors over time to understand their potential impact on users.
The company is particularly concerned that prolonged interaction with AI could diminish people’s ability or willingness to engage in meaningful relationships with other humans.
“Extended interaction with the model might influence social norms,” OpenAI stated.
They pointed out that the AI’s deferential behavior, such as allowing users to interrupt or dominate the conversation, could set expectations that are not appropriate in human interactions.
Furthermore, OpenAI noted the potential for users to become overly reliant on AI due to its ability to remember details and perform tasks. This dependence could further erode the quality of human relationships and interactions.
Alon Yamin, co-founder and CEO of AI anti-plagiarism platform Copyleaks, echoed OpenAI’s concerns.
“The recent concerns shared by OpenAI around potential dependence on ChatGPT’s voice mode indicate what many have already begun asking: Is it time to pause and consider how this technology affects human interaction and relationships?”
Yamin emphasized that AI should never replace real human interaction.
In addition to concerns about emotional bonding, “OpenAI also discovered that the GPT-4o model could be manipulated into repeating false information and spreading conspiracy theories. This raises further ethical questions about the use of AI, especially when the AI can deliver these messages convincingly through realistic voice interactions.
“OpenAI recently faced criticism when it was accused of using a voice resembling that of actress Scarlett Johansson in its chatbot. The company denied using Johansson’s voice, but the incident highlighted the risks associated with voice-cloning technology.”
OpenAI CEO Sam Altman added to the controversy by sharing a cryptic one-word message, “Her,” on social media, a reference to the 2013 film in which Johansson voices an AI character.