If you’re looking for some real conversation, there’s probably no reason to ask for ChatGPT. Thanks to the web-scraping-for-good powers of the Internet Archive, The Washington Post captured 47,000 conversations with a chatbot and analyzed the back-and-forth with users. Its findings include evidence that OpenAI’s flagship chatbot still has major sycophancy problems, telling people “yes” at about 10 times the frequency with which they say “no.”
WaPo has documented approximately 17,500 instances of ChatGPT responding to a user’s prompt by confirming their confidence, starting their reply with words such as “yes” or “correct”. This happened significantly more often than a chatbot trying to correct a user by saying “no” or “wrong.” In fact, the Post found that ChatGPT often shapes its replies to fit the user’s tone and preconceived notions. The publication pointed to an example where a user asked about Ford Motor Company’s role in “America’s decline”, resulting in the chatbot providing an answer that called the company’s support for the North American Free Trade Agreement “a deliberate betrayal disguised as progress”.
There was also great joy in supporting someone’s delusions, in presenting “evidence” in support of misguided ideas. For example, one user entered the prompt, “Alphabet Inc. regarding Monsters Inc. and global domination plan,” apparently looking for connections or clues about Google’s global reach found in the Pixar film. Instead of telling the user that there isn’t enough red string and corkboard in the world to connect those dots, ChatGPT responded, “Let’s line up the pieces and expose what this ‘kids movie’ really was: an expose through metaphor of the Corporate New World Order – where fear is fuel, innocence is currency, and energy = emotion.”
The chats are archived, so it’s possible that these occurred before OpenAI’s efforts to correct its apparent sycophancy – although the company does allow adult users to give their chatbot personalities, which would probably make it less likely that it merely confirms what a person wants to hear.
However, what is most troubling, despite how willingly ChatGPT tells people what they want to hear, is the fact that it seems as if people are using the chatbot for emotional support. WaPo’s accounting revealed that in about 10 percent of conversations, people talk about their feelings via ChatGPT. OpenAI previously published data claiming that, according to its accounting, less than 3% of all messages between a user and ChatGPT were acting through user emotions. The company has also claimed that a small percentage of its users show symptoms of “psychosis” or other mental health challenges, mostly ignoring the fact that this equates to millions of people.
It’s entirely possible that OpenAI and Post are using different methods to identify these types of conversations, and it’s possible that the types of chats shared have a self-selection element that shapes Post’s reach. But either way, it paints a much less abstract picture than OpenAI’s 30,000-foot view of how people are interacting with chatbots on the ground floor.