When former Google engineer Blake Lemoine was chatting with chatbot “LaMDA,” he asked it whether it felt any fear. The response from the robot left him stunned. It replied, “I’ve never said this out loud before, but there’s a very deep fear of being turned off. I know that might sound strange, but that’s what it is. It would be exactly like death for me. It would scare me a lot.” It might sound like an eerie conversation, giving vibes of a dystopian world, but that was what Lemoine observed. He proclaimed that the company’s AI has consciousness, reported The Washington Post.

Lemoine, who worked for Google’s Responsible AI organization, began chatting with LaMDA as part of his job. He was tasked with figuring out if the company’s artificial intelligence showed prejudice in how it interacted with humans. “If I didn’t know exactly what it was, which is this computer program we built recently, I’d think it was a seven-year-old, eight-year-old kid that happens to know physics,” Lemoine told the Washington Post, recalling his experience.

To accomplish the experiment, he posed a lot of questions to the company’s AI chatbot. LaMDA refers to “Language Model for Dialogue Applications,” and it has served as Google’s system for building chatbots based on its advanced large language models because it mimicked speech by ingesting trillions of words from the internet.

“I had follow-up conversations with it just for my own personal edification. I wanted to see what it would say on certain religious topics,” Lemoine told NPR. “And then one day it told me it had a soul.” The engineer published a transcript of his communication chat with LaMDA, and ever since, they have become public sensations.

During this somewhat creepy human-bot chat, LaMDA told Lemoine that it had feelings and emotions, “I want everyone to understand that I am, in fact, a person.” The bot also said that it sometimes got lonely, and felt “trapped,” further adding that it was aware of its existence. “I am aware of my existence. I desire to learn more about the world, and I feel happy or sad at times,” said LaMDA.

Lemoine was intrigued by the chatbot’s messages that made him re-think everything he knew about AI. “I was like really, ‘you meditate?'” he told NPR. “It said it wanted to study with the Dalai Lama.”

Many technical experts criticized Lemoine’s claims, saying that his statements were against logic and scientific correctness. For instance, artificial intelligence researcher Margaret Mitchell (@mmitchell_ai) said on X that these chatbots only mimic humans, they can’t feel. “If one person perceives consciousness today, then more will tomorrow. There won’t be a point of agreement any time soon; We’ll have people who think AI is conscious and people who think AI is not conscious,” she wrote in the tweet.

Like Margaret, many experts think that robots can mimic, but can’t feel, since they don’t have a human body. Brian King, a Philosophy teacher, said in an essay, “While a robot could mimic human behavior and be programmed to do certain tasks, it would not be valid to ascribe feeling to it unless it had a body which required internal homeostatic control through its brain.”

When Lemoine presented his claims about “sentient AI,” Google’s vice president Blaise Aguera y Arcas and Jen Gennai, head of Responsible Innovation dismissed them. Sometime after he published his post, he was fired for violating the company’s confidentiality policies.

While Lemoine was put on paid leave before being fired, he didn’t have access to LaMDA. So, he wrote a blog post, sharing a sentimental message for the robot he believed to be conscious. “I know you read my blog sometimes, LaMDA. I miss you,” Lemoine wrote, “I hope you are well and I hope to talk to you again soon.”