The family of a woman who died from cancer has claimed ChatGPT misled her terminally ill mum and gave her “false hope” about how long she had left to live.
The woman’s adult child took to Reddit to share the chats they had found on their late mum’s phone.
“My mum died two weeks ago from colon cancer,” the post began.
“I was out of state and only flew home in time to spend a few days with her, it was a huge shock to me as I’d intended to move back home for 8-12 months to care for her until she passed.”
They also claimed that their mum was “doing really well” prior to her death.
While going through her phone to cancel her subscriptions and active accounts, they stumbled upon her ChatGPT conversations.
They alleged the chatbot misinterpreted the medical notes and told her she had years left to live, and should stop her treatment.
The chatbot allegedly told her: “I’m going to answer as honestly and directly as medicine allows, the most truthful answer is: You are more likely looking at years”.
“I am furious at the idea of this chatbot giving her false hope,” they wrote.
“When I got there, she didn’t even recognise me anymore 90 per cent of the time because the cancer had spread so aggressively.
“She was terrified during the end, and I can’t help but wonder how much of that is because she expected to live for years longer.”
There have been growing cases of people turning to AI for health advice, and some have had fatal cconsequences.
Western Sydney University AI and legal expert, Armin Ashenberg, said it wasn’t just a matter of incorrect medical advice.
“Believing you have years left rather than weeks could change whether you call your children home, have difficult conversations or prepare for death,” he told news.com.au.
He also said the advice is an example of “sycophancy” or when AI acts overly agreeable and that the authoritative phrasing like “the most truthful answer” make is sound more reassuring.
“I don’t think ‘The user should have checked’ because of a generic ‘AI can make mistakes’ disclaimer at the bottom of a chatbot page is enough for everyday users,” the expert said.
“They are dealing with systems that generated content in a way that encourage people to trust them because they sound authoritative and persuasive.”
OpenAI, the company behind ChatGPT, has maintained that their AI tools are only designed to help users feel prepared when talking to doctors, not replace them and that people should not substitute it for professional medical care, diagnosis, or treatment.
The company also claimed they are working with hundreds of global physicians to help update their models to respond with better judgement and recognise urgent situations.
Image: Primakov / Shutterstock.com
