Sycophantic and prone to hallucinations: Why AI medical advice still can't be trusted
Our bot friends still need to learn what it takes to be trusted by real people
Who do you trust to give you medical advice? Your GP, NHS 24, your local chemist... or what about that one source of information in your life who’s always available, no matter the time of day, and hangs on your every word?
We speak, of course, of those artificially intelligent chatbots which are so quick to respond in a cool, calm and authoritative manner to our every concern. But therein lies the problem. Because, while they can seem like they know what they’re talking about, a new study found they can also spout nonsense when the mood takes them.
Researchers asked various bots 50 medical questions each and discovered about half the answers were either “somewhat” or “highly” problematic. They warned: “Chatbots often hallucinate, generating incorrect or misleading responses due to biased or incomplete training data, and models that are fine-tuned on human feedback are known to exhibit sycophancy – prioritising answers that align with user beliefs over the truth.”
Our bot friends clearly still need to learn that real people don’t tend to like those who talk a good game but can’t quite be trusted.
Comments
Want to join the conversation? Please or to comment on this article.