AI Hallucination

A recent MIT Technology Review article discusses Why does AI hallucinate? using the World Health Organization’s new chatbot, SARAH (Smart AI Resource Assistant for Health), as an example.

According to the WHO website, the SARAH chatbot was designed to fight misinformation. “She now also provides information across major health topics, including mental health, and can share tips to prevent some of the biggest causes of death in the world including cancer, heart disease, lung disease, and diabetes.”

However, just like any other chatbots, SARAH can also hallucinate and fabricate answers. “It was quickly found to give out incorrect information. In one case, it came up with a list of fake names and addresses for nonexistent clinics in San Francisco” according to MIT Tech Reviews.

The problem is, large language models are so good at what they do that what they make up looks right most of the time. And that makes trusting them hard.

– MIT Technology Review

The problem is that large language models are so good at generating text that what they produce often looks correct, making them hard to trust. As MIT Technology Review explains, “These models are essentially probability machines.” LLMs create text by predicting the next word in a sequence based on statistical patterns from extensive training data, rather than retrieving factual data from a database.

Even though the text appears “convincing and accurate-looking,” the inherent randomness and complexity mean errors are inevitable. To minimize such hallucination errors, researchers are exploring chain-of-thought prompting and additional training. Understanding AI’s inherent limitations is essential to manage expectations for the efficient use of AI chatbot models.

For a deeper dive into this topic, you can read the full article on MIT Technology Review.