A recent OpEd editorial in the Star Tribune writes that the ChatGPT is an unreliable interlocutor in its current state. “Everything it tells you has to be considered provisional. You may as well parse a search engine’s long lists of possibly relevant links from the start, because you’ll end up there anyway.”
We’ve spent some time with ChatGPT, version three. This is not the latest version; its creator OpenAI — a San Francisco-based research enterprise with both nonprofit and for-profit components — has released version four behind a paywall. Nor is it the only example of a “large language model” — Microsoft and Google are among those implementing their own. But it is the most widely accessible currently.
Among our wide-ranging interactions, ChatGPT almost instantly summarized the precedents and legal doctrines in a Supreme Court ruling, and it offered a sophisticated discourse comparing the Talmudic and Jesuit religious traditions. A widely curious person might have trouble finding a fellow human so willing to join the perambulations of the mind.
The editorial board wrote that it gave them much a lots of inaccurate information, which they have documented as follows.
But ChatGPT also confidently misinformed us that Minneapolis has a moderate climate; that Ely and Hoyt Lakes are actually in southeastern Minnesota, where there is also an array of military installations; and that the Fleetwood Mac song “Dreams” was written and sung by the band’s Christine McVie, rather than by Stevie Nicks. The times we asked it to cite sources, our double-checking showed that it made up some of them. When we pointed that out, it apologized and made up different ones instead.
But the danger isn’t so much that AI chat will give you bad information. It’s that it will give you enough apparently good information that you won’t recognize its errors.
The editorial ends with a conclusion: “For our part, as we’ve recommended for other technologies, we’d allow some time and space for things to play out. Specific regulatory needs will emerge soon enough.”