Is AI Filling In for Humans?

|

,

In The Conversation Magazine, Ambuj Tewari examines a subtle shift in how some AI surveys are conducted. Instead of asking real people, researchers are having AI respond on their behalf. Tewari, a machine learning researcher at the University of Michigan, explains that while these AI stand-ins are cheaper, they do not accurately reflect what people truly think.

Tiwari notes that fewer people are answering surveys these days, and reaching enough real respondents is expensive. One provider, for example, charges tens of thousands of dollars for a 10-minute survey of 1,000 participants. To cut costs, some researchers are now using AI in place of humans, a practice called synthetic surveys, or silicon sampling.

Below are the key points from Tiwari’s article:

  • How it works. According to Tiwari, a pollster gives a model like ChatGPT a short profile of an imaginary person. The model answers as that person would. By changing the profile and using the model’s built-in randomness, researchers can produce thousands of different replies, fast and cheap.
  • Why it’s a problem. A survey is meant to measure what people actually think. Tewari compares it to a thermometer. You would not trust one that guessed your temperature by asking an AI. He points out that Models also pick up hidden biases from their training data, and can misrepresent groups that are underrepresented online.
  • A useful comparison. Tiwari acknowledges that AI uses made-up data in many fields, such as self-driving cars and medicine. But there, the data is tested against the real world and fixed if it fails. A fake survey is different. In Tewari’s words, it “is running a simulation of public opinion.” A wrong result may go unnoticed until it has shaped policy or business choices.
  • Where AI does help. Tiwari thinks that AI can still improve surveys without replacing people. It can write clearer questions. It can translate surveys into other languages. It can sort and summarize written answers. Some teams now mix smaller human surveys with AI analysis.

In a related story from TIME magazine shows the same worry in the job market. Writing in TIME, Rebecca Schneid highlights on recent comments from OpenAI’s Sam Altman.

  • A surprising admission. Altman told TIMES that he was “delighted to be wrong.” He had expected AI to wipe out many entry-level office jobs by now. That has not happened as fast as he predicted.
  • The picture is mixed. Schneid notes that the reversal is striking, since Altman had long warned about job losses. Even so, layoffs continue, and some companies are still cutting roles.

Tewari’s perspective on surveys is straightforward. AI can assist survey administrators, but it shouldn’t replace the individuals whose opinions the survey aims to capture.

Acknowledgement: This summary was generated using AI writing tools as part of my AI Explorations process. For more, see the original articles linked below.

Related Resource Links