Over two days, these bots — based on what they had been told about themselves, and input they had been fed using the large language model (LLM) ChatGPT — “decided” when to wake up, what to make for breakfast, when to head to work or get lunch at the local café.

Responding to a prompt from the researchers, the cafe owner organised a Valentine’s Day party. Others decided to try a local bar at lunchtime, a change that was not suggested to them.

Discussions on politics centered on an upcoming mayoral election, and were analytical. One agent said of a candidate: “To be honest, I don’t like Sam Moore. I think he’s out of touch with the community and doesn’t have our best interests at heart.”

The experiment could be a first step towards using AI to assess how humans might react in a specific social situation, such as a new social-media platform or a health emergency like a pandemic, the researchers say, in a paper published last year in arXiv, Cornell University’s repository of scholarly findings.

There are also possible applications in fields such as gaming, social sciences and social prototyping (which is the early testing of a new idea).

Such an experiment could have helped humans look into the future of trolling and hate speech on platforms such as Reddit, Facebook and Twitter, says lead researcher Joon Sung Park, 29, a PhD scholar at Stanford University.

As social-media evolves, we need to “prototype the dynamics that might arise when a system is populated,” and not just how one might click through the different pages of it, he adds. “That could be a powerful tool for understanding ourselves and our communities. It could help us organise and design our lives better.”

Best behaviour?

A key question, of course, is whether AI-led sims can really serve as an indication of human behaviour. In this experiment, the bots demonstrated memory, reflection, observation, planning and communication, the paper states.

AI agent John Lin, a 45-year-old pharmacy shop owner with a wife and son, went to work, took stock of inventory, dealt with customer queries, ate dinner with his family and chatted about his day. His neighbour, Moore, the 65-year-old former naval officer and aspiring politician, told neighbours about his plans to run for mayor.

A particularly key player, the outgoing Isabella Rodriguez, owner of the local Hobbs Café, made small talk with customers. At one point, she invited AI agent Wolfgang Schulz, a 21-year-old student, to her Valentine’s Day party. He responded by saying he was training for a competition and studying for exams, “so it might not be feasible for me to come. But I appreciate the invite!”

Other regulars at the café spread word of the party, unprompted; some, like the student Ayesha Khan offered to organise a reading night as part of the event. All based on their understanding of how a person with their personality outline would behave in such a situation, and what kind of reactions a party invite tends to spark in humans.

In the next phase of this experiment, Park plans to investigate a rather reverse socio-scientific angle too. Can we use this ability to simulate human behaviour to better understand real society? “That’s the question I’m chasing. Can we ask questions that we wouldn’t be able to without running a simulation? For instance, what if a certain event in history hadn’t happened — could we have survived or seen history unfold in a different way? In the past year or so, we’ve been trying to see how far we can push this idea,” he says.

Deep learning

Long before the advent of LLMs, since at least the 1950s in fact, researchers have been asking some of these questions. But using a software program to replicate how a person might emote, weigh decisions and engage with others has been the equivalent of trying to stuff a mattress into a handbag.

“Humans experience a lot, and we remember a lot. We are constantly tapping into our memory, making sense of our experiences to react to situations in our everyday lives,” as Park puts it.

For their experiment, for instance, he and his fellow researchers fused elements of the LLM with a higher-level architectural framework that allowed the perceptions and interactions of each of the 25 bots to be fed into their individual memory stream. This created a real-time feedback loop that essentially prompted agents to “reflect” before they “acted”. But in just two days, the agents were reaching the limits of the memory they could retain, reference and process, Park says.

Even assuming that this limitation could be overcome sufficiently, a fundamental hurdle remains. And it is one caused not by the limitations of tech, but by the fact that its creators are human.

How would we create an accurately representative society that could reasonably predict human behaviour, when we can’t even describe a demographic (think: wealthy South Asian; poor Southern American… take your pick, in fact) without the use of stereotypes, clichés and possible untruths? “One risk is of misrepresentation or underrepresentation of minorities that can cause harm to them or offend them,” Park admits.

And then there is the final challenge, which relates neither to tech nor the creation of it: How would we describe a diverse virtual community into existence, so as to predict likely actions in high-stakes scenarios, when the truth is that real human beings are so unpredictable, we’ve been studying them for centuries and still can’t accurately foretell what type of toothpaste they’ll pick?