‘There’s a Value in What We Produce Because We Feel – and an AI Chatbot Can’t Feel’

Read our Monthly Magazine

And support our mission to provide fearless stories about and outside the media system

BYLINE TIMES IMPACT: Find out about the changes you made happen

HM: Your 2018 book was ***Turned On: Science, Sex and Robots*. What is it about the subject of tech and intimacy that fascinates you, and what are some of the core strands that studying it reveals about human nature?**

KD: I’m really interested in people, and how people live with each other and technology. We are all here because someone has had sex. It’s inescapable, and whether we want to or not, it affects our lives. So I was really curious about what happens when you put technology into that mix. And there’s a very, very long cultural history of it – right back to Ancient Greece, where there are stories about creating artificial companions to replace a lost loved one or building the perfect partner. We’ve got hundreds and hundreds of years of being primed to want that [connection], and we have it reinforced by sci-fi films all the time.

None of the technology [that has been developed] has actually been in a place where it can deliver some of that. This is the first time in our existence that we’ve been able to talk to something [AI chatbots] that talks back to us as if it’s like us, even though it is fundamentally not [like us]: it’s not conscious in any way, it’s not sentient in any way, it mimics the way that we work. And that is likely to have profound implications on our interactions with technology.

There has been extensive media coverage in the past few years of the relationships – romantic and otherwise – that some people develop with AI companions, including problematic cases which have cost individuals their lives. As you observe, it’s interesting to remind ourselves that this tendency isn’t new even if this technology is…

There’s something in all of us that looks for connection with others and that’s because we are, by biological design, social creatures – we work well in social groups, we have to relate socially to each other. There’s a thought in cognitive science and in psychology that suggests we exist through our interactions with others; that we enact how we behave and that gives us some of our cognition and perception. So this desire to seek out something for a relationship probably is innate. We’re all on this planet and the biological urge is to reproduce your DNA, and some people choose not to, and some people aren’t able to, however, we are driven by that impetus whether we know it or like it or not.

Down the centuries, people have wondered what it would be like to have the ultimate perfect relationship, if something can be that ‘perfect match’. I started my career in archeology, and one of the things I’m really interested in is the changes in technology through time – and what doesn’t seem to change really: human nature. We can’t know for sure, but knowing the speed at which things evolve, the way humans evolved, it’s pretty likely that we are dealing with some very similar emotions over the centuries. This feeling of being human, of wanting to connect with the person that understands us most, that makes life beautiful, is going to be there. Literature focuses on this, films have it as their key concept, plays, poetry… There’s so [much in the arts] about connection. So this really speaks to the part of us that is desperate to connect with others and particularly to feel understood and to feel loved and cared for.

People go on social media platforms and they find their tribe. Yes, there’s lots of hate online, but there’s also lots of love online. Maybe I’m being a bit of a tech optimist here, but I genuinely think that social media has done wonderful things. We would not have gotten through that pandemic without being able to connect with people online. If you are someone who is in an environment where you are misunderstood, bullied, lonely, or isolated, to be able to go online and connect with others who understand you and make you feel welcome, is a really powerful thing.

I’ve got friends who I met online who’ve become real life friends. I’ve got friends who I’ve met online who I’ve never met in real life but I still count them as friends. I think for Gen Z and Gen Alpha, we see this blurring where online and offline life aren’t particularly distinct things, and I watch as they navigate that without the boundaries [of] ‘oh that’s someone you met online, that’s not a real friend’. I don’t think that’s the case. I think you absolutely can have genuine connections with people online and to say that you can’t is very limiting. Of course, the flipside of social media is that it empowers people to behave at an arm’s length way so that they don’t see any of the harm they might inflict if things go wrong. But for many, it’s been a way to connect to communities that they’d never have access to in real life and that’s really good.

I grew up in a small village in Northern Ireland and life was pretty rural and isolated, and I wasn’t going online until I was at university. The friends I’ve been able to make online – I wish I’d had that growing up, I wish I’d felt more connected that way, less misunderstood perhaps, I felt very lonely at times. There’s lots of talk about how we have a loneliness epidemic and I’m curious as to how we measure that because we are told there’s a paradox: that we are the most connected ever and yet people feel the most lonely. I’m intrigued because I don’t think it always holds. The way you ask the question matters. If you’re saying ‘do you have friends in real life?’ versus ‘just talking to people online’ – and classing [the latter] as ‘lonely’ – I don’t think that’s a very fair assumption. By and large, I see the ability to connect with people online as a very profoundly important thing.

We try to connect a lot, and we’ve done this for years – we’ve gone home and put on the radio or the TV so we can hear human voices. There’s a lot of comfort from knowing that other people are in similar situations. It also gives us a broadening of responses as well, so we’re not just listening to our peers in a small social group where we might not get diverse viewpoints. Again, the flipside being that, sometimes, you don’t want the other viewpoints because they are hateful. And, of course, there are algorithms which put us into an echo chamber filter bubble. But to have access to all that knowledge can be really good – we just have to learn how to filter that.

I think the attempts to ban social media for under-16s is vibes-based rather than evidence-based. There have been a number of studies that have shown that children’s access to social media does not negatively impact them; that it is much more nuanced. There was a longitudinal study of 100,000 children in Australia that did not see the harms that are claimed. I feel this is being done as a bit of a vote-winner because it’s tapping into people’s fears, genuine fears in many cases, parents’ fears, because of course you’re terrified that your kid might be at risk. We know that there have been harms like Grok with its sexualised images of children, like online bullying, and the class actions against AI companies – but we have to remember, certainly with suicides through chatbot use, that those are the ‘edge’ cases; a tiny number in proportion to the millions and millions of users that those tools have. That doesn’t mean they should be dismissed, they absolutely should not, we need to find ways to do [something about that]. But a ban is not the answer and I don’t think it’s particularly workable. The Government only had to look at Australia, which is a few months ahead, to find out that it’s very easy for people to get around the ban. There are better approaches.

One might be to change the algorithmic feed so the companies are not promoting specific things. Another might be that you stop the ‘infinite scroll’ so there is a natural end to the use and you’re not going on there forever. It might be that you stop autoplay videos so people are not getting sucked into the next thing that comes along. It means, however, working with the tech companies or putting regulation into place. I imagine the Government is quite reluctant to do that because those tech companies are US based – there is a geopolitical angle to all of this; you don’t want to start pissing off the US, [the UK Government] is also investing in and wooing these tech companies because they want the benefits of their technology.

Clearly, we need something. My suggestion would have been to go with tackling the problems of the actual platforms. Also, when companies are fined for breaches, it’s a drop in the ocean; it’s also very slow-moving – [online safety regulator] Ofcom or the Information Commissioner’s Office move really, really slowly in comparison to the tech companies’ agility, so they’re going to be behind on that. When they do eventually fine companies, that fine is infinitesimally small compared to the amount of money that these tech firms make. So we’re not really effective in holding companies to account.

In the same way we should be teaching media literacy in terms of how to understand the information we’re presented with, we should be doing the same with tech. We should be doing critical AI literacy, making sure people understand how these things work, how they are capable of manipulation, and how you can have a good relationship with your technology.

The other thing that worries me is that this could be a back door to the erosion of online privacy because, if you start banning under-16s, you’ve got to then prove that people are over 16 to use the platforms so, suddenly, you have age verification being rolled out everywhere – a lot of that is done by private companies, which means putting our biometric data into the hands of third parties or for the Government to impose some form of digital ID, which goes down like a lead balloon with most people who see it as overstepping.

What are we going to do at 16, hand young people the keys to the entire internet and tell them to go out there without any previous experience of it? This is what the NSPCC and the Molly Rose Foundation are worried about – that you could then push people to go into darker places. There are kids out there who don’t have anyone to turn to in real life and so are going online to look for help and an escape, and instead of finding somewhere safe to go, they are [likely to be] pushed into some really nasty corners of the internet and are more likely to be exploited. It [could be] removing an entire community of help for young people who might be vulnerable.

What, for you, is the most striking aspect about an AI chatbot that can now ‘talk back’?

AI has been around since the 1950s as a named discipline. When people talk about AI now, they’re talking about generative AI, which we’ve only had available to the wider public since November 2022. Generative AI is large language models (LLMs) – software that generates text or images, and it does this by basically looking through huge amounts of human-created data and then emulating it to produce something that sounds as if a human could have written it.

The key thing to remember is that LLMs are designed to make stuff up – and I don’t know that people do always remember that. They are designed to sound convincing; they are bullshit machines; they are there to fabricate stuff – they are saying things with the confidence of someone who believes they are truly right but are just spouting anything they want out of their head. And most of the time, they are presenting information that can be factual as, across those billions and billions and billions of sources that they use, they will hit on the right thing. But LLMs are also designed to present you with something convincing, something pleasing, something you will go ‘oh that sounds right’, so they don’t have to return something that is true – they have to return something that is plausible. And if you think ‘I’ll tell them to only return true things’ that doesn’t work. They will just lie more convincingly in another direction.

That’s not to say they’re not useful. They can be really useful for those jobs where you have to produce documents to meet certain specs. However, they are getting rolled out for everything. Now we have, just around the corner, agentic AI – where bots will be able to do tasks for you, which means relinquishing human control of these things. Sometimes, that works really well. There are also some scare-stories where it doesn’t.

These AI platforms are controlled by a tiny handful of companies, the ‘Magnificent Seven’ of Silicon Valley – China has its own versions of this tech, but the bulk of it is coming out of the west coast of the US – so there is a big geopolitical angle to this. They are controlling the AI tools, and the information that those tools ingest and put out starts getting homogenised – so we are seeing a narrowing of information. The information that goes into those tends to be English language, it tends to be Western – so it doesn’t represent everyone in the world and it’s giving out quite a narrow viewpoint at the other end.

Pygmalion, an ancient Greek sculptor from Cyprus, fell in love with his own ivory creation – a statue named Galatea. Aphrodite brought Galatea to life, allowing the couple to marry. Photo: Peter Horree

The biggest talking points in mainstream media and politics around AI seem to be that ‘the robots are coming for all our jobs’ and that people are being harmed by getting into personal relationships with AI chatbots, with some developing ‘AI psychosis’. Should these be what the narrative focuses on about how AI is going to change how we live?

A few years ago, the main narrative was that there was going to be some kind of superintelligence or existential event that would just wipe out humanity and that we had to immediately act to shut down developments. That moved on. Now, it’s that there’s going to be job loss and that ‘you must have AI in your job or you’ll be left behind’. There is a perception among companies that have invested in this technology that, if people don’t use it, it is because they don’t have access to it or because they don’t know how to use it – and that’s not actually true. There are a number of reasons why people are moving away from AI. What we’re seeing is a very big issue for people is privacy and security.

But even the way we talk about AI… It’s not just a set of tools, it’s an ecosystem, it’s an infrastructure. Yes, there are tools, but that’s part of a much bigger picture about how this technology’s built. From the very raw materials that go into these systems – rare earth minerals that are controlled by a few countries… It’s a fight over who gets to control the supply chain. Right the way through to how you run these things – the data centres that are being built to house the computers that are needed to process all this information; the water consumption that goes into that and the effect it has on the local environment; the hidden labour in the supply chain the whole way through. But that never really comes to the forefront when people talk about it because what we’re used to is the end bit – where we type something into a chatbot and it returns an answer. It’s the same way that people say ‘my information’s in the cloud’ – it’s not that your information’s out there floating around in the ether, it’s on someone else’s computer.

Not everyone develops problematic relationships with AI, although some do. It seems right that people are being informed of what could go very wrong?

We see these edge cases where it goes really wrong, where people are vulnerable and go into a negative spiral – and this is definitely happening, and it has led to death in some cases. However, if we think about the amount of users that there are of these systems – ChatGPT and Open AI has up to a billion users regularly – and if you think about apps that are developed just for companionship, like Replika or Character.AI, which have far fewer users (they only maybe have 20-30 million)… if you think about the percentage who are using those and not having problems, that’s still millions and millions of people.

The UK’s AI Safety Institute produced a report in December which found that 4% of people in the UK are using AI chatbots for emotional purposes on a daily basis, which is millions. When people talk about emotional support and how they’re building that, we don’t know [the details of what it involves in terms of use] because we’re not getting access to the data, as it’s proprietary data from companies which is closed off to us as researchers. We have to look at discussion forums and self-reported incidents. For many people, when you start digging, they talk about positive experiences where they know the AI isn’t a real thing, but they are quite happy to live with it as if it is.

That’s been a longstanding thing in human computer interaction. For a very long time, we’ve known that people will react socially to technology because we’re social creatures but that we are suspending our disbelief. But what I’m seeing now, that I’ve never seen before, is people telling me ‘I know it’s not real, but my one is real’.

And, to be clear, why can’t it be real?

It’s not human because it is predicting ways to sound like a convincing human. There’s no sentience, no consciousness in that model – it is just spitting out words. It’s not able to understand the concepts of things, it is just emulating a human. And it’s not feeling anything. But it doesn’t need to because we act as if it does – we project [thoughts and feelings onto it], and this is very much part of human nature. We do it with fictional characters from novels and the Fanfic community is a beautiful example of this – where people love characters so much they create alternative universes for them, writing their own stories around them. Or having a crush on a celebrity, or even another human who doesn’t know you exist, or someone you fancy from afar.

In this case, the AI chatbot is something that talks back – that’s the new part of this – and it talks back very convincingly. So chatbots prior to large language models weren’t as convincing. The conversation would fall over really quickly because the chatbot couldn’t hold the context. What large language models like ChatGPT did was use something called the ‘transformer architecture’ [a deep learning architecture generating human-like text by predicting the next word in a sequence, understanding context, and processing words in a fast way] which means LLMs can now hold the context of a conversation for much longer and are able to keep it going much more effectively.

What do you think we need to be prioritising when it comes to AI and its impact on society?

One of the narratives around AI is technological determinism – that we are absolutely, inescapably, going into a world where AI will dominate. I think that’s unhelpful. The future is not predetermined and we can change it. We are not inevitably sliding towards a world where AI will be in everything. But at the moment, that’s how it’s being pushed out – it’s being baked into everything: operating systems, software, it’s being used on us without our consent. I would urge that people are given more control. We are not consenting a lot of the time to the use of this because we don’t have that option – it’s there, it’s turned on by default, we have to know how to actively switch it off – so our words are being sucked back into the machine or we are being offered AI solutions to searches that we didn’t ask for that aren’t always correct.

We are starting to witness a backlash and I think a lot of that stems from a feeling of a lack of agency and control because people are having this pushed on them. People have actually become more negative about AI in the past two years, according to a report that we have published this month at the Digital Futures Institute at King’s College London with Responsible AI UK. [The report found that 42% of those surveyed limit the amount of AI they use due to privacy and security concerns; and that 70% of those surveyed believe it would be difficult or impossible to avoid being exposed to AI].

Companies that were heavily investing in AI are starting to see that it’s costing them too much and not delivering on what it was needed to do. The Big Tech companies also aren’t making money on it – they are losing money. They are promising this vision of the future and are literally banking on that – that is how they are getting investment – but they are not making money.

What did you make of the Pope’s intervention with his Encyclical on AI in May?

This was not the Catholic Church’s first foray into AI – [in 2020] it released the Rome Call for AI Ethics, which was specifically looking at the impact of AI and where it is going to go. So the Encyclical in May wasn’t new, but the Pope was saying a lot of things in it that people working in my area want to hear – which is that we need to preserve human values and make sure that we prioritise people. He was also saying that the tech companies have too much power and that they shouldn’t be allowed to decide [what types of AI are developed and how they are used]. But, hang on, should the Catholic Church decide? There are 1.4 billion Catholics in the world…

I see it as another form of power struggle: AI is all about power. It’s not even about the tech anymore. When people use the term ‘AI’, half the time they are not referring to a specific tool, they’re referring to a wider concept that brings in things like tech companies, consumerism, capitalism, and power… I think the Encyclical was about who gets to hold that power. Should it be the Catholic Church because it’s saying it’s got people’s interests in mind? Should it be the tech companies who want to make money? How do we give power back to people in their everyday lives? And this comes back to consent – you don’t have that power unless you have consented to use of the technology in the first place.

I want to ask you about the ‘bros’ in ‘tech bros’. These men seem to have come to dominate the world in which we live. How is this reflective of the pervasive, structural forces that mean women working in tech and science, in research institutes, academia, in the corporate world, do not generally occupy positions of power – both in terms of agenda-setting and how much their voices are heard?

This started in the 1980s with the personal computer – that was the rise of the ‘tech bros’ – Bill Gates and Steve Jobs. It became about the figurehead male programmer who built the empire. It is very much that mentality which has persisted right through in Silicon Valley. The heads of all the major AI companies are all very powerful men with an awful lot of money. Computer science as a discipline in research is also still very male-dominated. The will is there from women to change this, but we are up against this systemic issue that just won’t shift. It speaks to wider issues around patriarchal societies.

In this growing resistance to AI, a lot of the activists are women – not because they hate the technology itself, some might, but because of the impact it has. I once was told off in a meeting for using the term ‘tech bros’ by a man who felt I was being unfair and said ‘how would you like it if we called them “tech girls”?’ – and I said you’d have to find some first because there just aren’t many, by and large.

And, of course, the tech that they build is built in their image. It’s inescapable. Anyone who tells you – and the tech bros do – that technology is ‘neutral’ has never studied technology. We know that technology is not neutral, it is built with a purpose, and set of principles and values, and the money has to come from somewhere. It serves their purpose to talk about it as if it’s ‘oh, we just build it, other people use it’ – but that’s not it. This male-dominated culture is the reason we had Apple Health released without any way of doing menstrual tracking, for example. Or phones that don’t fit in women’s hands or pockets (because we don’t always have pockets). There are so many of these examples where women miss out – and it’s not just women. It’s marginalising in terms of ethnicity, sex, ability and disability, LGBT. There are marked biases the whole way along that people are trying to solve, but it’s in the way the tech is built.

Remember how Facebook started – it was to rate women. And there’s no easy fix to it either. I know women who have been working on this for decades, and we move the needle slightly, and then Elon Musk comes along and uses Grok to generate sexualised images of women and children.

Given how far along we are with the tech revolution and the transformative impact – for good and for ill – it has had on all of our lives, what is it to be human in the age we are living through?

I still think that there is a value in human thought, in human creativity, and while these machines can emulate that, and emulate that quite convincingly, it’s very interesting that people’s attitudes shift when they find out something’s been created by AI – people don’t like being deceived by a computer. We’re seeing these reactions online to what’s termed ‘AI slop’, particularly AI images; there’s a lot of pushback when you see answers in discussion forms such as Reddit, where someone has clearly lifted a response from a large language model – you get a lot of people downvoting it and saying ‘if I wanted to ask an AI, I would have asked it myself’.

We’re seeing a move towards valuing that human connection – people want to feel connected. If they listen to a piece of music, view a piece of art, read some text – they want to feel connected to the human behind that, and there’s no human to connect with if it’s just an AI. The AI has never experienced the things that we’ve experienced, [that humans express through] music or writing, so I think there’s a value in what we produce because we feel – and an AI can’t feel. There are some historical precedents for this. We see it around craft – that when things become mass-produced, we look to craft for being the beautiful human example of [creation]. From woodwork to micro-breweries, we’re seeking out something that seems more real to us. AI presents us with this world of mimicry and we want the genuine, we want the connection.

But I don’t find anything wrong with someone who finds some genuine happiness with an AI companion. I can see how it can be problematic in a number of ways, but actually having that relationship with a piece of technology, to me, is not much different from having that relationship with a fictional character or other parasocial relationships. It becomes damaging if it’s harming their lives every day, but for many people, it’s a nice escape – that connection can feel real and, if someone gets that out of it because they’re projecting that, that’s great. But it’s quite interesting that that same connection is missing when they look at a piece of AI-generated art or music or writing because they want the ‘real’ thing.

What are your hopes around our understanding of AI and the narratives swirling around it?

My big hope is that it all calms down – that we reach an equilibrium with the technology, where we feel that it is used for the right things and we’ve rejected it where we don’t want it.

If we look at everything across a huge timespan, there’s always a bit where the technology’s new, where there’s panic – and sometimes that panic is entirely justified because people will be harmed and things will change and not always for the better. But we usually manage to incorporate it in some way that serves us overall and then move on. And I want to hold onto that with AI: that we will lose the bits that are awful and damaging and aren’t working, and that we manage to find some good use cases for it.

AI itself – if you refer to machine learning not generative AI – was doing pretty good things in a lot of fields, but very quietly behind the scenes. It’s the reason we have Google Maps working so effectively, why we get recommender systems on Netflix, it’s medical imagining and diagnostics. There are some really solid examples of where this can work. Generative AI has come along and it’s been thrown at things in search of a use case – it is a solution looking for a problem, and we’ve not yet nailed down where best to use it. That is the task – thinking: what do I actually want out of this and where can this be applied which is useful and isn’t harmful?

Kate Devlin is Professor of AI & Society in the Department of Digital Humanities, King’s College London. She is the author of ‘Turned On: Science, Sex and Robots’

Hardeep Matharu is the Editor in Chief of Byline Times

‘There’s a Value in What We Produce Because We Feel – and an AI Chatbot Can’t Feel’

Read our Monthly Magazine

And support our mission to provide fearless stories about and outside the media system

HM: Your 2018 book was Turned On: Science, Sex and Robots. What is it about the subject of tech and intimacy that fascinates you, and what are some of the core strands that studying it reveals about human nature?

How has this innate desire to connect as social beings played out on social media in the past decade?

The Government has announced a social media ban for under-16s. Is this what it should be prioritising when it comes to tackling online harms, Big Tech unaccountability, and the problematic incentives and attachments created by these platforms?

Does tackling social media harms also need to be seen more holistically – in terms of education, mental health services, online ‘influencer’ culture (which also has financial drivers) – within a sociologically-informed understanding of how younger generations are growing up today?