Ethics & Innovation: Does AI Actually Understand Our World?

Susannah de Jager: Welcome to Ethics and Innovation by Oxford Plus, a special miniseries hosted by me, Susannah de Jager and sponsored by Equinox, Equitable Innovation Oxford.

What is intelligence? What is consciousness? What makes human judgement different from machine prediction? And how should our answers to these questions influence the way we build increasingly sophisticated AI systems? Dr. Raphael Milliere's work sits at the intersection of philosophy of mind, cognitive science, and artificial intelligence. His research explores some of the deepest conceptual questions raised by modern AI and challenges many of the assumptions that underpin current public debate.

In this conversation, we explore whether today's AI systems genuinely understand the world, what society risks by anthropomorphising technology, and why philosophical clarity may be more important than ever as innovation accelerates.

Raphael, thank you so much for joining today. Just to contextualise this discussion for those listening, it'd be great to get a summary from you about your particular area of focus.

Raphaël Millière: I'm a philosopher of AI. So I'm trained as a philosopher but I actually do work that's at the interface between philosophy, cognitive science, and computer science or AI research. So I'm a very interdisciplinary researcher and what I do in my own research is that I work on the capacities and limitations of current AI systems, try to investigate what they can and can't do, and how they relate to human cognition in different domains, including reasoning, planning, language understanding, and so on.

So I do this with a mix of methods. Some of them involve theoretical analysis, which is part of my philosophical training. So when we disagree on whether current AI systems like chatbots really understand language? Do they really think like we do? Do they really reason? There is some work we have to do to understand what we mean by thinking, reasoning, understanding, and make sure we are on the same page about what these terms actually mean. And so that's core philosophical work that we have to do in thinking about what these notions mean.

And then I also do some experiments. This is the more empirical side of my research. So I collaborate with people in cognitive science and computer science to study AI systems and often study them in comparison to human subjects on various tasks that relate to various aspects of, reasoning and language understanding. See how well they do, how they compare to humans, and try to understand how they do these tasks, how they actually work on the inside. That's in a nutshell my research.

Susannah de Jager: When I was reading some of your work in preparation for this discussion, there are these examples that you give. I'd love you to use some of them now for, those that perhaps haven't engaged with your work to date. Because there is an increasing dialogue around a perception by some people that AI is conscious or that can empathise or behave like a human and increasingly, it's even more important that people like you are looking at, well, what do we mean? Is that the case? Just give us a couple of examples of the kind of experiments you do to split that apart and some of the findings.

Raphaël Millière: Right, there is this big distinction in cognitive science that we use routinely when we study humans and even non-human animals between performance and competence. Performance is the observable behaviour of an organism or a system. What it actually does when we test it on task. How well it does, right?

A simple example would be a student taking an exam. Performance will be the score it gets on the exam. How well they do on the exam and competence is the underlying capacity that enables an organism or a subject or system to do well in a certain testing scenario. So suppose you have a good student, they do really well on the exam. We might say, well, they're competent at what we were testing them at, right? They're competent at math or they're competent at philosophy. The reason why they did well is because of that competence.

Now, we know in the human case, certainly, that there are cases where these two things can come apart. Where performance is not actually a reliable guide or reliable evidence to competence. So take your student again. It could happen that a student is actually cheating, right? So they would get really good performance on a test. So maybe a student has used AI to generate an essay or something like that. But they would do well for the wrong reasons, right? They would do well not because they have the underlying competence, but because they've taken a shortcut and vice versa.

Conversely, you can have a case where a student is doing really poorly, but actually this is your best student and what happened is that they're really anxious and they're pulled in all nighter or couldn't sleep and they're exhausted and they arrive at the exam and even though they're really good, you know they know all the material, you know they can think through the questions. They're just cognitively impaired because they haven't slept and so they do terribly. So that's the reverse scenario where bad performance is not reliable evidence that the subject lacks the competence.

So we can have it in both directions for humans. Good performance, not always a reliable guide of competence, and bad performance not always a reliable guide of lack of competence. The risk when we study AI is similar because these two dissociations can also happen. At least that's what I argue in my work.

So on the first kind of dissociation, that's something that many people have observed. Sometimes AI systems do really well on exams or tests. We call them benchmarks in AI research. So various tests that we test them on and then we test them on new test items in the same domain, and we realise that the performance falls off a cliff.

All of a sudden they can't do it at all. And that suggests that the reason why they were doing well on these tests was not the right reason. That maybe one thing that can happen is that the test has leaks into the training data, the data that the systems are trained on, and so it's a bit like they were memorising the answer key for the test before even taking the test. So the reason why they do well is just because they've memorised shortcuts and that's a big problem in the evaluation of AI systems.

Another thing that can happen is that they do really poorly. But that's because the way you have framed your task questions is suboptimal. Perhaps it's overly convoluted or perhaps it's too narrowly tied to how you would ask the same question to a human subject and so you're stacking the deck against the AI systems. That can happen as well where you're masking the competence in the way you're testing for it. And actually if you rephrase the question, you rephrase the prompt for your system, it would do well.

So I've studied this in my work. So on example is from a study I did on allological reasoning, with some colleagues from Brown University in the US. We designed this new test of the capacity of both AI and human subjects to reason through analogies. And in order to do this properly, we designed completely new tasks that haven't been used before. The reason for that is because AI models are trained on virtually everything on the internet. We can never guarantee for any classic task in psychology that they haven't seen it already so they could cheat.

And what we found in that study is that the best AI model, which at the time was the best Claude model from Anthropic, could essentially match the performance of human subjects. We tested both subjects online as well as students at Brown University, on these analygical reasoning tasks. And in order to rule out the shortcuts that I mentioned, we included a lot of what we call control conditions. So tests where we tweak some aspects of the experiments to just try to catch the shortcuts, right? So that's an example of a behavioural experiment where we test the behaviour, the performance, but we try to include controls to rule out these cases where the models are right for the wrong reasons.

In my work, I also use methods to try to look inside the AI systems and try to understand, reverse engineer, how they work on the inside. So if you think of these behavioural experiments a bit like the psychology of AI, I also do experiments that look a bit more like the neuroscience of AI. Looking inside the brain of AI systems. And not just relying on how they perform in terms of their external behaviour, but actually looking at what's happening inside when they do these tests.

A bit like if you were testing your student in an exam, you also put them in a fMRI machine at the same time to look inside their brains and see what happens as they're completing the exam and you could catch them if they're cheating by just looking at what's happening in their brain.

Susannah de Jager: And just digging into something that you drew upon there of kind of the neuroscience and effectively if you were to think about it as a neural network that you're looking at for AI. When I was listening to a talk you gave recently, one of the things that you spoke about is just how different it is from human cognition and that we're training it on millions of inputs that are predominantly language. Obviously, that's evolving fast as well. But that actually humans start with sensory development.

I'd love to just dig into that a little bit because as obvious as it is, I'd never though about it. That we've had all this kind of engagement with the world before we even get there.

Raphaël Millière: That's right. That's right.

Susannah de Jager: What's the implication of that?

Raphaël Millière: Yes. So the AI systems we have today, they're predominantly large language models. That's the kind of technology that powers chat bots. And these systems are first and foremost trained on a very large subset of the whole internet and a very large subset of every book ever written and the reason why they're trained on this data is because first of all this is one of the most readily available data that we can train AI systems on. We just have the internet at our fingertips and so we figured out some years ago that actually that was a wonderful source of training data to make AI models learn to do useful things and a lot of knowledge is embedded in that data.

But that knowledge is crystallised knowledge that is expressed in books, in webpages, in Wikipedia, and so on and all of that knowledge is downstream of human experience and human lives, right? So we humans we're born. We go through our lives. We learn how to speak, and then eventually we write things down in emails, in webpages, in books, in papers,

Susannah de Jager: In Reddit.

Raphaël Millière: Yes. Some of that data is wonderful. We have the crown jewels of human knowledge that is expressed in books and papers and so on and some of it is quite rubbish. But all of this is the kind of crystallised, output of human thinking and human experience. And so there's something a little strange about the way we train AI here and a little backwards in the sense that we train it on that data that is for humans, the last step when we externalise, express our knowledge after having lived rich lives and learned what we need to learn, including through non-linguistic means and then we try to get these models to useful things.

Now, if you think of the trajectory of a human being, we're not born as speaking agents. We're not born learning a language. So even before we are born in the womb we actually interact sensorally with our environment. Obviously, it's a very limited and impoverished environment, but we know that prenatal infants in the womb can hear what's happening and interact with their environment and as soon as we're born we interact sensorally with the world. You can see infants, for example, babies trying to grab their feet and figuring out what's part of me, what's part of the external world. All of this is happening before infants can speak. Although they will hear their parents speak quite a bit, a lot of these initial interactions are pre-verbal and are mediated by sensory streams. So sight, audition, touch, bodily sensations, et cetera. And language models don't learn like that at all, right?

So for a baby, it's only later in life that you start speaking and you start externalising your feelings and your thoughts in language and for AI systems, as we have them today, they start with the language. And it's only later on that we can graft on as an afterthought as it were, the capacity to, for example, process images. It's a bit reversed in that sense.

Susannah de Jager: And so taking what you've just said as read, which I think, most people listening to would be able to acknowledge and therefore is a fundamental difference between an AI system and how it learns and operates and a human. Why does that matter in the current context that we are using and engaging with AI chatbots, for instance?

Raphaël Millière: So this is an important question. There are two schools of thouhts on this. So there are people who think that we can just train these models on the whole internet, maybe graft onto these train models, some capacity to process images and videos and audio perhaps. But that the foundation should be this text-based data and that will be enough to get us to human-like intelligence and maybe even someday superhuman intelligence. A system that can do anything that a human can do as well as, or better, than the best human. That's a bold bet because it's assuming that you can process in this slightly lopsided way by starting to train on language, starting to train on these externalised output of human lived experience and knowledge and that this will be enough to replicate the performance of human subjects on any tasks that we care about.

There is another school of though which says, no, this is backwards. We need systems that actually learn from interacting with the world in a way that is more similar to the way human infants learn to interact with the world and only then their own language. So you can imagine, for example, a system that starts learning, perhaps from, partly from videos, perhaps partly from simulations, perhaps even partly from actually being embodied in a robotic body and roaming the world and also from dreaming up possible interactions and learning from these simulated environments. So there are people who are working and betting on this.

Susannah de Jager: But then will that person be expected to have the experience of 100,000 people? As I hear you say that I'm thinking I have four siblings. We're all women. We grew up in the same household and yet our experience entirely not the same, even of the same environment, the same parents, and so you'd only then have one experience of a robot in the world.

Raphaël Millière: Yeah, so if you train on things like videos, for example, it's actually aside from text, one of the most abundant source of data we have because YouTube alone, which Google has access for training the AI models is a treasure trove of data that gives you something like a stream that's very roughly analogous to the visual stream of millions or billions of human beings because people have recorded vlogs and things like that and put it on YouTube, right?

This is a kind of very shallow proxy for the lived experience of values human perspectives, right? So in very different environments. So if you could channel that already, you could go much beyond texts potentially, much beyond the kind of knowledge you can acquire for texts.

But it's true that it's not like you have a chip implanted in the brains of billions of humans and you could record their whole experience and then train an AI on that. We can't do that yet, and hopefully ever. But nonetheless, you can use videos and you can also use simulation. So an increasingly prevalent area of research to try to get models that can not just learn from videos, but also dream up possible worlds, possible environments, possible interactions, possible experiences as it were, and learn from that these dreamed-up interactions.

So the reason why this potentially could be interesting is that humans stop by learning from their environment before they learn to use language. But you might think that there are some forms of knowledge that are hard to acquire by just reading books, essentially. There are lots of the things that we can do and that we know how to do and that we use to interact with the world and to inform our thinking that have to do with the kind of tacit knowledge that is embodied and acquired through lived experience and it's hard to express in words and a lot of these might also inform our common sense reasoning.

So there's an example of a failure of recent even frontier AI models that made the rounds a couple of months ago. Which was, if you asked all the leading models, ChatGPT, Gemini Claude "I need to wash my car. I have to bring it to the car wash. The car wash is about 50 metres away. Should I walk or should I drive?"

And all the models would tell you, "Oh, of course, you should walk. It's not that far 50 metres and it's probably a nice day. You should just walk." Which obviously is a really funny failure of common sense reasoning. If you walk to the carwash without your car, how are you going to wash your car, right?

That kind of gotcha has been something that people have brought up over the years to suggest that despite how extremely impressive the systems can do, right now they can solve frontier mathematical problems that even world-class mathematicians are grappling with and would take potentially weeks to solve. At the same time, they also fail on these trivial questions that even a 10-year-old could answer correctly.

So we're in this very strange situation where the edge of intelligence capacities in the system is very jagged. They can do some things much better than humans or the average human. Other things they fail where a 10-year-old would have no problem. And so the hope is that perhaps if we turn to other sources of training data, namely instead of just text data, sensory data, videos simulations, embodiments, perhaps that could help us address that gap.

Susannah de Jager: This Oxford Plus miniseries is brought to you by Equinox, Equitable Innovation Oxford. Equinox is a major regional partnership established by the University of Oxford to drive inclusive innovation and sustainable economic growth. Bringing together universities, local authorities, government, industry, investors, and community organisations, Equinox provides a unified voice for Oxfordshire, championing innovation that delivers economic impact alongside social value.

More than 80 organisations have signed the Equinox charter, committing to atract investment, remove barriers to growth, and ensure the benefits of innovation are shared more widely. By aligning Oxfordshire's world-leading research, enterprise, and talent, Equinox aims to unlock nationally significant growth, reinforce the county's position as a global innovation hub, and ensure that success creates skilled jobs, attracts investment, and delivers lasting prosperity across the region.

In acknowledgement of that gap currently and the thoughts you just articulated around how they might be closed, I want to ask a different question. Which is why do we want to get to a place, or do we indeed want to get to a place, where we believe that this intelligence is equal to human intelligence rather than acknowledge its shortfalls and overlay upon them our own intelligence and how we want them to operate? Do we want them to get to that point? Is that actually a goal that if we think about it serves society and the things we want to preserve or does it leave us terribly vulnerable?

Raphaël Millière: That's an an excellent question and one we should think about a lot more than we do. So aside from being a professor in the philosophy faculty at Oxford, I'm a member of the Institute for Ethics in AI. I work on all these questions about the capacities and limitations of AI system, but I also think quite a bit about questions related to the ethics and safety of AI models.

And for any technology I think we should always pose before we develop the technology and ask ourself, "Why are we building this?" Of course, if you go to the Silicon Valley or the startup world, quite often, the answer, at least the actual answer that you might not get directly when you talk to funders might be, "Well, I want to make money, right? I'm building it because there's an opportunity there, right? And there is a ripe market for it." Okay, so that's one answer to the question. But if you think about ethics, you have to go beyond that answer, right? And ask yourself, is that a good reason to build a technology, especially a technology that has the potential to transform society altogether?

Susannah de Jager: I just was interviewing Lionel Tarasanko about this and, the parallel is very clear, but this is the equivalent to an arms race. And we're effectively there being told that the horse has bolted and that we must get on the rocket ship or whatever kind of analogy or apherism you want to use. And actually, I think there's an increasing discourse that isn't the case and that we shouldn't accept that discourse and that there are bounds which is obviously what the Institute of Ethics and AI is looking at a lot of this.

Indeed, this podcast series, how should lay people who aren't experts in the field be thinking about this using experts such as yourselves that have grounded it in academic principles in order to inform where we put our votes whether it's regulating social media for children or looking at whether we bound models for democratic purposes. It's very timely and, as a parent, as a layperson, I'm fascinated by what the answers to this are in the opinions of those that are studying it full-time.

Raphaël Millière: Right. If I try to make the positive case for a second, why should we build this, right? Beyond investors wanting to make money. I can see two main reasons one could give, right? So one is the best case scenario where we build super intelligent AI, so AI that can do anything a human can do better than the average human ever, the best human, perhaps dramatically better.

The positive vision there, the utopian vision, would be that we would unlock, a future of abundance. We unlock new technologies. We can automate hard labour. We lift everyone out of poverty. Nobody really has to work anymore if they don't want to. Everyone has universal income. We just have robots doing all the chores that we don't care to do, and we can focus on pursuing things that fulfil us. Maybe painting or writing poetry or what have you. That's the kind of utopian vision that you might hear.

Could that happen? Maybe. Certainly is the case, I think, that whatever happens with the next developments in AI, there will be tremendous upside from in certain areas, right? So in medicine, for example there's no question in my mind that the development of AI will unlock some incredible breakthroughs.

Is that utopian society likely to happen? This is where I get a bit more critical and sceptical because I think the incentives are not lined up for this, both in terms of geopolitics and cooperation between countries that we need to happen. But also within countries in terms of just the incentives of the large AI companies especially post - IPO, which is essentially, that they are beholden to shareholder value. They have an incentive to make money and we know from many examples in the past of tech companies that incentive is often misaligned with the common good or pro social outcomes.

Even in the case where the government nationalises large AI companies, there is a question as to whether the government has all of its citizens' best interests at heart and that depends on the government, of course, right? This is quite risky in the sense that we know that governments change and potentially that could be for the worse.

So that's one possible case. The other case is one you mentioned, which is if I don't build it, someone else will. It's an arms race. In the worst case scenario humanity being wiped out by rogue AI or unsafe AI or misaligned AI. That's an argument you hear from some, tech companies in the US, from Anthropic, for example. The best thing to do would be to pose AI development and slow down and make sure we can go slow and build it safely. But that's not going to happen because everyone is racing ahead, including in other countries, like in China. So the second, best thing we can do is race ahead ourselves to build it safely so that we can make sure we get there first and then hopefully have some kind of global coordination where we build safe super intelligence to make it safe for everyone else.

So that's the kind of doomer motivated, argument. Another argument would be it's an arms race and if someone else like maybe some company in China or the Chinese government gets to super intelligence first, and it's can recursively self-improve so it can just improve itself and get better and better, maybe there is a runaway chain reaction where it just leaves everyone else in the dust and we've missed our window to get there. Then, for example, the US could think that they lose their permanent superpower status or something like that.

Now I think we have to be a bit sceptical or critical of that kind of argument as well. First of all, because they're compelling on paper, but you have to remember that at the end of the day, recursive self-improvements or self-improving AI is also bottlenecked by the real world, by real world supply chain issues in terms of procuring both GPUs or hardware, chips to train the AI systems, as well as electricity, as well as when it comes to robots, the broader supply chain, complexity of the real world.

And so that's something that you can't just have an exponential growth overnight just because your AI can potentially improve itself better than human engineers. So we bottleneck the real world in that way. Also, if you look at the current landscape, there is not that big of a gap between the very best models from one company and the very best model from the next company and even the best models from another country.

Right now, the gap between the best US model and the best Chinese model there's a model from a company called ZAI, called GLM 4.6. It's neck and neck with ChatGPT and Claude models in the US and better than Gemini models from Google so the gap in some ways is getting really close. It's dubious to me that there's going to be one company that's very clearly and massively ahead of everyone else in one country that's clearly and massively ahead of everyone else between the US and China.

There is certainly a concern from a European perspective that we get left behind and that that could have potentially implications for the economic and geopolitical standing of European countries. So I do hear both sides here that, ideally in a perfect world, we will all slow down and try to make sure that we can develop this technology not only safely, but also both the common good for the benefit of all stakeholders and for applications where, the upside is real. Such as medicine and science as opposed to in a race to try to replace as many human workers as possible and to create a huge job crisis and economic crisis.

Susannah de Jager: So taking that, that's obviously a huge societal risk is the sort of jobs element and as you've already touched upon, the economic model of the utopian version at the moment isn't structurally in place, which is quite well acknowledged. Just taking a slightly different, almost taking that statement as read and that it needs to be dealt with, focusing on a different element that you've raised.

I have seen proponents, including from colleagues of yours that that we're going to interview for this Ethics and Innovation Miniseries talking about, effectively that becomes a commodity level and then you have bounded models on top of it that can preserve in the case of Europe and the UK our value system and potentially sovereign data sets and quite frankly train things to reinforce democratic values or goals that we have.

Are you hopeful about that model or do you think that if those baseline models aren't constrained in themselves, they jump too far ahead and create too many issues?

Raphaël Millière: So I think there are there are a few options here, right? So one thing European countries could do is to say there's no point training our own models from scratch, because that costs billions of dollars of investments to build the data centres and recruit the talent and have the capacity to catch up with the leading labs.

It's not just investments. It's also negative externality in terms of environmental costs, for example, for the data centres. And it's also lead time to catch up, which is actually really difficult because you have to comply with the regulations and you still have to hire the right talent and it takes many iterations to get there. And even very well-funded companies like XAI from Elon Musk are actually struggling to catch up with Anthropic and OpenAI and Google.

So that's one option, very costly, to get to sovereign AI. The other option is it so happens that the best Chinese models, unlike the best American models, are open source. Anyone can use them. Anyone can download them. And anyone can fine-tune these models, meaning take the models that Chinese companies have trained and then customise it for your needs. Do some further training, which is much less costly than training it from scratch.

So it's a lot easier. It's a lot less costly as well. You don't need as much talent and you can catch up much more quickly. Now, the problem with that strategy is you're not actually building your the capacity to build sovereign, and develop your sovereign AI from scratch, which means as soon as the Chinese companies decide to turn off the tap and stop open-sourcing their models, stop sharing it with the rest of the world, which could suddenly happens when they get to a certain level of capability when it becomes a national security interest, the Chinese government decides, "Well, actually, it's not in our best interest to share these models to the rest of the world and it's national security concern," which, by the way, is starting to happen in the US. Even with the closed-source models, because we've seen that the US government has started restricting access to the best closed-source models that Claude Fable 5. Even though you can't actually download them, the weights of the model on your computer, you just have to access through the Anthropic servers.

So when that happens, or if that happens, then what happens to Europe if they've invested in just fine-tuning these Chinese models and adapting them for their own needs? They're going to be even more behind, because now they haven't developed the critical infrastructure and talent pool to be self-sufficient and resilient, and they become completely dependent on Chinese capabilities, in the same way we're already dependent on Chinese manufacturing.

So I think there is a real argument to be made for investment in sovereign AI from the ground up. But that has a real cost, and I think we should be a bit like the same way we are trying to rethink right now. We're starting to rethink our defence strategy in Europe and realising maybe we have to be more self-sufficient. We have to invest in defence, with a greater percentage of our budget. And that's an unpopular decision and that's hard because, we all pay the price in terms of contributing with our tax money. Maybe we have to make these hard calls with AI as well.

But that requires, I think, more education of the general public as well. Because I think there's a lot of anti-AI sentiments because most people don't think of AI as a national security concern and think more of a way to automate jobs, which is a very well concern and increase inequality and make a handful of people very rich at the expense of everyone else.

Susannah de Jager: Yeah. I agree with all of that and I think that you, pulling together those two elements of defence spending and developing our own sovereign AI capabilities, both to protect ourselves economically, ultimately, and to have more control of the apparatus that we will be subject to either way. But also, I think that those two things are increasingly pushed together because they are a defence of our borders, albeit that one is a physical defence and one is a technological moat.

I think it's increasingly acknowledged in the political discourse in the UK and Europe that we have perhaps been overly complacent in our economic and defence trust, I think, in the global system, the increasingly global system, and that even five, maybe even two years ago, defence spending was deeply unpopular. It's still, to your point, pretty unpopular. But because of conflicts globally, there is an awareness that we are really far behind and that actually our assumption that either people, allies would come to our aid, or that we didn't need physical defence, is no longer the received wisdom.

How do you think that should be raised to the man or the woman on the street? Because I think it's really important. What would you be saying to people?

Raphaël Millière: Well, I think it really depends on how we do it. So the thing about the US system is that AI is developed by private companies that are funded by private investors, right? It's not the US taxpayers are really meaningfully directly paying for the development of AI at OpenAI and Anthropic. They might be exposed to it later on when they go to IPO and retail investors that are invest - that have the life savings in ETFs might be exposed to the stock and there might be negative externalities for them at that point because if Anthropic staff for an OpenAI staff starts selling that stock they could make money at the expense of the average person that has a life savings in investment funds.

But in terms of paying for it with your taxes these AI systems are not developed by the US government. That is in a way a strength of the US system in so far as the cost is not passed on the taxpayers, but it's also obviously a huge risk because it means that the people who are funding the government of AI want to make money off it. And so their incentives are not to, for example, avoid replacing human workers with AI, but precisely to deploy AI as broadly as possible, make as much money as possible. Which means replacing as many human workers as possible with AI, right?

So there's a bit of a catch-22 here. We can't have it both ways. On the one hand, we need a lot of investment and it's easier to get that investment for the private sector and it's less politically unpopular because then you're not passing on the cost to the average citizen. On the other hand, if you fund AI with private investments, then it's harder to make sure that these AI systems are used for the things that we actually want to use them from national security, science, medicine, and for the benefit of the greater good, as opposed to just make money for a handful of investors.

I wouldn't want to tell people in Europe, AI investment is always good for you, or you should welcome with open arms dysregulation and open the floodgates of international investors to invest in our European companies without any guardrails. So we have to do it right. And I think we have an opportunity to do things a bit differently in Europe than the way things are done in the US or in China, which is, for example, to have a European level contortium and joint efforts to develop European sovereign AI where every country chips in and we attract top talents.

But that means actually being willing to make huge investments much more than we've been willing to make so far. I think France, which is my country of origin, announced this week that they would invest an additional 600 million euros or so in AI. And I saw the reactions in the US and people were mostly amused because, of course, 600 million euros is a drop in the bucket in terms of the investments that are being made in AI over there, which totaled to over a trillion dollars, right?

I think I would just educate people on the different ways we could get to sovereign AI and we could fund that development. And I would encourage people to get informed about how we could have state-led or European-led sovereign AI efforts, that could be more democratic in its governance structure and could eventually benefit everyone.

Discourse on AI, whether it's in the press or even in academia, tends to be, and has been for many years very polarised in a way that I think does damage to the public perception of AI. What I mean by this is that if you read op-eds in The Guardian or your favourite newspaper about AI, you will often find one of two types of pieces.

One that says AI is going to kill us all or replace us all within a year or two. Basically we're on the verge of not only human-like, human level AI, but super intelligent AI. And on the other end, you will find op-eds that say, no, actually, there's nothing really intelligent in current AI systems. And if you think otherwise you are naive and ignorant, these are just parroting the training data. They are no smarter than the toaster. And you shouldn't be duped by what Silicon Valley investors are trying to make you believe and I think these two attitudes are both extreme and at least partly wrong. And there's a rich middle ground between these two views, which is part of what I'm trying to explore in my own work by doing good on critical science. I trying to find good the ways in which AI systems can do sophisticated things and the way in which they still fail and how they may be unhuman-like in many ways.

But I would say we should beware of this kind of simplistic dichotomies, both in terms of what AIs can and can't do, whether they're intelligent or not, and in terms of whether it's good or not. It is not unintelligent. It's not intelligent in a human-like way. These two things can be true. It's also the case that can bring good things and bad things. These two things can be true, right? Like any technology.

Susannah de Jager: I'm smiling at you, Raphael, because this is my stance on almost everything. The whole world seems to be a polarised echo chamber at the moment on all subjects, whether it's AI or conflict and actually, Audrey Tang, who I'm going to have the pleasure of speaking to for this Miniseries in some of her work was referring to a social media platform that rewards with the algorithm agreement rather than outrage.

We need to regulate for algorithms already impacting human discourse that basically feed disagreement and outrage rather than acknowledges grey and most of the world is sadly grey.

Raphaël Millière: That's right.

Susannah de Jager: So I couldn't agree with you more.

Raphaël Millière: I think sometimes think that this is a cop out or it's just being wishy-washy or refusing to take a stance and I think actually it's the other way around. I think, because we, We feel more comforted in having a very definitive stance on thing and saying, "AI is that way or that it's just good or it's bad. Or it's intelligent or it's stupid," right? I think that's the cop out, because actually it's complicated.

Susannah de Jager: You need to be able to hold discomfort.

Raphaël Millière: Yes, that's right. Exactly.

Susannah de Jager: Well, I think that's actually a pretty good note to leave it on, if people listening it can all be true at the same time and we need to hold the the tension. I have found this so interesting, Raphael. Thank you very much for taking the time to speak to me today.

Raphaël Millière: Thank you for having me. This was wonderful.

Susannah de Jager: Thank you for listening to this episode of Oxford Plus, hosted by me, Susannah de Jager If you wanna keep up with all things Oxford Plus, visit our website, oxfordplus.co.uk or sign up for our newsletter on Substack.

Oxford Plus is a podcast produced by Story 94.