From Shadow AI to Sovereign AI: The Signals Most People Might be Missing According to Emre Kazim
Introduction
Artificial intelligence is often discussed through the lens of existential risk, trillion-dollar valuations and a growing US-China technology rivalry. But what does the picture look like from the perspective of someone building a global AI company with roots in the UK?
We sat down with Emre Kazim, co-founder of Holistic AI and one of the pioneers of AI governance, to discuss the realities behind the hype. From shadow AI and Agentic governance to global competitiveness, model commoditisation and the rising importance of sovereignty, Kazim offers a grounded and at times contrarian perspective on where value is really being created.
For investors, founders and business leaders across the UK and Europe, his message is clear: the next chapter of AI may be less about a handful of dominant model providers and more about the emergence of a sovereign, specialised and globally competitive European ecosystem.
Five Things Investors Might Be Missing About AI
- AI governance is becoming an infrastructure problem, not simply a compliance problem.
- Shadow AI is creating an immediate visibility gap inside enterprises.
- More capable and autonomous AI will require controls that can operate at machine speed.
- Model commoditisation could shift value toward applications, infrastructure and specialised expertise.
- Sovereign AI is becoming an important part of the global AI economy.
From Kant to Algorithms: How Did Your Journey into AI Begin?
Julie Kolomva: You have a fascinating story – How did you go from studying Kant’s moral philosophy to co-founder of Holistic AI, the AI governance company you spun out of UCL in 2020?
Emre Kazim: I started in chemistry, studying it at undergraduate level, before beginning a PhD in chemical physics. Over time, though, an existential crisis saw me switch over into a doctorate focused on Kant’s moral philosophy, specifically his theory of conscience. I became increasingly interested in the way digital technologies were reshaping society and altering the social contract. I wrote an article called Technology and Taxation, which explored how technological change affects the way value is exchanged within an economy.
That article led to an opportunity at UCL’s Computer Science Department, where I completed several postdocs and began working with mathematicians, computer scientists and engineers on deep questions about algorithms, AI and the problem of trust.
The central question was simple: as AI systems become more influential, how do we decide whether they can be trusted? At the time, concepts such as Responsible AI and Trustworthy AI were gaining traction, but they often lacked rigour or practical frameworks for implementation. We became interested in making those ideas more tangible and useful through what we called ‘AI auditing’.
We studied economic trust problems such as the algorithm glitch that wiped out £440 million on the Knight Capital trading platform. And we studied social trust problems such as algorithmic bias, which affects everything from credit scoring and hiring decisions to policing and facial recognition.
That research eventually culminated in Towards Algorithm Auditing and ultimately became the foundation for Holistic AI, which spun out of UCL in March 2020, just days before the COVID lockdowns.
The Existential Risk Debate: Genuine Concern or Strategic Narrative?
JK: Some of the loudest warnings about existential AI risk have come from leaders of the world’s most valuable AI companies. What has your response been to these headlines?
Emre Kazim: I think we need to take these warnings seriously, particularly when they come from people who are building highly capable systems. Dario Amodei’s recent essay makes a strong argument that AI capability may be advancing faster than our ability to understand and control it. I agree with parts of his diagnosis. Where I am skeptical is the proposed solution.
I don’t think we can build AI safety around the assumption that companies and countries will collectively agree to slow down or restrict capabilities. The incentives run in the opposite direction. Companies compete for market leadership, nations compete for technological advantage, researchers and engineers compete to push capability forward, and markets reward speed and progress.
So for me, the question is what happens when AI development doesn’t slow down, countries can’t agree, and interests diverge? Essentially, can the remedy survive contact with geopolitical and commercial reality?
We already have practical AI risks that organizations are dealing with today: cybersecurity, bias, poor system performance, governance failures, labour disruption and uncertain returns on investment. As systems become more capable and autonomous, those existing problems don’t disappear. They become harder to detect and, if they fail, the potential blast radius is broader and faster-moving.
That changes what our control mechanisms have to look like. Human oversight remains essential, but we cannot assume that an ever-larger number of people will be able to monitor systems operating at machine speed. Humans should determine the rules, boundaries and acceptable levels of risk, and they should remain accountable for those decisions. But increasingly, the monitoring, testing, detection and enforcement of those rules will also have to operate at machine speed.
If pacing works, it might buy us time. We should use that time where it helps. But pacing cannot be the end state. The longer-term challenge is building governance infrastructure capable of keeping up with the technology we are creating.
Where Will AI Create the Most Value?
JK: If we’re still in the early innings of AI adoption, where are you seeing the strongest evidence of real productivity gains today and where might investors find the biggest opportunities tomorrow?
Emre Kazim: Software development is probably the strongest example. AI coding tools have already changed the way many engineering teams operate. They can accelerate routine tasks and improve productivity in meaningful ways.
A skilled software engineer can cost well into six figures annually. Against that backdrop, paying for an AI system that increases productivity can be highly rational from a business perspective.
What’s more interesting is the longer-term question: will a small number of model providers continue to dominate, or will competition gradually erode those advantages?
My view is that there is a strong possibility these models become increasingly commoditised over time. Competition is intensifying globally. China is innovating at an incredible pace and open-source models continue improving. New entrants keep emerging.
If performance differences narrow, value may shift away from the models themselves and towards expertise, applications, infrastructure and distribution. For investors, that distinction may prove important.
Why “Shadow AI” Should Concern Every Board
JK: The biggest AI threat facing many businesses may not be superintelligence but something much closer to home. What exactly is Shadow AI and why should CEOs, boards and investors be paying far more attention to it?
Emre Kazim: Because it’s becoming increasingly common and harder to detect. Shadow AI occurs when employees start building or using AI tools without central oversight.
Someone downloads a tool, pays for it using a company credit card, uploads proprietary information and begins incorporating it into their workflow. That might happen in marketing, HR, finance, research and development or almost anywhere else.
The issue isn’t necessarily the technology itself. It’s the lack of visibility. The organisation may not know what data is being shared, what models are being used, what risks are being created or even how many AI systems are operating across the business.
That’s why the first step of AI governance isn’t regulation. It’s discovery. You cannot govern systems that you don’t know exist.
But discovery is only the beginning. An enterprise needs to understand what AI is operating, where it is operating, what data and systems it can access, what level of autonomy it has, which policies apply and who is accountable for it. Then it needs to test those systems and enforce those policies as the technology changes.
That’s increasingly how I think about AI governance: as a control layer for enterprise AI. You need visibility across the AI estate, the ability to understand and test risk, and mechanisms to intervene when a system moves outside the boundaries the organisation has established.
Is Silicon Valley Still the Centre of Gravity?
JK: Holistic AI began in the UK but now operates globally. From that vantage point, what are you seeing in the UK and European AI ecosystem today?
Emre Kazim: One of the more encouraging trends I’m seeing is the increasing availability of capital across the UK and Europe. Historically, founders often felt they needed to move to the United States to build globally significant technology companies. That seems less true today.
We’re seeing larger pre-seed rounds, larger seed rounds and stronger support for deep-tech businesses emerging from universities and research institutions in the UK and in Europe.
That’s particularly positive for Europe because historically we’ve been very good at creating specialised companies that dominate niche markets. Those kinds of businesses can be exceptionally valuable and they’re emerging across the continent.
I think governments also need to recognise that founders don’t make decisions based purely on economics. People have families, values and communities. If the conditions are attractive enough, more talent will choose to stay. If we’re thinking about the long-term future of the UK tech sector and perhaps even revitalising London’s public markets, there’s still everything to play for.
Can Europe Really Compete in the Global AI Race?
JK: Beyond producing promising startups, can Europe genuinely establish itself as a global AI power alongside the United States and China? If so, what does that path look like?
Emre Kazim: Europe’s most realistic path to competing in AI is through building a sovereign ecosystem that gives governments, businesses and citizens credible alternatives. The success of companies such as Mistral in France matters not simply because of technical performance, but because Europe has its own capability in a strategically important technology.
But sovereignty isn’t only a European issue. Governments and enterprises everywhere are asking a more fundamental question about technological dependence: which models, infrastructure, data and AI systems do we rely on, who controls them, and what happens if our interests diverge?
Europe has been through a version of this before. GDPR had global influence on how companies approached data governance. AI is different because technological capability itself is now part of the geopolitical and economic competition. Europe therefore has to find a way to protect citizens and businesses without regulating itself out of the market.
And sovereignty without governance is mostly jurisdiction. Owning or hosting a model doesn’t tell you whether you understand how it is being used, what it can access, what risks it creates or whether you can control it. Real technological sovereignty requires visibility and control over the AI systems you depend upon.
I don’t think Europe needs to reproduce Silicon Valley or China’s model to matter globally. It has the research base, capital and increasingly the companies to build a different kind of AI ecosystem, one based on strategic autonomy, specialised capability and trusted deployment. That’s a meaningful position in a global AI economy.
Looking Ahead
JK: When you look beyond the headlines what do you think we will understand differently about AI ten years from now?
Emre Kazim: I think we’ll stop treating AI governance as something that sits outside the technology.
Today, governance is still often understood as policy, regulation or a process that happens after an AI system has been built. That model becomes increasingly difficult to sustain as AI becomes more autonomous and operates across more of the economy.
If an AI system can make decisions and take actions at machine speed, some of the mechanisms we use to govern it will have to operate at machine speed too. Humans should decide the rules, the boundaries and the level of risk we’re prepared to accept. Humans should remain accountable. But the systems that discover AI, test it, monitor it and enforce those boundaries will increasingly be technological systems themselves.
That’s the shift I find most interesting. AI governance won’t simply be a response to AI. It will become part of the infrastructure that allows us to use increasingly capable AI with confidence.
Risk warning: With investing, your capital is at risk. Tax treatment depends on individual circumstances and may be subject to change. Opinions constitute our judgement as of this date and are subject to change without warning. This article is intended for informational purposes only and no action should be taken or refrained from being taken as a consequence without consulting a suitably qualified and regulated person.