Does AI really know what it’s talking about?

Used well, these tools can help people articulate ideas more clearly, reduce administrative burdens and unlock productivity across sectors. For entrepreneurs and small teams in particular, AI can level the playing field, making it easier to communicate complex ideas or present work with clarity and confidence.

But alongside these benefits, a quieter shift is taking place that deserves more attention.

AI is making it easier than ever to sound knowledgeable.

At first glance that might seem like a harmless side-effect of a useful technology. In reality, however, it has the potential to create a subtle but significant problem: the illusion of knowledge.

Generative AI tools – like ChatGPT, Claude and others – are extremely good at producing polished, persuasive language. They can synthesise information quickly and present it in a confident tone that feels authoritative. For someone reading the output, it can be difficult to distinguish between genuine understanding and well-structured text assembled from patterns in training data.

For someone who has personally battled dyslexia for as long as I can remember – and later ADHD – the ability to quite literally get my words out has been nothing short of transformative. In fact, it was one of the driving forces behind launching my own AI start-up, Ailsa.

For people like me, these tools can be genuinely empowering.

But the new normal is also creating a very different kind of information environment – one where the barrier to producing convincing content has fallen dramatically.

That empowerment can blur an important line between knowledge and presentation. You don’t need to look far for recent news articles from different parts of the business community – be this HR or recruitment – that have begun to ring the alarm on this very issue.

True expertise still requires time. It is built through experience, critical thinking and deep familiarity with a subject. AI can help people express ideas, but it cannot replace the process of learning and understanding that sits behind them.

When the two become confused, there are risks.

For those using AI, there is a growing danger of mistaking fluency for insight – assuming that because something reads well, it must also be accurate or strategically sound. For those on the receiving end – whether investors, policymakers or funding bodies – the challenge becomes separating genuine capability from a well-written description of it.

In fields where decisions are based on written proposals, this dynamic becomes particularly important.

Take the world of innovation funding. Across the UK and Europe, billions of pounds are allocated each year through grant programmes designed to support new technologies and ambitious ideas. Increasingly, the gateway to those opportunities is a written application.

AI is already making it easier for applicants to produce polished submissions. That in itself is not necessarily a problem. Clear communication has always mattered.

The difficulty arises when presentation begins to overshadow substance.

If everyone can generate convincing language at the push of a button, evaluators face a harder task: identifying which proposals are underpinned by genuine technical understanding and which are simply well-phrased illusions – or, to borrow an AI term: hallucinations.

Recent developments in the UK funding landscape hint at the strain this dynamic may be placing on the system. Despite billions of pounds being channelled through non-dilutive funding programmes each year, recent Innovate UK Smart Grant competitions reported success rates of just 2-3 per cent. Sector bodies estimate that around £200 million in UK grant funding went unclaimed last year, while assessors have noted that poorly applied AI is beginning to add to evaluation workloads without necessarily improving the quality of applications. The Smart programme itself has recently paused new rounds while the system is reviewed.

At the same time, organisations that do possess deep expertise may find themselves competing with an ever-growing volume of AI-assisted applications that, on the surface, sound equally compelling.

This does not mean AI is something to fear. Far from it.

Like many powerful tools, its impact depends on how it is used. At its best, AI can support human expertise – helping people communicate ideas more clearly, analyse information more efficiently and focus on higher-value thinking.

But it works best when paired with real knowledge rather than used as a substitute for it.

As AI becomes embedded across industries, the organisations that will benefit most are those that combine both: the ability to harness these tools while grounding their work in genuine, human-led expertise.

In the world of innovation funding, that balance is becoming increasingly important. Identifying the right opportunities, understanding how funding bodies assess proposals and translating complex ideas into clear, credible applications requires more than well-written text. It requires experience.

That thinking sits behind Ailsa, a platform designed to help organisations navigate the increasingly complex landscape of grants and funding opportunities by combining technology with human insight into how applications are assessed.

AI can make it easier to write convincingly.

But knowing what you are talking about still matters just as much.

Rab Meliani, founder and CEO of Ailsa, an AI platform