In 2025, Newfoundland and Labrador saw two reports that suffered from made-up citations likely generated by AI. In September, CBC reported that the Education Accord NL final report contained at least 15 citations for non-existent documents. Then, in November, The Independent confirmed that Deloitte had delivered a report costing nearly CDN$1.6 million with at least four false citations. Previously, Deloitte had issued a partial refund for an AUD$440,000 report commissioned by the Australian government as a result of AI-generated errors.
Large language models (LLMs) are by nature prone to “hallucinations” because they work in a probabilistic manner: answers are mostly generated based on likelihood. But awareness of their inner workings is lagging behind their adoption. Hence, mishaps pop up everywhere. A British Columbia lawyer was reprimanded for submitting a brief that contained made-up jurisprudence. Ironically, a Dutch researcher and medical doctor saw a paper about AI retracted over incorrect links to the medical website PubMed.
These hallucinations, however, are not a problem per se. Police detectives deal with unreliable witnesses on a daily basis, yet crimes are still solved. The issue is that AI’s fluent style can make answers seem more credible than they actually are. A 2024 study found participants rated simpler AI summaries more credible (4.72 versus 4.47 on a seven-point scale) and trustworthy (4.70 versus 4.42) than human-written ones. Interestingly, participants were more likely to believe that the AI summaries were written by a human than the actual human ones (4.80 versus 4.29). In other words: we tend to believe that “clear writing” equates “truth.”
And that is a problem, even when AI does not confabulate. Professor Kiron Ravindran from Spain’s IE Business School observed that AI use creates students that are “incompetent experts”: “I received submissions that read like graduate-level work, but when I asked basic questions about methodology or reasoning, many students struggled to respond satisfactorily.”
Such graduates will not have the skills to steer AI in an optimal direction, which will get them into trouble if they have to compete on the job market with those who have, quite literally, done their homework and now possess the meta knowledge to use AI as a cognitive enhancer, rather than a full replacement.
This phenomenon, often called “deskilling,” is dangerous, and aviation learned that lesson the hard way. In 2009, Air France flight AF447 crashed, in part, because pilots had become overly reliant on automation, something researcher Lisanne Bainbridge warned of as early as 1983. The subsequent report recommended extra training for manual aircraft handling.
Smart systems work better when controlled by smart humans. Bainbridge’s words seem almost prophetic in the context of effectively using AI: “Perhaps the final irony is that it is the most successful automated systems, with rare need for manual intervention, which may need the greatest investment in human operator training.”
Ravindran is not the only one noticing the problem of “shallow learning.” In 2025, Shiri Melumad and Jin Ho Yun combined the results of seven experiments with 10,462 participants in total to show that LLM users, on average, retain less knowledge than those who use traditional web search engines. This is because LLMs present easy-to-digest summaries, whereas someone using Google or DuckDuckGo has to do the work of digestion themselves.
Yet AI holds the promise of empowering workers and even making research more equitable. A recent policy article in Science, alongside several critical observations, noted that “LLM use alters authors’ citation behavior, seemingly steering them toward a more diverse knowledge base.”
But to fully deliver on AI’s promises of empowerment and equity, it is not enough to just give workers access to an LLM.
Currently, AI does not reduce the difference between workers with varying levels of performance. A British study from 2024 suggests that: “ChatGPT use increased productivity in all tasks” but “did not affect productivity differentials between gender, age, educational or occupational groups.” Simply put, everyone is more productive with AI, but existing differences do not disappear.
This might be because AI does not take away an advantage that skilled researchers have: not taking any answer for granted. This trait is called “epistemic vigilance,” a concept that Dan Sperber and his co-authors defined in a 2010 seminal paper as a “suite of cognitive mechanisms" that mitigate “the risk of being accidentally or intentionally misinformed.” This vigilance is a crucial skill for police detectives, scientists and journalists.
But vigilance is not a universal trait. A study by Carita Kiili, Leena Laurinen and Miika Marttunen from 2008 found that only a small minority (12 percent) of students (n=25) cross-checked their sources when doing research. Not much has changed since then: in 2021, Joel Breakstone and colleagues found that 96 percent of students (n=3,446) were unable to correctly identify ties between a climate website and the fossil fuel industry.
This resonates with my own experience. Having lectured in research skills since the early aughts, I found it necessary to explicitly teach what I christened the “Easter egg rule,” after a Dutch children’s song telling you how many to eat: one egg is no egg, two eggs is half an egg, three eggs is an Easter egg. Similar voraciousness is required when doing research.
The good news is that research skills can be taught. The not-quite-as-good news is that the story for instilling epistemic vigilance looks to be more complex.
Learning how to do research is relatively straightforward. The basics: know your tools and what they can do. The often-overlooked step before that, however, is to know what you want to know. To paraphrase Douglas Adams’ Hitchhiker’s Guide to the Galaxy: you won’t find the answer if you don’t know what your question really is.
As a result, instilling appropriate doubt is where the real challenge lies. But we are better at teaching people to act skeptically than to actually be skeptical. In a 2000 paper, Peter A. Facione concluded that when it comes to critical thinking, “skill and disposition are two separate things in people.” That is to say, knowing how to ask an AI pointed questions does not mean that a person will actually do it.
Epistemic vigilance likely forms at a very young age, as studies of children as young as three have shown. For example, four-year-olds are more likely to trust people whom they perceive as knowledgeable than three-year-olds. And six-year-olds are capable of making more nuanced assessments of someone’s credibility than children of four or five.
In 2016, Christopher R. Huber and Nathan R. Kuncel asked themselves the question whether going to college teaches critical thinking. They concluded that “the present study demonstrates that college students learn critical thinking skills, but this does not guarantee that they retain these skills long after college or apply them in other contexts.”
To the best of my knowledge, we lack solid evidence that vigilant attitudes can be meaningfully and persistently changed later in life. Which raises the question: How helpful is it to teach adults better research skills, if the likelihood that they will use these faculties seems to have been shaped so early in life?
Two solutions can help mitigate, though perhaps not solve, the problem.
The first is to offer continuous training: not just an initial skills workshop but frequent boosters to make sure epistemic vigilance stays top of mind, an approach called inoculation theory.
The second is to use protocols. This means implementing workflows that include thorough checklists, the likes of which have prevented plane crashes and unnecessary medical deaths.
However, compliance is an issue, which in itself might be an indicator of how hard it is to instill epistemic vigilance. A 2025 study on surgical checklist adherence found an overall compliance rate of 73 percent. And as far as boosters go: they work, but there is uncertainty about their long-term effects, as this 2025 paper discusses.
Then there is the equity aspect. Continuous training and protocols help ensure a minimum standard, but they raise the floor rather than the ceiling. AI empowerment is already — and will continue to be — a skill necessary for career success and socio-economic mobility. To promote social equality and strengthen Canada’s future economic prospects, early intervention may be needed. If epistemic vigilance is indeed difficult to influence except at a very young age, then preschools are not just childcare centres; they might well be Canada’s most important strategic assets in the AI age.
