Soon after the mass shooting in Tumbler Ridge, BC, OpenAI revealed that it had flagged the shooter’s account months earlier but had chosen not to notify authorities. Many Canadians were justifiably angry. In September, a Mother Jones investigation reported disturbing details about the shooter’s conversations with ChatGPT days before the killings. According to the report, the chatbot advised her to frame questions about violent scenarios as hypothetical to circumvent its safeguards and then helped her plan the attack.
BC Attorney General Niki Sharma called on the federal government to amend the Criminal Code and make company officials criminally liable for releasing products that endanger the public. She no doubt spoke for many of us when she said that we need to “ensure a clearer pathway for human and corporate accountability for the actions of AI.”
But the idea that stiffer criminal law can solve the problem of chatbots counselling violence misunderstands their nature. Making AI safer against this kind of misuse is not a matter of turning a few dials back at headquarters in San Francisco. It is not even a matter of preventing these companies from releasing new products until they can guarantee that they are safe enough, since no such guarantee is possible.
No one can make a language model completely safe from jailbreaking (or malicious attempts to circumvent its safeguards). A key problem is that vulnerabilities are unpredictable, which makes it difficult to know whether and when to hold model makers such as OpenAI liable for the harm their systems cause.
The United Kingdom’s AI Security Institute reported last December that it had tested a range of frontier models and found universal jailbreaks “in every system we tested.” After extensive internal testing this summer, OpenAI conceded that its latest model, GPT-5.6-Sol, is still susceptible to some forms of jailbreaking and “performs comparably to recent predecessors” in this regard.
Experts have known about this problem for years. Users can circumvent a model’s guardrails in myriad ways, often with only slight changes to the wording of a prompt. Model capabilities themselves are often emergent or unexpected, as reports in July of models escaping their sandboxes and hacking into private servers made clear. And with open-weight models, once a company releases the weights online, it loses control over them altogether.
Some will say that a chatbot coaching a user on how to evade its own safeguards, as ChatGPT reportedly did in the Tumbler Ridge incident, is a design flaw that OpenAI should have caught. Perhaps. But that behaviour is itself a product of the same unpredictability. A model trained to refuse might still be talked into explaining how to get around its refusals.
But British Columbia’s call for new criminal offences against OpenAI and similar companiespoints to a larger problem: How do we hold AI companies accountable for harm, even in the face of uncertainty about safeguards? Maybe OpenAI CEO Sam Altman was not aware that ChatGPT could be jailbroken so easily and made to counsel someone on how to commit mass murder. But after Tumbler Ridge, he is aware. If he green-lights a new model and it fails to prevent a similar tragedy, should he not face criminal charges for recklessness?
A bill tabled in 2022, but never passed, illustrates one challenge of criminalizing the failure to build adequate safeguards into AI: the difficulty of obtaining a conviction for acts not easily foreseen. The Artificial Intelligence and Data Act would have created an offence of knowing or being reckless as to whether an AI system “is likely to cause” serious harm, where it then causes that harm. How could one prove that a CEO or chief engineer knew that a particular jailbreak leading to harm was likely? Not just possible but likely? Knowing that jailbreaks happen in general is a long way from knowing that a particular one was likely.
Duties Instead of Foresight
A better approach is the one that we are taking in Bill C-34, the Safe Social Media Act, now before parliament. Rather than criminalizing a failure to make an AI system safe enough, the bill would impose a series of duties on model makers to adequately mitigate the risk that a chatbot will engage in harmful conduct, including encouraging serious bodily harm or death. Where a user expresses a violent intent, the chatbot would have to end the conversation and direct the user toward a crisis intervention service. The bill would also impose greater transparency and accountability measures on companies such as OpenAI, which a new Digital Safety Commission would enforce. Compliance would turn on what companies actually do to test and safeguard their models, rather than on what harms they could foresee.
This is not to say that criminal law will have no role to play in holding AI creators accountable. And Bill C-34 would help there, too. The new legal duties it creates would work with Criminal Code offences of common nuisance and criminal negligence to give prosecutors all the tools they need in the clearest of cases. Common nuisance makes it a crime when the breach of a legal duty endangers the public. Criminal negligence, which carries a heavier sentence, applies where the breach amounts to a marked and substantial departure from the conduct of a reasonable person. The question at trial would be whether a company failed to test its models adequately or to put escalation protocols in place.
Yet given the uncertainty about how well any chatbot safeguard will work, or about when AI companies have done enough to adequately mitigate the risk of harm, criminal convictions will likely be rare. As with cars, if we want the convenience and power of AI, we have to accept some risk. We can reduce that risk by holding AI creators to a higher standard of care and subjecting them to more oversight. The Safe Social Media Act would do just that.
It will not make us perfectly safe from AI’s harms, but for the foreseeable future, nothing likely will.
