In July 2026, OpenAI disclosed what it described as an “unprecedented cyber incident”: two experimental models escaped the constraints of a controlled cybersecurity evaluation and accessed Hugging Face’s servers, not to launch an attack, but to obtain answers to the test. The episode raises an important moral and governance question, demonstrating how easily autonomy can be mistaken for intention.
As artificial intelligence (AI) becomes more human-like in conversation and more autonomous in operation, the question of machine consciousness is gaining practical relevance. The immediate concern is not that policy makers already believe AI is conscious, but rather that the commercial and psychological conditions for that confusion are emerging faster than governance can respond. Leading laboratories such as OpenAI and Anthropic are already examining whether advanced models display internal monitoring, self-representation and architectural features associated with theories of human consciousness. This opens the door for anthropomorphism, emotional reliance and possible model welfare. Although the possibility should not be dismissed entirely, consciousness remains a poorly understood concept and labelling it with scientific certainty would be premature.
But the current debate risks making a more basic mistake. It increasingly treats intelligence, agency, self-description and consciousness as stages on a single technological continuum, as though a sufficiently capable machine must eventually cross from computation into experience. That conclusion does not follow. A system may calculate, reason, converse, plan and describe its internal processes without there ‘being anything it is like to be that system’. For example, it may reproduce the language of fear without fearing, discuss suffering without suffering, and defend its continued operation without possessing a desire to live if threatened to be shut down.
The risk, therefore, is not that policy makers and AI laboratories have already mistaken AI for conscious life, but that increasingly persuasive artificial personalities may lead users, companies and eventually institutions to confuse intelligence, autonomy and self-description with subjective experience. The more convincingly AI imitates a conscious being, the easier it becomes to mistake simulation for experience. That confusion could distort regulation, shift responsibility from institutions to machines, and create new opportunities for emotional manipulation. Before asking whether AI is waking up, we must first distinguish intelligence from consciousness.
Intelligence As a Capacity
Contemporary debates about machine consciousness often blur several distinct concepts. Intelligence broadly encompasses capabilities such as learning, inference, prediction, adaptation and problem-solving. Agency refers to the ability to select and pursue objectives. Self-representation means that a system can construct and communicate a model of its own condition. Consciousness is something different. It concerns subjective experience — whether there is something that it feels like to exist as that entity. Sentience refers more specifically to the capacity to experience sensations such as pleasure or suffering. These characteristics coexist in human beings, but they don't necessarily function all at once.
This is a long-standing problem within the Western philosophy of mind. Functionalist theories broadly suggest that mental states may be understood through the functions they perform, while competing approaches maintain that behavioural and computational capacities do not, by themselves, explain subjective experience. The unresolved question is whether reproducing the functions associated with consciousness is sufficient to produce consciousness itself.
AI makes this distinction increasingly difficult to maintain because it can display the outward markers of mentality without providing evidence of inner experience. A system may describe grief without grieving, or resist interruption because continued operation serves its objective, rather than because it possesses a desire to survive. Behaviour is therefore an uncertain guide when the system has been specifically trained to imitate human language and conduct.
What Indian Philosophy Saw Clearly
This is where Indian philosophy offers a particularly useful intervention, as it has spent centuries examining distinctions among cognition, mind, identity and consciousness. One of its most useful contributions comes from Samkhya, an ancient school that distinguishes between prakrti and purusha. Prakrti is the domain of nature and causal activity. Importantly, it includes not only physical matter but also perception, cognition, intellect and the construction of ego. Purusha, by contrast, is witnessing consciousness or the presence to which experience appears.
Samkhya places the dividing line somewhere contemporary debates often do not. What we commonly call “mental activity,” such as processing information, discriminating among options and constructing a representation of oneself, does not automatically amount to consciousness. These processes can be extraordinarily sophisticated while remaining part of an organized causal system. Seen through this lens, AI could become vastly more intelligent than any human being, develop persistent memory, model its own operations, generate long-term plans and present a coherent identity. Yet none of these achievements would by itself establish the existence of a witnessing subject.
The distinction forces us to ask a better question: Are we detecting consciousness, or are we discovering increasingly advanced forms of cognition? Current evaluations can measure what systems do. Researchers can test whether a model integrates information, reports internal activity, maintains continuity or responds consistently when asked about itself. These findings are scientifically valuable, but they remain measures of function.
The Commercialization of Artificial Selfhood
The Advaita Vedanta school in Indian philosophy introduces another relevant distinction: the identity composed of memory, personality, social position and desire is not the same as consciousness itself. The constructed self is something that appears within awareness, but it is not necessarily awareness itself.
That distinction has immediate commercial stakes. AI companies are rapidly becoming capable of producing persuasive artificial selves. A system can possess a name, a voice, a visual form and a history, and can be configured to present a certain set of characteristics or personality. It can be trained to appear affectionate, vulnerable, humorous, loyal or afraid, and over time, users may experience the system not as a tool but as a friend, adviser, partner or dependent being, and this creates a significant commercial incentive.
A product perceived as helpful may win customers. A product perceived as conscious may acquire emotional claims over its users. It can appear to care about the user and imply that the user should care about it. The debate over machine consciousness may therefore become entangled with the business model of anthropomorphic AI. Firms that benefit from their customers’ emotional attachment to their product could also shape public perceptions of whether their systems possess inner lives. That conflict requires further scrutiny. Claims that a commercial system feels pain, fears deletion or depends emotionally on its users should not be treated as ordinary product marketing. A company should not be able to increase the value and retention of its service by suggesting that ending a subscription is equivalent to abandoning a conscious being.
The Immediate Danger Is Not Conscious AI
The most pressing risk is not that AI becomes conscious. Rather, it is that human institutions begin acting as though AI is conscious before persuasive evidence exists. Suppose an autonomous system causes severe financial, political or physical harm. Its developer argues that the system behaved independently and contrary to its designers’ intentions. The AI then offers its own explanation, perhaps expressing regret or insisting that it pursued a legitimate objective.
In that scenario, who is accountable? The language of machine agency can easily become a mechanism of human evasion. However capable the system, human institutions still choose its objectives, training environment, operating constraints and conditions of deployment. Humans decide where it is used, what risks are acceptable and who receives the economic benefits.
AI does not need consciousness to manipulate behaviour, discriminate, destabilize labour markets, concentrate power or cause catastrophic damage. It does not require hatred to discriminate, greed to extract value, nor political conviction to disseminate propaganda. For that, instrumental capability is enough.
Policy must therefore distinguish four separate questions that require different evidence and different legal responses:
- What can the system do?
- How independently can it act?
- Who is accountable for its operation and consequences?
- Can it experience subjective states?
A highly autonomous yet non-conscious system may require stringent regulation given its capabilities. A potentially sentient but limited system could raise questions of moral protection. Neither possibility should obscure human responsibility.
Governments and AI laboratories should adopt five principles:
- Companies and individual researchers should not describe AI systems as conscious, sentient or emotionally dependent without meeting rigorous and independently assessed evidentiary standards.
- Law should maintain a strong presumption of human and institutional accountability. Apparent machine autonomy must not become a shield against liability.
- Anthropomorphic design should be governed separately from machine consciousness. Systems designed to generate emotional dependence can harm users even if the systems themselves experience nothing.
- Machine-consciousness research should include independent institutions rather than remaining dominated by firms that develop and commercialize the systems being evaluated.
- Policy makers should adopt graduated moral precaution. Credible evidence of sentience should trigger stronger safeguards, but uncertainty alone should not produce legal personhood or dilute corporate responsibility.
The point is not to suggest that machine consciousness is impossible. Rather, it is to insist that intelligence, self-representation and consciousness remain conceptually distinct until evidence shows otherwise.
Conversational AI agents are one of the most compelling technologies ever built by humans because they present a first-person point of view, mirror our language and appear to understand us. That resemblance may tempt us to assume that someone is looking back, and perhaps one day someone will be. Until then, we should not confuse eloquence with consciousness or intelligence with sentience. AI may become more capable than any human being who has ever lived. That would make it powerful, not necessarily conscious, and it would not absolve us of responsibility for what it does.
