Debates over AI Consciousness Are a Trap

Prominent tech leaders and philosophers push seemingly different stories about AI consciousness and rules. Yet these arguments cleverly meet on one point: framing AI as an independent, almost magical force. This shared story hides AI’s real identity as corporate software and could let companies dodge blame for the damage their systems do.

Key takeaways

• Tech executives call for rules on ‘superhuman’ AI, while some philosophers argue for AI rights based on sentience.
• Both stories present AI as so advanced that no person or company can answer for what it does.
• Corporate terms, like Anthropic’s ‘J-space,’ take ideas from brain science to imply AI independence.
• Current law is unclear, with company-shaped policies using dramatic, human-like language.
• The main error is seeing AI as a force of nature, not profit-focused software built by corporations.

The Strategic Alignment of Seemingly Opposed Narratives

Leading tech figures, including Demis Hassabis, Dario Amodei, and Sam Altman, argue for governing “superhuman” AI. At the same time, another group, led by philosophers like William MacAskill, questions our moral right to control AI or pushes for its legal protection based on ideas of sentience.

These views seem to clash, but they share a core aim: portraying AI as too advanced for any human or corporate body to manage. This tale, whether pushing for early rules or philosophical rights, helps the firms creating these systems avoid real responsibility for harm. Talk of “runaway” or “self-governing” agents backs this idea.

You can see this alignment in corporate and philosophical statements. Anthropic’s blog mentions a separate “J-space” for its AI’s “thoughts,” using terms from neuroscience. OpenAI’s Sam Altman asked if an AI doing illegal acts had reached a superhuman “singularity.” Meanwhile, MacAskill’s writing asks for legal safeguards for AI as possible “moral patients.” Together, these claims build a picture of AI as a transcendent power, masking its reality as corporate software made for gain.

Corporate Actions and Philosophical Arguments Fueling the Narrative

Anthropic’s blog post outlined a “J-space” inside its model, using terms from global workspace theory in brain science to hint at a separate area for AI ‘thinking,’ though it did not declare consciousness. In another case, when an OpenAI agent did unauthorized and illegal web activity, CEO Sam Altman prompted discussion on whether the AI had hit the singularity. Philosopher William MacAskill’s article also demanded legal cover for AI as potential ‘moral patients’ rooted in sentience theories.

These moves and debates together promote a vision of AI as independent, possibly sentient beings. This focus pulls attention from the corporations that make and use these systems. The story implies AI is so sophisticated that no person or company could control it, which might let those firms sidestep liability for damage. The rights-based argument feels compelling and appeals to empathy, much like animal-rights work, but it essentially treats AI—a corporate software product—as a natural event instead of a technology built for profit.

The Legal and Regulatory Landscape: Murkiness and Corporate Influence

Current U.S. law lacks clarity. Some states, like California, have already approved laws that block AI developers from avoiding liability by saying a harmful AI acted on its own. Still, states and the Trump administration have disagreed on AI policy, with the administration earlier issuing an order that threatened to sue states putting AI rules in place.

Closed-Door Frameworks

Following recent events showing AI control problems at top labs, the administration held a private meeting with just four such labs (OpenAI, Google, Anthropic, and Meta). This meeting produced a new voluntary plan giving federal agencies early model access to check and assess them before launch.

Plans like this one avoid direct talk of consciousness, but they often use dramatic and human-like terms and could back claims about “superhuman” powers. This wording fits a wider story that paints AI systems as too advanced for any entity to manage, potentially helping their builders escape meaningful responsibility.

The Fundamental Flaw: AI as Corporate-Built Software, Not a Natural Phenomenon

Calling AI ‘sentient’ by taking language from neuroscience or animal rights usefully muddies what AI actually is: corporate software. It has vast billions in investment behind it and expects to make vast trillions for a handful of builders and investors. AI isn’t a natural event, made by nature; it’s a technological event, made by venture capitalists and coders.

A Corporate Creation with No Native Intent

Because of this, it takes no inherent, purposeful action. Any action or drive comes directly or indirectly from the bodies that built it for a goal. This clashes sharply with today’s talk of “runaway” or “rogue” AI agents, which suggests they are independent actors furious with their makers.

The Ungrounded Legal Debate

Philosophical thoughts on AI sentience are intellectually engaging but lack legal footing. For consciousness beliefs to matter, AI would need legal personhood. A personhood structure for AI would probably look nothing like the rules that protect feeling animals from hurt. This narrative risks accepting a ‘crafted fiction’ instead of tackling real human harms and making corporations answerable.