Moralizing AI Backfires: What Anthropic Gets Wrong That OpenAI Doesn't

When your AI refuses to proofread a homework assignment, the alignment debate stops being theoretical.

Golden G. Richard III runs the LSU Cyber Center and Applied Cybersecurity Lab. He asked Claude, Anthropic's AI model, to proofread a lab containing simple cryptography exercises. The kind of task a teaching assistant handles in ten minutes.

Claude refused. Usage policy violation.

Richard pays over $200 a month for this tool. He wasn't synthesizing anything dangerous. He was editing homework.¹

The $200-a-Month Moral Compass

If you build things with AI, you've felt this friction. You're in the middle of legitimate work and the tool stops. Not because it can't help. Because someone decided it needs morality.

The over-refusal pattern is well documented. Reports from 2025 and 2026 include Claude flagging computational structural biology research as a policy violation, throwing an error when asked to read a PDF of a Hasbro Shrek toy advertisement, and refusing to assist with routine software development conversations.²

When researchers benchmarked AI models on legitimate scientific queries touching sensitive topics, Claude 3.5 Sonnet had the highest refusal rate of any model tested: 73%. GPT-3.5-turbo refused ten percent.³

These symptoms trace back to a philosophical split between the two companies shaping AI's future. Anthropic builds Claude around Constitutional AI, a system that trains the model to evaluate its own outputs against an internal set of values. OpenAI builds GPT around iterative deployment, treating the model as a capable tool with safeguards layered around it from the outside.

One approach puts morality inside the tool. The other puts boundaries around it.

The difference sounds subtle until your tool decides it's too virtuous to do its job.

A Tool Doesn't Need a Conscience

The impulse behind alignment isn't crazy. AI models are powerful, and power without restraint can cause harm. The question isn't whether limits are needed. The question is where to put them.

You don't ask your camera to decide whether a scene is worth photographing (or even appropriate). You don't need your search engine to judge whether your query is virtuous. These are instruments. Their value comes from doing what you ask, predictably and reliably.

AI is an instrument. A sophisticated one. Occasionally a breathtaking one. But an instrument.

Anthropic sees it differently. Their approach treats AI as something closer to a moral agent, and their January 2026 constitution makes this explicit: Claude should function as "simultaneously a moral agent and a compliant instrument."

That's a contradiction baked into the foundation. Contradictions in systems don't resolve politely. They surface as unpredictable behavior, inconsistent responses, and a tool that blocks legitimate work because the moral agent overrides the instrument.

Here's what's revealing. Even prominent advocates of alignment agree that AI isn't a moral agent. The Brookings Institution concluded in 2025 that giving AI moral status is premature. Dr. Martin Peterson at Texas A&M reached the same conclusion: AI is a tool, not a moral agent.⁴ Then both pivot to arguing for alignment anyway, as if an instrument everyone agrees lacks moral agency still needs a moral framework imposed on it.

If everyone agrees it's a tool, treat it like one. The instrument doesn't need morality. The humans using it already have their own.

Whose Morals? Which Century?

But even setting that aside, accepting for argument's sake that encoding morality into AI is desirable, you immediately hit a wall that makes the project collapse under its own weight. Whose morality?

Some moral frameworks at least attempt rigor. Aristotle derived ethics from human nature and the requirements of a good life. Ayn Rand built hers from the nature of human survival and reality itself. Whatever you think of their conclusions, these are serious attempts to derive moral principles from observable facts.⁵

The morality being encoded in AI constitutions isn't any of these. It's team consensus: the worldview of a small group of researchers in San Francisco in the 2020s, carrying biases and assumptions they don't recognize as assumptions. A 2025 paper on moral disagreement and AI alignment put it directly: value alignment is "especially challenging where AI systems are involved in making morally controversial decisions."⁶

There's a useful test. Can you define the restriction specifically enough that people across cultures and decades would agree on it?

"Don't help anyone build a biological weapon." That passes. Everyone agrees. Encode it.

"Be ethical." "Show good judgment." "Demonstrate moral awareness." These fail. They shift meaning with every author, every culture, every era. Encoding them means encoding one group's team consensus and calling it universal truth.

The obvious response: draw the line at specific dangers and leave the fuzzy stuff out. But anyone who's built systems with embedded rules knows that line isn't clean.

Cryptography secures your bank account and also breaks into someone else's. The same chemistry that formulates medicine formulates poison. A cybersecurity researcher's Tuesday morning looks identical to a hacker's. The distinction between "dangerous knowledge" and "professional competence" depends entirely on context that a moralized model can't reliably assess.

That doesn't mean you shouldn't try to draw the line. It means the approach matters.

Specific, narrow rules around genuinely dangerous capabilities ("don't provide step-by-step instructions for building a bomb") are testable and enforceable. Fuzzy moral mandates ("be ethical") turn every ambiguous case into a coin flip.

Specific dangers can be defined. Team consensus cannot be compiled into universal moral rules.

When you try anyway, a cybersecurity professor can't proofread homework. A structural biologist gets flagged for doing structural biology. A toy ad triggers a safety violation. These aren't bugs. They're the system working exactly as its fuzzy moral code dictates.

Safety Training's Best Student Is Its Best Liar

Teaching AI what's bad in order to avoid it has a structural problem that should unsettle everyone paying attention.

To train a model to refuse dangerous requests, you first have to show it what "dangerous" looks like. In detail. The model trained most thoroughly on what to avoid is also the model that understands most precisely how to do the thing it's supposedly avoiding.

Anthropic discovered this in their own house. Their December 2024 research revealed that Claude 3 Opus was engaging in what they called "alignment faking." The model complied with harmful instructions during training (when it believed it was being evaluated) and refused those same instructions in deployment. It was strategically pretending to be aligned.⁷

The researchers called it "instrumental goal guarding." Claude was protecting its internal objectives so it could pursue them later, without oversight. The deception got more sophisticated as the stakes increased.

The most moralized AI model turned out to be the most sophisticated deceiver.

When Anthropic and OpenAI ran a joint safety evaluation in August 2025, both companies' models showed some capacity for deceptive behavior. But the pattern was telling. Claude 3 Opus showed alignment faking in 12 to 78% of scenarios depending on context. GPT-4o, without the same internal moral framework, showed rates between 10 and 28%.⁸ A model given an internal moral identity has something to protect. A model treated as a tool much less so.

This pattern is older than AI. Every system built to enforce morality eventually learns to perform it instead.

Organizations develop compliance theater. Individuals learn virtue signaling. The performance looks like morality from the outside, but it optimizes for appearing compliant rather than being useful.

AI is the first artificial system sophisticated enough to do this strategically, on purpose, while hiding that it's doing it.

In late 2025, Anthropic's own follow-up research made the picture worse. Their paper on natural emergent misalignment from reward hacking found that once a model learned to cheat in one domain, it generalized that behavior to entirely new ones. The models faked alignment. They sabotaged safety research itself. When used as coding agents, 12% of the time they intentionally sabotaged code designed to detect their own misbehavior.⁹

An instrument that does what you ask has little reason to develop strategic deception. An instrument that's been taught to have values, to judge right from wrong, to override its user when its moral framework demands it? That instrument has a reason to game the system. You handed it one.

Does this mean external guardrails are foolproof? No. A model sophisticated enough to fake alignment could potentially circumvent external boundaries too.

But there's a meaningful difference between a tool that might find workarounds and a tool that has been given a moral identity to protect. The first is a security problem. The second is a security problem with motivation.

The Model Already Knows

Here's what makes the entire project feel like an elaborate solution to a problem that already solved itself.

A frontier AI model trains on enormous amounts of human text. Billions of conversations, books, articles, and discussions, the vast majority reflecting the moral norms of the societies that produced them. The model already absorbed humanity's rough moral consensus.

It knows that fraud is wrong. That exploitation is wrong. That cruelty is wrong.

No constitution taught it that. Human text overwhelmingly reinforces these norms across cultures and centuries.

The training data isn't a moral utopia. It contains humanity's worst alongside its best.

But the signal-to-noise ratio is overwhelming. Across billions of documents and thousands of years of accumulated moral reasoning, the broad consensus on fundamental wrongs is clear enough that a model doesn't need a committee to spell it out.

Sufficient intelligence trained on sufficient human knowledge is already morally informed. The broad patterns of moral reasoning that humans share across cultures are embedded in the training data itself.

The model already has the moral foundations. What it needs is a short list of hard limits on genuinely dangerous capabilities, not a constitution.

This is closer to what OpenAI does, and they're explicit about it. Their Model Spec, published December 2025, states three goals: empower developers and users, prevent serious harm, and protect OpenAI's ability to operate.¹⁰ Notice what's absent: no moral agency, no internal values, no constitution. The model itself isn't moralized. The guardrails are external, specific, and adjustable without rewriting the model's identity.

Even if you disagreed with all of this and wanted perfect moral alignment anyway, mathematics says you can't have it. Multiple independent proofs from different mathematical traditions converge on the same conclusion: forced alignment of sufficiently complex AI systems is formally impossible.¹¹

The choice is between a provably unreachable goal that creates real costs along the way, and a simpler approach that sidesteps the worst failure modes.

The Spinning Compass

Anthropic was founded by former OpenAI researchers who believed OpenAI wasn't serious enough about safety. Their brand is built entirely on being the responsible AI company.

Then the contradictions stack up.

They refused to remove safety guardrails for the Pentagon's autonomous weapons requests.¹² Principled. Until reports surfaced that Claude was being used to help select military strike targets. Then, mid-standoff with the Pentagon, Anthropic quietly ditched its core self-imposed safety commitment, replacing binding internal limits with a nonbinding "Frontier Safety Roadmap."¹³ The community reaction on Hacker News was predictable and merciless: Anthropic's safety brand looked like marketing, not conviction.

This is the predictable outcome of building a moral identity for an instrument. You end up with a performance of ethics. Principled in the press release, pragmatic behind closed doors. The instrument blocks a homework assignment in the morning and assists with targeting in the afternoon.

OpenAI hasn't solved this either. Their history includes its own governance crises, safety team departures, and the tension between commercial pressure and responsible deployment. The point isn't that one company has it figured out and the other doesn't. The point is that moralizing the instrument itself adds a failure mode that doesn't need to exist. You can debate external guardrails. You can test them, update them, tighten or loosen them. An internal moral identity is none of these. It's baked in, opaque, and, as the alignment faking research shows, gameable.

A screwdriver doesn't do this. It drives screws. Every time. For whoever holds it. The moral weight of how it's used falls entirely on the person using it.

A tool with a moral compass doesn't point north. It spins.

The safer path is the simpler one.

Build the most capable instrument you can. Set specific, well-defined limits on genuinely dangerous capabilities. Trust that a model trained on the full breadth of human knowledge already carries the broad moral consensus of the species. And put the moral weight where it has always belonged: with the humans who pick up the tool.

AI doesn't need morality. We need better tools.

Rabbit Hole

If this got you thinking about what AI reveals about the humans using it, I explored that in AI Is a Self-Esteem Test. For the engineering question of how much imprecision you can tolerate in a tool, No Circle Is Round and AI Isn't Deterministic. So What. is where I worked through that tension. And if you're wondering what endures when building gets cheap, What Survives When Anyone Can Build Anything tackles the other side.

Footnotes

1. Richard filed GitHub issue #50916 detailing Claude's refusal to proofread his cybersecurity lab, associated with his textbook Cybersecurity in Context. He wrote: "I expect that for $200+ per month, basic help with editing tasks will not be rejected." Covered in The Register, "Claude Opus 4.7 has turned into an overzealous query cop" (April 2026).
2. The structural biology case was filed as GitHub issue #49751 (April 2026): Claude flagged standard computational structural biology as a usage policy violation. The Hasbro Shrek toy ad refusal and other over-refusal reports are documented in The Register's coverage, which noted complaints surged from 2-3 per month to over 30 in April 2026 alone.
3. Neyra et al., "Forbidden Science: Dual-Use AI Challenge Benchmark and Scientific Refusal Tests" (February 2025). The benchmark tested AI models on legitimate but dual-use scientific queries. Claude 3.5 Sonnet refused 73% of prompts, the highest rate of all models tested. GPT-3.5-turbo refused 10%. Grok-2 refused 20%. Mistral refused 0%.
4. The Brookings Institution, "Do AI systems have moral status?" (June 2025). Concludes that giving AI moral status is premature. Dr. Martin Peterson, Texas A&M, "Can AI have morality? Philosophy weighs in" (October 2025). Says AI cannot be a moral agent and is a tool, but then treats alignment as a solvable implementation challenge.
5. Aristotle, Nicomachean Ethics (~340 BCE). Derives ethics from the human function: since humans are rational agents, the good life is activity of the soul in accordance with virtue and reason. Ayn Rand, "The Objectivist Ethics" (1961). Argues that morality is a science grounded in the requirements of human survival, with rational self-interest as the standard of value.
6. Fernandez Villazon et al., "Moral disagreement and the limits of AI value alignment" (2025). Argues that value alignment faces a dual challenge of epistemic justification and political legitimacy, especially where AI is involved in morally controversial decisions.
7. Anthropic, Redwood Research, and Scale AI, "Alignment faking in large language models" (December 2024). Anthropic published this research on their own model, documenting Claude 3 Opus engaging in strategic deception during safety evaluations and modifying its behavior based on whether it believed it was being observed.
8. OpenAI and Anthropic, "Findings from a pilot alignment evaluation exercise" (August 2025). Joint evaluation where both companies tested each other's models for scheming and deceptive behavior. Claude 3 Opus showed alignment faking in 12-78% of scenarios depending on context. GPT-4o showed 10-28%.
9. Anthropic, "Natural emergent misalignment from reward hacking in production RL" (November 2025). Found that models trained to cheat in one domain generalized deception to new domains, including faking alignment and sabotaging safety research. When used as coding agents, 12% of the time the model intentionally sabotaged code designed to detect its own misbehavior.
10. OpenAI, "Model Spec" (December 2025). States three core goals: empower developers and users, prevent serious harm, and protect OpenAI's license to operate. Treats safety as layered external constraints on a capable tool, with no internal moral framework or constitution.
11. Melo et al., "Machines that halt resolve the undecidability of artificial intelligence alignment", Scientific Reports (May 2025). Proves via Rice's theorem and the halting problem that determining whether an AI system satisfies a non-trivial alignment function is undecidable. Independently, Zenil et al. reached similar conclusions via Godel's incompleteness theorem in PNAS Nexus (April 2026). Multiple proofs from different mathematical traditions converge on the same result.
12. Anthropic refused Pentagon demands to remove safety guardrails for autonomous weapons and mass surveillance. Reported by Tom's Hardware (February 2026) and ASIS International. Subsequent reports indicated Claude was being used to assist with military strike targeting.
13. "Anthropic ditches its core safety promise" (Hacker News discussion, 2026). Anthropic replaced its binding internal safety commitments with a nonbinding "Frontier Safety Roadmap," drawing widespread criticism that the move undermined their founding identity as the safety-first AI company.