Who Gets to Teach AI Right From Wrong?

Yesterday, I showed you a chart suggesting that today’s leading AI models have something that looks surprisingly like a worldview.

That naturally raises another question.

Where did those values come from?

At first, the answer might seem obvious. After all, large language models are trained on enormous amounts of text from books, websites, research papers and countless other sources.

Surely they’re just reflecting what they read.

But that’s only part of the story. Because it doesn’t explain why Claude might refuse a request that Grok answers.

Or why Gemini sometimes responds differently than ChatGPT.

Or why nearly every major AI model tends to sound remarkably thoughtful, measured and polite.

That doesn’t happen by accident. Those behaviors are learned.

And that’s where building AI gets a whole lot more complicated.

AI’s Learned Behaviors

Today’s AI systems don’t simply absorb information during training.

Once they learn language, researchers begin teaching them how they should behave.

One common technique is called reinforcement learning from human feedback.

In simple terms, people review thousands of responses, deciding which answers are more helpful, more accurate and less harmful. The AI is then rewarded for producing the kinds of responses humans prefer.

Over time, those preferences become part of the model itself.

That sounds straightforward until you start thinking about the kinds of preferences an AI needs to consider.

For example, a parent might want a very different kind of answer than a physician. A therapist might prioritize compassion, while a lawyer might prioritize precision. And a teenager might simply want encouragement.

In other words, the same question could have several reasonable answers depending on the values behind it.

That means AI companies aren’t just teaching computers how to answer questions. They’re deciding what a good answer actually looks like.

And that’s a much harder problem than teaching a model to write computer code or summarize a document.

It’s also one reason why many of the world’s leading AI companies are employing philosophers alongside engineers.

These folks aren’t being hired to write software, but to think through questions that people have debated for thousands of years.

Questions like:

When should honesty outweigh kindness?
When should safety outweigh personal freedom?
And when should an AI refuse to answer a question?

Anthropic decided to tackle these questions in an unusual way.

Instead of relying entirely on human reviewers, the company developed what it calls Constitutional AI.

Researchers gave Claude a written set of guiding principles inspired by sources like the Universal Declaration of Human Rights and other broadly accepted ethical frameworks. Claude then critiques and revises its own responses against those principles before producing a final answer.

It’s a little like giving an AI a conscience.

Not because the model understands morality the way people do. But because it’s been taught to weigh its responses against a consistent set of values.

The company has since gone even further.

Earlier this year, Anthropic published research examining more than 700,000 real-world conversations with Claude.

Instead of asking what values researchers intended to teach the model, they asked what values was Claude actually expressing.

The researchers identified more than 3,300 distinct values across those conversations.

Some appeared exactly where you’d expect. Historical accuracy, professional responsibility and intellectual honesty were all represented.

But the most interesting discovery is that Claude wasn’t applying the same value system to every conversation.

When discussing relationships, it emphasized empathy and mutual respect. When helping someone solve a technical problem, it prioritized accuracy and competence. And when talking about sensitive topics, it leaned toward harm reduction and personal safety.

Rather than applying the same values to every conversation, the model adjusted its priorities based on the situation.

That doesn’t mean it was making moral judgments like a human would.

But it was doing something remarkably similar.

And that’s a remarkable place for the AI industry to be in already.

Here’s My Take

Yesterday’s chart showed that today’s leading AI models appear to share a worldview that’s distinct from almost every country on Earth.

Today we’ve taken a step closer to understanding why.

Those values didn’t simply emerge from the internet. They’re being shaped through thousands of design decisions made by researchers, engineers, ethicists and even philosophers.

That’s inevitable. Because the moment an AI starts giving advice instead of simply retrieving information, someone has to decide what good advice looks like.

It’s one of the biggest challenges facing the entire industry today.

But I believe it’s only a temporary one.

Right now, millions of people are essentially using the same AI.

But I’m convinced the next evolution of artificial intelligence won’t be about building one model for everyone.

It’ll be about building a different model for every person.

The next generation of AI won’t simply remember your previous conversations. It’ll learn how you prefer to think, how you like information presented to you and even what level of risk you’re comfortable with.

That has the potential to make AI far more useful than anything we’ve seen so far.

And in our next issue, I’ll show you why the biggest AI companies are already laying the groundwork for this future.

Regards,

Ian KingChief Strategist, Banyan Hill Publishing

Editor’s Note: We’d love to hear from you!

If you want to share your thoughts or suggestions about the Daily Disruptor, or if there are any specific topics you’d like us to cover, just send an email to [email protected].

Don’t worry, we won’t reveal your full name in the event we publish a response. So feel free to comment away!