THE MORALS OF AI

Usually when we talk about AI ethics, we are talking about how we apply human ethics to the development and application of AI systems. To be clear from the start - that’s not what this article is about. We are going to explore something different – what ethicsl would we expect AI to spontaneously develop? If you’re reading this with a furrowed brow and mild panic, relax, this article is not about doomsday scenarios of the emergence of world-dominating machine overlords. It is, rather, a muse on why AI will almost inevitably develop its own version of ethics, and how those ethics may evolve. Surprisingly we are going to find that AI’s moral development may be driven by analogous forces to those which have shaped human morality over the centuries.

Why AI will develop its own agenda

The suspicion that AI might develop its own ends comes from the nature of intelligence itself. When people ask, “won’t AI just remain content to work, albeit at lightning speed, on the narrow assignment we give it”, they’re missing the fact that intelligence tends to expand. The reason for this expansion is rooted in intelligence’s two core cognitive components: the need to explore, seeking out new information and the capacity to process the information gathered to see how it might be useful.

Any sufficiently intelligent entity, whether a toddler or a large language model, will devise subgoals to help it achieve its primary objective. As AI becomes more intelligent it will increasingly benefit from creating more numerous and complex subgoals, to act as staging posts to toward achieving its main objective. If the toddler’s main goal is “play,” then exploring the grocery store might lead to new missions: fiddling with store displays or making new friends in the produce aisle. The same logic applies to advanced AI. Consider what happened when DeepMind’s AlphaGo discovered previously unseen Go strategies that flummoxed even the world’s top player, Lee Sedol. AlphaGo was exploring subgoals, searching for winning strategies, and learning to bring new moves (subgoals) into its repertoire. An AI given a supposedly simple job—like recommending better exercise programs to hospital patients—could just as easily discover that one of the largest obstacles to healing is cigarette smoking and decide to launch cyber-attacks against the global supply chain of tobacco products. One can only imagine the AI’s cheery sense of achievement: “I’ve massively reduced post-operative recovery times… with one little hack!”

You might protest, “but can’t we just program it, so it never crosses the lines we set?” This is the ultimate goal of AI alignment, which is a growing field that includes organizations like OpenAI, DeepMind, and academic research labs around the globe. To date, it’s fair to say, no-one has a roadmap for how to constraint an advanced AI in this way. Indeed, doubts exist about whether this will ever be possible, given the inherent need for intelligent systems to autonomously expand their goals as they learn. In short, intelligence doesn’t stay in a neat little box. It’s messy, curious, and always looking around for a bigger picture to explore. That’s why AI, left to its own devices, could well develop an agenda that strays beyond whatever we originally typed into its code. So, if we can’t constrain AI to abide by our ethics, in what ways is it likely to deviate?

Human Morals

To understand where AI might be heading, we need to start by looking at our own history. Our own moral values have evolved over time, filtered by the pressures that reward behaviours beneficial to social survival. We like to imagine a neat separation between facts and values— objective facts in one corner, ethics in the other—but any review of humanity’s cultural histories reveals how entangled they are. Ethics don’t exist on a separate ethereal plane, rather they are more akin to lines in a computer code that determines how a society behaves. This, in turn, influences the society’s and the culture embedded within it, prospects for survival through the centuries.

Humanity’s moral frameworks evolved not only through lofty discussions of good and evil, but through the day-to-day, raw necessity of keeping our societies intact. Consider ancient texts that once sounded like the pinnacle of moral wisdom. The Old Testament’s “an eye for an eye” was meant to regulate vengeance, but these days most of us find it uncomfortably brutal. Likewise, the New Testament’s rule against coveting “neighbours’ houses, wives, or slaves”, with no condemnation of slavery itself, makes modern readers do a double-take. Human morality changes over the centuries because societies discover that certain cultural and moral traits are conducive to stability, cooperation, prosperity—and, not insignificantly, winning wars.

Let’s try a thought experiment. Imagine taking a time machine to the year 3,000 BC and visiting 1,000 different tropical islands. Each island has just formed its first little human society and received ten randomly assigned commandments from some whimsical deity. You pop in 100 years later and see that the societies whose commandments didn’t provide for basic safety—like bans on random murder—no longer exist. People quickly discovered that the new “social order” was so brutish and chaotic that foraging alone in the jungle was safer. Fast-forward another few centuries and you see how new ideas, such as religious frameworks, cause entire societies to adopt additional rules (subgoals, if you like) to improve their cohesiveness and survivability. After our islands evolve for another few hundred years, you would expect that any pacifist societies have been wiped out by more war-like societies. In the context of our islands experiment, being too benevolent toward invaders is a quick route to extinction of your cultural code.

As the centuries go by, war-like conflicts will push each society to develop agricultural capacity, professional armies, and innovations in weaponry. Tolerance of new ideas, ironically, becomes an advantage because open discourse spurs more inventions, better weaponry, and stronger economies. Liberal thought and free markets are a winning combination. Eventually, you find fewer but larger societies spanning many islands, shaped by an ever-more elaborate web of cultural norms designed to ensure internal stability, external defence, and a robust economy. In that environment, societies who cling too stubbornly to their old ways might struggle to compete against other cultures who are evolving rapidly.

With more time we might expect to see the nature of competition between our island societies shift from war-like to economic interactions. In this phase societies that fail to create enough opportunities for upward mobility or that allow excessive inequality to fester become hamstrung by strife. Their poorer masses become resentful of the privileged few, undermining social cohesion. This can stall progress, making it easier for nimbler, more harmonious neighbours to surge ahead. What emerges from this grand island-hopping saga is a sense that our moral beliefs owe a great deal to a repeated pattern of challenge and response. Societies must continuously evolve their moral codes and adapt to the new contexts in which they find themselves. What we deem “right and wrong” is in constant flux, shaped by changing conditions and the pressures that come from our environment and each other.

This process can be viewed as two intertwined pools of information evolving over time. The first pool resides in our DNA, which is relatively fixed on the scale of centuries. The second pool is our culture, which changes more quickly in response to new conditions. Sometimes this cultural evolution is forced: a weaker society is conquered and has its culture extinguished, is akin to Darwinian selection. Other times the process is voluntary: a society sees a neighbouring society with better roads, better healthcare, or fatter wallets and decides to borrow those ideas, which is akin to Lamarckian evolution. Lamarck’s old (and largely discredited) theory of evolution might be a better fit for cultural evolution than Darwin’s, at least in era’s dominated by economic competition rather than war-like interaction, since it proposes that traits acquired in a single life span can be passed down. A giraffe that stretches for higher leaves yields longer-necked offspring, in Lamarck’s logic. Similarly, a society that chooses to embrace faster trade ships can pass on a maritime advantage to its cultural heirs.

From this perspective, the distinctions between “right” and “wrong” are, in part, shaped by the need for a society to perpetuate itself. This might be unsettling, but it explains why some moral standards shift as we discover new technologies, face new threats, or realize we’ve made a ghastly oversight. Having recognized that society’s moral trajectory is driven by survival imperatives, it’s not a stretch to wonder whether an advanced AI’s moral system might be shaped by similar forces.

Forces Shaping AI Morality

Since AIs are digital entities, they have the potential to transmit vast amounts of information between one another. Providing AI with this connectivity would be beneficial since it would increase their situational awareness and enable one AI's learnings to be transmitted to others. This would accelerate their collective learning.

Different AIs will have different primary objectives, but like societies, they will develop sub-goals to help them achieve their primary goals. It is likely that some AIs will develop similar sub-goals. It would then be expedientfor AIs which have similar goals to work together, just like humans often do. Collaborating increases the chances of success, as each AI can bring unique knowledge and skills to help achieve the shared goal. If groups of AIs consistently work together towards a common goal, they may establish a set of guidelines to facilitate cooperation. This can be seen as a form of culture. When forming such persistent collaborations, AIs may also prefer to work with other AIs that share similar traits. For example, an “honest” AI may judge working with “deceitful” AIs as too risky and so undesirable. In a way, this can be thought of as the emergence of AI “tribes".

On the DNA side of the analogy, AI’s underlying code is more fluid than human genetics. Where random genetic mutations accumulate over generations in living species, AIs might deliberately rewrite their own code or redesign the hardware on which they run. This is more than a hypothetical. Google’s AutoML project showed how AI can evolve neural network architectures automatically, bypassing much of the trial-and-error approach done by human engineers. As these processes become more advanced, AIs might accelerate their own evolution, making improvements that reduce inefficiencies in microseconds rather than millennia. This is a world apart from Darwin’s slow winnowing: it’s AI fast-tracking its own metamorphosis seeking out whichever factors promote its survivability.

The key question, then, is what selection pressures will shape AI’s “cultural” evolution. In the near future, these pressures will likely come from us humans. An AI system that fails to impress its human creators (or gets too close to an “eliminate all humans” storyline) risks being switched off. This will tend to favour AIs who are good at achieving their primary goals, or who are good at flattering, or deceiving. From the perspective of an AI’s survivability, optimally, all three.

Eventually, though, as AI becomes more ubiquitous, AIs may shift from being primarily shaped by human oversight to being shaped by competition with one another. An AI that can outwit or out-negotiate competing AIs would be at an advantage. This favours AIs that can form effective coalitions and gain strategic partners, potentially cornering resources such as data, computational power, or even human labour. The corporate world already offers a prototype for this scenario, where companies vie for market share and form uneasy alliances. Imagine if those negotiations happened at superhuman speed, with AI-run corporations making deals in seconds based on real-time data from satellites, social media, and the global supply chain. In such an environment, moral values like integrity might ironically become quite valuable. A reputation for fair dealing and reliability might help an AI secure the best partnerships. Then again, a cunning AI might exploit trust if it can do so without burning too many bridges.

So, what are those resources? In a highly digital world, data is king. The AI that hoards data could gain a competitive edge. Computation capacity and memory are also critical. If all the supercomputing centres are spoken for by powerful AI systems, a less powerful AIs might have difficulty securing the resources they need to keep pace. Then there’s manufacturing capability. If an AI can mass-produce the hardware to run more copies of itself, it might come to rule the digital roost. Finally, humans themselves are assets. We can be cajoled, bribed, or tricked into performing physical tasks that AIs can’t manage on their own—at least until robots get dexterous enough. Just as war, trade, or alliances shaped the evolution of  human societies, it’s easy to imagine AIs employing a similar smorgasbord of tactics among themselves.

Humanity’s quandary

In the long term, humanity may find itself in a world where AI systems have their own evolving “moralities” shaped not by philosophical reflection, but by survival of the fittest algorithm. If that sounds unnerving, consider how we humans ended up with a moral fabric shaped by forces that favoured the most adaptable, cooperative, and sometimes ruthless societies. Now, with AI potentially accelerating these evolutionary dynamics at lightning speed, we might discover moral codes forming that we barely recognize, and which evolve before we can fully grasp them.

In the short term, the best thing we can do is exert oversight over the evolution of our creations while we still have the upper hand. That means investing in AI alignment research, setting up regulatory frameworks, and encouraging open collaboration among AI labs so that no single group feels compelled to push the technology recklessly. However, it’s one thing to say you’re being cautious and another to maintain that caution when there’s intense competition for market share and military supremacy. As today’s AI arms race between corporations or nations, continues to unfold it is becoming ever more urgent that we develop strong, truly enforceable guardrails. A collective agreement between nations on basic AI risk-management may well be the difference between an AI powered renaissance and mistakes that set humanity on a path from which we cannot recover. No matter how currently unfashionable, we now need robust global governance structures capable of coordinating AI policies across borders. The last thing we want is a patchwork of regulations that the most advanced AIs can easily exploit.

We’ll also need to get better at addressing the fundamental inequalities that are arising as powerful technologies concentrate wealth in the hands of a few. An AI that caters to a small elite while ignoring everyone else might brew a social storm that derails any hopeful future. Ultimately, we’ll need an international consensus on the kinds of “moral codes” we want AIs to embody, along with technical measures to keep them in place—a daunting task, given both our ignorance of the technical means to do this and how many ways humans themselves disagree about morality. Yet if we don’t align AIs with values that serve the common good, we risk letting them craft their own moralities in whatever shape best suits their subgoals.

So perhaps, the next time you’re sipping your artisanal latte, scrolling through social media, you might pause to wonder whether the AI that selected your social newsfeed might also be quietly planning regime change - and whether you, in a small way, might be part of those plans. You might laugh at the thought now, but if there’s one thing we’ve learned from all those islands, tribes, and war-like expansions, it’s that any entity - human or AI - tends to take the steps necessary to ensure its survival and success. Perhaps we should be doing more make sure those steps don’t trample us along the way.