Newsletter | Pope Leo XIV Is a Big-Time Systems Thinker

photo by Edgar Beltrán

When an emerging technology is so significant that it promises (threatens?) to transform societies, the way we live and even what it means to be human, it is no surprise that religious leaders will feel compelled to weigh in. Pope Leo XIV did just that last month, joining the dialogue on AI with the release of his encyclical, Magnifica Humanitas.

As AI is increasingly becoming the dominant technology of our time, seeping into the products and services that shape our lives – and thus our health – we’re particularly interested in how it develops and in what guides that development. Leo’s standing as a prominent religious leader – and one who is seemingly independent from many of the conflicts that plague other stakeholders – makes his perspective intriguing.

Magnifica Humanitas is simultaneously reasonable and radical. In examining the relationships among technology, economics, politics, society and humanity, it offers straightforward, often matter-of-fact critiques of how market capitalism and concentrations of power in multinational corporations have led to structural inequalities – and how these same structures could shape the development and deployment of AI toward negative social outcomes. And yet challenging the very system that governs how we, as a now global society, choose to build the future feels like a radical position.

Leo starts by noting that any technology isn’t inherently good or evil, but rather like a liquid, it takes the shape of the vessel into which it is poured. As he puts it, “technology is never neutral, because it takes on the characteristics of those who devise, finance, regulate and use it.” He then goes on to critique those characteristics – both in a structural sense and in terms of the values and paradigms that are instantiated in the structures and the rules that govern the societal adoption of technology.

Building on the writings of his predecessors, Leo addresses injustice and inequality, noting that they result from the system itself: “injustices do not arise solely from the wrong choices of individuals, but also from structures, mechanisms and economic and cultural systems that produce inequality almost automatically.” He goes to say that when new technologies are introduced, believing that they will automatically benefit everyone is “to ignore the evidence. Unless transformations at the design stage prioritize the prevention of new and further disparities, technological progress will inevitably produce structural inequalities.” He urges political leaders to take responsibility for outcomes rather than outsourcing their duty to the market: “More than ever, in the age of AI and robotics, it is no longer possible to rely solely on the ‘invisible hand’ of the market. Politics has the task of orientating economies and technologies to the common good, promoting dignified work, social inclusion and an equitable distribution of the benefits of innovation.” This duty, Leo recognizes, has become even more challenging as technology development and deployment has increasingly fallen under control of private interests whose power often exceeds state actors.

At one level, Leo is focused on the broad set of rules that make up our economic system and the failure of governments to uphold the common good when the system yields unjust results. At another, he challenges the paradigms from which those rules – and our fascination with AI – are implicitly derived. In particular, he questions the elevation of efficiency, the need to “correct” imperfect humans and the treatment of humans as resources.

Invoking his immediate predecessor Pope Francis and echoing technology critics such as Neil Postman, Leo decries the primacy of technocracy and efficiency:

“In his Encyclical Laudato Si’, Pope Francis denounced the growing dominance of a technocratic paradigm in our globalized world: the tendency to let the logic of efficiency, control and profit alone shape personal, social and economic decisions. This makes it clear that technology is not simply a tool. When it becomes the standard by which everything is judged, it begins to dictate what matters and what can be discarded, reducing creation to an object of exploitation and human beings to mere cogs in a system driven toward ever greater efficiency.”

He even calls out our obsession with GDP as a key culprit, noting that it and related metrics “almost systematically neglect aspects essential to the overall wellbeing of people and the environment,” and calling for the development of alternatives. (There’s some progress on that – see below.)

Underpinning this technocratic paradigm is what Pope Leo calls an “anti-human” vision that doesn’t respect the natural limitations of humans but rather sees them as “errors to be corrected.” He doesn't mince words about where he sees this paradigm leading:

“More gravely, the pervasive technocratic paradigm in which we are immersed, and that is amplified by the digital revolution and AI, threatens to normalize an anti-human vision. In that vision, the fullness of life is equated with having more, reducing weakness, eliminating uncertainty and exerting total control. When efficiency becomes the ultimate measure of value, human beings are tempted to see themselves as a project to be optimized rather than as persons called to relationship and communion.”

Interestingly, he even takes an implicit swipe at the current wellness and longevity movements, noting that “the human desire for fullness of life is at risk of being misled by deceitful goals, such as the prospect of a technology that promises to free us from all weakness, and models of wellbeing that leave behind entire populations.”

Perhaps most troubling to Leo is – in an age where platforms delivers “eyeballs” and “engagement” to advertisers, where products are designed to become craveable or even addictive – what he calls the “post-humanist mentality that tends to regard the human person as an object to be manipulated or a resource to be optimized, removing all safeguards against the unchecked pursuit of profit.” He even includes a paragraph on data extraction as a form of neo-colonialism, comparing people in developing worlds whose data are being used, among other purposes, to train AI models to “rare earths” being mined.

All in all, Pope Leo’s encyclical makes the case that a technology shapes society not from its nature but rather because of its nurture. His work is thus a cry for more responsible parenting. It’s a thoughtful work that elevates beyond the circumstances of the moment to explore the deep patterns, to identify root causes and get to the heart of the matter. He argues that what a society chooses to build and how it builds it is crucial:

“We are called to reflect on the great ‘construction sites’ of our era and ask: What are we building? As technological development rapidly transforms languages, relationships, institutions and forms of power, we believers must and can choose which projects to work on and in what manner, so as to safeguard and value the grandeur of humanity that has been given to us as a gift. This is a choice not only for our future but also for our present, since artificial intelligence and other emerging technologies are already part of our daily lives.”

From our perspective at Building H, where we’ve drawn many of the same conclusions in our pursuit of a world where technological progress is aimed with intention at improving people’s health, Pope Leo’s diagnosis feels spot on. And yet we know how daunting it is to take on the deep patterns of our economic system, let alone the paradigms from which they arise. Silicon Valley, Wall Street and Washington might all turn deaf ears to Leo’s pleas. But we congratulate him for making them. That’s what we need leaders to do.

Read the full newsletter.