NORBERT BIEDRZYCKI

Can We Trust the Algorithms? The Ethical Challenge of Artificial Intelligence

Can We Trust the Algorithms? The Ethical Challenge of Artificial Intelligence

In August, hundreds of thousands of British students received their A level results. Because exams had been cancelled by the pandemic, grades were calculated by an algorithm designed by the exam regulator, Ofqual. Around 40 percent of results came out lower than teachers had predicted, and students from schools that historically performed worse were hit hardest. Within days students were protesting outside the Department for Education, the government reversed course and the algorithm was abandoned.

In February, a court in The Hague stopped the Dutch government from using SyRI, a system that combined data from multiple agencies to flag citizens for welfare fraud investigation. The court found that it violated the right to privacy under the European Convention on Human Rights, in part because nobody affected could understand how it reached its conclusions.

In June, the case of Robert Williams became public. He was arrested in front of his family in Detroit after a facial recognition system wrongly matched his driving licence photo to surveillance footage.

Three countries, three systems, the same lesson. Algorithms are now making or shaping decisions that change people’s lives, and our ability to understand and question them has not kept up.

The trust gap

Joseph Weizenbaum, who built one of the first chatbots in the 1960s, warned in Computer Power and Human Reason that there is a difference between deciding and choosing. A computer can decide, in the sense of calculating. Choosing requires judgment, and judgment carries responsibility. Almost half a century later his distinction reads like a description of our problem.

Trust in an algorithmic decision depends on three questions that most deployed systems cannot yet answer.

Is it fair? Does it treat comparable people comparably, and how would anyone know?

Can it be explained? Can the organization tell a person why the decision went the way it did?

Who is responsible? When it goes wrong, is there a human or an institution that answers for it?

In the A level case, the answer to all three turned out to be no. That is why a statistically defensible model collapsed politically within a week.

Why this is becoming a business issue

Until recently, AI ethics was discussed in universities, think tanks and corporate responsibility reports. That is changing. Recruitment, credit scoring, insurance pricing, fraud detection and customer service are all moving toward automated decisions. Each of these is a place where a company can now face the same collapse of trust that Ofqual faced.

The consequence is that ethical AI is moving from corporate social responsibility into strategy and risk. A model that cannot be explained cannot be defended to a regulator, a court or a journalist. A model that cannot be defended will eventually be switched off, often after the damage has been done.

The question for leadership teams is not whether their AI is accurate. It is whether they could stand in front of the people affected and explain it.

Leadership question: Can organizations deploy AI at scale without being able to explain or justify the decisions it makes?

Read next

  1. Ethical AI: From Principles to Accountability, 2020–2026

  2. The Ethics of AI and Human Autonomy: Are We Giving Away Too Much Control?

    Ethical AI series, written in 2026

  3. The Accountability Gap: Who Is Responsible When AI Makes Decisions Independently?

    Ethical AI series, written in 2026