Μake us preferred on Google On 10 November 2026, the Australian government released recommendations following an incident involving an OpenAI artificial intelligence agent gaining access to government websites, and described the information it had received as inadequate. A leak of individual medical records has not been established. But a question of power arises: who is told, in time, that a privately developed system has overstepped its limits?
The government announced plans to introduce independent checks and mandatory incident reporting. This legislative intention acknowledges something fundamental: safety cannot depend solely on assurances from those who build and sell the technology.
Who really exercises power in a democracy? The obvious answer would be the institutions that derive their legitimacy from citizens. Yet in the digital age, an increasing share of our daily lives is shaped by decisions made outside the democratic process: by companies that control communication platforms, critical infrastructure and artificial intelligence systems.
Technology is not merely a tool. It determines which information reaches us, which choices are presented as important and, in some cases, how applications, job candidates or citizens are assessed. When these decisions have social consequences, the debate is no longer simply about innovation. It is about power.
In a democracy, state power is subject—at least institutionally—to rules, parliamentary scrutiny, judicial protection and electoral accountability. A private company, by contrast, is not elected by citizens. That does not automatically make its conduct arbitrary or unlawful. It does, however, expose a gap: its decisions can affect millions of people without giving them a corresponding say or means of scrutiny.
Artificial intelligence makes the issue more complex. If an automated system unfairly excludes a candidate from a job, reproduces discrimination or uses data without a lawful basis, who takes responsibility? The company that developed it? The organisation that chose it? The person who accepted its decision? Responsibility cannot disappear behind the words “the algorithm decided”.
This is not merely a hypothetical risk. In the SCHUFA case, the Court of Justice of the European Union ruled in 2023 that when a bank relies decisively on an automated credit score, safeguards governing automated decisions apply. It did not ban all scoring. But it showed that a number determining someone’s access to credit is not simply a neutral piece of information.
The significance of this example extends beyond banking. If an organisation points to the score while the supplier insists it merely provides information, a citizen may be left facing a decision for which no one can be identified as responsible. A human signature is not enough when the substantive judgement has already been handed over to the system.
The problem is not confined to remedying individual harm. It also concerns who sets the rules before harm occurs. When public services depend on private digital systems, technological dependence can become political dependence. A state may formally retain its decision-making powers while lacking the knowledge or practical ability to scrutinise the technology on which it relies.
There is, of course, a serious counterargument. Private enterprise has accelerated research and created tools with real benefits for health, education, accessibility and productivity. An excessively rigid regulatory framework can favour already powerful businesses, which have the resources to comply, while shutting out smaller innovators. The answer, then, is not to treat every technological advance as a threat.
The European framework does not leave citizens entirely unprotected. Subject to certain conditions, the GDPR provides protection against solely automated decisions with serious consequences, alongside safeguards for human intervention and the ability to challenge a decision. The AI Act adds rules on oversight and explanations of certain decisions, with implementation phased in over time. What matters, however, is whether someone who has been wronged can actually use these safeguards. A right on paper does not guarantee that anyone will hear their objection.
The answer lies in proportionate and effective accountability. The person responsible for oversight must have the knowledge, time and genuine ability to disagree. A public service that purchases a system must be able to scrutinise its use. And an affected citizen must have access to a review process, not be sent back and forth between the organisation and its supplier.
The deeper question is political: can democracy remain meaningful when crucial choices affecting social life are transferred to systems that most citizens neither understand nor can challenge? Engineers and legislators alone cannot provide the answer. It requires public debate about the limits of private power and citizens’ right to know, to scrutinise and to push back.
Progress is not measured solely by what a technology can do. It is also measured by whether society retains the ability to decide how it will be used, by whom and for whose benefit. If challenging a decision that shapes our lives requires us first to obtain permission from those who designed the system, who ultimately holds the power?
