Published Sep 15, 2026 | 9:35 AM ⚊ Updated Sep 15, 2026 | 10:14 AM
Image used for representational purpose. (AI generated)
Synopsis: The warnings about the harms AI could unleash are growing starker, with even the bosses now calling for a slowdown. AI governance becomes crucial against this backdrop. The choice must not be between control and empowerment, but about ensuring that responsibility gets assigned to institutions and people with knowledge, capability, influence and ability to prevent or minimise harm.
The first week of September saw an Anthropic researcher resign on a stark note.
“AI could kill us all by the end of the decade,” warned 28-year-old Jacob Coxon, who had previously worked at OpenAI.
Coxon said he had “spent the last three years doing pretraining research at both OpenAI and Anthropic.”
“Neither company is acting responsibly. They are racing straight to self-improving superintelligence and gambling with our lives,” Coxon grimly noted.
A few days later, OpenAI’s Sam Altman and Elon Musk backed Anthropic CEO Dario Amodei’s calls for slowing down the pace of AI development, acknowledging the harm that AI without proper guardrails could unleash.
But then what does an AI slowdown mean? As Ed Zitron, CEO of EZ Primary Research, was quoted as saying, “Right now we are very thin on what a ‘slowdown’ means.” This is what brings AI governance increasingly into the spotlight.
AI governance is currently being presented as a contest between two perspectives.
One argues that increasingly powerful systems must remain under strict human control through independent oversight and democratic accountability. The other argues that the danger lies not in the technology but in its concentration, and that broad access to AI capabilities will distribute power, accelerate innovation and deliver prosperity.
This framing is attractive, but incomplete.
The central problem of AI governance is neither how to balance safety and innovation, nor how to choose between control and empowerment. It is about ensuring that responsibility gets assigned to institutions and people with knowledge, capability, influence and ability to prevent or minimise harm.
We assign greater responsibilities to those best positioned to foresee, prevent and manage harm.
Food manufacturers bear obligations that consumers do not. Airlines bear responsibilities that passengers do not. Financial institutions shoulder duties that ordinary depositors cannot reasonably assume. These arrangements exist because knowledge, control and the ability to prevent harm are unevenly distributed.
Artificial intelligence presents an even starker version of the same problem.
Most users cannot inspect training data, reproduce safety evaluations, assess evolving capabilities, understand model vulnerabilities or assess wider social effects. Yet proposals for governing AI continue to rely heavily on disclosure, transparency and user choice.
Giving people access to increasingly powerful technologies does not necessarily equip them to evaluate or manage the risks those technologies create.
The lesson from decades of experience in fields such as food safety, finance and telecommunications is that empowerment works best when it rests on robust institutional foundations. Consumers feel protected, though not always, because standards exist, products are tested, supply chains are traceable, regulators can intervene, and responsibility can be assigned to those who possess the greatest knowledge and control. Choice occurs within this protective environment.
We, therefore, argue that the AI governance debate must not be about choosing between the Pro-Human AI Declaration and Zuckerberg’s The Future Is for Everyone manifesto. We need a framework that distributes the benefits and capabilities of AI broadly while placing progressively greater duties on those who design, deploy and govern systems whose impacts are consequential, systemic or difficult to reverse.
The Pro-Human AI Declaration frames AI governance as a defence of human agency against a “race to replace” humans in creative, caregiving, advisory, and decision-making roles. It identifies principles that advocate meaningful human control over AI, independent oversight of highly autonomous systems, restrictions on superintelligence development until proven safe, and liability and accountability for AI firms. It seeks constitutional constraints on frontier AI.
The declaration aligns its vision with EU-style AI governance, human-centric AI principles, labour-oriented governance, and democratic accountability debates. To gain legitimacy, it shares on its website a list of endorsers, which includes influential organisations as well as individuals. It also shares the result of a poll in which 72% believe AI companies should be legally responsible for harms and 69% want superintelligence prohibited until proven safe.
While the declaration correctly identifies the growing asymmetry between human institutions and increasingly capable AI systems, its insistence on meaningful human control and independent oversight reflects an important principle. Responsibility cannot be delegated entirely to machines. However, it does not discuss the framework for allocating responsibility among developers, deployers, governments, researchers, workers and affected communities within an AI-rich society.
Its strong proposals also face an enforcement problem. If governments treat frontier AI as a strategic asset, a declaration without binding institutional mechanisms cannot by itself alter their incentives. The declaration, therefore, offers a valuable constitutional direction but leaves unresolved both the distribution of responsibility below the level of the state and the international mechanisms through which commitments might become effective.
Zuckerberg’s manifesto, published on Meta’s website, presents “individual empowerment as the source of prosperity.” Its central claim is that the principal danger lies in the concentration of AI capability, rather than in technology.
It proposes widespread access to powerful AI, open-weight models, innovation as the central purpose of superintelligence, and balance of power through distribution rather than restriction.
Openness cuts both ways — once a model’s weights are released, they cannot be recalled, and the safeguards built into them are, by design, easier to strip out than those in a closed system, a point the International AI Safety Report makes as description rather than speculation.
Zuckerberg’s defence of broad access reflects an important democratic instinct. Yet the manifesto tends to treat empowerment as both a social objective and a governance mechanism. The two are not the same.
Zuckerberg’s argument tends to position ‘control’ and ’empowerment’ as rival governance strategies. That framing is incomplete.
In systems characterised by deep information asymmetries, ordinary users cannot reasonably evaluate training data, safety evaluations, emergent capabilities or systemic consequences. A governance framework built primarily on diffusion, therefore, risks transferring responsibility toward those least able to exercise it while leaving the actors who create and deploy frontier capabilities insufficiently accountable for their broader effects.
While widespread access can potentially reduce monopolistic control, encourage experimentation and distribute productive capability, it does not establish that diffusion will control systemic misinformation, correlated model failures, cyber or biological misuse, labour-market disruption, cumulative environmental effects, personalised manipulation or cross-border externalities.
Access is a distributive property and not necessarily a safety property.
The Seoul Declaration, adopted by ten countries and the EU in 2024 and updated since, identifies “safety, innovation, and inclusivity” as interrelated goals. It recognises the need for a “risk-based approach to maximise the benefits and address the broad range of risks from AI”. It also recognises that organisations developing and deploying frontier AI must share the responsibility for safety and seeks voluntary safety commitments from influential firms.
The International AI Safety Report 2026 goes further in one respect: it does not call for a pause on frontier development, but neither does it treat openness as inherently safe. Instead, it advocates “defence-in-depth” — layered technical and institutional safeguards. Its defence-in-depth approach shares one feature with mature international risk regimes: it assumes that no single commitment, safeguard or institution can adequately contain systemic risk.
A parallel effort from Shanghai, following the first meeting of the World AI Cooperation Organisation (WAICO), proposes tiered, category-based governance and treating AI-driven education as a primary response to job displacement. WAICO’s framework is a layered structure of rules on generative AI services, mandatory content labelling, and technical standards.
Other areas of shared understanding are:
* Independent scientific assessment — testing by shared expert bodies rather than a single company’s board or a purely political mandate — sits at the centre of both the Seoul framework and the safety report.
* The Centre on International Cooperation’s guidance note on AI governance proposes a global roadmap built on three pillars: managing AI risks, distributing AI benefits, and aligning AI rules. Together, they point toward a governance system that links safety, inclusion, and regulatory coherence rather than treating them as separate objectives.
* Regulatory interoperability — minimum global baselines with mutual recognition, so a model evaluated once need not be re-evaluated in every market — has backing from a December 2025 standards statement signed by sixty-five countries.
All the above-mentioned initiatives suggest that the future of AI governance does not lie in choosing between control and empowerment. These principles do not yet amount to a coherent global framework. They do, however, indicate that the debate is slowly moving beyond the choice between control and access toward a more fundamental question: how should responsibility be allocated in societies where knowledge, capability and influence are increasingly unevenly distributed?
Building on our earlier conversation on food safety regulation, we identify the following broad characteristics of an AI-shaped society:
* Choices that benefit individual firms, including automation and cost reduction, can generate collective harms such as society-wide deskilling, homogeneous reasoning, correlated errors, degraded human oversight, etc.
* Harm frequently originates upstream in model design, data selection, evaluation, deployment architecture and commercial strategy, yet responsibility is often transferred downstream to users with the least information and control. This is responsibility inversion.
* Consumers cannot directly observe many important hazards, e.g., whether evaluation results are representative, whether hidden system changes have occurred, whether a model is vulnerable to a particular attack, etc.
* Information does not equal capacity, e.g., a warning that AI can make mistakes is of limited value given that model limitations are dynamic, probabilistic and context-dependent.
* Vulnerability is unevenly distributed, as people differ in bargaining power, income, institutional support and exposure to automated decisions.
Under these conditions, empowerment is a legitimate social objective, but it is not an adequate risk-control mechanism. The more consequential, systemic and irreversible an AI application becomes, the less its governance can depend on disclosure, user choice or corporate self-restraint, and the more it must rely on upstream duties, independent assessment, layered safeguards and publicly legitimate institutions.
Citizens should have authority over governance purposes, protected values and acceptable trade-offs, while expert institutions assess technical means and evidence under conditions of public accountability and contestability.
Public opinion shapes AI governance in at least four ways:
* Provides political authorisation for legislation.
* Grants or withholds the social licence for deployment.
* Shapes how failures are interpreted and responsibility assigned.
* Affects whether citizens accept the authority of regulators, safety institutes and standards bodies.
Recent empirical research finds that perceived AI risk and institutional trust predict support for regulation. Greater perceived risk and trust in government are associated with stronger support for intervention, while greater trust in AI companies and AI technology is associated with less support for restrictions.
The legitimacy deficit surrounding AI governance is already manifesting in public settings, where younger people are confronting influential leaders with their anxiety about AI’s effects on their work and lives.
The BBC reports several such incidents. For example, the students’ reaction to Eric Schmidt is not only representative of young people, but also consistent with wider public unease. A Pew Research Center survey found that 50 per cent of American adults were more concerned than excited about the increasing use of AI in daily life, compared with only 10 per cent who were more excited than concerned.
While trust is at the centre of many conversations, it is difficult to manufacture and increasingly scarce in complex societies. The groups expected to live longest with AI’s labour-market, educational and political consequences do not yet see themselves as authors of the institutional settlement.
A more durable approach, therefore, is to focus on responsibility rather than trust.
Citizens need not possess detailed technical knowledge or place unconditional confidence in firms, regulators or experts. They must be able to shape the purposes and limits of governance, contribute evidence about experienced harms, and see that actors with the greatest ability to prevent harm bear corresponding responsibilities.
As discussed earlier, ‘constitutional control’ and ‘individual empowerment’ paradigms both risk overlooking a more fundamental principle that we have outlined in the previous section: responsibility should be with people and institutions who have knowledge, capability and can prevent harm. We articulate the following five governance principles:
Not all AI systems create the same level of risk, uncertainty, or dependence. Low-risk systems used for drafting, translation, education support, or routine assistance should generally be governed through transparency, privacy protection, consumer rights, and competition safeguards.
More consequential applications in domains such as employment, education research, insurance, finance, healthcare, and public administration require stronger obligations, including auditability, traceability, meaningful human review, and clearly assigned liability.
Regulatory intensity should increase with system capability, the magnitude and systemic reach of possible harms, and the difficulty of reversing those harms once they occur.
The emerging lesson from food safety, aviation, finance, and other complex systems is that responsibility must exist at multiple levels simultaneously.
Developers should bear responsibility for design choices, evaluation practices, model security, and foreseeable misuse. Deployers should bear responsibility for how systems are used in specific contexts and for the consequences of those applications. Regulators should bear responsibility for maintaining adaptive oversight and ensuring accountability mechanisms remain effective.
Independent scientific institutions should bear responsibility for testing, monitoring, and generating evidence that neither governments nor firms can credibly produce on their own.
Citizens should participate in determining societal priorities and acceptable trade-offs, but they should not be expected to manage risks they cannot reasonably understand or control.
Layering should not fragment accountability. Responsibilities must be sufficiently explicit that developers, deployers and regulators cannot each attribute failure to another part of the chain.
AI systems may optimise, advise, recommend, automate, and in some contexts make decisions faster or better than people. Yet responsibility for those decisions cannot be delegated entirely to machines.
Human authority requires more than a symbolic “human in the loop” process. Humans must retain the practical ability to understand, challenge, suspend, override, and, where necessary, terminate the operation of systems whose consequences affect human lives. This is not simply a safety requirement. It is a constitutional principle.
Responsibility becomes impossible to assign when authority disappears.
The history of risk regulation suggests that effective systems evolve through monitoring, reporting, learning, adaptation, and revision. Food safety did not emerge from a single law. Aviation safety did not emerge from a single inspection regime. Financial regulation did not emerge from a single disclosure requirement.
AI governance should be understood similarly.
Standards will need revision. Evaluations will need updating. New risks will emerge. Existing assumptions will be challenged. In highly uncertain environments, institutional capacity to learn may prove more important than any specific rule.
Much contemporary discussion focuses on trustworthy AI. Trust matters, but trust must not be the primary foundation of governance. Citizens should not be required to trust technology companies blindly. Nor should they be expected to trust regulators, governments, or experts unconditionally.
A stronger foundation is accountability.
People do not need complete knowledge of a food supply chain because accountability exists throughout that chain. Passengers do not need to understand aircraft engineering because systems of accountability exist around aircraft engineering.
Similarly, AI governance should be designed so that responsibility is visible and enforceable. We expect trust to emerge because of effective accountability rather than as a prerequisite for it.
The governance principles outlined above need to be put into practice through institutions that ensure that humans are in control, AI capabilities are accessible, and society can trust the producers, deployers, evaluators and regulators. The first steps should be:
* Independent evaluation centres and incident-reporting systems must generate credible knowledge about model capabilities, near misses and emerging harms.
* Explicit responsibility maps, graduated liability and compensation mechanisms must prevent developers, deployers and regulators from transferring accountability to one another or to users.
* Human authority must be protected through escalation, contestability, suspension and, wherever possible, reversibility.
These capacities must not remain confined to a small number of wealthy states; a global AI safety-infrastructure fund, along the lines of the Global Climate Fund, should support evaluation, regulatory expertise and public-interest research in countries otherwise unable to participate effectively.
A governance system built around responsibility would not seek simply to maximise either control or empowerment. It would ensure that those with the greatest knowledge, influence and ability to shape outcomes bear proportionately greater duties.
Citizens would retain agency but would not be expected to carry risks they cannot reasonably assess or control. This alignment cannot be achieved through declarations or advocacy alone. It requires institutions capable of independent evaluation, explicit allocation of responsibility, meaningful human intervention, public contestability and continuous learning.
The central challenge is therefore not to choose between human control and individual empowerment. Control without distributed agency risks paternalism and concentration; distributed agency without upstream responsibility risks externalising uncertainty onto individuals and societies unable to evaluate it.
A legitimate AI order must distribute ordinary capabilities and benefits broadly while subjecting consequential, systemic and difficult-to-reverse capabilities to progressively stronger obligations.
Its success should be judged by a simple test: whether responsibility grows alongside capability or is transferred to those least able to exercise it.
Also Read:
Casino Capitalism: Inside the financial architecture powering the AI boom
Fresh AI concerns: Did researchers push Claude to the brink of biological misuse?
What AI can’t teach: The power of serendipity in learning
Note: Generative AI tools were used in a developmental editing role during revision of this article. The authors remain responsible for all ideas, interpretations, examples, and final content.
(Edited by R Rajesh Kumar.)
