EU’s AI Act: Why do we need it?

Part 5: Dilemmas of the digitalised environment and principles of a new social contract

Despite a widespread recognition of worries about the growing dominance of Big Tech, governments have hesitated to regulate these powerful companies by law because ‘tech companies are both targets as well as tools for governments’ (Bradford 2023, p. 13). In practical terms, they are used by almost all governments for enhancing national security and promoting national wealth, while in ideological terms they are venerated symbols of an innovative economic and social environment, where rewards for creativity lead to the enriching of the whole society.

In this blog, I look at the concept of a social contract and argue that in the context of AI any new social contract must engage not only with the usefulness of digital technologies but their potential dangers too, and build in safeguards against this potential as a matter of urgency.

Dilemmas in a new social contract for the AI Age

The EU’s move to pass an AI Act was recognition that the terms of the existing social contract, making for an ethical balance of power in society, have come under tremendous strain by digital technology. Its earlier promise, particularly of the Internet being a democratic place, has been overwhelmed by its colonisation by commercial interests that have a seductive but, nevertheless, insidious hold upon the data, attention and even pocket of the ordinary user.

Unsurprisingly, this means the search for a new social contract has picked up speed amongst political and cultural elite. Important contributions have come from the UN platform, including from the International Organisation of Employers (IoE), the Secretary General of the United Nations and other conversations and discussions facilitated by UN and its agencies, and through other academic contributions (Ashrafian 2022; Bankins and Formosa 2021).

Waning hopes of the digital dividend

In its deliberations on a new social contract, the IoE has recognised that the ‘digital dividend’ has not fully translated into ‘decent work and higher productivity’ (IoE 2023) while the Secretary General, António Guterres, has highlighted that ‘inequality defines our time’ (Guterres 2020). Speaking in 2020, the Secretary-General specified that the 26 richest people in the world held as much wealth as half of the global population put together. He mentioned gender, age, ethnicity, race and disabilities as some of the other factors that exacerbate inequalities. In the same year, another important contribution, this time with special reference to the AI age, was from Professor N. Choucri, called Social Contract 2020.

In line with the global nature of the digital economy and the remit of the United Nations, these calls for a new social contract are made on a world level. They emphasise the need for greater equality, inclusivity, sustainability and solidarity.

Continuing faith in liberation through digitalisation

An important feature of these calls is the visualisation of digital technologies as a major driver for achieving equality and other democratic ends. Due to their efficiency and economic potential, digital technologies are said to have brought about the Fourth Industrial Revolution. Hence, the Secretary-General, in his 2020 speech, urged that ‘the remaining four billion people’ (Guterres 2020) should be connected ‘to the Internet by 2030’. Similarly, Professor Choucri’s Social Contract 2020 calls for ‘enabling and providing applications of AI to assist decision making for all critical functions – notably the provision of public services, performance of civic functions, and evaluation of public officials – supported by a Center for National Decision Making and Data (NDMD).’ Mwamadzingo et al. (2023) from the ILO as well, while recognising that ‘technological change, pertaining especially to new digital devices and tools such as artificial intelligence’ has failed to live up to its promise of enhancing productivity and cutting down the drudgery of work, still stress that ‘such innovations are needed’ (p. 36).

AI’s opacity and potential for loss of human control

At the same time, Social Contract 2020 states clearly that the powerful advances in AI ‘remain opaque ‘ and that AI is a domain of knowledge ‘where technological innovation interacts with the potential for a total loss of human control’ [ital added]. This is in line with other commentators on the subject and the acknowledgement in the Proposal for the AI Act that in AI we are dealing with a powerful but potentially very dangerous technology so that a certain amount of freedom to conduct business and of arts and sciences needs to be restricted ‘when high-risk AI technology is developed’ (p. 11).

 

Fig. 1: The advantages and dangers of unexamined AI use on a wide scale

Under the circumstances, it may seem pragmatic to accept AI as ‘already here’ and try to make it more ethical, as the only remedy available for controlling its impact upon the human race, but as Sætra et al (2021) argue ‘[T]he structures that allow Big Tech to gather data as they do now [which create autonomous AI] are contingent; they are not essential or necessary to the social and economic environment’ (p. 24). Bareis and Katzenbach assert that quite often it is the elite who are ‘Talking AI into Being’ (2022).

Those who continue to advocate increasing the reach of the digital to the whole population of the world continue to consider all forms of digital technology as a force for liberation from poverty and oppression. Yet if these moves succeed, they can potentially put AI’s power in the hands of technological and political elite at a time when the IoE stated that ‘too many governments have failed to sufficiently strengthen governance systems, [and] address corruption’ (IoE 2023).

Exposing large numbers of people to the digital forces identified by Deffains (see ‘Part 4: The digitalised environment and the threats to the existing social contract’ in this series of blogs), whether through such political elite, or through Big Tech, carries considerable risk.

Nevertheless, technology in the modern imagination remains closely connected to markers of improved living standards and growth of wealth and these new technologies continue to trigger those connections in popular, and often in academic, discussions too. However, the potential power that these technologies have of controlling individual thought and behaviour, coupled with their need for far less human input than ever before, means that the threats of the concentration of power and control in very few hands and the loss of real freedom for the rest is a real danger that must be engaged with urgently. Consequently, drawing upon a broad range of philosophical and political theories, I explore below some concepts that I argue must be incorporated in a new social contract to protect individuals and vulnerable communities from the severe asymmetries of power between the ordinary human person and Big Tech and authoritarian regimes in control of high-risk digital technologies.

Salient features of a new social contract for the AI age

Respect for the principle of contracting for survival

According to D’Agostino et al (2023) in a social contract ‘the real issue is “the problem of justification”—what principles can be justified to all reasonable citizens or persons’ [ital added].

Thomas Hobbes in his Leviathan (1651) asserted that one such principle, on which most reasonable people would agree, is to not ask a human being to acquiesce to their own destruction without resistance: ‘no man can transferre, or lay down his Right to save himselfe from Death, Wounds, and Imprisonment, (the avoyding whereof is the onely End of laying down any Right,)’.

Does such a danger exist in relation to unexamined and rapid AI use?

Like most commentators on the subject, the Proposal to the AI Act stated that the ‘AI Act is to ensure a high level of protection of health, safety and fundamental rights enshrined in the Charter’, indicating that these areas of people’s lives can be threatened by AI technologies.

Similarly, Professor Choucri’s Social Contract 2020 evokes the Precautionary Principle as the very first principle to undergird the new social contract. The Precautionary principle states that ‘if there is any chance at a technology causing catastrophic harm, and there is no scientific consensus suggesting that the harm will not occur, then those who wish to develop that technology or pursue that research must prove it to be harmless first (see Epstein 1980)’ (Sullins 2023).

Thus, the consensus seems clear that AI uses should be proven to be harmless first, and proven to the satisfaction of most reasonable people around the world, before their mass use is pushed forward.

In his 2016 book, Homo Deus, Professor Yuval Noah Harari carried out a broad overview of the several possible impacts of new technologies, including AI. He pointed out that humans have two kinds of abilities that are used in employment: physical and cognitive. If machines continue to replace these two kinds of labour (and AI is now beginning to replace parts of cognitive labour too) by performing it more efficiently, he asked, ‘What will conscious humans do’? (Harari, 2016, p. 370).

Not only this, but with wearable tech and smarthomes monitoring, measuring and guiding our lives in a far more efficient manner than we can ourselves do, and in some cases smart devices looking after our health from inside our bodies, ‘we may reach a point when it will be impossible to disconnect from this all-knowing network even for a moment. Disconnection will mean death’ (Harari 2016, p. 401).

If we do reach the Hobbesian point where opposing or opting out of this network is clearly seen as necessary for human survival, will we still have the understanding and control to do so if we continue to increase the spheres of our lives where digital systems are constantly exerting ‘wider perceptual control’ (Sales 2019) upon us, making critical thinking and debates difficult to carry out?

A new and just social contract must, therefore, insist on a critical and transparent debate on AI before its widespread adoption so that vulnerable populations around the globe, suffering from knowledge and power deficits, do not walk into agreeing to their own destruction without resistance.

Combating the astronomical asymmetries of power and knowledge

The access to the scientific and commercial knowledge driving the AI revolution, e.g., information about data centres that power the Internet or the rules and laws governing the deployment of hardware and software in different countries, is distributed extremely unequally around the globe. This puts the highly educated professionals in these fields in a position of tremendous power vis-à-vis the citizens, or even nations, that do not have access to academic centres of excellence in these fields, creating global economic inequalities caused by inequitable access to knowledge.

Further, as Allen (2019) points out, many researchers maintain close relationships with Big Business for funding and hence supporting the interests of business, rather than of the ordinary citizen, becomes woven into the very heft and weave of scientific education too.

This inequality of knowledge, as Professor Zuboff (2018) has argued, in the AI economy converts into an ‘absence of reciprocities’ with far-reaching consequences. The ‘surveillance capitalists’ in charge of the digital products and services very often do not depend on ‘the masses’ for employees, or even for their direct income. They employ a handful of highly educated people controlling a large number of machines. As a result, the bargaining power of labour is hugely diminished even in countries with well-developed education systems that do provide general access to scientific knowledge.

The only need that digital capitalism has for the masses is for their ‘data’ or information. It is able to extract this data by providing the masses with ostensibly ‘free’ services such as social media that the user ends up paying for by their personal data.

Morreale et al. (2023) demonstrate how the users of various digital systems contribute personal data and information in a way that is equivalent to extracting their ‘humanness’ to feed the machines in machine-learning AI systems.

“As individuals often do not know how, or do not have the power, to avoid interacting with these systems, they are essentially reduced to ‘labour units assigned to work with tools made available to them by their employers’. Their loss of autonomy and unwaged contribution of their ‘humanness’ for developing new products for corporates is equivalent to the unfair alienation of labour in the Marxian theory of surplus value.”

Under the circumstances, the terms ‘user’ and ‘consumer’ become linguistic strategies that obscure the true relationship of exploitation between the individual user and the corporate provider. A justice of UK Supreme Court, Lord Sales, describing the contracts for digital services, said in 2019 that these contracts provide ‘access to digital platforms on their [the providers’] terms requiring access to your data, and on their very extensive contract terms excluding their legal responsibility’ (Sales 2019).

The new social contract must recognise the individual’s autonomy, and the ‘choice mechanisms [of] … exit, voice, and loyalty’ (Zuboff 2019) as fundamental rights of human individuals to be protected. It must empower all individuals to maintain their human dignity, defined as ‘respect for another human as an “equal citizen” or “moral person”’ (Darwall 2017, p.194). This may even entail government provision of non-digital services allowing the consumers to vote with their feet as ideal market competition scenarios ought to do.

Further, if the political space opened up by the AI Act for ensuring human-centric developments in AI field is to be meaningful, it is to be recognised that breaking up the monopolies of Big Tech is complicated by knowledge asymmetries between political institutions tasked with oversight of these companies and the companies themselves. This is further exacerbated by even more limited resources in many middle- and low-income countries.

Therefore, a new social contract will provide for knowledge equity globally and ensure that the adaptation of AI follows, rather than precedes, the development of relevant knowledge and capabilities amongst people and regulatory authorities.

Development as freedom

As stated earlier in this blog series, a social contract is ultimately about power relations in society. In the modern era, technology has emerged as a significant site for power contestations. As the driver of ‘development’ and growth, the position of technology has been almost unassailable in the past 2-3 centuries. Consequently, labour forces have had to adjust and adapt their skills to fit in with technology as it has evolved and changed.

However, as Professor Zuboff and others have shown, with digital, particularly AI, technologies, things may be ‘different this time’. Professor Zuboff has shown that with AI and similar technologies ultimately the mass of the people may not be needed either as employees or as customers for those who control these technologies. This greatly reduces the power of the masses to bargain with the ‘surveillance capitalists’ in charge of the technologies.

Taken with the other potential dangers they pose – the circumscribing of humans to ever narrower categories of employment, the concentration of wealth and shocking financial inequality, manipulation of the will of the individuals, device-managed isolation of people, the spread of fake news and other forms of political confusion – AI technologies threaten to reconfigure power relations in society in unprecedented ways.

Therefore, their lightening uptake, particularly with their potential for ‘loss of human control’ (Choucri 2020), without adequate public information and debate violates fundamental democratic principles. The gap in the knowledge of the ordinary person is such that Veale and Borgesius criticised the EU’s public consultation on the AI Act in 2020 by saying, ‘[I]t is unclear whether limited existing efforts to include stakeholder representation will enable the deep and meaningful engagement needed from affected communities’ (2021). Yet, the EU’s consultation was the first large-scale effort to take the civil society’s voice on the subject seriously.

Further, stressing the need for clearer information to be made available to the relevant stakeholders on AI, Zhu et al. (2022) proffered the assessment that ‘humans and societies perceive trust in AI in intricate ways, which does not necessarily closely match the trustworthiness of a particular AI system’.

In these circumstances, Professor Amartya Sen’s exhortation from two decades ago to reconceptualise ‘development as freedom’ is a relevant concept. The rationale behind this call was that measuring development in terms of monetary wealth means that it is possible for ‘an organized and politically influential group of “industrialists”’ to ensure their profits are well protected even though ‘significant sacrifice’ is imposed on the ‘population at large’ (p. 121). With the growth of Big Tech we are seeing this happen.

The push to tie-in the entire human population to digital systems, on the basis of their purported economic efficiency, ignores this new reality of the threat to human freedom in a blind bet that the earlier liberatory potential of the internet will somehow come through despite later developments.

A new social contract must push back against this blind faith and examine whether the digital can still deliver on the meaningful freedoms that ‘include the liberty of acting as citizens who matter and whose voices count, rather than living as well-fed, well-clothed, and well-entertained vassals’ (Sen 1999, p. 288). Sen’s economic expertise did not prevent him from recognising that, without dismissing the value of a market economy, we must remember that ‘the more immediate case for the freedom of market transaction lies in the basic importance of that freedom itself’ (p. 112).

 

Fig. 2: Digital technologies and the threats to individuals’ freedom in a context of little knowledge and power

 

It is a salutary reminder that the world is facing a climate crisis that has resulted partly from a narrative of Progress that we now find was a very partial, and sometimes erroneous, account of human development. Yet, its sway did allow callous destruction of entire communities, knowledges and ways of life, that was wanton and harmful. In science, as in other branches of knowledge, narrow, but repeatedly mined channels of knowledge, can produce an illusion of deep knowhow, but ultimately, as Professor Sen reminds us, the Buddha’s ‘speeches on nonextremism’ (p. 112) may have vital lessons to teach us today and even the utilisation of  AI technologies that do not threaten addiction or manipulation must be done in consultation with a well-informed public. The freedom of the individual to be informed and consulted on all issues affecting their capability to live a meaningful life must be a fundamental part of the new social contract.

Rawlsian principles of justice

John Rawls’ idea of a fair social contract prioritised between two principles of justice – ‘the priority [was] liberty’ ((Rawls quoted on p. 44 of Kukathas and Pettit), and below that came ‘maximizing the advantage of the worst off, no matter how this may affect the advantages of all others’ (Sen 1999, p.286). A new social contract must secure the priority of liberty at the individual level first. This would also serve to maximize the advantages of the worst off in the current scenario where between ordinary citizens, the knowledge elite and the socio-political elite, the ordinary citizens are massively disadvantaged in relations of power.

The rapid integration of AI technologies in both industry and services without much warning threatens the ‘primacy of liberty’ by severely restricting the freedom of the individual to make use of her best capacity to pursue self-determined goals in life. Instead, she is forced

into ever-decreasing spheres of productive labour, coerced into keeping up with the ‘speed and shock’(Feenberg 2024), uncertainty and anxiety of sudden rapid change, which is often beyond human capacity to cope with. The number of people seeking support for mental health and related issues in England, for example, has seen a significant rise in the past five years as political–economic conditions have felt uncertain and volatile to the ordinary people.

Such conditions must trigger the second principle of first securing the advantages of the ‘worst off’. Economically, as Xavier Oberson has shown ‘AI and robots are not only replacing workers in the industrial sectors but are used more and more in service activities … [moreover] many workers will not have sufficient time or skills to adapt to the constantly evolving automation and economic changes.’ (p. 11)

Under these circumstances, Oberson warns that a ‘triple negative effect’ may result in the long term. It will be brought about by the loss of jobs, impacting on the reduction of income tax paid to the State, at a time when the need of the unemployed for support and retraining would be going up.

Though controversial, proposals to tax and redistribute income from the use of AI, such as proposals for a universal basic income, must be urgently discussed. They will go at least some way to addressing the economic concerns when experts are warning that ‘[So] far, the evidence is this: The fourth industrial revolution seems to be creating fewer jobs in the industries than previous revolutions’ (Klaus Schwab quoted in Oberson 2024, p.12) and they do not expect this trend to reverse.

Moreover, the new social contract must take into account more recent scholarship showing that the modern industrial revolutions, though effected in the West, were significantly helped by wealth drawn from many different places. Consequently, a fair distribution of wealth must be done on a planetary basis. As the graphic below shows, the current problems of poverty and underdevelopment in large parts of the world are crying out for a more equitable distribution of the existing levels of wealth rather than prioritising further growth or increasing production.

Fig. 3: Distribution of wealth in the world, 2021

Source: Chart: The Global Wealth Pyramid | Statista

The need of the hour is to capture all those who may be marginalised under the new technological dispensation by economic deprivations as well as the new but significant threatened deprivations of loss of freedom, dignity and wellbeing from AI use too. A new and just social contract must, therefore, provide a certain minimum protection to all against these conditions even if ‘a Millian “steady state” of zero real growth’ (Leif 2021) is the price to pay.

Human–technology relationship, innovation and Gandhian ‘mind-body-heart human’ vision

Since the fall of the Berlin Wall in 1989 and the diminishing of the challenge of Communism as a system of government, efforts to combat poverty and promote development have often been translated into promoting ‘innovation’ in all societies.

An influential definition of innovation is from Joseph Schumpeter (Becker et al 2012). His ideas of innovation, defined as ‘new combinations’ (Becker et al 2012, p. 924), and entrepreneurship, defined as the ability to push new combinations through against resistance, have received much attention. These are supposed to be intrinsically connected to socio-economic development.

In the 21^st century, Schumpeterian innovation has brought together technological research in academia and its conversion into marketable commodities as an almost unassailable shibboleth of economic development stories. Despite their novel mechanisms of exploitation, however, digital technologies in the hands of business still produce societal relationships of ‘haves’ and ‘have-nots’ that need redressing.

As Maskus points out, it is not clear that ‘a patent regime alone can stimulate innovation’(Maskus 2012, p. 263). The narrative of technology as solely responsible for the phenomenal growth in Britain’s wealth in the modern era stands corrected by recent scholarship.  This scholarship shows that wars, violence and empire had a seminal role in allowing technology to become a tool that could draw wealth to Britain while making erstwhile rich countries like India poor (Wardley-Kershaw and Schenk-Hoppé 2022).

In fact, technology-led growth seems always to produce huge wealth inequalities, which when challenged, lead to a race for ever greater production of wealth to accommodate ever higher standards of material comfort for a larger and larger number of people. This is often accompanied by ‘a dialectic of domination, alienation, and other crises of modern life’ (Allen 2019, p. 116).

A new and just social contract must therefore accommodate ‘possibilities for a radical paradigm shift with qualitatively different values and a different framework and way of living’ (Allen 2019, p. 99) where ‘rules of exchange are not defined by commerce but depend on the ability to share and build on the work of others’ (Lessig 2011) aiming at a richer experience of humanness.

This new social contract envisions a new human–tech relationships that acknowledges the always-partial nature of human knowledge. As such, this view, while recognising the utility of technology to improving human lives, emphasises its inherently ambiguous impact on global ecology. The new social contract places humans in a relationship of mutual dependence rather than confrontation with nature and therefore calls for mindful limits on consumption rather than extolling limitless economic growth.

Some of these are already available to us in the form of a ‘harmoniously integrated, mind–body–heart human’ vision of Gandhi as articulated by Dougals Allen with its ‘ key criterion of the need for us to significantly lower our level of consumption’(Allen 2019, p. 128), the ideas of Herbert Marcuse as developed by Andrew Feenberg (Feenberg 2024) and Feenberg’s ideas of a technical politics to humanise and democratise technology (Kirkpatrick 2020), and a more communal sharing of knowledge as embodied in the Creative Commons movement (Lessig 2011) that aims to keep the Internet as an inclusive free space rather than one colonised by commercial interests.

The new social contract takes the human being as currently constituted to be an organism that gathers and processes knowledge in a variety of ways, which, for example, was reflected in Alan Turing’s acceptance of telepathy (Turing 1950) as the most persuasive of arguments against ‘thinking machines’ in his famous article that otherwise extolled being ‘supercritical’ (see, Blog 1 in this series ‘The age of Artificial Intelligence (AI)). It values diversity of learning ways amongst humans.

Placing this human organism in sympathy with its natural environment, the new social contract recognises meaning-creation through ethics and aesthetics as a public good that ensures that self-interest in individuals, entities or institutions does not overwhelm collective good.

References

Allen, D. (2019) Gandhi after 9/11: Creative Nonviolence and Sustainability. Oxford: Oxford University Press. DOI: 10.1093/oso/9780199491490.

Ashrafan, H. (2023) ‘Engineering a social contract: Rawlsian distributive justice through algorithmic game theory and artifcial intelligence’, AI and Ethics 3, pp.1447–1454. https://doi.org/10.1007/s43681-022-00253-6

Bankins, S. and Formosa, P. (2021) ‘Ethical AI at work: the social contract for Artificial Intelligence and its implications for the workplace psychological contract’ in M. Coetzee and A. Deas (eds), Redefining the Psychological Contract in the Digital Era: Redefining the Psychological Contract in the Digital Era. Switzerland: Springer Nature.

Bareis, J. and Katzenbach, C. (2022) ‘Talking AI into being: the narratives and imaginaries of national AI strategies and their performative politics’. Science, Technology, & Human Values, 47(5), 855-881 [online]. Available at: https://doi.org/10.1177/01622439211030007 (accessed 12.12.23).

Becker, M. C., Knudsen, T. and Swedber, R. (2012)  ^‘Schumpeter’s Theory of Economic Development: 100 years of development’, J Evol Econ 22, pp. 917–933. DOI 10.1007/s00191-012-0297-x. Published online: 23 October 2012.

Bradford, A. (2023) Digital empires: the global battle to regulate technology. Oxford: Oxford University Press.

Carlsson, V. and Rönnblom, M. (2022) ‘From politics to ethics: transformations in EU policies on digital technology’, Technology in Society, 71 [online]. Available at: https://doi.org/10.1016/j.techsoc.2022.102145 (accessed 8.3.24).

Choucri, N. (2020), ‘UN75: Social Contract 2020 – Toward Safety, Security, & Sustainability for AI World’. The United Nations [online]. Available at: https://www.un.org/en/academic-impact/un75-social-contract-2020-toward-safety-security-sustainability-ai-world#:~:text=Social%20Contract%202020%20helps%20steer,evaluation%20of%20public%20officials%20%E2%80%93%20supported (accessed 25 February 2024).

D’Agostino, F., Gaus, G. and Thrasher, J. (2024) ‘Contemporary approaches to the social contract’, The Stanford Encyclopedia of Philosophy (Spring 2024 Edition), Edward N. Zalta and Uri Nodelman (eds.), forthcoming [online]. Available at:  https://plato.stanford.edu/archives/spr2024/entries/contractarianism-contemporary/ (accessed 9.3.24).

Darwall, S. (2017) ‘Equal dignity and rights’ in R. Debes (ed.) Dignity: A history. OXFORD PHILOSOPHICAL CONCEPTS. New York, 2017 and online edition Oxford Academic, 6e [online]. Available at: https://doi.org/10.1093/acprof:oso/9780199385997.003.0009, Pages 181–202 (accessed 20.3.24).

Deffains, B. (2022) ‘Rethinking the social contract in the digital age’ in Ytheir, J.M. (co-ordinator), Economic reason and political reason: deliberation and the construction of public space in the society of communication. London: John Wiley & Sons, pp. 227–258.

European Commission (EC) (2021) Proposal for a Regulation of the European Parliament and of the Council laying down harmonised rules on artificial intelligence (Artificial Intelligence Act) and amending certain union legislative acts {SEC(2021) 167 final} – {SWD(2021) 84 final} – {SWD(2021) 85 final}. Available at: https://artificialintelligenceact.eu/the-act/ (Accessed: 16 November 2023).

Feenberg, A. (2024) ‘Andrew Feenberg: Marcuse and the New Left – Book Launch’ [online]. Available at: https://www.youtube.com/watch?v=5faYzlSTORE&ab_channel=SFU%27sVancityOfficeofCommunityEngagement (accessed 10.3.24).

Guterres, A., ‘Secretary-General’s Nelson Mandela Lecture: “Tackling the Inequality Pandemic: A New Social Contract for a New Era” [as delivered]’ 18 July 2020 [online]. Available at: Secretary-General’s Nelson Mandela Lecture: “Tackling the Inequality Pandemic: A New Social Contract for a New Era” [as delivered] | United Nations Secretary-General (accessed 10.3.24).

Harari, Y. N. (2017) Homo Deus. Kindle edn. Vintage.

Hobbes, T. (1651) Leviathan, or, the matter, forme, & power of a commonwealth ecclesiasticall and civil. [online] Available at: https://en.wikisource.org/wiki/Leviathan/The_First_Part (Accessed: 26 January 2024).

International Organisation of Employers (IoE), ‘A new social contract: deliberations of employers’, February 2023 [online]. Available at: https://www.ioe-emp.org/news/details/employers-position-on-a-new-social-contract

Interview with Lawrence Lessig, February 2011 [online]. Available at: https://www.wipo.int/wipo_magazine/en/2011/01/article_0002.html (accessed 25.3.24).

Kirkpatrick, G. (2020) Technical politics: Andrew Feenberg’s critical theory of technology. Manchester: Manchester University Press.

Kukathas, C. and Pettit, P. (1990) Rawls: A theory of justice and its critics. Cambridge: Polity Press.

Lord Sales at the Sir Henry Brooke Lecture for BAILII, London: Algorithms, Artificial Intelligence and the Law [2019] UKSpeech 191112 (12 November 2019).

Maskus, K. E., (2012) Private rights and public problems: the global economics of intellectual property in the 21^st century. Washington, DC: Peterson Institute for International Economics.

Morreale, F., Bahmanteymouri, E., Burmester, B., Chen, A. and Thorp, M. (2023) ‘The unwitting labourer: extracting humanness in AI training’, AI & Society [online]. Available at: https://doi.org/10.1007/s00146-023-01692-3 (accessed 20.3.24).

Mwamadzingo, M., Koyi, G., Watt, M. and Waziri, K. M. (2023) ‘The world of work and the new social contract’, International Journal of Labour Research, 12 (1-2), pp. 20-40 [online]. Available at: https://www.ilo.org/wcmsp5/groups/public/—ed_dialogue/—actrav/documents/publication/wcms_907017.pdf

Oberson, X. (2024) Taxing Artificial Intelligence. Cheltenham: Edward Elgar. https://www.e-elgar.com/shop/gbp/taxing-artificial-intelligence-9781035307548.html

Sætra, H.S., Coeckelbergh, M. and Danaher, J. (2022) ‘The AI ethicist’s dilemma: fighting Big Tech by supporting Big Tech’, AI Ethics, 2, 15–27 [online]. Available at: https://doi.org/10.1007/s43681-021-00123-7 (accessed 15.3.24).

Sen, A. (1999) Development as freedom. Oxford: Oxford University Press.

Sullins, J. (2023) ‘Information Technology and Moral Values’, The Stanford Encyclopedia of Philosophy , Edward N. Zalta and Uri Nodelman (eds.) [online]. Available at:  https://plato.stanford.edu/archives/fall2023/entries/it-moral-values/.

Turing, A. M. (1950) ‘Computing machinery and intelligence,’ Mind, vol. 49, pp. 433–460. Available at: Article 1 in Google search here (Accessed: 30 October 2023).

Veale, M. and Borgesius, F. M. (2021) ‘Demystifying the draft EU Artificial Intelligence Act:  analysing the good, the bad, and the unclear elements of the proposed approach’ in Computer Law Review International, 22 (4), pp. 97-112. 10.9785/cri-2021-220402

Wardley-Kershaw, J.  and Schenk-Hoppé, K. R. (2022) ‘Economic growth in the UK: the inception’, World, 3(2), 162174. https://doi.org/10.3390/world3020009.

Wenar, L. (2021) ‘John Rawls’, The Stanford Encyclopedia of Philosophy (Summer 2021 Edition), Edward N. Zalta (ed.) [online]. Available at: https://plato.stanford.edu/archives/sum2021/entries/rawls/ (accessed 19.3.24).

Zhu, L., Xu, X., Lu, Q., Governatori, G., Whittle, J. (2022) ‘AI and ethics—operationalizing responsible AI’. In Chen, F., Zhou, J. (eds) Humanity Driven AI. Cham.: Springer. https://doi.org/10.1007/978-3-030-72188-6_2.

Zuboff, S. (2019) ‘Surveillance Capitalism and the Challenge of Collective Action’, New Labor Forum, 28:1, 10-29 [online]. Available at : https://journals.sagepub.com/doi/epub/10.1177/1095796018819461 (accessed 16.3.24).

Photo by Steve Johnson on Unsplash

EU’s AI Act: Why do we need it?

Part 3: The need for a new social contract

The feather-like controls mentioned as ‘panopticism’ by Michel Foucault in his famous Discipline and punish appeared in a startlingly new iteration in the second decade of the 21^st century and technology was very much at the heart of the process.

To grasp the nature of the current relationship between technology and society, against the background of the various contestations looked at in Part 2, it is helpful to begin with a key argument made by the philosopher of technology Andrew Feenberg who has posited that ‘technology as currently designed is a key element in the maintenance of political hegemony’ (Kirkpatrick 2020, p.71).

Hegemony is taken here in the sense used by Gramsci to mean ‘a form of domination that works through cultural media of representation’ (Kirkpatrick 2020, p. 71). When a particular group or class succeeds in making its own version of cultural meanings accepted by the majority, that group or class is hegemonic in that culture. It is in this sense that Feenberg’s Critical Theory of technology considers modern technology, along with the socio-economic formations of capitalism, to be hegemonic. By virtue of this role, then, technology, ‘as currently designed’, is considered a main hurdle in achieving a society where every individual is empowered to fulfil their potential, which is the main goal of socialist and other progressive movements for change.

Fig. 1: Cultural hegemony

Digital technology, however, seemed to promise a different iteration of technology at the turn of the century. In the early years of the 21^st century, the internet was considered a force for good, a vehicle for promoting an open and democratic global society where ordinary individuals could enjoy freedoms of speech and association, and come together to foster genuine popular solidarity, free from the physical constraints and political oppressions of the non-virtual world. The Arab Spring of early 2010s was considered a typical flowering of the Internet’s democratic and liberating potential.

In the world of realpolitik, with the astronomical growth of digital technology from the end of 20^th century, political elite around the world had to take a position on the ‘rise of mass internet use, followed by the ongoing absorption of much of society into social media’ (Kirkpatick 2020, p. 8).

The growth of the economic imperative and the eclipse of the political process

Surveying the EU’s policy on digitalisation in the period 2010–2020, Carlsson and Rönnblom (2022) show that ‘Since the late 2010s, the EU has promoted a “Digital Europe”’ (p. 2).

They note that between 2010 and 2016 this promotion was focussed on ‘good governance and services to citizens as important democratic values related to digital technology’ (p. 4). However, from 2016 a change occurred. From 2016 to 2020 their analysis showed that the values of ‘economic growth and competitiveness’ (p. 4) began to dominate. At the same time, there was a shift in the policy discourse from politics to ethics while the role and influence of private sector in the EU’s policy grew manifold.

Fig. 2: Shift in the EU’s digital policy, 2010–2024

Based on Carlsson and Rönnblom (2022) and Bradford (2023)

Similarly, Bareis and Katzenbach, also writing in 2022, found the influence of corporates to be equally powerful in media discourse on AI in all those societies where AI technology was beginning to dominate discussions of economic growth. From the 2020s, almost all economically powerful countries or political groupings like China, USA and Europe began to experience this phenomenon. Bareis and Katzenbach suggest that the shift in discourse from politics to ethics gave a disproportionate amount of influence to the corporates because regulatory frameworks ‘foregrounding ethics as a normative framework (Jobin, Ienca, and Vayena 2019)’ help corporates escape regulation even while providing governance of a sort.

From around 2020, however, voices of concern at this political power of the corporates – without corresponding accountability – began to be heard in prominent fora. Like Carlsson and Rönnblom (2022) these voices suggested that the co-optation of the political elite in the digital, especially AI, policy context was leading to an emasculation of ‘the space for collective action and the articulation of conflicts’ (p. 7) that is crucial to the functioning of the democratic process.

The difficulties in problematising the influence of AI technologies

Yet, it seemed difficult to capture the political harms of a technology whose growth seemed driven by popular demand. Graeme Kirkpatrick (2020), in discussing Feenberg’s Critical theory of technology, points out that existing categories of critique seem to be inadequate for capturing the problematic nature of AI technologies. Unlike the technology of the industrial era, AI is not brutal or overtly exploitative. On the contrary, it is related to leisure pursuits as much as it is implicated in work practices. In fact, there is an ‘emphasis … on the smooth flow of consumerist experience [that often ends in] …resolving any trace of antagonism [between the individual user’s interests and those of the corporate providers] in positive, pleasurable experiences’ (p. 116)

It took time to establish that the devices that are ‘natural’, easy to use and seamless to incorporate into both work and leisure practices have a ‘manipulative aspect’ that can ‘reach through symbolic communicative processes and steer people into preferred courses of action’ (Kirkpatrick 2020, p. 114).

This Foucauldian feather-light control of the individual user can be said to have dangerously shifted the balance of power between the mass of ordinary people, often termed ‘users’ or ‘consumers’ of social, economic and even political services, and the small group of ‘Big Tech’ firms who command resources that are beyond the reach of even a majority of governments. The realisation that the backing of this immense corporate power by the political elite leaves little room for guarding the freedom and agency of the ordinary individual began to elicit calls for a new ‘social contract’ to redress this imbalance of power in society.

The AI Act and the re-creation of space for the political

It is against this background that the proposal for legislation to define and limit the role of AI technologies in the AI Act became a radical break from the global political preference for ethics over politics to regulate a technology that was fast gaining unprecedented leverage over the lives of ordinary individuals through both work and play.

According to Ernest Laclau and Chantal Mouffe’s concept of ‘radical democracy’, making visible those power relations in society that marginalise and oppress given groups and individuals is at the very heart of democracy. The ‘capacity of political discourse to produce radical re-articulations of dominant ideas’ (Kirkpatrick 2020, p. 72), so that the limits of what may be proposed and done about changing a given mode of reality, is crucial to democratic politics with all its emancipatory potential.

This process became possible when the civil society, including individuals and concerned organisations, was invited to articulate their views on regulation of AI by the EU in 2020. After the public consultation and following a trialogue amongst the European Commission, the European Parliament and the European Council, the final draft of the Act stated that it sought to promote the

‘uptake of human centric and trustworthy artificial intelligence, while ensuring a high level of protection of health, safety, fundamental rights enshrined in the Charter, including democracy, rule of law and environmental protection against harmful effects of artificial intelligence systems in the Union and supporting innovation.’

By acknowledging that the protection of fundamental rights, democracy and the rule of law are considerations that have at least an equal, if not greater, weight than commercial reasons, the AI Act created a space where the power of Big Tech or the handful of corporations commanding the vast majority of resources and the technical knowhow of the globalized digital sector may at least be contested by or, on behalf of, the ordinary citizens.

Ultimately, politics is about power relations in society. Writing in December 2021, Sætra et al. stated that in the context of AI ‘Big Tech holds much power, while others are perceived to be relatively powerless, or exposed, to this power’ (p. 21). The AI Act was, therefore, urgently needed to bring back politics in the form of State’s regulation of the power of the corporates in the digital sphere, with special relation to the growth of AI. If this space is to be utilised to its full potential, however, an understanding of a new social contract that focuses on the special conditions of the AI Age needs to be clarified and disseminated to the mass of the people as we are all affected by it.

The next blog in the series looks in detail at the environment threatening the existing social contract.

References

Bareis, J. and Katzenbach, C. (2022) ‘Talking AI into being: the narratives and imaginaries of national AI strategies and their performative politics’. Science, Technology, & Human Values, 47(5), 855-881 [online]. Available at: https://doi.org/10.1177/01622439211030007 (accessed 12.12.23).

Bradford, A. (2023) Digital empires: the global battle to regulate technology. Oxford: Oxford University Press.

Carlsson, V. and Rönnblom, M. (2022) ‘From politics to ethics: transformations in EU policies on digital technology’, Technology in Society, 71 [online]. Available at: https://doi.org/10.1016/j.techsoc.2022.102145 (accessed 8.3.24).

Kirkpatrick, G. (2020) Technical politics: Andrew Feenberg’s critical theory of technology. Manchester: Manchester University Press.

Sætra, H.S., Coeckelbergh, M. and Danaher, J. (2022) ‘The AI ethicist’s dilemma: fighting Big Tech by supporting Big Tech’, AI Ethics, 2, 15–27 [online]. Available at: https://doi.org/10.1007/s43681-021-00123-7 (accessed 15.3.24).