EU’s AI Act: Why do we need it?

Part 4: The digitalised environment and the threats to the existing social contract

Social contract is an important conceptual tool often used to delineate the basic features of a legitimate government, the justifications for it and the nature and scope of its role vis-à-vis the governed. It ‘is concerned with public justification, i.e., “of determining whether or not a given regime is legitimate and therefore worthy of loyalty” (D’Agostino 1996, 23)’ (D’Agostino et al. 2024). Hutan Ashrafian identifies the fact that the concept had been known in ancient times too (Ashrafian 2023, p. 1448), however, it was revived by Thomas Hobbes (1588–1679) in England in the 17th century, and has since been utilised most prominently by John Locke (1632–1704), Jean Jacques Rousseau (1712–1778) and John Rawls (1921–2002).

AI use is spreading at a time of loss of trust in governments and institutions and deep polarisation in society. Digitalisation is in part both a cause and effect of this environment. We are faced with, what Anu Bradford has called a battle of ‘digital empires’ (Bradford 2023). This battle involves both ‘horizontal’ and ‘vertical’ value clashes. The horizontal clashes are battles between governments that represent democratic vs authoritarian cultural values, while the ‘vertical’ battles are between governments and Big Tech, whose domain of influence stretches across territorial boundaries. Under the circumstances, a broad consensus on what constitutes legitimate power distribution in society must ground itself in certain principles drawn from across a wide range of philosophical and political ideas.

What follows is an exploration, first, of the threats to the existing social contract in the current environment, and, second, the next blog in the series explores some essential ideals to underpin a new and just global social contract that can make for a fair ‘division of advantages that arises through social cooperation’ (D’Agostino et al. 2024).

Digital technology and the threat to social contract

Deffains (2022) argues that our current ‘social contract’ – the implicit understanding we have of the legitimate role of government and of the rights and duties of individuals and entities forming society – was forged during a period when ‘work filled all the space’ (Deffains 2022, p. 234). People’s identities were often linked to their work, as was their level of safety and security against risks such as accidents, illnesses and unemployment. A progressive vision and implementation of this social contract envisaged a ‘mutualization of risks’ within national boundaries. It created a ‘solidarity’ amongst the people on principles akin to Rawls’ veil of ignorance. Since no one knew what risks they might face in future, they agreed to a sharing, or ‘redistribution’, of their advantages with those more afflicted, to ensure they had some level of security should they be so afflicted too.

Fig. 1: John Rawls’ idea of the ‘veil of ignorance’

Deffains sees this veil of ignorance being torn asunder by the latest technological developments. On the one hand, automation is forcing a larger number of people into ‘less routine tasks that are more autonomous but less protected’ (p. 234), on the other, ‘increasingly sophisticated processing’ (p. 243) of individual data seems to be bringing nearer the prediction of risk for each individual separately in terms of their health risks, employment risks and so on. Under these circumstances, solidarity in society is ‘confronted with the promise of increasingly individualized solutions’ to life’s risks (p. 245). Once people know their own level of risks, will they still stand in solidarity with those facing greater levels as is the case at the moment, or will each individual be expected to carry their own burden of risk?

This level of risk prediction, however, is not merely a threat to social solidarity, it also threatens the individual. The gain of knowledge about themselves is bought by individuals at a cost of their freedom. The lightening spread of ever more user-friendly devices like smartphones and even electronic toothbrushes means the ‘user–customer’ is under constant ‘surveillance’. Their data is being ‘constantly analyzed and dissected [and]… manipulated’ (p. 243) for the creation of more paid-for services and a ‘mental substitution system’ where our devices think for us, guiding, persuading and nudging us ‘to act progressively according to standardized and normalized patterns (eat well, sleep well, do sports, do not smoke)’ (p. 244). The surveillance here is of the finest Foucauldian variety so that we rarely feel our freedom slipping away with the drip-drip suction of our data.

Fig. 2: The ways in which AI affects individuals and social solidarity (based on Deffains 2022)

The role of Big Tech

Yet, taken together, the situation represents a serious threat to freedom, democracy and perhaps even the very existence of humanity. Michon and Hyppolite (2018) analysed the business practices of the American ‘Big Tech’, or the few technological giants who dominate the digital tech industry that include Apple, Amazon, Alphabet (owner of Google), Meta, Microsoft, Intel, Oracle, Cisco, IBM and Qualcom (they focus on the American firms though more recently some Chinese firms have joined the group of huge international tech operators too) to conclude that ‘the American tech giants have acquired dominant positions, which they now use to hamper competition’. Following these practices and leveraging the respect for science and technology as the ‘foundation of humanity’s march toward progress’ (Deffains 2022, p. 248), these firms have added significant political and popular influence to their economic power without corresponding accountability.

Their services, often seemingly free to the user, e.g., Google searches and Facebook communication, provide them with the ‘data’ that is the bread and meat of their businesses, helping them devise new products that further increase their reach. While competition and anti-trust laws struggle to keep pace with the emerging business practices of digital operators, which are different enough from more traditional business practices to often evade the reach of the existing laws, the power of Big Tech grows and spreads in the lives of ordinary individuals who now increasingly work through digital media as well as conduct leisure and social activities through them.

Both Deffains, and Michon and Hippolyte warn that these firms have their own visions of the future that, if realised, can translate their economic power into an overlordship that can potentially stretch from political control to what Feenberg has called ‘psychic oppression’ (Feenberg 2024, 30:34) of the individual. Writing of GAFA (Google, Apple, Facebook and Amazon), Deffains says that ‘they also have a dominant and universal vision (identity, currency, power, etc.) which is dangerous if left unchecked’. Michon and Hippolyte point to the fact that the tech giants are careful to hoard a substantial amount of their ‘unparalleled’ profits, unlike other similar companies that tend to reinvest, while a much higher ratio of their R&D budget is spent on developing new products that are likely to extend their reach into ever newer areas, e.g., driverless cars, augmented reality and solar-powered drones.

Fig. 3: Comparison of R&D expenditure in percentage of revenues

Other ethical issues

Other ethical problems about AI and Big Data include ‘platform and monopoly power [10-12], social media and its effects on human relations [13], and issues related to human cognition, polarization, filter bubbles, and pressure to conform [1,4,5,14] … privacy and surveillance [15,16], the manipulative power of tech [17,18], algorithmic bias resulting in racist and discriminatory AI systems [6,8,19,20] , and the carbon footprint of AI and Big Data [21-23]’ (Sætra et al 2021, p. 16).

The next blog in the series explores some fundamental principles that need to be incorporated into the new social contract to mitigate the effects of these threats and safeguard liberty and justice for people everywhere.

References

D’Agostino, F., Gaus, G. and Thrasher, J. (2024) ‘Contemporary approaches to the social contract’, The Stanford Encyclopedia of Philosophy (Spring 2024 Edition), Edward N. Zalta and Uri Nodelman (eds.), forthcoming [online]. Available at: https://plato.stanford.edu/archives/spr2024/entries/contractarianism-contemporary/ (accessed 9.3.24).

Deffains, B. (2022) ‘Rethinking the social contract in the digital age’ in Ytheir, J.M. (co-ordinator), Economic reason and political reason: deliberation and the construction of public space in the society of communication. London: John Wiley & Sons, pp. 227–258.

Feenberg, A. (2024) ‘Andrew Feenberg: Marcuse and the New Left – Book Launch’ [online]. Available at: https://www.youtube.com/watch?v=5faYzlSTORE&ab_channel=SFU%27sVancityOfficeofCommunityEngagement (accessed 10.3.24).

Michon, A. and Hyppolite, P. A., ‘Big Tech dominance (2): A barrier to technological innovation?’, December 2018 [online]. Available at: https://www.fondapol.org/en/study/big-tech-dominance-2-a-barrier-to-technological-innovation/ (accessed 10.3.24).

Sætra, H.S., Coeckelbergh, M. and Danaher, J. (2022) ‘The AI ethicist’s dilemma: fighting Big Tech by supporting Big Tech’, AI Ethics, 2, 15–27 [online]. Available at: https://doi.org/10.1007/s43681-021-00123-7 (accessed 15.3.24).

EU’s AI Act: Why do we need it? Part 4: The environment for the new social contract